A binary response can be modelled with P(y = 1 | x) = ℓ−1(w0 + w1x) for different links ℓ. The logit is the canonical one; the probit uses the Gaussian CDF (12.25); the complementary log-log (12.26) is asymmetric: it approaches 1 much faster than it leaves 0. The data here come from a complementary log-log truth. The probit and logit curves are almost the same up to a rescaling of w, so they cannot be told apart; the complementary log-log fits better, but only with enough data and only in the tails. All fits are maximum likelihood, by Fisher scoring (IRLS), which is Newton's method for the canonical logit link and a weighted least squares iteration with expected weights for the others.
Data: 600 inputs uniform on [−3.2, 2.4]; truth P(y = 1 | x) = 1 − exp(−exp(−0.4 + 0.95x)) (dashed). Deviance is 2 × the negative log likelihood here (the saturated model has likelihood 1 for binary data); AIC adds 4 for the two parameters. The log scales show the probability of a 1 (lower tail) or of a 0 (upper tail), where the links differ most.
code/ch12_generalized_linear_models.py (LINKS, glm_fit, bernoulli_deviance).