Interactive demo · Probabilistic Machine Learning · Chapter 11

Regularisation paths on the prostate data

Log prostate-specific antigen is regressed on eight clinical measurements for 67 men (standardised), and the fit is tested on 30 others. Choose ridge, the lasso or the elastic net and slide the penalty λ. The top left shows every coefficient along the whole path, the top right the coefficients at the current λ against least squares, and the bottom the training and test error. The lasso path is piecewise linear: the vertical ticks mark the critical values of λ where a variable enters or leaves.

Method

Strength log10(1 + λ)

Ridge part of the elastic net μ

Try these

Objective: ½‖y − Xw‖2 + λ‖w‖1 (lasso), ½‖y − Xw‖2 + ½λ‖w‖22 (ridge), and the lasso objective plus ½μ‖w‖22 (elastic net). The intercept is not penalised. At λ = ||XTy||∞ = 58.44 the lasso solution is zero.

In the lecture notes