The log PSA of 67 men is regressed on eight standardised clinical measures. Ridge regression (3.41) adds λ∑βj2 to the residual sum of squares; the lasso (3.52) adds λ∑|βj|. Move the slider and watch the coefficients: ridge shrinks every coefficient smoothly and shrinks most along the principal directions in which the inputs vary least (bars, left), while the lasso sets coefficients exactly to zero, and a coefficient is nonzero exactly when its inner product with the residual has reached λ (bars, right).
Ridge is computed from the eigendecomposition XTX = VD2VT of the centred training inputs, λ is found from df(λ) by bisection, and the whole lasso path is computed exactly by least angle regression with the lasso modification (Algorithm 3.2a). The intercept is ȳ − xTβ, never penalised. Test error: mean squared error on the 30 test cases.
code/ch03_linear_regression.py (ridge_svd, lam_for_df, lars, lasso_at_s, Figures 3.8 and 3.10); the coefficients shown here equal the Python ones.