A linear model y = wTx with eight inputs, only three of which matter (true weights 3, 1.5, 0, 0, 2, 0, 0, 0), is fitted by minimising ½∑(t − wTx)2 + (λ/2)∑|wj|q (9.19). With q = 2 (ridge, weight decay) every coefficient shrinks smoothly and none is ever exactly zero; with q = 1 (the lasso) coefficients hit zero one after another as λ grows, leaving a sparse model. The right panel shows why: the minimum of the error inside the region ∑|wj|q ≤ η (9.20) sits at a corner when the region has corners.
Inputs are standardised on the training set and the targets centred, so no bias is needed. The lasso is solved by cyclic coordinate descent (soft thresholding at λ/2), ridge in closed form, (XTX + λI)−1XTt. Validation error is measured on 200 fresh points from the same model (noise standard deviation 2).
code/ch09_regularization.py (lasso_data, lasso_cd, ridge, lq_constrained_2d, supplementary Figure 9.S1); the coefficients shown here equal the Python ones.