A degree-d polynomial has d + 1 weights and is fitted by minimising the training MSE (1.17). With d + 1 = N weights it passes through every training point, so the training MSE is exactly 0, yet the curve swings wildly between the points. The test MSE, measured on 500 fresh points, falls and then rises again: this U shape (Section 1.2.3) is why we judge a model on data it was not fitted to. The grey dotted line is the noise floor σ², the smallest test MSE any predictor can reach.
The 21 points are a stand-in for the book's data: x = 0, 1, ..., 20 and y = f(x) + σ z with f a quadratic and z standard normal. The random sample draws N inputs uniformly on [0, 20] from a fixed pool of 200. The test set has 500 random inputs. The polynomial is written in Chebyshev polynomials of x/10 − 1 (the same function space as the powers of x, but stable up to degree 20) and fitted by a Householder QR least-squares solve; the degree is limited to N − 1.
code/ch01_introduction.py (poly_fit, fig_1_7).