Projection pursuit regression models f(x) = ȳ + Σm gm(ωmTx) with unit directions ωm and smooth ridge functions gm. Terms are added one at a time. For each term the page alternates two steps: smooth the residuals against v = ωTx (a penalised cubic spline with the degrees of freedom you set), then move ω by a Gauss–Newton step, a weighted least-squares regression of the working response vi + ri/g′(vi) on xi with weights g′(vi)2 (halving the step until the fit improves). The arrows in the left panel are the fitted directions and the small dots the path of the first one.
Each ridge function is a cubic B-spline with ten interior knots at equally spaced ranks of v, penalised by λ∫g″(v)2dv, with λ set so that the trace of the smoother equals df (bisection); it is centred over the training points and continued linearly beyond them. “Best of nine starts” tries the least-squares direction and eight angles kπ/8 for every term and keeps the smallest residual sum of squares. The test error is the mean of (f − f̂)2 over the test inputs (2000 points; 1000 for the Exercise 11.5 model, 10,000 for the sum of sigmoids).
code/ch11_neural_networks.py (RidgeSmoother, ppr_single, ppr_fit, ppr_demo_data).