An additive model y = α + f1(x1) + … + fp(xp) + ε is fitted by cycling a one-dimensional smoother over the inputs: term j is replaced by the smoothing spline of its partial residual y − α − Σk≠jfk, then centred (Algorithm 9.1). Each update minimises the penalised criterion PRSS (9.7) over one function with the others fixed, so PRSS falls at every step; the residual sum of squares need not. Step through the updates, change the smoothness of the terms, and see how correlated inputs make the terms trade a straight line back and forth.
Each term is an exact cubic smoothing spline with a knot at every distinct value (ties carry their counts as weights); λj is found so that trace Sj = df. α is the mean of y and every update is centred. Simulated data: y = sin(2x1) + x22/2 + 0.5ε with x2 = ρx1 + √(1 − ρ2) z, 100 points. Ozone: cube root of ozone against temperature, wind and radiation (111 days).
code/ch09_additive_models_trees.py (backfit, SmoothingSpline, ozone_additive, correlated_pair).