Interactive demo · Statistical Learning: Theory and Methods · Chapter 5

Smoothing splines: degrees of freedom and the eigenvalues of the smoother

The smoothing spline minimises Σ(yi − f(xi))2 + λ∫f″(t)2dt; its fitted values are f̂ = Sλy with Sλ = (I + λK)−1. In the eigenbasis of K (the Demmler–Reinsch basis) the spline keeps the two linear components untouched and shrinks the k-th component by ρk = 1/(1 + λdk); the effective degrees of freedom are df = trace Sλ = Σρk. Set df with the slider (the matching λ is found by root finding), pick an eigenvector to see how much of it survives, and let leave-one-out cross-validation choose df.

Data

Effective degrees of freedom df = trace S

Eigenvector shown k

Standard-error band

Try these

Ties in x are merged: the distinct values carry the counts as weights and the mean responses (Exercise 5.12), so S acts on the distinct values. λ is reported in the units of the data and, in brackets, for x rescaled to [0, 1] (λ/range3), the scale used by the book. Band: ±2σ̂ times the pointwise standard deviation at the distinct x, σ̂2 = RSS/(N − df). CV uses the shortcut (5.27); GCV replaces each Sii by df/N. The eigen-decomposition is computed once per data set by Jacobi rotations.

In the lecture notes