Interactive demo · Nonparametric Statistical Inference · Chapter 5

B-splines, regression splines and smoothing splines

A spline of order M is a piecewise polynomial of degree M − 1 with M − 2 continuous derivatives at its knots. The B-splines Bi,M, built from indicators by the Cox–de Boor recursion, are a basis of such splines with small supports (top panel). A regression spline uses a few knots and ordinary least squares, so its smoothness is set by the number of knots. A smoothing spline puts a knot at every data point and minimises Σ(Yi − r(xi))2 + λ∫r''2: the fit is β̂ = (BTB + λΩ)−1BTY, a ridge-type linear smoother whose degrees of freedom fall from the number of distinct x's (interpolation) to 2 (the least squares line) as λ grows.

Data

Method

Order M of the B-splines

Interior knots k (regression spline, basis panel)

Penalty λ (smoothing spline)

Try these

The covariate is rescaled to [0, 1] for the smoothing spline, so λ has no units. Ωjk = ∫Bj''Bk'' is computed exactly by two-point Gauss–Legendre quadrature on each knot interval. The smoothing spline is cubic whatever M says. The leave-one-out score (5.35) is exact for both methods; GCV replaces Lii by ν/n.

In the lecture notes