A spline of order M is a piecewise polynomial of degree M − 1 with M − 2 continuous derivatives at its knots. The B-splines Bi,M, built from indicators by the Cox–de Boor recursion, are a basis of such splines with small supports (top panel). A regression spline uses a few knots and ordinary least squares, so its smoothness is set by the number of knots. A smoothing spline puts a knot at every data point and minimises Σ(Yi − r(xi))2 + λ∫r''2: the fit is β̂ = (BTB + λΩ)−1BTY, a ridge-type linear smoother whose degrees of freedom fall from the number of distinct x's (interpolation) to 2 (the least squares line) as λ grows.
The covariate is rescaled to [0, 1] for the smoothing spline, so λ has no units. Ωjk = ∫Bj''Bk'' is computed exactly by two-point Gauss–Legendre quadrature on each knot interval. The smoothing spline is cubic whatever M says. The leave-one-out score (5.35) is exact for both methods; GCV replaces Lii by ν/n.
code/ch05_nonparametric_regression.py (bspline_basis, bspline_deriv, omega_matrix, SmoothingSpline, spline_select, regression_spline).