Fifty points are fitted by least squares on a cubic B-spline basis with knots at quantiles of x. How uncertain is the fitted curve? The least squares (maximum likelihood) answer is μ̂(x) ± 1.96·[h(x)T(HTH)−1h(x)]1/2σ̂. The nonparametric bootstrap refits the curve to B resamples of the pairs and reads the 2.5% and 97.5% points at each x; the parametric bootstrap instead adds N(0, σ̂2) noise to the fit and agrees with least squares as B grows. The Bayesian posterior under β ~ N(0, τI) is shrunk towards zero for small τ and becomes the least squares answer as τ → ∞. Only the bootstrap can also account for choosing the knots from the data.
The noise variance is the maximum likelihood estimate σ̂2 = RSS/N, used by all four methods (as in the book). Percentile bands use the k-th smallest and k-th largest of the B curves at each x, k = round(0.025B) (the fifth for B = 200). With adaptive knots, each resample chooses K = 1, …, 8 by leave-one-out cross-validation, computed from one fit with the hat-matrix shortcut. The fifty points are artificial data of the kind used in the book (whose own are not distributed).
code/ch08_model_inference_averaging.py (SplineLS, bootstrap_curves, percentile_band, choose_K_loocv).