Interactive demo · Probability and Statistical Inference · Chapter 20

Kernel regression with a confidence band

The Nadaraya–Watson estimator r̂(x) = Σwi(x)Yi averages the responses with kernel weights; the bandwidth h sets how far it looks. Here h is chosen by leave-one-out cross-validation, which needs only one fit: the left-out residual is (Yi − r̂(xi))/(1 − Lii). The noise level σ comes from successive differences of the responses, and the band r̂ ± q σ̂ (Σwi2)1/2 covers the smoothed curve E r̂, not r itself. Try the WMAP spectrum of the book's Example 20.23, the motorcycle data (where the noise is far from constant) and local linear regression at the edges.

Data

Estimator

Bandwidth h

Confidence band

Try these

Gaussian kernel throughout. The bandwidth runs over 121 values spaced evenly on the log scale (2 to 150 for WMAP, 0.5 to 10 ms for the motorcycle data, 0.03 to 1.5 for the glass data). The band uses m = (b − a)/ω with the effective kernel width ω = 3h and q = Φ−1((1 + (1 − α)1/m)/2). The effective degrees of freedom are tr(L) = ΣLii.

In the lecture notes