Interactive demo · Nonparametric Statistical Inference · Chapter 4

Regressograms and kernel local averages

The simplest smoothers average the responses Yi whose covariates lie near x. A regressogram averages over fixed bins of width h; a kernel local average r̂(x) = Σℓi(x)Yi uses the weights ℓi(x) = K((xi − x)/h) / ΣjK((xj − x)/h), a window that moves with x. The pink sticks show the weights at the point x0. The lower panel is the leave-one-out cross-validation score, an estimate of the prediction risk, against h: too small and the fit chases the noise, too large and it flattens the structure.

Data

Smoother

Bandwidth (bin width) h

Show the weights at x0

Try these

The bandwidth runs over 60 values evenly spaced on the log scale (2.9 to 200 for LIDAR, 2.1 to 200 for CMB, 49 to 800 grams for BPD; the end points avoid the integer spacings of the data, where a boxcar window would sit exactly on a data point). For the regressogram, h is the bin width: the range of the xi is cut into round(range/h) equal bins. For compact kernels h is the half-width of the window; for the Gaussian kernel it is the standard deviation. In the cross-validation score a point with no other point in its window (or bin) cannot be predicted and is left out; the readout counts such points. The effective number of points at x0 is 1/Σℓi(x0)2.

In the lecture notes