Interactive demo · Nonparametric Statistical Inference · Chapter 6

Kernel density estimation: bandwidth, cross-validation and a band

The kernel estimator f̂(x) = (nh)−1 Σ K((x − Xi)/h) puts a bump of mass 1/n on every observation. The bandwidth h matters far more than the shape of K. The cross-validation score Ĵ(h) = ∫f̂2 − (2/n) Σ f̂(−i)(Xi) is an unbiased estimate of the risk minus ∫f2, computed here exactly from the pairwise distances. The Normal reference rule 1.06σ̂n−1/5 is the best bandwidth for a Normal f: watch it fail on a density with sharp features.

Data

Kernel

Bandwidth h

Try these

Kernels: Gaussian; Epanechnikov (3/4)(1 − u2) and boxcar 1/2 on [−1, 1]. Their bandwidth grids are 2.2 times the Gaussian one, so that equal slider positions give similar smoothness. The score uses the convolution K*K in closed form. The band (Gaussian and Epanechnikov) uses that √f̂ has nearly constant standard deviation √(∫K2/(4nh)) and takes its constant from the tube formula with κ0 = (length/h)√(∫K′2/∫K2); it covers Ef̂, a smoothed f. ISE = ∫(f̂ − f)2 over [−4, 4].

In the lecture notes