Interactive demo · Statistical Learning: Theory and Methods · Chapter 6

Choosing the bandwidth of a kernel density estimate

A Gaussian kernel density estimate f̂(x) = (1/N) ∑ φλ(x − xi) has one tuning constant, the bandwidth λ. Least-squares cross-validation picks the λ minimising LSCV(λ) = ∫f̂2 − (2/N) ∑ f̂−i(xi), an unbiased estimate of the mean integrated squared error up to a constant; likelihood cross-validation maximises the average leave-one-out log density. When the data come from a known density the exact integrated squared error is drawn next to the LSCV curve, so you can see how well cross-validation tracks it, how the rules of thumb behave on a two-peaked density, and why rounded measurements drive cross-validation to λ → 0.

Data

Simulated sample size N

Bandwidth λ (log scale)

Try these

Bandwidths run over 241 log-spaced values from sd/100 to 2 sd. Silverman's rule: 0.9 min(sd, IQR/1.34) N−1/5; normal reference: 1.06 sd N−1/5, the AMISE-optimal λ when f is normal. Simulated data: 0.7 N(0, 1) + 0.3 N(2.5, 0.52).

In the lecture notes