Interactive demo · Statistical Learning: Theory and Methods · Chapter 6

Local regression at the boundary: the equivalent kernel at work

One hundred points with Y = sin(4X) + noise (standard deviation 1/3) are smoothed by fitting, at each target x0, a constant (Nadaraya–Watson), a line or a parabola by kernel-weighted least squares. Every such fit is a weighted sum f̂(x0) = ∑ li(x0) yi; the weights li(x0), the equivalent kernel, are drawn as stems. Near the boundary the local line bends its equivalent kernel (some weights turn negative) so that ∑(xi − x0)li = 0, which removes the first-order bias that the plain weighted average cannot avoid.

Local polynomial degree d

Kernel

Window

Width

Target point x0

Try these

Metric window: K = D(|x − x0|/λ) with λ the half-width (the standard deviation for the Gaussian). Nearest-neighbour window: λ is replaced by the distance to the k-th nearest xi. The bias and standard deviation use the true curve and σ = 1/3; LOOCV refits without each point in turn.

In the lecture notes