Interactive demo · Statistical Learning: Theory and Methods · Chapter 13

DANN: neighbourhoods that follow the class boundary

Near a class boundary a round neighbourhood mixes the classes. DANN looks first at the kM Euclidean neighbours of the query x0, computes their pooled within-class covariance W and between-class covariance B, and measures distance by D(x, x0) = (x − x0)TΣ(x − x0) with Σ = W−1/2[B* + εI]W−1/2, B* = W−1/2BW−1/2 (13.9). Distances along the direction that separates the local class means are inflated, so the k nearest in this metric line up along the boundary. Click on the left panel to move the query point.

Data

Query point x₀ (or click the picture)

Neighbours for the metric kM

Neighbours for the vote k

Rounding ε

Try these

Ring data: 200 training points per class (orange: N(0, I) conditioned on 4 < |x|² < 9; blue: N(0, I)) and 1,000 test points. Strip data: 200 uniform points, class by the side of x₁ = 1/2 with 10% of labels flipped, and 2,000 test points. Test errors are recomputed with the current kM and k.

In the lecture notes