Each of the 120 training points belongs to one of three classes. The classifier stores them all and, for a new point, returns the class frequencies among its K closest stored points (16.1); the colours show the majority label everywhere in the plane. With K = 1 the regions are the Voronoi cells of the training points and the training error is zero; as K grows the boundaries smooth out and the training error rises, while the test error (600 fresh points) follows a U shape. Click in the plane to place a query point and see its neighbours and vote. The second slider stretches the vertical axis, which changes the Mahalanobis metric (16.2) and with it who counts as a neighbour.
Data: 120 training points (40 per class, each class a mixture of two Gaussians) and 600 test points drawn from the same distribution, simulated. The distance is d(x, x′)2 = (Δx1)2 + s2(Δx2)2, that is the Mahalanobis distance with M = diag(1, s2). A tie between classes goes to the tied class that owns the nearest neighbour. The training error counts each training point as its own nearest neighbour, so it is 0 at K = 1.
code/ch16_exemplar_based_methods.py (knn_proba, knn_predict, knn_error_curve).