Interactive demo · Statistical Learning: Theory and Methods · Chapter 13

K-means and LVQ: where should the prototypes go?

A prototype classifier stores a few labelled points and gives a new x the class of the nearest one. K-means places R prototypes in each class by clustering that class alone, so the other classes have no say. LVQ1 (Algorithm 13.1) starts from the K-means prototypes and visits training points at random: the nearest prototype moves a step ε(xi − m) towards xi if the classes agree and the same step away if they do not, with ε falling linearly to zero. Grey rings are the K-means prototypes, filled discs the LVQ prototypes, and the thin lines their paths.

Data

Prototypes per class R

LVQ learning rate ε₀ (falls linearly to 0)

LVQ passes through the data (0 = K-means only)

Random start (K-means and LVQ seed)

Try these

One pass is N LVQ steps (N = 300 for the three-class data, 200 for the mixture data). Test errors use 9,000 test points (three-class data) or the 10,000 test points of Chapter 2 (mixture data). "Separation" is the mean distance from a prototype to the nearest prototype of another class.

In the lecture notes