Interactive demo · Probabilistic Machine Learning · Chapter 9

Fisher's linear discriminant: which direction to project on?

Project two-dimensional data onto a line with direction w. Fisher asks for the line that pulls the projected class means apart while keeping each projected class tight: it maximises J(w) = (m2 − m1)2 / (s12 + s22) (9.29), and the maximiser is w ∝ SW−1(μ2 − μ1) (9.42). PCA ignores the labels and keeps the direction of largest variance. Turn the dial, or jump to the two optimal directions, and compare.

Data set

Direction of the line angle in degrees

Try these

Both coordinates are standardised. J is computed on the data exactly as in (9.29), with the sums of squares sk2 (not divided by the class sizes), so only the ratios between directions matter. “Overlap” is the training error of the best threshold on the projection placed halfway between the projected class means. The data are the heights and weights of 199 adults (Davis; one record with the two entries swapped is removed), and a simulated set built so that the largest-variance direction carries no class information.

In the lecture notes