Interactive demo · Statistical Methods for Modern Biology · Chapter 12

Learning by heart, cross-validation and the curse of dimensionality

Fit linear discriminant analysis to n samples with more and more features. The error on the training data keeps falling, even when the labels are pure coin flips; the leave-one-out error, which judges each sample by a rule fitted without it, does not. With a few informative features the honest error first falls and then rises again as noise features are added. The last view shows the most common way to fool yourself: choosing the features on all the data and cross-validating afterwards.

View

Data

Samples n

Informative features m and shift δ

Noise features in the pool

Simulated data sets averaged

Scoreboard

Try these

In the lecture notes