Two classes in the plane with known densities f0, f1 and P(Y = 1) = π. The Bayes rule h*(x) = 1 exactly when πf1(x) > (1 − π)f0(x) has the smallest possible error L(h*). Any other rule pays an excess risk that has an exact formula: L(h) − L(h*) = ∫ |πf1 − (1 − π)f0| over the set where h and h* disagree. Fit LDA, QDA and logistic regression to a sample, see where they disagree with the Bayes rule (shaded for the method you pick), and watch how their excess risk falls with the sample size, or stops falling when the method's model is wrong.
Class 1 has nπ points (rounded) and class 0 the rest. Integrals are sums over a 181 × 181 grid of spacing 0.08 on [−7.2, 7.2]². The curves average 40 data sets for each n; every data set comes from a seeded generator, so the same settings always give the same numbers.
code/ch22_classification.py (gauss_scenario, gauss_truth, gauss_sample, plugin_rules, excess_risks, learning_curve).