Regress the 528 × 11 indicator matrix Y of the vowel classes on a basis H of the ten inputs, giving fits Ŷ = SY. The eigenvectors θℓ of YTŶ/N, normalised by the class proportions, are the optimal scores: the first is constant, the others turn each class label into a number that regression on H predicts as well as possible (eigenvalue αℓ2 = 1 − residual). The fitted scores ηℓ(x) are discriminant functions; classify to the nearest class centroid with weights 1/[rℓ2(1 − rℓ2)]. With the linear basis this is LDA; with a richer basis it is flexible discriminant analysis, and a ridge penalty makes it penalised discriminant analysis.
Columns of H are divided by their training standard deviations; the constant is never penalised. Spline knots sit at equally spaced quantiles of each training input. Coordinates are scaled by √wℓ, so distances in the plot are the classification distances (12.54).
code/ch12_svm_flexible_discriminants.py (OptimalScoring, additive_ns_basis, demo_vowel_fit, fig_12_10); the error rates here are those of the Python functions.