An SVM with the radial kernel K(x, x′) = exp(−γ‖x − x′‖²) scores each patient by f(x) = β0 + ∑i∈S αiK(x, xi), a sum of bumps centred at the support vectors. A small γ makes wide, gentle bumps and a nearly linear score; a large γ makes narrow bumps that can fit every training patient. The ROC curve of the score on the 207 training patients keeps improving as γ grows, as in Figure 9.10; on the 90 test patients it does not, as in Figure 9.11. Every fit below is computed in your browser.
297 patients with complete records; 13 predictors coded as 16 columns (ChestPain and Thal as dummies), each standardised with the training means and standard deviations; the same random split into 207 training and 90 test patients as in the course's Figures 9.10 and 9.11. The SVM is fitted by a small SMO solver (the algorithm of LIBSVM, which R's svm() and scikit-learn's SVC use); LDA uses the pooled covariance. A positive score predicts heart disease.
code/ch09_support_vector_machines.py (heart_data, roc_points, auc, lda_score, Figures 9.10 and 9.11).