Interactive demo · Applied Statistical Learning in R and Python · Chapter 9

Radial kernels on the Heart data: training and test ROC curves

An SVM with the radial kernel K(x, x′) = exp(−γ‖x − x′‖²) scores each patient by f(x) = β0 + ∑i∈S αiK(x, xi), a sum of bumps centred at the support vectors. A small γ makes wide, gentle bumps and a nearly linear score; a large γ makes narrow bumps that can fit every training patient. The ROC curve of the score on the 207 training patients keeps improving as γ grows, as in Figure 9.10; on the 90 test patients it does not, as in Figure 9.11. Every fit below is computed in your browser.

Kernel

Radial width γ (log scale)

Cost C′ (log scale)

Try these

297 patients with complete records; 13 predictors coded as 16 columns (ChestPain and Thal as dummies), each standardised with the training means and standard deviations; the same random split into 207 training and 90 test patients as in the course's Figures 9.10 and 9.11. The SVM is fitted by a small SMO solver (the algorithm of LIBSVM, which R's svm() and scikit-learn's SVC use); LDA uses the pooled covariance. A positive score predicts heart disease.

In the lecture notes