Interactive demo · Probabilistic Machine Learning · Chapter 17

Support vector machines: the kernel width and the regulariser C

A soft-margin support vector machine with the RBF kernel exp(−γ‖x − x′‖2) is fitted in your browser by solving its dual (17.85)–(17.87) with sequential minimal optimisation. The left panel shows the decision function f(x) = ∑n αnỹnK(xn, x) + w0 (17.93): the solid line is the boundary f = 0, the dashed lines the margins f = ±1, and ringed points are support vectors (αn > 0). The right panel is the 5-fold cross-validation error over a grid of (C, γ); click it to jump to a setting. Large γ (narrow kernels) needs small C (strong regularisation), and the reverse.

Data

Kernel precision log γ

Regulariser log C

Try these

Data: 100 two-moons points with noise 0.15 (my own simulation of the toy data of Figure 17.17), test error on 1000 fresh points; and the 200-point two-class mixture of Hastie, Tibshirani and Friedman (2009), the data of Figure 17.18, test error on 2000 points of its test set, Bayes error 0.208. Labels are ±1 (red −1, blue +1). The cross-validation maps were computed once in Python on a grid with steps of 1 in log C and 0.5 in log γ; the fit itself is recomputed live. Logs are natural; the shading is blue where f > 0.

In the lecture notes