Interactive demo · Statistical Learning: Theory and Methods · Chapter 12

The SVM path: every cost C at once

The support vector machine minimises ∑i [1 − yi f(xi)]+ + (λ/2)‖β‖2 with λ = 1/C. Its solution is f(x) = [∑i αi yi K(x, xi) + α0]/λ with every αi in [0, 1], and the KKT conditions say that αi is the slope of the hinge at yif(xi): 1 inside the margin, 0 outside, anything in between exactly on it. As λ decreases only the multipliers of the points on the margin move, and they move linearly, so the page computes the exact solution for all values of C by following the breakpoints, the path algorithm of Section 12.3.5. Slide C to move along the path.

Kernel

Cost C = 1/λ (log scale)

Try these

Filled black dots are on the margin (y f = 1); thick rings have αi = 1 (inside the margin or misclassified); faint rings have αi = 0 and do not affect the fit. The test error uses the 10,000 test points of Chapter 2; the purple dashed curve is the Bayes boundary and the dashed line in the bottom panel the Bayes rate.

In the lecture notes