The support vector classifier (9.12)–(9.15) makes the margin M as wide as possible, but lets observation i cross into the margin by a fraction εi of it, as long as the slacks add up to at most a budget C. With C = 0 nothing may cross, and the result is the maximal margin hyperplane of Section 9.1, if the classes can be separated at all. As the budget grows, the margin widens, more observations sit on or inside it, and every one of them becomes a support vector. The data are the 20 points of the book's R lab, as they are and after the lab shifts one class by 0.5 to make the classes separable. Click on the plot to add points of your own.
The page solves the equivalent cost form, minimise ½‖β‖² + C′ ∑ ξi (the cost of R's svm(), the C of scikit-learn's SVC), with a small SMO solver, and searches for the C′ whose total slack equals your budget: then M = 1/‖β‖ and εi = max(0, 1 − yi f(xi)). A budget below the smallest total slack that any hyperplane can achieve is infeasible, so the slider starts there. The numbers are the R lab's observation numbers.
code/ch09_support_vector_machines.py (budget_fit, budget_direct, hard_margin, Figures 9.3 and 9.7) and code/ch09_lab.R (the data).