MARS builds a regression surface from hinge functions (x − t)+ and (t − x)+. Each forward step multiplies a term already in the model by a reflected pair on a new input, with the knot t at an observed value, choosing the parent, input and knot that most reduce the residual sum of squares. The large model is then pruned one term at a time, and generalised cross-validation, which charges c extra parameters per knot, picks the size. Step through the forward pass, then move along the backward sequence.
Knots: every observed value of the input among the parent's nonzero points, at most every L points (minspan) and not among the 8 most extreme values of the input (endspan), Friedman's (1991) rules. Each input appears at most once in a product. Backward pass: delete the term whose removal increases RSS least; GCV = (RSS/N)/(1 − M/N)2 with M = r + c(r − 1)/2 for r terms. Tensor data: y = (x1 − 1)+ + (x1 − 1)+(x2 − 0.8)+ + σε (9.21), 100 standard normal inputs.
code/ch09_additive_models_trees.py (class MARS: fit, _score, _backward; mars_spans, mars_demo_data).