Interactive demo · Statistical Learning: Theory and Methods · Chapter 9

Growing and pruning a classification tree on the spam data

A large tree T0 is grown on the 3065 training messages by greedy binary splits that most reduce the deviance (or the Gini index) of the children. It is then pruned: for each penalty α the subtree Tα minimises the misclassification count plus α times the number of leaves (9.16). These subtrees are nested and the weakest-link algorithm lists all of them, with the breakpoints α1 < α2 < … Move α, compare training, test and 10-fold cross-validated error, apply the one-standard-error rule, and grow the tree with a larger weight on the genuine emails.

Splitting criterion

Weight on email (loss L01)

Penalty α (member of the sequence)

Try these

Growing: a node is split if it has at least 10 messages and a deviance of at least 1% of the root's, and each child needs 5. Split points are midpoints between consecutive distinct values; an observation goes left when its value is below the split. Pruning uses the (weighted) misclassification count. Cross-validation: 10 fixed folds of the training messages; for each member T(k) of the sequence the fold trees are pruned at √(αkαk+1). Test set: the 1536 messages of the book's split. Spam is the positive class for sensitivity.

In the lecture notes