Interactive demo · Applied Statistical Learning in R and Python · Chapter 8

Bagging against a random forest on the Heart data

150 patients train, 147 test. Both ensembles grow deep classification trees (Gini splits, no pruning) on bootstrap samples and take a majority vote. Bagging lets every split choose among all p = 18 input columns; the random forest draws a fresh set of m columns at each split. With m < p a strong input cannot dominate every tree, so the trees differ more from one another, and averaging them removes more variance. The out-of-bag (OOB) error predicts each training patient with only the trees whose bootstrap sample left it out.

Random forest: columns per split m

Number of trees B

Random seed

Try these

ChestPain and Thal enter as one 0/1 column per level (4 + 3 columns), so p = 18; m = 4 is about √18 and matches the book's 4 of 13 inputs. Both ensembles use the same bootstrap samples for a given seed. A tree's split chooses, among the allowed columns, the cut that most reduces the Gini index (8.6); a node stops when it is pure. Dashed line: the single tree pruned by cross-validation (Figure 8.6).

In the lecture notes