Course · Python code

Python code that reproduces the figures

One module per chapter of Hastie, Tibshirani and Friedman, The Elements of Statistical Learning (2009), with one function per figure, linked to the lecture notes. The code is written for this course with numpy, scipy, pandas, matplotlib and scikit-learn, and reads the book's own data sets.

How to run it

  1. Install Python 3 with numpy, scipy, pandas, matplotlib and scikit-learn.
  2. Run one chapter with python3 ch03_linear_regression.py, list its figures with --list, make one with --fig 3.8, or run every chapter with python3 make_all.py --figures.

The book's data sets are downloaded once from its website into a local cache. Each figure is written to img/chNN/ as PNG, SVG and PDF. Shared helpers are in common.py.

Chapter 1. Introduction

Module: ch01_introduction.py · Notes: Introduction

4 figures

Chapter 2. Overview of Supervised Learning

Module: ch02_overview_supervised_learning.py · Notes: Overview of Supervised Learning

11 figures

Chapter 3. Linear Methods for Regression

Module: ch03_linear_regression.py · Notes: Linear Methods for Regression

21 figures

Chapter 4. Linear Methods for Classification

Module: ch04_linear_classification.py · Notes: Linear Methods for Classification

15 figures

Chapter 5. Basis Expansions and Regularization

Module: ch05_basis_expansions.py · Notes: Basis Expansions and Regularization

22 figures

Chapter 6. Kernel Smoothing Methods

Module: ch06_kernel_smoothing.py · Notes: Kernel Smoothing Methods

17 figures

Chapter 7. Model Assessment and Selection

Module: ch07_model_assessment.py · Notes: Model Assessment and Selection

14 figures

Chapter 8. Model Inference and Averaging

Module: ch08_model_inference_averaging.py · Notes: Model Inference and Averaging

19 figures

Chapter 9. Additive Models, Trees, and Related Methods

Module: ch09_additive_models_trees.py · Notes: Additive Models, Trees, and Related Methods

15 figures

Chapter 10. Boosting and Additive Trees

Module: ch10_boosting.py · Notes: Boosting and Additive Trees

21 figures

Chapter 11. Neural Networks

Module: ch11_neural_networks.py · Notes: Neural Networks

16 figures

Chapter 12. Support Vector Machines and Flexible Discriminants

Module: ch12_svm_flexible_discriminants.py · Notes: Support Vector Machines and Flexible Discriminants

15 figures

Chapter 13. Prototype Methods and Nearest-Neighbors

Module: ch13_prototypes_nearest_neighbors.py · Notes: Prototype Methods and Nearest-Neighbors

14 figures

Chapter 14. Unsupervised Learning

Module: ch14_unsupervised_learning_a.py ch14_unsupervised_learning_b.py · Notes: Unsupervised Learning

44 figures

Chapter 15. Random Forests

Module: ch15_random_forests.py · Notes: Random Forests

13 figures

Chapter 16. Ensemble Learning

Module: ch16_ensemble_learning.py · Notes: Ensemble Learning

9 figures

Chapter 17. Undirected Graphical Models

Module: ch17_undirected_graphical_models.py · Notes: Undirected Graphical Models

10 figures

Chapter 18. High-Dimensional Problems: p ≫ N

Module: ch18_high_dimensional_problems.py · Notes: High-Dimensional Problems: p ≫ N

16 figures