Course · Python code

Python code that reproduces the figures

One module per chapter of Bishop and Bishop, Deep Learning: Foundations and Concepts (2024), with one function per figure, linked to the lecture notes. The code is written for this course with numpy, matplotlib and PyTorch, and every model is small enough to train on an ordinary CPU.

How to run it

  1. Install Python 3 with numpy, scipy, matplotlib and PyTorch (CPU build).
  2. Run one chapter with python3 ch06_deep_neural_networks.py, list its figures with --list, make one with --fig 6.4, or run every chapter with python3 make_all.py.

Each figure is written to img/chNN/ as PNG, SVG and PDF. MNIST is downloaded once into a local cache, and trained models are cached so reruns are quick. Shared helpers are in common.py; the driver is make_all.py.

Chapter 1. The Deep Learning Revolution

Module: ch01_deep_learning_revolution.py · Notes: The Deep Learning Revolution

7 figures

Chapter 2. Probabilities

Module: ch02_probabilities.py · Notes: Probabilities

14 figures

Chapter 3. Standard Distributions

Module: ch03_standard_distributions.py · Notes: Standard Distributions

16 figures

Chapter 4. Single-layer Networks: Regression

Module: ch04_single_layer_regression.py · Notes: Single-layer Networks: Regression

6 figures

Chapter 5. Single-layer Networks: Classification

Module: ch05_single_layer_classification.py · Notes: Single-layer Networks: Classification

14 figures

Chapter 6. Deep Neural Networks

Module: ch06_deep_neural_networks.py · Notes: Deep Neural Networks

18 figures

Chapter 7. Gradient Descent

Module: ch07_gradient_descent.py · Notes: Gradient Descent

14 figures

Chapter 8. Backpropagation

Module: ch08_backpropagation.py · Notes: Backpropagation

7 figures

Chapter 9. Regularization

Module: ch09_regularization.py · Notes: Regularization

17 figures

Chapter 10. Convolutional Networks

Module: ch10_convolutional_networks.py · Notes: Convolutional Networks

19 figures

Chapter 11. Structured Distributions

Module: ch11_structured_distributions.py · Notes: Structured Distributions

13 figures

Chapter 12. Transformers

Module: ch12_transformers.py · Notes: Transformers

15 figures

Chapter 13. Graph Neural Networks

Module: ch13_graph_neural_networks.py · Notes: Graph Neural Networks

11 figures

Chapter 14. Sampling

Module: ch14_sampling.py · Notes: Sampling

19 figures

Chapter 15. Discrete Latent Variables

Module: ch15_discrete_latent_variables.py · Notes: Discrete Latent Variables

16 figures

Chapter 16. Continuous Latent Variables

Module: ch16_continuous_latent_variables.py · Notes: Continuous Latent Variables

19 figures

Chapter 17. Generative Adversarial Networks

Module: ch17_generative_adversarial_networks.py · Notes: Generative Adversarial Networks

13 figures

Chapter 18. Normalizing Flows

Module: ch18_normalizing_flows.py · Notes: Normalizing Flows

8 figures

Chapter 19. Autoencoders

Module: ch19_autoencoders.py · Notes: Autoencoders

14 figures

Chapter 20. Diffusion Models

Module: ch20_diffusion_models.py · Notes: Diffusion Models

16 figures

Appendices A to C

Module: appendices.py · Notes: Mathematical appendices

10 figures