Mathematical foundations, demonstrations and code
A course that follows Kevin Murphy's Probabilistic Machine Learning: Advanced Topics (MIT Press, 2023) and its online supplement chapter by chapter: the probability, statistics, graphical models, information theory and optimisation behind modern probabilistic machine learning; exact and approximate inference (Kalman filtering, message passing, variational inference, Monte Carlo, MCMC and sequential Monte Carlo); Bayesian prediction with generalized linear models, Bayesian neural networks and Gaussian processes, and learning under distribution shift; deep generative models (VAEs, autoregressive models, flows, energy-based models, diffusion models and GANs); discovery of latent structure; and decision making, reinforcement learning and causality.
Lecture notes that prove what the book states. The book is a research-level survey and leaves many steps to the reader. The notes follow it section by section, with its numbering (supplement sections are marked as such), and supply the argument in a "Filling the gap" box: 682 derivations in all, from the Kalman smoother, belief propagation and the evidence lower bound to Hamiltonian Monte Carlo, particle filtering, the reparameterisation trick, the change-of-variables formula for flows, score matching, the diffusion ELBO and probability-flow ODE, GAN divergences, the Dirichlet process and the InfoNCE bound. The book has no exercises, so every chapter has practice problems of our own with worked solutions (106 in all). Where we found a slip in the book, the notes say so and show the evidence.
The book's figures, reproduced. The Python code rebuilds 471 figures, one function per figure, from bundled data sets and from simulations; where the book uses data that cannot be obtained offline, a stand-in is used and the caption says so. Deep generative models are trained at CPU scale on small data sets.
Demonstrations you can run. 96 browser demonstrations let you change a proposal, a step size, a number of particles, a noise schedule, a kernel or a prior and see the effect at once. The notes open each one at the case being discussed, and each links back to the notes.
Prerequisites. The introductory volume, covered by the companion course Probabilistic Machine Learning: An Introduction: probability, linear algebra, optimisation, linear models and deep networks. Chapters 2 to 6 review the foundations at the level the rest of the book needs; the notes link the introductory course wherever a proof is already given there.
The book. The book is available from its web page, with the online supplement. Keep it open next to the notes: section, equation and figure numbers match.
Companion courses. Each notes page lists the overlapping sections of Probabilistic Machine Learning: An Introduction, Bayesian and Probabilistic Machine Learning, Foundations of Deep Learning, Probability and Statistical Inference, Nonparametric Statistical Inference and the other courses on this site, and links them.
Why a model-based approach to machine learning, inference as inverting a generative process, and the four kinds of task (prediction, generation, discovery, action) that organise the book.
1 figures reproduced · 1 gap-filling derivations · 2 practice problems with solutions
The distributions the book relies on, Gaussian joint distributions and Gaussian conditioning, the exponential family and its moment and cumulant identities, change of variables, Markov chains, and divergence measures between distributions.
23 figures reproduced · 29 gap-filling derivations · 3 practice problems with solutions
Bayesian and frequentist statistics side by side: conjugate, noninformative and hierarchical priors, empirical Bayes, model selection and checking, Bayesian hypothesis testing, and inference with missing data.
30 figures reproduced · 39 gap-filling derivations · 3 practice problems with solutions
Directed and undirected graphical models, conditional independence and d-separation, Markov random fields and conditional random fields, the comparison of the two, and structural causal models.
17 figures reproduced · 25 gap-filling derivations · 3 practice problems with solutions
KL divergence, entropy and mutual information, source coding and channel coding, the information bottleneck, and algorithmic information theory.
13 figures reproduced · 27 gap-filling derivations · 3 practice problems with solutions
Automatic differentiation, stochastic and natural gradient methods, the Bayesian learning rule, bound optimisation and EM, Bayesian optimisation, derivative-free search, conjugate duality, proximal methods, dynamic programming and optimal transport.
25 figures reproduced · 29 gap-filling derivations · 3 practice problems with solutions
A map of the inference algorithms in Part II: common inference patterns, exact versus approximate methods, and how to evaluate an approximate posterior.
6 figures reproduced · 7 gap-filling derivations · 3 practice problems with solutions
The Kalman filter and smoother, the extended and unscented Kalman filters, assumed density filtering and other Gaussian approximations for state-space models.
14 figures reproduced · 24 gap-filling derivations · 3 practice problems with solutions
Belief propagation on chains and trees, loopy belief propagation, variable elimination, the junction tree algorithm, inference as optimisation, and MAP estimation for discrete graphical models.
14 figures reproduced · 9 gap-filling derivations · 3 practice problems with solutions
Variational inference: the evidence lower bound, gradient-based and coordinate ascent VI, richer variational families, tighter bounds, the wake-sleep algorithm and expectation propagation.
17 figures reproduced · 30 gap-filling derivations · 3 practice problems with solutions
Monte Carlo integration, sampling from simple distributions, rejection and importance sampling, and the main techniques for reducing Monte Carlo variance.
9 figures reproduced · 15 gap-filling derivations · 3 practice problems with solutions
Markov chain Monte Carlo: Metropolis-Hastings, Gibbs sampling, auxiliary-variable methods, Hamiltonian Monte Carlo, convergence diagnostics, stochastic-gradient MCMC, reversible jump and annealing.
22 figures reproduced · 26 gap-filling derivations · 3 practice problems with solutions
Sequential Monte Carlo: particle filtering, resampling, proposal distributions, Rao-Blackwellised particle filters, SMC samplers and particle MCMC.
15 figures reproduced · 18 gap-filling derivations · 3 practice problems with solutions
How predictive models are evaluated: proper scoring rules, calibration, and distribution-free uncertainty through conformal prediction.
5 figures reproduced · 7 gap-filling derivations · 3 practice problems with solutions
Bayesian generalized linear models: linear, logistic and probit regression with conjugate, Laplace and variational posteriors, and multilevel (hierarchical) GLMs.
18 figures reproduced · 29 gap-filling derivations · 3 practice problems with solutions
The building blocks of deep networks as differentiable circuits (layers, normalisation, attention) and the canonical architectures built from them.
3 figures reproduced · 15 gap-filling derivations · 3 practice problems with solutions
Bayesian neural networks: priors over functions, posterior approximation by Laplace, variational, ensemble and MCMC methods, generalisation, online inference and hierarchical BNNs.
17 figures reproduced · 28 gap-filling derivations · 3 practice problems with solutions
Gaussian processes: Mercer kernels, exact regression, non-Gaussian likelihoods, sparse and scalable approximations, kernel learning, the link with deep networks, and GPs for time series.
30 figures reproduced · 26 gap-filling derivations · 3 practice problems with solutions
Learning when training and test data differ: covariate, label and concept shift, detecting and adapting to shift, multi-task and continual learning, and adversarial examples.
12 figures reproduced · 21 gap-filling derivations · 3 practice problems with solutions
A map of deep generative models: the main families, what generative modelling is for, and how sample quality and likelihood are evaluated.
3 figures reproduced · 6 gap-filling derivations · 3 practice problems with solutions
Variational autoencoders: the amortised ELBO and reparameterisation trick, generalisations, posterior collapse, hierarchical VAEs and vector-quantised VAEs.
9 figures reproduced · 16 gap-filling derivations · 3 practice problems with solutions
Autoregressive models: NADE and MADE, causal convolutions, transformers and large language models.
3 figures reproduced · 8 gap-filling derivations · 3 practice problems with solutions
Normalizing flows: the change-of-variables formula, coupling, autoregressive, residual and continuous-time flows, and their applications.
4 figures reproduced · 16 gap-filling derivations · 3 practice problems with solutions
Energy-based models: maximum likelihood with MCMC, contrastive divergence, score matching, noise contrastive estimation and related training methods.
8 figures reproduced · 16 gap-filling derivations · 3 practice problems with solutions
Diffusion models: denoising diffusion, score-based models, the stochastic and probability-flow differential equations, faster samplers, guided conditional generation and discrete diffusion.
11 figures reproduced · 20 gap-filling derivations · 3 practice problems with solutions
Generative adversarial networks: learning by comparing distributions, divergences and integral probability metrics, training dynamics and mode collapse, conditional GANs and inference with GANs.
13 figures reproduced · 15 gap-filling derivations · 3 practice problems with solutions
A map of Part V, discovery methods: latent variable models, structure learning and representation learning, and what it means to discover structure in data.
1 figures reproduced · 2 gap-filling derivations · 2 practice problems with solutions
Latent factor models: mixtures, factor analysis and probabilistic PCA, sparse and non-Gaussian priors, topic models and independent component analysis.
22 figures reproduced · 19 gap-filling derivations · 3 practice problems with solutions
State-space models: hidden Markov models and their learning, linear dynamical systems, switching and nonlinear models, changepoint detection and structural time series.
21 figures reproduced · 21 gap-filling derivations · 3 practice problems with solutions
Graph learning: latent variable models for networks (block models, relational models), and learning graphical model structure from data, from Chow-Liu trees to the PC algorithm and NOTEARS.
18 figures reproduced · 18 gap-filling derivations · 3 practice problems with solutions
Nonparametric Bayes: the Dirichlet process and its mixtures, Pitman-Yor and hierarchical processes, the Indian buffet and beta processes, completely random measures, and point processes.
15 figures reproduced · 18 gap-filling derivations · 3 practice problems with solutions
Representation learning: comparing representations with CKA and CCA, autoencoding and contrastive objectives, InfoNCE, non-contrastive methods, and identifiability theory.
8 figures reproduced · 14 gap-filling derivations · 3 practice problems with solutions
Interpretability: inherently interpretable models, local surrogates, attributions and Shapley values, sanity checks and evaluation of explanations, and user studies.
8 figures reproduced · 11 gap-filling derivations · 3 practice problems with solutions
Decision making under uncertainty: Bayesian decision theory, influence diagrams, A/B testing, contextual bandits, Markov decision processes and active learning.
15 figures reproduced · 29 gap-filling derivations · 3 practice problems with solutions
Reinforcement learning: exploration, temporal-difference learning, Q-learning and maximisation bias, policy gradients with baselines, actor-critic and trust-region methods, model-based RL, off-policy evaluation and the deadly triad, and control as inference.
9 figures reproduced · 29 gap-filling derivations · 3 practice problems with solutions
Causality: potential outcomes and graph surgery, randomised trials, confounder adjustment with inverse weighting and double machine learning, instrumental variables and the LATE, difference in differences, sensitivity analysis with Austen plots, and the do-calculus.
12 figures reproduced · 19 gap-filling derivations · 3 practice problems with solutions
One module per chapter and one function per figure, with readable implementations of Kalman filters and smoothers, belief propagation, variational inference, importance sampling, MCMC and particle filters, Gaussian processes, VAEs, flows, energy-based and diffusion models, GANs, hidden Markov models, Dirichlet process mixtures and bandits.
The first volume, with the proofs these notes link to. See also Bayesian and Probabilistic Machine Learning for Chapters 8 to 12 and 28, Foundations of Deep Learning for Chapters 16 and 20 to 26, and Nonparametric Statistical Inference for Chapter 31.