Interactive demo · Bayesian and Probabilistic Machine Learning · Chapter 2

Conjugate priors and sequential Bayesian updating

A prior is conjugate to a likelihood when the posterior has the same form as the prior, so that updating only changes its parameters. Pick a model below, add observations one at a time, and watch the prior (grey, dashed) turn into the posterior (red) while the maximum likelihood estimate and the MAP estimate move. Every curve is computed from the closed-form updates of Sections 2.1 to 2.3.

Model

Try these

Coin flips, Gaussian data and the Student t components are seeded, so every visitor sees the same pictures. The grey dashed curve is always the prior, the red curve the posterior.

In the lecture notes

Chapter 2 notes: Section 2.1 (binary variables, the beta distribution, sequential updating), Section 2.2 (the Dirichlet distribution), Section 2.3.6 (Bayesian inference for the Gaussian, gamma and normal-gamma priors) and Section 2.3.7 (Student's t-distribution).
Python code: code/ch02_probability_distributions.py (functions beta_update, dirichlet_update, gauss_mean_posterior, gauss_precision_posterior, normal_gamma_posterior, student_pdf; Figures 2.2, 2.3, 2.5, 2.12 to 2.15) and the checks in code/checks_ch02.py. The demo's numbers equal the Python ones for the same data.