Interactive demo · Bayesian and Probabilistic Machine Learning · Chapter 3

Bayesian linear regression: sequential learning, predictive band and equivalent kernel

The prior p(w) = N(0, α⁻¹I) is turned into the posterior N(m_N, S_N) one data point at a time. Add points and watch the posterior over the weights shrink, the sampled functions agree, and the predictive band narrow near the data. The line model reproduces Figure 3.7 (two weights, so the posterior can be drawn); the sine model uses nine Gaussian basis functions and a bias, as in Figures 3.8 and 3.9.

Model

View

Data points N

Prior precision α (log₁₀)

Noise precision β (log₁₀)

Gaussian basis functions M

samples of y(x, w) predictive mean ± 1 sd true function

Try these

Line model: data from t = −0.3 + 0.5x + noise (sd 0.2), x uniform on (−1, 1), as in Section 3.3. Sine model: t = sin(2πx) + noise (sd 0.2), x uniform on (0, 1). The equivalent-kernel view uses 11 Gaussian basis functions (s = 0.2) and 200 equally spaced x in (−1, 1), as in Figure 3.10.

In the lecture notes

Chapter 3 notes: Section 3.3 (Bayesian linear regression), 3.3.1 (the posterior and sequential updating), 3.3.2 (the predictive distribution), 3.3.3 (the equivalent kernel).
Python code: code/ch03_linear_regression.py (functions posterior, predictive, equivalent_kernel; Figures 3.7 to 3.11) and the checks in code/checks_ch03.py.