Interactive demo · Probabilistic Machine Learning · Chapter 11

Sequential Bayesian linear regression

A straight line y = w0 + w1x + noise with true coefficients (−0.3, 0.5) and noise standard deviation 0.2. The prior on (w0, w1) is a zero-mean Gaussian with precision α. Slide the number of observed points N: the left panel shows the Gaussian posterior (one and two standard deviation ellipses, with the true value as a cross), the right panel the data, lines sampled from the posterior, the posterior mean line and the 95% posterior predictive band. Shift the inputs away from zero and the posterior becomes a thin tilted ridge; switching centring on removes the tilt.

Observed points N

Prior precision α

Noise standard deviation σ

Shift the inputs by

Centre the inputs

Try these

Posterior: SN = (αI + XTX/σ2)−1, mN = SNXTy/σ2 (11.121, 11.122 with a zero-mean isotropic prior). The data are 100 draws with x uniform on [−1, 1], fixed once; the first N are used. The shifted inputs are x + shift, plotted on the shifted axis.

In the lecture notes