A straight line y = w0 + w1x + noise with true coefficients (−0.3, 0.5) and noise standard deviation 0.2. The prior on (w0, w1) is a zero-mean Gaussian with precision α. Slide the number of observed points N: the left panel shows the Gaussian posterior (one and two standard deviation ellipses, with the true value as a cross), the right panel the data, lines sampled from the posterior, the posterior mean line and the 95% posterior predictive band. Shift the inputs away from zero and the posterior becomes a thin tilted ridge; switching centring on removes the tilt.
Posterior: SN = (αI + XTX/σ2)−1, mN = SNXTy/σ2 (11.121, 11.122 with a zero-mean isotropic prior). The data are 100 draws with x uniform on [−1, 1], fixed once; the first N are used. The shifted inputs are x + shift, plotted on the shifted axis.
code/ch11_linear_regression.py (posterior, predictive, Figures 11.20 to 11.22).