A Gaussian process with the squared exponential kernel σf2 exp(−(x − x′)2/(2ℓ2)) and Gaussian noise of standard deviation σy is fitted to a handful of points. The left panel maps the log marginal likelihood log p(y | X, θ) (17.52) over the length scale and the noise level; the right panel shows the posterior mean and a band of ±2 predictive standard deviations for the point you pick. Click in the map, move the sliders, or press Climb to run gradient ascent with the exact gradient (17.54) from where you are. With the 7-point data there are two hills: a wiggly low-noise explanation and a smooth high-noise one.
Data: 7 points (Figure 17.9) and 20 points (Figure 17.8), each simulated from this Gaussian process with (ℓ, σf, σy) = (1, 1, 0.1), because the book's own points are not available. Every number is computed exactly in the page with a Cholesky factorisation, as in (17.47)–(17.49); the map has 56 × 44 grid points. Climb takes gradient steps in (log ℓ, log σy) with step halving until the gradient vanishes; the white line is its path.
code/ch17_kernel_methods.py (gp_posterior, se_logml, fit_se_hyper, fig_17_9).