A coin shows N1 heads and N0 tails; with a uniform prior the exact posterior is Beta(N1+1, N0+1), so every approximation can be scored against the truth. The blue curve is exact. Pick a method (or all of them) and watch how it fails: the MAP point has no spread, a coarse grid is blocky, the Laplace approximation in θ is symmetric and leaks outside [0, 1], the same approximation in the logit is better, the variational Gaussian is mode seeking, and MCMC converges only if the proposal is sensible. The table gives the divergence KL(p ∥ q) of each approximation from the posterior and its estimate of the log evidence log p(D).
The prior is uniform on θ (Beta(1,1)). The variational family is a Gaussian on the logit α = log(θ/(1−θ)), fitted by gradient ascent on the ELBO with a fixed quadrature (a deterministic stand-in for the stochastic ADVI of the notes). KL is computed numerically on a fine grid; for the Laplace approximation in θ the Gaussian is truncated to [0, 1] and renormalised. MCMC is random-walk Metropolis on the logit from α = 0, with the first 20% of the samples discarded, and the seed is fixed.
code/ch07_inference_overview.py.