The target p is a zero-mean Gaussian with standard deviations σ1, σ2 = 1 and correlation ρ. The approximation q has a diagonal covariance diag(v1, v2). Minimising the forward KL D(p‖q) gives the marginal variances, vi = Σii (teal, too wide). Minimising the reverse KL D(q‖p) gives vi = 1/Λii = Σii(1 − ρ2) (gold, too narrow). The right panel is the plane of (log10 v1, log10 v2) coloured by the chosen KL; your own q (pink) is set by two sliders.
KL divergences are in nats. The colour scale is log(1 + KL), so the minimum is the darkest point; it sits exactly at the teal marker for the forward KL and at the gold marker for the reverse KL.
code/ch05_information_theory.py (diag_projections, kl_gauss).