Draw z ~ N(0, 1), map it to the plane by a curve g(z), add noise: p(x|z) = N(x | g(z), σ2I) (16.80). The marginal p(x) (16.81) is a blurred copy of the curve. Click anywhere in the left panel to choose a point x: the top-right panel shows which z could have produced it (the posterior), a Gaussian q(z) you control, and the identity ln p(x) = ELBO + KL (16.57). Then replace the circle by a small neural network g(z, w) and train it by maximising the likelihood estimate (16.83), with a quadrature grid or with random prior samples.
Network: g(z, w) = V tanh(az + b) + c with 16 hidden units. Training maximises (1/N)Σn ln Σk πk N(xn | g(zk, w), σ2I): "grid" uses K equally spaced zk on [−4, 4] with πk ∝ N(zk|0, 1); "Monte Carlo" draws K new zk ~ N(0, 1) at every step with πk = 1/K, which is (16.83). The ELBO, the posterior and ln p(x) use 801 grid points on [−5, 5].
code/ch16_continuous_latent_variables.py (nlvm_data, nlvm_init, nlvm_train, elbo_parts, best_gauss_q, supplementary Figures 16.S5 to 16.S7); with the same seeds the training run and the ELBO values here equal the Python ones.