Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 24

A product of experts, sampled by Langevin dynamics

Expert A has four Gaussian wells at (±1, ±1); expert B is the soft constraint “x1 is positive”, EB = log(1 + exp(−5x1)). Adding the energies multiplies the densities (24.4): the product keeps the two right-hand wells. The chains below run x ← x − (ε²/2)∇E(x) + εz (24.18), optionally with a Metropolis accept/reject step. Choose the experts, the sampler and the step size, then run the chains.

Energy

Sampler

Step size ε

Try these

400 chains start from N(0, 1.8²I). The monitor shows, for the two-well-pair energies, the fraction of chains with x1 > 0 against its exact value (by quadrature); for the standard Gaussian well it shows the sample variance against the exact stationary variance 1/(1 − ε²/4) of the unadjusted chain (a = 1). Chains that blow up are removed from the plot but counted.

In the lecture notes