A velocity field f(z, t) (a small tanh network trained in Python) moves points by dz/dt = f (18.27) from a standard Gaussian at t = 0 to the two moons at T = 1. Along each path the log density changes at the rate d ln p/dt = −Tr(∂f/∂z) (18.28): where the flow contracts (negative trace) density piles up. To find ln p(x) the page integrates backwards from x at time t to t = 0 and adds up the trace. The trace can be computed exactly, or estimated by Hutchinson's εT(∂f/∂z)ε (18.29)–(18.30) with a random probe ε held fixed along each path: unbiased, but noisy. Click the left panel to follow one point.
Velocity: f(z, t) = W3 tanh(W2 tanh(W1[z, t] + b1) + b2) + b3 with 32 units per layer, fitted by flow matching: f((1−t)z0 + tx1, t) was regressed on x1 − z0 (Lipman et al. 2022). Its Jacobian is exact: W3 diag(1−h2²) W2 diag(1−h1²) W1. The density map is computed on a 50 × 50 grid over [−3, 3]² each grid point is integrated back to t = 0 with the chosen number of steps (4 network evaluations per step). Probes are drawn from a seeded generator, Gaussian or Rademacher (±1), one set per grid point, fixed along the path. "Mass in the box" is the midpoint sum of the density.
code/ch18_normalizing_flows.py (train_cnf, CNF.push, CNF.logp, hutch_probes); the weights are in data/ch18-cnf.json and the page reproduces CNF.logp to rounding error.