A diffusion model destroys data by adding a little Gaussian noise at every step, zt = √(1 − βt) zt−1 + √βt εt (20.3). Because each step is linear and Gaussian, t steps collapse into one: zt = √αt x + √(1 − αt) ε with αt = ∏(1 − βτ) (20.6)–(20.8). Blue points are run through the chain one step at a time, orange points jump straight to step t; the two clouds have the same distribution. The lower panel asks when the reverse step can be modelled by a Gaussian: it computes q(zt−1 | zt) exactly for a one-dimensional mixture (Figures 20.3 and 20.4).
300 data points from the spiral (each arm a chain of small Gaussians). The circles are the 2-standard-deviation circles of q(zt | x) = N(√αtx, (1 − αt)I) for three data points. Lower panel: q(zt−1) is a fixed three-component mixture (red), the reverse density (blue) is proportional to q(zt | zt−1) q(zt−1), and the dashed curve is the Gaussian of Exercise 20.7, mean zt + β(zt/2 + s), variance β + β²(1 + h), with s and h the first two derivatives of ln q(zt−1) at zt.
code/ch20_diffusion_models.py (schedule, forward_chain, forward_closed, reverse_1d, forward_demo_run, supplementary Figures 20.S1 and 20.S2); with the same seed the points here equal forward_demo_run.