A network g(z, w, t) with three hidden layers of 64 units was trained in Python with Algorithm 20.1 to predict the noise in zt = √αtx + √(1 − αt)ε for points x of a three-arm spiral; its weights run here, forward only. Generation starts from white noise and removes noise step by step. Algorithm 20.2 takes a step towards the predicted clean point and adds fresh noise; the DDIM update with η = 0 adds none, so the map from zT to x is deterministic and survives much larger steps. Switch to the exact predictor E[ε | zt], which the network approximates, to separate the network's error from the sampler's.
Steps run at S times spread evenly over 1..200. Algorithm 20.2 on a sub-sequence uses β = 1 − αt/αt′ between consecutive times. DDIM: x̂ = (z − √(1 − αt)g)/√αt, then z′ = √αt′x̂ + √(1 − αt′ − s²)g + sε, with s = η√((1 − αt′)/(1 − αt))√(1 − αt/αt′); η = 1 is the stochastic sampler with the variance (20.17). Quality is the mean of ln p(x) under the true spiral density: real data score real data score −1.51minus;1.53.
code/ch20_diffusion_models.py (train_denoiser, np_denoiser, sample_alg202, sample_ddim, demo_sample, supplementary Figures 20.S3 and 20.S4); with the same seed the points here equal demo_sample. Weights: data/ch20-denoiser.json.