The data are eight Gaussians (standard deviation 0.05) on a circle of radius 2. A generator g(z, w) (2–32–32–2, tanh) maps two-dimensional Gaussian noise to the plane; a discriminator d(x, φ) (2–64–64–1, tanh) tries to tell its samples from the data. Both are trained here, in your browser, by alternating Adam steps on mini-batches of 64, with forward and backward passes written out by hand. The background shows the discriminator (red: judged real, blue: judged fake); the blue dots are 1000 generated samples from fixed latent points.
Losses. Minimax: d minimises the cross-entropy (17.6) and g maximises it, i.e. minimises mean ln(1 − d(g)). Non-saturating: g minimises −mean ln d(g) instead (17.10). Least squares: d is fitted to 1 on data and 0 on samples, g minimises mean (d(g) − 1)²/2. WGAN: a real-valued critic maximises mean d(data) − mean d(g), kept Lipschitz by clipping every weight to [−0.05, 0.05] or by the penalty 0.1·mean(‖∇xd‖ − 1)² at random points between data and samples. A sample counts as high quality if it is within 3 standard deviations (0.15) of a mode centre.
code/ch17_generative_adversarial_networks.py (ring_gan). With the same settings and seed, the losses and weights here equal the Python ones.