Interactive demo · Foundations of Deep Learning · Chapter 17

Training a GAN on a ring of Gaussians

The data are eight Gaussians (standard deviation 0.05) on a circle of radius 2. A generator g(z, w) (2–32–32–2, tanh) maps two-dimensional Gaussian noise to the plane; a discriminator d(x, φ) (2–64–64–1, tanh) tries to tell its samples from the data. Both are trained here, in your browser, by alternating Adam steps on mini-batches of 64, with forward and backward passes written out by hand. The background shows the discriminator (red: judged real, blue: judged fake); the blue dots are 1000 generated samples from fixed latent points.

Loss

Generator learning rate

Discriminator learning rate

Discriminator steps per generator step

Generator starts

Iterations

Run

Try these

Losses. Minimax: d minimises the cross-entropy (17.6) and g maximises it, i.e. minimises mean ln(1 − d(g)). Non-saturating: g minimises −mean ln d(g) instead (17.10). Least squares: d is fitted to 1 on data and 0 on samples, g minimises mean (d(g) − 1)²/2. WGAN: a real-valued critic maximises mean d(data) − mean d(g), kept Lipschitz by clipping every weight to [−0.05, 0.05] or by the penalty 0.1·mean(‖∇xd‖ − 1)² at random points between data and samples. A sample counts as high quality if it is within 3 standard deviations (0.15) of a mode centre.

In the lecture notes