Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 26

DiracGAN: how gradient descent plays a two-player game

The data are a point mass at 0, the generator is a point mass at θ, and the discriminator is the linear function D(x) = φx. With the zero-sum losses LD = −l(0) − l(θφ) and LG = l(0) + l(θφ) the only equilibrium is (φ, θ) = (0, 0), yet no player ever settles there. The vector field is orthogonal to the position, so the continuous dynamics keep the radius √(θ2 + φ2) fixed; simultaneous gradient descent multiplies the squared radius by 1 + h2l′(θφ)2 at every step, so it spirals outward; alternating descent circles the equilibrium on a closed orbit until the step is too large. A penalty (γ/2)φ2 on the discriminator gradient adds damping and the game converges. All trajectories start at (φ, θ) = (1, 1).

Update rule

Game l(z)

Step size h

Gradient penalty γ

Iterations

Try these

Logistic: l(z) = −log(1 + e−z), so l′(0) = 1/2 (the book's choice); linear: l(z) = z, so l′ = 1 (a Wasserstein-type game). Simultaneous: both players use the old iterate; alternating: the generator uses the updated discriminator; continuous: fourth-order Runge–Kutta with step h on the ordinary differential equation φ̇ = −∇φLD, θ̇ = −∇θLG. The spectral radius is that of the linearisation at the equilibrium (the update map with l′ replaced by l′(0)); a run is stopped when the radius exceeds 108.

In the lecture notes