Interactive demo · Foundations of Deep Learning · Chapter 10

Adversarial examples: the fast gradient sign method

Change every pixel of a digit by ±ε in the direction that increases the network's error, x′ = x + ε sign(∇xE(x, t)) (10.11), and a confident, correct prediction can flip, although no pixel moved by more than ε. The gradient is computed here by backpropagation through the trained network (the same network as in the feature-map demo). Compare with random signs of the same size: direction, not size, is what matters.

Digit (MNIST test set)

Perturbation

Target class

Size ε (pixel values in [0, 1])

Try these

E is the cross-entropy of the true label t. The iterative version takes 10 steps of size ε/10 and keeps every pixel within ε of the original; the targeted version instead lowers the cross-entropy of the chosen class. Pixels are clipped to [0, 1]. Random signs use a fixed seed.

In the lecture notes