Change every pixel of a digit by ±ε in the direction that increases the network's error, x′ = x + ε sign(∇xE(x, t)) (10.11), and a confident, correct prediction can flip, although no pixel moved by more than ε. The gradient is computed here by backpropagation through the trained network (the same network as in the feature-map demo). Compare with random signs of the same size: direction, not size, is what matters.
E is the cross-entropy of the true label t. The iterative version takes 10 steps of size ε/10 and keeps every pixel within ε of the original; the targeted version instead lowers the cross-entropy of the chosen class. Pixels are clipped to [0, 1]. Random signs use a fixed seed.
code/ch10_convolutional_networks.py (np_input_grad, fgsm, supplementary Figure 10.S2). The perturbed logits computed here equal Python's to about 10−13.