The network f(X) = β0 + ∑k βk max(0, wk0 + ∑j wkjXj) (10.1) predicts a baseball player's 1987 salary from 20 standardised inputs. It is trained here, in your browser, by RMSprop on the squared error (10.23), with the gradients of (10.29)–(10.30), on the 176 training players of the R lab, and judged on the 87 test players. Compare the network's test error, epoch by epoch, with least squares and the lasso: Section 10.6's question of when deep learning is worth it.
Initial weights are uniform on ±1/√(fan-in), as in torch. RMSprop divides each gradient by the running root-mean-square of its past values (decay 0.99). With "full" every step uses all 176 players and, without dropout, a run is deterministic; minibatches of 32 make 6 steps per epoch, drawn at random as in stochastic gradient descent. Dropout zeroes each hidden unit with probability φ during training and scales the survivors by 1/(1 − φ). The inputs are standardised over all 263 players, as in the lab.
code/ch10_lab.R (the same network written out in base R) and ch10_lab.py (PyTorch); the full-batch reference run is demo_net_train in ch10_deep_learning.py.