Some entries of the standardised USArrests matrix are hidden (one variable in each chosen state). Algorithm 12.1 starts from the column means and then repeats two moves: fit the best rank-M approximation to the filled-in matrix, and copy its values into the hidden cells. Step through it: the points (true value across, imputed value up) move towards the diagonal, and the error on the observed entries, the objective (12.14), can only go down. The dashed lines show what mean imputation gets and what the same rank-M approximation would get if nothing were missing.
Each iteration: Xapp = best rank-M approximation of the filled-in matrix (projection of its rows on the top M eigenvectors of its cross-product matrix, the same as the SVD); hidden cells ← Xapp. MSS is the mean squared error of Xapp on the observed cells; the run stops when the relative decrease (MSSold − MSS) / mean(x²) falls below 10−7, the labs' rule. "R lab" and random patterns use R's scaling (n − 1); "Python lab" uses scikit-learn's (n) and starts MSSold at mean(x²), as that lab does.
code/ch12_unsupervised_learning.py (Figures 12.5 and 12.6) and code/ch12_lab.py.