Interactive demo · Applied Statistical Learning in R and Python · Chapter 12

Matrix completion by iterated principal components

Some entries of the standardised USArrests matrix are hidden (one variable in each chosen state). Algorithm 12.1 starts from the column means and then repeats two moves: fit the best rank-M approximation to the filled-in matrix, and copy its values into the hidden cells. Step through it: the points (true value across, imputed value up) move towards the diagonal, and the error on the observed entries, the objective (12.14), can only go down. The dashed lines show what mean imputation gets and what the same rank-M approximation would get if nothing were missing.

Hidden entries

Number of states with a hidden value

Random pattern number

Rank M

Try these

Each iteration: Xapp = best rank-M approximation of the filled-in matrix (projection of its rows on the top M eigenvectors of its cross-product matrix, the same as the SVD); hidden cells ← Xapp. MSS is the mean squared error of Xapp on the observed cells; the run stops when the relative decrease (MSSold − MSS) / mean(x²) falls below 10−7, the labs' rule. "R lab" and random patterns use R's scaling (n − 1); "Python lab" uses scikit-learn's (n) and starts MSSold at mean(x²), as that lab does.

In the lecture notes