The singular value decomposition writes any matrix as a sum of rank-one pieces, X = s1u1v1t + s2u2v2t + …, with the singular values s1 ≥ s2 ≥ … ≥ 0. Keeping the first k pieces gives Xk. Its squared error is exactly the sum of the squared singular values that were dropped, and the Eckart–Young–Mirsky theorem says that no other matrix of rank k comes closer. The three heat maps show X, Xk and the residual X − Xk (gold negative, blue positive), following Holmes and Huber (2019), Section 7.6.
The SVD is computed in your browser by one-sided Jacobi rotations. A competitor picks a random k-dimensional column space (a seeded Gaussian matrix, orthonormalised) and then uses the best matrix with that column space, the projection of X onto it, so it is already as good as that space allows. The turtle, athlete and wine data are standardised with R's scale (divisor n − 1); the athletes' running times are turned into speeds.
code/ch07_multivariate_analysis.R (Figures 7.15 and 7.16, and the matrices of this page).