Standardising gives every column mean 0 and variance 1 but leaves the correlation alone. Whitening goes further: a matrix \(\mathbf W\) with \(\mathbf W\boldsymbol\Sigma\mathbf W^{\mathsf T}=\mathbf I\) turns the centred data into data with identity covariance. With \(\boldsymbol\Sigma=\mathbf E\mathbf D\mathbf E^{\mathsf T}\), PCA whitening \(\mathbf W_{\rm pca}=\mathbf D^{-1/2}\mathbf E^{\mathsf T}\) rotates onto the principal axes and rescales; every \(\mathbf R\mathbf W_{\rm pca}\) with \(\mathbf R\) orthogonal also whitens. Turn the rotation and watch the mean squared displacement \(\frac1N\sum_n\|\mathbf W\mathbf x_n-\mathbf x_n\|^2\) (bottom panel): its minimum is ZCA whitening, \(\mathbf W_{\rm zca}=\boldsymbol\Sigma^{-1/2}\), the whitened data closest to the original.
Covariances use the divisor \(N\). The eigenvector signs are fixed so that \(\mathbf E\) is a rotation with a positive first entry; then ZCA is the rotation \(\mathbf R(\varphi^*)=\mathbf E\). Height and weight: the 88 men of the Davis (1990) study (R package carData), a stand-in for the book's data set.
code/ch07_linear_algebra.py (whitening, standardize, fig_7_7).