Principal component analysis looks for the direction u along which the data vary most. Project every point onto a line through the mean: the variance of the projections, uTSu, and the mean squared distance from the points to the line always add up to the total variance Tr S, so the line that maximises the first also minimises the second (Bishop 2006, Sections 12.1.1 and 12.1.2). The other views show the principal axes, whitening, the generative model of probabilistic PCA, the EM algorithm for PCA and the maximum likelihood noise variance (Section 12.2).
S is the covariance matrix with divisor N (12.3), λ1 ≥ λ2 its eigenvalues and u1, u2 its unit eigenvectors. In two dimensions probabilistic PCA uses M = 1 latent variable, so its noise variance is the single discarded eigenvalue λ2; in D dimensions it is the average of the D − M discarded ones (12.46).
pca, whiten, ppca_ml, ppca_loglik, em_pca_step, fig12_6, fig12_9, fig12_12); the ten points of the EM view are those of EM_PCA_POINTS, read off the book's Figure 12.12.