Interactive demo · Applied Statistical Learning in R and Python · Chapter 12

PCA of the USArrests data: scaling, units and signs

Principal components look for the directions in which the data vary most. Variance depends on units, so without scaling the variable with the largest numbers wins: Assault, counted per 100,000 residents, has a variance of 6945 against 19 for Murder. Turn scaling off, then change the unit of Assault, and watch the first loading vector swing from Assault to UrbanPop. With scaling on, nothing you do to the units changes the picture. The sign buttons show why two programs can print loading vectors of opposite signs and both be right.

Variables

Assault counted

Components shown

Signs

Try these

The components are the eigenvectors of the sample covariance matrix of the centred variables (the correlation matrix when they are standardised with the n − 1 standard deviation, as R's prcomp(scale = TRUE) does). Arrows: the loadings, read on the orange top and right axes; codes: the scores, read on the bottom and left axes. "Book" signs make each loading vector sum to a positive number, which gives Table 12.1.

In the lecture notes