Resample the observations with replacement (a bootstrap), redo the PCA, and repeat. Each grey dot is one resample. A component whose eigenvalue is well separated from its neighbours comes back almost unchanged. When two or three eigenvalues are nearly equal, a small change in the data rotates the corresponding eigenvectors inside the subspace they span, so their individual loadings cannot be interpreted; the subspace itself, measured by its largest principal angle, stays stable. This is the warning attached to Figure 7.26 of Holmes and Huber (2019, Section 7.7.3), made visible.
Every resample uses correlation PCA (centre and scale) for the real data and covariance PCA for the simulated data, and each bootstrap eigenvector is given the sign that agrees best with the original one. The angle between two unit vectors a, b is arccos |atb|; the largest principal angle between two subspaces with orthonormal bases A, B is the arccosine of the smallest singular value of AtB. The simulated data are Gaussian with covariance eigenvalues (3, 1 + δ, 1, 0.4) and fixed eigenvectors; random numbers are seeded, so a class sees the same picture.
code/ch07_multivariate_analysis.R (Figure 7.26).