Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 28

ICA and the rotation angle

Two independent unit-variance sources are mixed linearly and the mixtures are whitened (PCA). What is left is a rotation by an unknown angle θ. Rotate the whitened data and look at the ICA objective, the average negative log-likelihood of the rotated data under a logistic prior. If the sources are non-Gaussian the objective depends on θ and the true sources sit at a stationary angle (repeating every 90°); for Gaussian sources it is flat. The dashed line marks the true angle. Watch the sign: with a super-Gaussian prior, uniform sources are at a maximum of the objective, not a minimum.

Sources

Objective

Rotation angle θ (degrees)

Try these

400 samples of each kind of source, mixed by a fixed matrix and whitened. The objective is the average of ∑j 2 log cosh(zj/(2s)) + log 4s with s = √3/π (unit-variance logistic prior), z = R(θ)x. The kurtosis objective is −(|kurt z1| + |kurt z2|), which is lowest where both outputs are as non-Gaussian as possible.

In the lecture notes