The data are document counts of the twelve words that say most about which newsgroup family a message comes from (comp, rec, sci, talk): this gives a joint distribution p(x, y) of word X and family Y. The bottleneck chooses a stochastic clustering p(z|x) to minimise βI(Z;X) − I(Z;Y), by iterating p(z|x) ∝ m(z) exp(−D(p(y|x)‖p(y|z))/β). Large β punishes memory of x and gives one cluster; as β falls the clusters split, one transition at a time, and the point (I(Z;X), I(Z;Y)) climbs the information-plane curve.
Information is in nats. The sweep starts at β = 10 and falls geometrically to about 0.003 in 44 steps, each step starting from the previous solution; the slider picks one step. The dashed line is the ceiling I(X;Y), reached only when no information about Y is lost. The cluster slots that nobody uses stay empty.
code/ch05_information_theory.py (ib_fit, ib_curve); data: data/ch05-word-class.json.