Both algorithms alternate between the latent labels Δi and the parameters. EM replaces each label by its responsibility γi and then maximises: its path (white) climbs the log-likelihood surface in the plane of the two means and stops at a mode, or at a saddle if it starts on the symmetry line μ1 = μ2. Gibbs sampling draws each label from Bernoulli(γi) and then draws the means: its iterates (dots) wander around the mode and describe the posterior. Algorithm 8.4 of the book draws each mean with variance σ̂k2; the exact full conditional under a flat prior has variance σ̂k2/nk. Compare the two, and watch the labels switch.
EM starts from π = 0.5 and both variances equal to the sample variance (the book's recipe) and runs to convergence. The Gibbs samplers 'printed' and 'exact' fix π, σ12, σ22 at their maximum likelihood values, as in the book. The 'full' sampler also draws π ~ Beta(1 + n2, 1 + n1) and each (μk, σk2) from its normal-inverse-gamma conditional (prior mean &ybar;, κ0 = 0.01, a0 = 2, b0 = 1). Summaries discard the first 10% of the iterations (at most 100). A label switch is an iteration with μ1 < μ2.
code/ch08_model_inference_averaging.py (em_mixture, gibbs_mixture, rao_blackwell_density).