Twenty points lie near two straight lines. A mixture of two linear regressions (14.34) explains each target by one of two lines, chosen with fixed probabilities πk. EM alternates an E step, which shares every point between the lines by its responsibility γnk (14.37), shown as the red and blue parts of the bars, and an M step, which refits each line by least squares with those weights (14.42). The fitted mixture still predicts two values of t at every x, even where no data lie. A mixture of experts (14.53) removes this by letting a gate πk(x) choose the line according to x.
One EM iteration is an M step followed by an E step, so the bars always show the responsibilities of the lines on screen. With the book's start, iterations 0, 30 and 50 give the three columns of Figure 14.8; β starts at 1/var(t) (divisor N − 1).
mixture_linear_regression_em, mixture_of_experts_em, gate_newton, fig14_8, fig14_9); checks in code/checks_ch14.py.