Gaussian mixtures, animated

A companion gallery to scikit-learn's mixture module and its example gallery. Each static figure there becomes a live, in-browser animation: the same EM and variational-Bayes algorithms (a faithful port of sklearn/mixture) run as you watch, so the estimation dynamics behind each plot are visible, not just the converged result.

These demos are the unsupervised Gaussian-mixture counterpart of the mixture-of-experts models in the rest of this section: no gate and no inputs, just density estimation over points in the plane. The seven demos below cover every example in the scikit-learn mixture gallery. They run entirely client-side (no data leaves your browser) and reimplement the exact steps used by sklearn.mixture.GaussianMixture and BayesianGaussianMixture: the EM E- and M-steps, the four covariance parameterisations, all four initialization strategies, the negative-log-likelihood density contours, the BIC/AIC information criteria, the EM-versus-variational comparison, and the Dirichlet / Dirichlet-process weight priors. The mixture-of-experts package extends exactly these estimators to the supervised, gated setting, and the research connections tie them to work on universal approximation, model selection, and simulation-based inference.

1 · Covariance types

The single most characteristic scikit-learn mixture figure: how the covariance parameterisation changes what a component can represent. spherical forces circular blobs, diag forces axis-aligned ellipses, tied shares one shape across all components, and full lets every component take its own orientation. Watch each type converge by EM on the same anisotropic clusters; the readout reports the mean log-likelihood, the parameter count, and the BIC.

Gaussian mixture · covariance parameterisations

EM fits K Gaussians; the 1σ and 2σ covariance ellipses are drawn as they settle. Points are coloured by their most likely component.

Covariance Components K 3

The maximisation of each component covariance follows sklearn's _estimate_gaussian_covariances_{full,tied,diag,spherical} exactly. Illustrative in-browser EM.

Reproduces · GMM covariances · Gaussian mixture ellipsoids

2 · Density estimation

A Gaussian mixture is also a flexible density estimator. The default data is exactly scikit-learn's example: a spherical Gaussian centred at (20, 20) plus a stretched zero-centred Gaussian, fitted with two full-covariance components. The white iso-lines are the negative log-likelihood contours (as in the sklearn contour plot), and the shaded field is the density; you can also switch to two moons, a sine band, or three blobs.

Gaussian mixture · density estimation

EM with full covariances; the contours are the negative log-likelihood −log p(x) = −log ∑_k π_k N(x | μ_k, Σ_k).

Data Components K 2

The likelihood grid and negative-log-likelihood contours use the same log-sum-exp mixture likelihood sklearn evaluates in score_samples; contours are drawn by marching squares. Illustrative in-browser EM.

Reproduces · Density estimation for a GMM

3 · Model selection by BIC

How many components? scikit-learn's selection example sweeps the number of components and the covariance type and picks the model with the lowest Bayesian information criterion. This demo runs that sweep live: a bar per K, the selected model highlighted, with the same BIC and AIC formulas (-2·loglik + p·log n and -2·loglik + 2p).

Gaussian mixture · BIC / AIC model selection

For each K = 1..8 the best of several EM restarts is scored; the lowest-BIC model (green) should recover the true number of clusters.

Covariance

Parameter counts match sklearn's _n_parameters for each covariance type. Illustrative in-browser model selection.

Reproduces · Gaussian mixture model selection

4 · EM versus a Dirichlet process

The canonical "why Bayesian" figure. Both models are given five components to fit data that really has two clusters (the exact sklearn dataset: two correlated Gaussians). The left panel is a classical GaussianMixture fitted by EM, which is forced to use all five components and so splits the clusters arbitrarily. The right panel is a BayesianGaussianMixture with a Dirichlet-process prior, which activates only as many components as it needs and leaves the rest empty. Redundant components are not drawn, exactly as in the sklearn example.

Gaussian mixture (EM) versus Dirichlet-process (variational Bayes)

Both start with K = 5 on the same two-cluster data; watch EM keep all five while the Dirichlet process collapses to the two it needs.

Data and the 2·√2·√λ confidence ellipses follow plot_gmm.py exactly; the Dirichlet-process panel is the variational estimator with a stick-breaking weight posterior. Illustrative in-browser EM / VB.

Reproduces · Gaussian Mixture Model Ellipsoids

5 · Initialization methods

How EM is initialised changes how many iterations it needs to converge. On four easy clusters, compare the four scikit-learn init_params strategies: kmeans (default), k-means++, random_from_data, and random. Orange diamonds mark the initial means; the crosses are coloured by the final assignment; the iteration counter shows the convergence cost. As in the sklearn example, the cheaper initializations tend to need more iterations.

Gaussian mixture · initialization methods

The initial component means (orange diamonds) come from the chosen strategy; then EM runs to convergence and reports the iteration count.

init_params

The four seeders mirror sklearn's _initialize_parameters (k-means, greedy k-means++, random data points, random responsibilities). Illustrative in-browser EM.

Reproduces · GMM Initialization Methods

6 · Sine curve: EM versus Dirichlet process

Fitting a mixture to data that is not a mixture of Gaussians: points along a noisy sine curve, with no true number of components. Compare an EM GaussianMixture with 10 components, a Dirichlet process with a low concentration (γ₀ = 0.01, which favours a few broad components capturing the big picture), and a Dirichlet process with a high concentration (γ₀ = 100, which uses many fine components). This reproduces scikit-learn's sine-curve example.

Gaussian mixture on a sine curve · EM and Dirichlet process

Ten available components; the Dirichlet-process concentration decides how many are actually used.

Model

Settings follow plot_gmm_sin.py (10 components, full covariance, low vs high weight_concentration_prior). Illustrative in-browser EM / VB.

Reproduces · Gaussian Mixture Model Sine Curve

7 · Concentration prior (variational Bayes)

The flagship Bayesian example. A BayesianGaussianMixture started with more components than needed will switch off the ones it does not use, and how aggressively it does so depends on the weight concentration prior and on whether that prior is a plain Dirichlet distribution or a Dirichlet process (stick-breaking). Start from K = 8, choose the prior, sweep the concentration α, and watch the posterior mixture weights (bottom bars) collapse onto the components the data actually support. This is the Bayesian-nonparametric mechanism the mixture-of-experts package uses to select the number of experts automatically.

Variational Bayesian mixture · automatic component selection

Mean-field variational updates; the discrete stick-breaking (Dirichlet-process) weight posterior is exactly as in sklearn / the BNP-GLLiM package. Faded ellipses are components with negligible weight.

Weight prior Concentration α 1.00

The E[log π_k] driving the responsibilities uses the true Dirichlet / stick-breaking digamma expectations; component means and covariances use the conjugate (MAP) update. Illustrative in-browser variational Bayes; the Python BayesianGaussianMixtureOfExperts is the reference implementation and also offers the full Pitman-Yor process.

Reproduces · Concentration prior analysis of a variational Bayesian GMM

Notes and honesty

These seven demos reproduce all of the scikit-learn mixture examples (covariance types, density estimation, model selection, EM vs Dirichlet process, initialization methods, the sine curve, and the concentration prior), as faithful in-browser reimplementations for visualisation, ported from the public scikit-learn mixture source (commit cc50648): the EM E/M-steps, the four covariance parameterisations, k-means / k-means++ / random / random-from-data initialization, negative-log-likelihood contours (marching squares), BIC/AIC, and the Dirichlet / Dirichlet-process weight posteriors. The data-generating settings match each example's source exactly.
For the variational (Bayesian) demos the mixing-weight posterior is exact (the stick-breaking digamma expectations); the component means and covariances use the conjugate Normal-Inverse-Wishart MAP update rather than the full posterior. This reproduces the automatic component selection, but because it is a lighter inference than scikit-learn's full ELBO, the concentration and covariance priors in the sine-curve demo are set to reproduce the published figure's few-broad-versus-many-fine contrast rather than copied verbatim.
For real work use scikit-learn's own GaussianMixture / BayesianGaussianMixture (re-exported by this package). The mixture-of-experts estimators on the other pages extend these same algorithms to the supervised, gated setting.

References

scikit-learn: mixture module. user guide, example gallery, BayesianGaussianMixture.
F. Pedregosa et al. Scikit-learn: Machine Learning in Python. JMLR 12:2825-2830, 2011.
D. M. Blei and M. I. Jordan. Variational inference for Dirichlet process mixtures. Bayesian Analysis 1(1):121-143, 2006 (the stick-breaking weight posterior).