Modifications of the BIC for order selection in finite mixture models
How many components does a mixture have? An information criterion answers by trading fit (maximized log-likelihood) against a penalty on complexity. This demo fits a light EM to a 1D data histogram for every candidate order k = 1..8, then draws four criterion curves: AIC (flat penalty), BIC, and the paper's ν-BIC and ε-BIC, which inflate the BIC penalty by a negligibly small logarithmic factor to buy order-selection consistency under much weaker (non-differentiable, mild-moment) conditions. Watch the selected order and the Monte-Carlo P(k̂ = k₀) climb to 1 as the sample size grows, while AIC plateaus below 1.
Each candidate order k is fitted by a short in-browser EM to the current dataset, giving a maximized log-likelihood Lₖ. With dₖ = 3k − 1 free parameters (k−1 weights, k locations, k scales), the criteria we minimize are AIC = −2Lₖ + 2dₖ, BIC = −2Lₖ + dₖ log n, and the paper's ν-BIC = −2Lₖ + dₖ log n Ln∘ν(n) and ε-BIC = −2Lₖ + dₖ (log n)1+ε, where Ln(n) = log(e ∨ n) and Ln∘ν is its ν-fold composition. The fit term drops as k grows (more components fit better); the penalty rises, and where it wins sets the argmin. AIC's flat penalty barely resists extra components, so it over-selects; BIC's log n penalty puts a genuine minimum near k₀. The right panel is a small streaming Monte-Carlo: for a grid of sample sizes it repeatedly resamples, refits, and records how often each criterion recovers k₀. (Honest caveats. This is a 1D toy with a light single-restart EM, not the paper's asymptotic theory; the log-likelihoods and argmins are indicative, not the figure numbers. By design the ν/ε inflation factors are negligible, so at browser-scale n the three consistent curves nearly coincide with BIC, that is the point: the visible lesson is the penalty-vs-fit tradeoff and consistency, and you can push ν up or ε down to watch ν-BIC / ε-BIC collapse onto BIC. Laplace/t use a weighted-mean/median M-step surrogate. The paper proves consistency for Gaussian, non-differentiable Laplace, heavy-tailed t, and regression mixtures, plus a misspecification and a minimax-conflict result.)
Run the experiments
Every animation runs live in your browser. Click a button to run that experiment on the demo (it scrolls up and starts); drag any control to take over. Nothing is downloaded, it is generated on the fly.
Grow the sample size
Sweep n from 50 to 3000. Consistency is a large-n statement: watch each criterion sharpen its minimum onto the true order and the Monte-Carlo curves climb toward 1.
Vary the true order
Step the generating order k0 from 1 to 6 and see the criterion argmins and the consistency target track it.
Sweep the overlap
Push the component means from well separated to heavily overlapping: order selection gets harder and AIC over-selects more visibly.
Collapse nu-BIC onto BIC
Increase the iterated-logarithm depth nu: the penalty factor Ln^(nu)(n) shrinks toward 1, so the nu-BIC curve merges into BIC.
Shrink the eps inflation
Lower epsilon toward 0: the (log n)^(1+eps) penalty relaxes to log n and eps-BIC collapses onto BIC, illustrating that the modification is immaterial in practice.
Tour the component families
Cycle Gaussian (classical), non-differentiable Laplace, and heavy-tailed Student-t components: the paper's consistency holds across all three, unlike the classical BIC theory.
Resample the data
Draw fresh datasets from the same true mixture and watch the fitted log-likelihoods and selected orders wobble around the truth.
The idea in three steps
Choosing a mixture order is a bias-variance tradeoff dressed as a penalized likelihood. The trick of this paper is that the BIC penalty can be nudged by an immaterial logarithmic factor to make the argument go through under far weaker regularity.
Log-likelihood per order
For every candidate order k, maximize the mixture likelihood by EM. More components can only fit the sample better, so the maximized log-likelihood keeps rising with k: fit alone would always pick the biggest model.
A penalty on complexity
AIC adds a flat 2 per parameter and under-penalizes, so it over-selects. BIC adds log n per parameter, heavy enough to be consistent but only under strong smoothness and moment conditions (Keribin, 2000).
Consistency for free
Multiply the BIC penalty by Ln∘ν(n) or replace log n by (log n)1+ε. The factor is negligible in practice, yet it delivers order-selection consistency without differentiability and with only mild moments.
For the ν-BIC and ε-BIC penalties, the consistency theorems, the misspecification (KL-optimal order) result, and the limits on minimal penalties and minimax Hellinger risk, see Modifications of the BIC for order selection in finite mixture models (H. D. Nguyen & TrungTin Nguyen, Annals of the Institute of Statistical Mathematics, to appear).