Modifications of the BIC for order selection in finite mixture models
How many components does a mixture have? An information criterion answers by trading fit (maximized log-likelihood) against a penalty on complexity. This demo runs the paper's own four simulation scenarios (S-L1, S-L2, S-T1, S-T2: non-differentiable Laplace and heavy-tailed Student-t mixtures, the cases where the classical Keribin (2000) conditions fail), fitting every candidate order by multi-start EM and scoring it with AIC, BIC, the paper's ν-BIC and ε-BIC, and PanIC. The right panel shows the paper's published selection proportions (200 replications) with a live in-browser Monte-Carlo drawn alongside it. The honest headline: the modified criteria are not more accurate than BIC here, they buy the same behaviour under far weaker assumptions, while AIC keeps a small but persistent over-selection rate that depends on the penalty shape α(k).
Every criterion is minimized on the same scale, −2Lₖ + 2n α(k) rₙ, where Lₖ is the maximized log-likelihood from a multi-start EM and rₙ is the penalty shape: 2/n (AIC), log n / n (BIC), (log n)1+ε/n (ε-BIC), Ln∘ν(n) log n / n (ν-BIC) and n−1/2Ln∘2(n) (PanIC). The complexity α(k) is one of dim/2, dim²/2, k or k² with dim(𝕊ₖ) = 3k−1. The fit term drops as k grows; the penalty rises; where they cross sets the argmin. (What this shows, honestly. Three things are worth reading off the panels, and none of them is that the new criteria win. First, ε-BIC and ν-BIC do not beat BIC: at ε = 0.02 they nearly coincide with it, and at ν = 3 the iterated logarithm equals 1 so ν-BIC is numerically identical to BIC. At ν = 1 the penalty is much heavier and the criterion under-selects badly at n = 100. Their value is theoretical: the same practical behaviour, but provably consistent for non-differentiable and heavy-tailed families that the classical theory does not cover. Second, AIC's over-selection is real but modest and depends on the penalty shape: with α type I it sits near 11% at n = 10000 and stops falling, whereas with type IV it is under 5% and the paper itself notes that for some configurations AIC and BIC "can produce perfect or near perfect estimation of k₀ for large n". Switch α between I and IV to see this flip. Third, the live Monte-Carlo is small: a replicate at n = 10000 takes about a second, so it runs far fewer than the paper's 200 replications and its dashed curve is noisy. That is why the paper's published proportions are drawn as the solid reference. The browser EM is also a simplification of the paper's mixsmsn::smsn.mix and MixSAL::msal fits, and a weaker EM systematically understates AIC's over-selection, so treat the live curve as an illustration and the solid curve as the result.)
Run the experiments
Every animation runs live in your browser. Click a button to run that experiment on the demo (it scrolls up and starts); drag any control to take over. Nothing is downloaded, it is generated on the fly.
Grow the sample size
Sweep n from 100 to 10000. Consistency is a large-n statement: watch each criterion sharpen its minimum onto the true order k0. The live Monte-Carlo keeps accumulating in the right panel while you do it.
Flip the penalty shape and watch AIC change
Cycle alpha(k) through the paper's four forms. This is the key experiment: with type I the AIC keeps a persistent over-selection rate near 11% at n = 10000, but with type IV it is under 5%. Whether AIC over-selects is not a fixed property, it depends on the penalty shape.
Tour the paper's four scenarios
Cycle S-L1, S-L2, S-T1 and S-T2: non-differentiable Laplace and heavy-tailed Student-t mixtures with k0 = 2 and k0 = 3. These are exactly the pathologies that fail the classical Keribin (2000) hypotheses.
Collapse nu-BIC onto BIC
Raise the iterated-logarithm depth nu. At nu = 1 the penalty is large and the criterion under-selects at small n; by nu = 3 the factor Ln^(nu)(n) equals 1 and nu-BIC becomes numerically identical to BIC. That identity, not superiority, is the honest message.
Shrink the epsilon inflation
Lower epsilon toward 0: the (log n)^(1+eps) penalty relaxes to log n and eps-BIC converges onto BIC. The paper's own values are 0.02 and 0.10.
Break the scenario on purpose
Slide the separation multiplier below 1 to push the components together. The problem becomes near non-identifiable, every criterion under-selects, and the published reference curve is hidden because this is no longer one of the paper's settings.
Resample the data
Draw fresh datasets from the same scenario and watch the fitted log-likelihoods and selected orders wobble around the truth.
The idea in three steps
Choosing a mixture order is a bias-variance tradeoff dressed as a penalized likelihood. The point of this paper is that the BIC penalty shape can be nudged by a logarithmic factor so that the consistency argument still goes through when the components are non-differentiable or heavy-tailed. The nudge is chosen to be as small as the proof allows, so in practice the modified criteria behave like BIC rather than better than it.
Log-likelihood per order
For every candidate order k, maximize the mixture likelihood by EM with several random restarts. More components can only fit the sample better, so the maximized log-likelihood keeps rising with k: fit alone would always pick the largest order. A weak optimizer understates the likelihood at large k, which quietly flatters the criteria that would otherwise over-select.
A penalty on complexity
Every criterion has the form −2Lₖ + 2nα(k)rₙ. AIC takes rₙ = 2/n, which does not grow with n, so it is not order-consistent: in the paper's tables it keeps a small residual over-selection rate (about 11% at n = 10000 under α type I) that stops shrinking. BIC takes rₙ = log n / n, consistent under Keribin's (2000) smoothness and moment conditions, which the Laplace and Student-t families here violate.
Consistency under weaker assumptions
Multiply the BIC shape by Ln∘ν(n) or replace log n by (log n)1+ε. This is not cosmetic: the consistency theorem needs the penalty gap to satisfy (n/log n)(penl,n − penk,n) → ∞, and for plain BIC that quantity is the constant α(l) − α(k), so BIC is not covered. The extra logarithmic factor makes it diverge, which is exactly what buys consistency without differentiability and under only mild moment conditions. Numerically the change is tiny (at ε = 0.02 or ν = 3 it is invisible), so the gain is in the assumptions, not in the accuracy.
For the ν-BIC and ε-BIC penalties, the consistency theorems, the misspecification (KL-optimal order) result, and the limits on minimal penalties and minimax Hellinger risk, see Modifications of the BIC for order selection in finite mixture models (H. D. Nguyen & TrungTin Nguyen, Annals of the Institute of Statistical Mathematics, to appear).