Software Demos
Play with the ideas behind my research directly in the browser: mixture-of-experts regression and clustering, and my dendrogram-merging approach to choosing the number of components without model sweeps.
Fit the model once at Kmax experts, then drag the cut down the dendrogram: redundant experts merge, the effective count K̂ falls, and the fit barely moves. That is the essence of consistency without model sweeps — no refitting for every candidate number of components. In Joint MoE mode a single fit does all three at once: the gating clusters the points (colour), each expert carries a local regression (a tilted ellipse for its region and trend, plus the mean curve), and the dendrogram selects the number of components.
The idea in three steps
Choosing the number of mixture-of-experts components usually means refitting the model for every candidate value and scoring each fit. My work reads the answer off a single over-fitted model instead.
Deliberately use too many experts
Fit a softmax/location-gated Gaussian mixture of experts with a large Kmax. Several experts end up explaining the same region — the fit is good but redundant.
Cluster the mixing measure
Each expert is an atom of the mixing measure. Agglomerating atoms by their distance in parameter space yields a dendrogram whose merge heights reveal which experts are really the same.
Read off the components
Cutting the dendrogram merges redundant atoms into one, giving a consistent estimate K̂ of the true number of components — without ever refitting the model.
This is a lightweight, illustrative in-browser fit (a few EM iterations, average-linkage merging), not the full estimator from the paper. For the theory and experiments see Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts.