TrungTin Nguyen Mathematical Statistics ยท AI for Science
Software Demos

Software Demos

Interactive, in-browser demonstrations of the methods behind my papers, click, drag and explore. Everything runs client-side; no data leaves your browser.

Mixture of experts & dendrogram merging Bayesian nonparametric mixture of experts Universal approximation Convergence rates for MoE Variable selection with missing data Robust Bayesian parameter balancing GLLiM-ABC likelihood-free inference Molecular conformer aggregation (FGW) FACET fragment-aware conformers Medical vision-language graph alignment Spectrum-preserving token merging StructSAM token merging for SAM HyperRouter (sparse mixture of experts) CompeteSMoE (competition routing) Concentration of ABC (unidentifiability) Model selection for MoE (slope heuristic) BIC order selection for mixtures PSGaBloME penalized selection BSL with mixtures of experts Bayesian model-evaluation limits Minimum empirical risk (limit law) Risk bounds for mixture density

Find a demo

Catalogue

Each demo is a lightweight, illustrative version of a method from one of my papers, linked to the paper and (where available) the public code. More will be added over time.

New software contributions (scikit-learn extensions)

Mixture of experts. A Python package that extends scikit-learn's Gaussian mixtures to softmax- and Gaussian-gated mixtures of experts, ported from my NeurIPS 2023 and AISTATS 2024 papers. It ships a user guide, an examples gallery (with a live in-browser EM demo), and an API reference, in the spirit of the scikit-learn docs. Open the software section →

Gaussian mixtures, animated. A companion gallery that reproduces all seven scikit-learn Gaussian-mixture examples as live in-browser animations, covariance types, density estimation, BIC selection, EM versus a Dirichlet process, initialization methods, the sine curve, and the concentration prior, a faithful port of sklearn/mixture. Explore the Gaussian-mixtures gallery →

Mixture of Experts & Dendrogram Merging

Advanced~4 minMixed

Fit one over-stuffed mixture of experts, then drag a cut down a dendrogram and watch redundant experts merge until the right number appears.

One over-fitted mixture of experts: the gating clusters the inputs (point colour), each expert fits a local regression (mean curve + band), and the number of components is read off a dendrogram of mixing measures with no model sweeps. The data is generated from the selected gate's own mechanism (softmax-gated SGMoE, or Gaussian-gated), so each gate is fit well-specified; the sliders set the true number of experts K0, the sample size N, and the ε-contamination. The data starts ε-contaminated (Laplace outliers): the classical likelihood criteria AIC / BIC / ICL over-select, while the consistent dendrogram selection criterion (DSC, gold line) holds near the true count; drag ε to 0 for the clean case. You can also compare the paper's rate-weighted merge against average-linkage agglomerative clustering.

Mixture of expertsModel selectionClusteringRegression
▸ Open the interactive demoRead the paper (softmax-gated)Gaussian-gated variant

Bayesian Nonparametric Mixture of Experts · Merge-Truncate-Merge

Advanced~4 minIllustrative

Drag one "merge radius" slider and watch a Bayesian model collapse dozens of redundant experts down to the true count.

The Bayesian companion to the dendrogram demo above. A Gaussian-gated mixture of experts (GLLiM) is fit by variational Bayes with a Dirichlet-process prior at a large truncation, so it deliberately over-fits the mixing measure; the Merge-Truncate-Merge (MTM) post-processing collapses the redundant atoms, and repeating its randomization gives a full posterior over the number of experts. Drag the merge radius (the Bayesian analogue of cutting the dendrogram) and watch the posterior concentrate on the true count with calibrated uncertainty.

Bayesian nonparametricsMixture of expertsUncertainty quantificationInverse problems
▸ Open the interactive demoRead the paperJournal (JNPS)

Universal Approximation · Mixtures and Mixtures of Experts

Intermediate~3 minIllustrative

Drag one slider up and watch a mixture model's curve snap onto whatever target shape you pick.

Why these models can represent essentially anything. A finite location-scale mixture approximates any probability density, and a mixture of experts approximates any regression function (and conditional density), to arbitrary accuracy as the number of components K grows. Pick a hard target and a family, then drag K: the approximation hugs the target and the Lp error falls to zero, at a rate set by how smooth the target is.

Approximation theoryMixturesMixture of expertsConvergence rates
▸ Open the interactive demoRead the papers

Convergence Rates for Parameter Estimation in Mixtures of Experts

Advanced~4 minFaithful

Simulate a mixture of experts and read its parameter-recovery speed off a log-log plot, then add one surplus expert and watch the rate collapse.

How fast do the estimated experts approach the true ones as data accumulate? If you fit the right number of experts, the parameters converge at the fast parametric rate √N. But you rarely know the true number, and the moment you fit too many experts the rate collapses to something far slower, set by the interaction between the gating and the experts. Run a real simulation and read the rate off a log–log plot.

Parameter estimationConvergence ratesMixture of expertsIdentifiability
▸ Open the interactive demoRead the papers

Variable Selection with Missing-Not-at-Random Data

Advanced~4 minIllustrative

Crank up how often data go missing and watch naive methods misclassify points (red rings) while the MNAR-aware model holds.

Model-based clustering has to solve two problems at once: which variables matter for the clusters, and what to do about missing values when the missingness is informative. When a value is missing because of the cluster it belongs to (missing not at random), throwing the pattern away biases everything. This unified framework instead models the missingness, and recovers both the clusters and the relevant variables. Crank up the MNAR strength and watch the naive methods fail while the MNAR-aware one holds.

Missing dataModel-based clusteringVariable selectionMNAR
▸ Open the interactive demoRead the paperNeurIPS 2025

Robust Bayesian Parameter Balancing

Intermediate~4 minIllustrative

Drag noisy enzyme measurements and watch a Bayesian fit snap them into thermodynamically consistent parameters, then throw an outlier at it.

Reconcile noisy, inconsistent kinetic measurements into thermodynamically consistent parameters with calibrated uncertainty, and watch robust likelihoods shrug off outliers.

Bayesian inferenceUncertainty quantificationSystems biologyRobustness
▸ Open the interactive demoRead the paperCode (Julia)

GLLiM-ABC: Likelihood-Free Inference with Surrogate Posteriors

Advanced~4 minFaithful

Click a map to place a hidden sound source, then watch four ways of matching simulations fight to recover its blurry, multi-armed posterior.

When the likelihood is intractable but you can simulate data, approximate Bayesian computation (ABC) accepts parameter proposals whose simulated data look like the observation. The hard part is how to compare: which summary statistics, and which distance. This method learns a surrogate posterior once (a Gaussian mixture from an inverse-regression fit), then uses the whole distribution as a functional summary and compares surrogates with an L2 or optimal-transport distance. This is the paper's flagship multiple-hyperboloid experiment, a source-localization problem whose posterior is genuinely multimodal. Click the map and compare the four ABC discrepancies.

Approximate Bayesian computationSurrogate posteriorsOptimal transportMultimodal inference
▸ Open the interactive demoRead the paperCode (R)

Spectrum-Preserving Token Merging (PiToMe)

Intermediate~3 minFaithful

Watch a Vision Transformer merge redundant tokens layer by layer while an energy score shields the informative ones.

Transformers spend most of their compute on attention, which is quadratic in the number of tokens. You can merge redundant tokens to shrink that count, but merge the wrong ones and you throw away information. PiToMe adds an energy score that flags which tokens sit in a dense, redundant neighbourhood (safe to merge) versus which are isolated and informative (protect), and it comes with a spectral guarantee: the merged token graph keeps almost the same Laplacian spectrum. Step through the layers and watch PiToMe protect the informative tokens while naive ToMe merges them away and its spectral distance climbs.

Token mergingEfficient transformersSpectral graph theoryViT / CLIP / BLIP
▸ Open the interactive demoRead the paperCodeNeurIPS 2024

HyperRouter: Efficient Sparse Mixture of Experts via a Frozen HyperNetwork

Intermediate~3 minFaithful

Train two token routers side by side and watch the ordinary one collapse onto a few experts while the frozen-hypernetwork one stays balanced.

A sparse mixture of experts routes each token to a few of N experts, but a freely trainable router, optimized jointly with the experts, tends to collapse: it over-uses a handful of experts, the load imbalances, and the experts drift toward each other (representation collapse), so practitioners bolt on a load-balancing auxiliary loss as a crutch. HyperRouter instead generates the router weights from a frozen random hypernetwork applied to a tiny trainable embedding e: a 2D code drives the whole gate through a fixed nonlinear map that keeps experts diverse and the load balanced without any auxiliary loss. Train both routers side by side and watch the standard one collapse.

Sparse mixture of expertsHyperNetwork routerRepresentation collapseLoad balancing
▸ Open the interactive demoRead the paperCodeEMNLP 2023

CompeteSMoE: Effective Training of Sparse Mixture of Experts via Competition

Advanced~4 minMixed

Train two mixture-of-experts side by side and watch one router learn who wins while the other collapses onto a couple of experts.

A sparse mixture of experts routes each token to a few of N experts through a router, a cheap gate scored on a detached embedding dot-product x⊤Wr that has no idea what the experts will actually do. That indirection makes routers slow to learn and prone to collapse. CompeteSMoE removes the middle-man during training: every expert computes its output, each expert's own neural response is scored, and tokens go to the strongest responders (winner-take-all), exactly like neurons competing. A cheap linear router is then distilled to predict the competition winner, so at inference only the router runs, same cost as vanilla SMoE. Train a standard router and CompeteSMoE side by side and watch the distilled router learn the competition winner while the standard one collapses.

Sparse mixture of expertsCompetition routingRouter distillationSample efficiency
▸ Open the interactive demoRead the paperCodearXiv 2024

Conformer Aggregation by Fused Gromov-Wasserstein Barycenter

Advanced~3 minMixed

Spin one 3D pose of a molecule: the naive coordinate average lurches, but the optimal-transport aggregate never moves.

A molecule is not a single shape: it flexes into many 3D conformers, and a property predictor must fuse them into one representation. Averaging the raw atom coordinates fails, because two conformers can be the same shape in different orientations, and the mean collapses the atoms into an unphysical blob. This method aggregates instead with a Fused Gromov-Wasserstein (FGW) barycenter of the conformer graphs. Each conformer enters only through its structure matrix and its atom features, both invariant to rotating, translating or reflecting the molecule, so the aggregate is provably E(3)-invariant. Rotate a conformer and watch the naive mean swing while the FGW barycenter holds still.

Molecular property predictionOptimal transportE(3)-invarianceGeometric deep learning
▸ Open the interactive demoRead the paperFull textCode (PyTorch)

Fragment-Aware, Fast Conformer Ensemble Transformer

Advanced~4 minMixed

Shuffle a molecule's 3D shapes and watch the averaged fingerprint hold still while the expensive all-pairs alignment vanishes.

FACET extends the Fused Gromov-Wasserstein conformer-aggregation idea with two moves. First, fragment awareness: a RingPath decomposition splits a molecule's 2D graph into SSSR rings and maximal path linkers that meet at junction atoms, producing a per-atom 30-dim fragment-type count prior that is projected (Linear 30→64), concatenated to the 9 base atom features, run through a 2-layer GAT, and folded back as an additive bias on each conformer's intra-conformer Graphormer atom-atom attention. Second, fast aggregation: the predecessor pays an O(N²) price to align all N conformers by pairwise FGW; FACET instead trains a Graphormer to regress the true pairwise FGW distance into a metric encoding, then aggregates the frozen per-conformer encodings by a plain mean (enc.mean, the latent FGW barycenter). Attention is atom-atom inside a conformer; across conformers the aggregate is an average, which is permutation-invariant. That turns N(N−1)/2 optimal-transport solves into an O(N) encode-then-mean pass, a measured over 6x faster aggregation. Rebuild the ensemble and watch the encoding track FGW while the mean aggregate stays put under reordering.

Fragment-aware graph learningConformer aggregationOptimal transportPermutation invariance
▸ Open the interactive demoRead the paperFull textCode (PyTorch)

Extended Context Graph Alignment for Medical Vision-Language Models

Advanced~4 minMixed

Watch three views of a medical batch snap into agreement with one shared hub, and see crossed red matches turn green.

A medical vision-language model must line up three views of the same training batch, one vector per sample: the pooled visual patch features Zv, the LLM features of the original answers Za, and the LLM features of GPT-4 extended answers Zae. Aligning every pair of views costs K(K−1)/2 comparisons. ExGra-Med instead forms one barycenter (the elementwise mean of the three node-feature matrices) and aligns each view to that single hub, K comparisons, reusing the source view's kNN edges for the barycenter. A second-order QAP scores both a node cosine term and an edge-difference inner-product term, and the training signal is the bidirectional Hamming distance between the solver's correspondence V̂ and the identity ground truth V*. Toggle first vs second order and watch the crossed red matches snap back to green.

Medical vision-languageBarycenter feature meanSecond-order QAP (BBGM)Bidirectional HammingI-MLE
▸ Open the interactive demoRead the paperFull textCode (PyTorch)

StructSAM: Structure- and Spectrum-Preserving Token Merging for Segment Anything

Intermediate~3 minMixed

Watch a patch grid collapse redundant cells while amber edges and the prompt box stay protected, keeping boundaries crisp.

Token merging accelerates a Vision Transformer by collapsing redundant tokens, but the Segment Anything Model (SAM) breaks the recipe twice over: its image encoder mixes windowed and global attention, and its mask decoder needs a dense, prompt-conditioned feature grid for crisp boundaries. Applied off-the-shelf, ToMe / PiToMe-style destination heuristics erode object contours and leak prompt information as the merge rate rises. StructSAM (extending PiToMe) fixes this with three moves: a token-energy score from first-order feature gradients (high = boundary, low = flat), grid-based flatness screening that protects boundary and prompt tokens, and a merge-unmerge wrapper that restores the dense grid for the decoder. Step through the layers and watch StructSAM keep both the graph spectrum and the segment boundaries intact while naive ToMe degrades them at the same compute.

Token merging for SAMStructure preservationSpectral graph coarseningPrompt-aware mergingEfficient segmentation
▸ Open the interactive demoRead the paperOpenReviewCode (PiToMe base)

Revisiting concentration results for approximate Bayesian computation

Advanced~3 minFaithful

Shrink the ABC tolerance toward zero and watch the posterior mass collapse: onto a point, two spikes, or a diamond it can never resolve.

As the tolerance ε → 0, the ABC posterior concentrates, but it can only concentrate on the equivalence set Θ₀ of parameters that generate the same data law: everything a discrepancy cannot distinguish stays indistinguishable. This demo draws the prior (dotted) and the paper's limiting ABC posterior πε(θ) ∝ π(θ) w(|D∞(θ₀,θ)|, ε) in closed form over a fine grid, for the three normal models of Section 6: identifiable, finitely unidentifiable, and infinitely unidentifiable. Sweep ε and watch the mass collapse, onto a point, onto two symmetric points, or onto a one-dimensional L1 diamond.

Approximate Bayesian computationPosterior concentrationUnidentifiabilityLarge-sample theory
▸ Open the interactive demoRead the paperFull text

Model selection for mixtures of experts via the slope heuristic

Advanced~4 minMixed

Slide the penalty constant and watch the chosen number of experts tumble down a staircase until the data itself marks the right spot.

How many experts does the data support? Fit a Gaussian-gated mixture of experts for every candidate number of experts K, then let the slope heuristic (Birge-Massart) choose. The method knows the penalty shape (proportional to the model dimension) but not its multiplicative constant κ. Slide κ and watch the selected model walk down a staircase: over-fit (large K) → correct (K₀) → under-fit (small K). The dimension jump reads the right constant straight off the data, and the fitted conditional-mean curve on the left changes in lockstep.

Penalized model selectionSlope heuristicMixture of expertsNon-asymptotic oracle inequality
▸ Open the interactive demoRead the paperFull text

Modifications of the BIC for order selection in finite mixture models

Advanced~4 minMixed

Slide the sample size up and watch each information criterion lock its minimum onto the true number of mixture components.

How many components does a mixture have? An information criterion answers by trading fit (maximized log-likelihood) against a penalty on complexity. This demo fits a light EM to a 1D data histogram for every candidate order k = 1..8, then draws four criterion curves: AIC (flat penalty), BIC, and the paper's ν-BIC and ε-BIC, which inflate the BIC penalty by a negligibly small logarithmic factor to buy order-selection consistency under much weaker (non-differentiable, mild-moment) conditions. Watch the selected order and the Monte-Carlo P(k̂ = k₀) climb to 1 as the sample size grows, while AIC plateaus below 1.

Order selectionFinite mixturesConsistencyBIC penalties
▸ Open the interactive demoRead the paperFull text

PSGaBloME: Penalized Model Selection for Softmax-Gated Mixture of Experts

Advanced~4 minIllustrative

Drag a penalty dial and watch a mixture-of-experts model shed complexity in a staircase, snapping to the best model.

A softmax-gated mixture of experts (MoE) needs its hyper-parameters chosen from the data: how many experts, how complex, and which gating weights are worth keeping. This demo runs the paper's slope-heuristic mechanism live. Fit a family of MoE models by EM, then let a penalized maximum-likelihood criterion pick the complexity: the empirical contrast (negative log-likelihood) plus a penalty κ·D/n proportional to model dimension, with an ℓ1 term that sparsifies the gating so the selected model keeps a data-driven number of active experts. Watch the dimension jump: as κ grows, the selected complexity falls in a staircase, and the slope-heuristic rule κ̂ = 2κmin lands the selected model near the oracle.

Slope heuristicOracle inequalitySoftmax-gated MoEℓ1 sparse gating
▸ Open the interactive demoRead the paperFull text

BSL with Mixtures of Experts: a surrogate likelihood that bends

Intermediate~3 minIllustrative

Watch a single stretched ellipse fail to trace a crescent-shaped cloud while a mixture of little ellipses bends right around it.

Bayesian Synthetic Likelihood (BSL) replaces an intractable likelihood by a single Gaussian fitted to simulated summaries, one fit per MCMC step. When the summaries are crescent-shaped (the classic two-moons model), one Gaussian cannot bend to follow them. This paper swaps that Gaussian for a Mixture of Experts (a GLLiM / Gaussian-mixture surrogate estimated by EM), which hugs the crescent, and it is fitted once instead of every iteration. Drag the source, raise K, and watch the amber ellipse fail while the teal mixture wins, then read the amortization saving.

Bayesian synthetic likelihoodMixtures of expertsGLLiM / EMAmortized inference
▸ Open the interactive demoRead the paperFull text

On the large-sample limits of Bayesian model-evaluation statistics

Advanced~3 minMixed

Watch Bayesian criteria slide down a log-scale plot and lock onto a single limit line, unless the tempering schedule breaks the rule.

Bayesian information criteria (DIC, BPIC, WBIC) replace a maximised log-likelihood with a posterior expectation of the log-likelihood. This demo reproduces the paper's normal-model experiment (its Figures 1-2 in spirit): simulate n data points from N(θ₀, 1), plot the per-observation criterion against sample size n on a log scale for many replicates, and watch it approach (or fail to approach) the almost-sure limit −2 E[log p(X | θ₀)] = log(2π) + 1. The one thing that decides convergence is the tempering schedule: WBIC locks onto the limit exactly when n βn → ∞, and stalls above it otherwise. A second panel switches the likelihood to a geometric model, where the same DIC / WBIC almost-sure-limit story plays out for discrete data.

Bayesian model evaluationWBIC / DIC / BPICPower posteriorsAlmost-sure limits
▸ Open the interactive demoRead the paperFull text

The Asymptotic Law of the Minimum Empirical Risk

Advanced~4 minMixed

Reshape a risk valley from a sharp point to a flat plateau and watch the histogram of the best-fit error morph from a clean bell curve into a skewed one.

Fit a model by minimizing empirical risk and the smallest attainable risk, the minimum empirical risk (MER), is a random number. Its rescaled fluctuation √n (MER − min f) has a limit law you can use for inference: bootstrap confidence sets, goodness-of-fit tests, model selection. The catch is where the population risk is minimized. When the minimizer is a single point, the limit is a clean symmetric Gaussian and the bootstrap is valid. When the model is over-specified so the population argmin is a whole flat set S, the limit becomes the infimum of a Gaussian process over S: it detaches, shifts left, and skews, and naive Gaussian inference (and the standard bootstrap) break. Widen the valley and watch the histogram fall apart. A second mode applies the same idea to the paper's dependent-data mixture-of-experts model selection: a softmax-gated Gaussian MoE with non-i.i.d. inputs, choosing the number of experts by SWIC, BIC and AIC, where the √n MER rate makes the dependence-aware SWIC the consistent choice.

Empirical risk minimizationAsymptotic distributionOver-specificationGaussian-process infimum
▸ Open the interactive demoRead the paperFull text

Risk Bounds via the h-Lifted KL Divergence

Advanced~4 minMixed

Sharpen a fitted mixture until it dips to zero and watch the standard KL loss spike to infinity while the h-lifted loss stays finite.

To estimate a density f on a compact domain by a finite mixture q, the natural loss is the Kullback-Leibler divergence. But KL blows up to infinity wherever the fitted mixture touches zero, so classical risk bounds must assume every density stays bounded away from 0, an assumption real mixtures and real targets violate. This paper's fix is the h-lifted KL divergence, KLh(f‖q) = ∫(f+h) log[(f+h)/(q+h)], which floors the ratio by a positive lift h and stays finite for all continuous densities, while still controlling estimation: the maximum h-lifted likelihood estimator obeys an O(1/(k+2) + 1/√n) excess-risk bound with no strict-positivity assumption. Sharpen or shift the mixture components to open a valley where q→0 and watch the two losses diverge.

Mixture density estimationh-lifted KL divergenceOracle risk boundsBregman divergence
▸ Open the interactive demoRead the paperFull textCode (R / Python)