Applications

Methods meeting the real world

Where my mixture-of-experts theory, model-selection and optimization algorithms, simulation-based inference and efficient deep-learning methods leave the page and act on real data. Each entry names the actual dataset, the method applied, the scientific insight it produced and the impact it is meant to have, then links to the paper's own experiments section and to related work.

My research develops principled statistical machine learning: guarantees for mixtures of experts, model selection without arbitrary sweeps, and honest uncertainty when the likelihood is intractable. The point of that rigour is to be trusted on problems that matter. The applications below span systems biology, genomics and proteomics, medical AI, chemistry, planetary science, engineering and the efficiency of AI itself. Filter by area or search by dataset, method or keyword.

Systems biology

Kinetic models are central to systems biology, but the enzyme-kinetic parameters compiled from the literature are incomplete, inconsistent and measured under heterogeneous conditions, so their uncertainty is easy to underestimate.

Robust Bayesian estimationBRENDA glycolysis & TCA-cycle kinetics

Bayesian parameter balancing enables robust and consistent estimation of kinetic parameter uncertainty

Method. A fully Bayesian reformulation of parameter balancing that keeps kinetic and thermodynamic parameters consistent (Haldane relations, equilibrium data from eQuilibrator), with robust Student-t and skew-normal likelihoods for outliers and asymmetric noise and source-level random effects for heterogeneity across studies.

Insight. In synthetic glycolysis and TCA-cycle studies, Gaussian credible intervals under-cover as the noise turns heavy-tailed or skewed, while the robust likelihoods stay close to the nominal 95%. On raw BRENDA records, leave-one-source-out prediction (230 glycolysis and 155 TCA-cycle sources) shows that source random effects give the largest gain, raising 95% predictive coverage from 0.68-0.72 to 0.83-0.85 with little change in RMSE.

Impact. Calibrated, thermodynamically consistent parameter sets can be reused as priors in dynamic, flux and whole-cell models, so overconfident kinetic values do not propagate false certainty downstream.

Genomics and proteomics

Omics data are high-dimensional, heterogeneous and often incomplete, so how many sub-populations there are, and which variables define them, is itself the scientific question.

Dendrograms of mixing measures233 maize genotypes, leaf proteomics

Dendrograms of mixing measures for softmax-gated Gaussian mixture of experts: consistency without model sweeps

Method. Fit one over-specified softmax-gated mixture of experts and merge its experts along a dendrogram of the mixing measure: a consistent, sweep-free choice of the number of experts, with finite-sample rates for parameter estimation.

Insight. For 233 maize genotypes, predicting water use from leaf-protein abundances measured under water deficit (973 proteins, screened to 10), the dendrogram selects two experts, read as two broad genotype-phenotype response regimes, where AIC picks 18 and BIC and ICL pick one. In simulations it stays correct under ε-contamination, while AIC, BIC and ICL over-select as the sample grows.

Impact. One over-fitted model, instead of a sweep over model sizes, gives both the number of experts and an interpretable hierarchy of genotype groups: a practical summary of heterogeneity in omics-assisted trait prediction.

Bayesian order selectionRice Diversity Panel; maize leaf proteome

Voronoi-calibrated order selection for Bayesian softmax-gated mixtures of experts

Method. A post-processor (V-MTM) for Bayesian softmax-gated Gaussian mixtures of experts: fit one deliberately over-specified model, then merge and truncate its experts with a Voronoi loss to recover the number of experts, with an order-recovery guarantee, instead of fitting and comparing a model for every candidate order.

Insight. On the Rice Diversity Panel (373 accessions; flowering time and plant height from four genotype principal components) it selects six experts (90% credible set 5 to 7) where variational inference, BIC and ICL all select one; the experts recover the panel’s five classical subpopulations almost one-to-one (adjusted Rand index 0.754), and five-fold held-out predictive density is worst at one expert. On a maize leaf proteome measured under well-watered and water-deficit treatments (248 genotypes, 954 proteins) it selects two experts that reproduce the withheld treatment (adjusted Rand index 0.913 with five principal components), a case the paper notes is easy.

Impact. Choosing the number of experts from a single posterior fit, with its uncertainty, makes mixtures of experts practical for population-structured biological data, and the same question sets the size of sparse expert layers in modern deep learning.

Incremental stochastic MMMaize proteomics; Communities and Crime

Revisiting incremental stochastic majorization-minimization algorithms with applications to mixture of experts

Method. An incremental stochastic majorization-minimization algorithm for high-volume or streaming data that generalizes incremental stochastic EM, needs no explicit latent-variable representation and provably converges to stationary points; it fits softmax-gated mixture-of-experts regression, for which no stochastic EM algorithm is available.

Insight. On 233 maize genotypes under water deficit (973 leaf proteins, Lasso-screened to 11 inputs) it has the lowest test error, MSE 0.116 against 0.132-0.133 for SGD, Adam, AdamW, RMSProp and Sophia. On the Communities and Crime data (1,994 U.S. communities, violent-crime rate) the incremental variants give the best MSE, MAPE and NRMSE.

Impact. Processing one observation per iteration, with convergence guarantees, makes mixture-of-experts regression practical where batch algorithms that revisit the full dataset are too slow.

MM training and dendrogram selectionProtein-protein interactions (DIP)

Fast model selection and stable optimization for softmax-gated multinomial-logistic mixture of experts models

Method. A batch minorization-maximization algorithm with closed-form updates and guaranteed monotone ascent for softmax-gated multinomial-logistic mixtures of experts, finite-sample rates, and dendrograms of mixing measures that choose the number of experts from a single fit.

Insight. On protein-protein interaction prediction (protein pairs from the DIP database, described by amino-acid composition), the dendrogram picks three experts, where ICL picks two (F1 0.66) and AIC and BIC pick four for only marginal gains. The three-expert model reaches F1 0.80 (recall 0.88), ahead of naive Bayes, random forest, SVM, logistic regression and a three-hidden-layer MLP (F1 0.72-0.77).

Impact. Stable training and single-fit model selection give compact, interpretable mixture-of-experts classifiers for heterogeneous biological prediction tasks.

Variable selection under MNARArabidopsis thaliana transcriptome

A unified framework for variable selection in model-based clustering with missing not at random

Method. A penalized Gaussian-mixture clustering with a data-driven penalty for variable selection and an MNARz mechanism that models the dependence between missingness and the latent class, with proven selection and asymptotic consistency.

Insight. On the 1,267-gene Arabidopsis thaliana transcriptome (27 conditions from seven projects), it finds 18 clusters and keeps four projects as the core clustering variables, agreeing with earlier analyses on the role of iron signalling, while showing that two projects previously selected as drivers are, once missingness is modelled, explained by the core four.

Impact. Clustering with variable selection that stays reliable when whether a value is missing depends on the hidden group, a common situation in transcriptomics.

Medical AI and healthcare

Medical foundation models must be reliable and data-efficient, because expert-annotated clinical data are scarce and mistakes are costly.

Graph-aligned vision-language modelVQA-RAD, SLAKE, PathVQA

ExGra-Med: extended context graph alignment for medical vision-language models

Method. A multi-graph alignment of images, instruction responses and extended captions for a LLaMA-7B medical assistant, trained end to end with black-box gradient estimation on PMC-15M image-text data spanning chest X-ray, CT, MRI, histopathology and gross pathology.

Insight. Pre-trained on only 10% of the data, ExGra-Med matches LLaVA-Med trained on all of it across VQA-RAD, SLAKE and PathVQA (73.96% versus 73.37% overall) and gains 20.13% on VQA-RAD over LLaVA-Med at the same budget, which drops from 72.6% to 52.4%. It also outperforms BioMedGPT and RadFM on visual-chatbot and zero-shot classification tasks.

Impact. Less dependence on costly instruction-following data makes better-grounded medical vision-language assistants cheaper to build.

Chemistry and drug discovery

A molecule is not one shape but an ensemble of conformers; predicting its properties well means integrating that 3D structural information at scale.

E(3)-invariant conformer aggregationMoleculeNet + SARS-CoV-2 inhibition

Structure-aware E(3)-invariant molecular conformer aggregation networks (CONAN-FGW)

Method. Aggregate a molecule's conformers through a differentiable fused Gromov-Wasserstein barycenter, giving an E(3)-invariant representation, paired with an efficient distance-geometry conformer generator.

Insight. With five conformers per molecule it has the lowest error on the MoleculeNet sets ESOL, FreeSolv and BACE (aqueous solubility, hydration free energy, BACE-1 inhibition) and is second on Lipo. On SARS-CoV-2 inhibition (CoV-2, CoV-2 3CL) it has the best ROC-AUC using 5-10 conformers instead of the 200 used by CP3D-NDU.

Impact. Accurate property prediction from a handful of cheaply generated conformers lowers the cost of screening molecules for drug-relevant properties such as solubility and antiviral activity.

Fragment-aware conformer transformerMoleculeNet + MARCEL (Kraken, Drugs-75K)

FACET: a fragment-aware conformer ensemble transformer

Method. A graph transformer that fuses the 2D molecular graph, its chemical fragments and many RDKit-generated 3D conformers through fragment-aware attention guided by fused Gromov-Wasserstein distances, a fast learned substitute for costly geometric solvers.

Insight. It has the lowest error on the MoleculeNet sets ESOL, FreeSolv and BACE and is best or second best on all seven MARCEL targets (Drugs-75K and Kraken), while training 5-6 times faster than CONAN-FGW: 300 epochs on Drugs-75K take 214 GPU-hours instead of 1,107.

Impact. Scaling conformer-ensemble learning to 75,000 molecules and hundreds of thousands of conformers brings 3D-aware property prediction to large, chemically diverse sets, including organocatalysts and transition-metal complexes.

Planetary science and inverse problems

In many physical sciences the forward model is a simulator, not a likelihood: given parameters you can simulate data, but you cannot write down the density needed for standard inference.

Surrogate posteriors for ABCNontronite reflectance (Hapke model)

Summary statistics and discrepancy measures for approximate Bayesian computation via surrogate posteriors

Method. Learn Gaussian-mixture (GLLiM) surrogate posteriors by inverse regression and use them as functional summary statistics in approximate Bayesian computation, compared through L2 and optimal-transport distances; the resulting quasi-posterior converges to the true posterior under standard conditions.

Insight. The approach is most useful when the posterior is multimodal. Inverting a laboratory reflectance measurement of the mineral nontronite through the Hapke photometric model (10 viewing geometries), it recovers two parameter solutions that differ only in macroscopic roughness and reconstruct the observed signal almost equally well.

Impact. Keeping every plausible solution, rather than a single best fit, matters when reflectance data are inverted to characterise planetary surfaces, where different parameter sets can explain the same observation.

Engineering and emissions

Engineering measurements are often heterogeneous, mixing several nonlinear input-output relationships, so choosing how many experts to fit is a model-selection problem.

Slope-heuristic model selectionEthanol engine NO emissions

A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models

Method. Penalized maximum likelihood that chooses the number of experts, the complexity of the mean experts and the block-diagonal covariance structure in Gaussian-gated (GLoME) and block-diagonal (BLoME) mixtures of experts, with non-asymptotic oracle inequalities once the penalty exceeds a lower bound, calibrated from data by the slope heuristic.

Insight. On the Ethanol data (88 measurements of nitrogen-oxide emissions against the equivalence ratio of a single-cylinder, ethanol-fuelled engine), the slope heuristic selects four experts across 100 random initializations, consistent with an earlier analysis, and regressing NO on the equivalence ratio gives an easily interpretable clustering and mean curve.

Impact. Penalties calibrated from the data choose the number of experts without hand tuning, with finite-sample guarantees behind the choice.

Efficient AI and foundation models

Large models are powerful but expensive to train and run; reducing their compute without losing accuracy lowers the cost of using them.

Spectrum-preserving token mergingImageNet, Flickr30k/MS-COCO, LLaVA VQA

Accelerating Transformers with spectrum-preserving token merging (PiToMe)

Method. An energy score flags large clusters of similar tokens for merging while preserving the low-energy (unique) tokens, with a proof that the merging preserves the spectral properties of the token space under mild conditions.

Insight. PiToMe saves 40-60% of the FLOPs of the base models. Off the shelf, the average drop is 0.5% for ViT-MAE-H image classification (2.6% for earlier merging methods) and 0.3% for CLIP retrieval on Flickr30k (4.5%), with similar behaviour for LLaVA-7B on six visual question-answering benchmarks.

Impact. Merging tokens without retraining cuts the compute and memory of vision, vision-language and language Transformers while keeping accuracy, lowering the cost of running foundation models.

Sparse mixture-of-experts routingenwik8, WikiText-103; SST, IMDB, BANKING77

HyperRouter: towards efficient training and inference of sparse mixture of experts

Method. Generate the router's parameters from a fixed hypernetwork and trainable embeddings, a middle ground between training the router and freezing it, to learn a better routing policy while alleviating the representation collapse of sparse mixture-of-experts training.

Insight. On enwik8 and WikiText-103 language modelling and transfer to SST-2, SST-5, IMDB and BANKING77, HyperRouter outperforms SMoE and SMoE-Dropout routing, most clearly when few experts are used at inference: with a single expert it lowers WikiText-103 perplexity from 560.93 (SMoE-Dropout) to 65.17.

Impact. Keeping accuracy when fewer experts are activated at inference is where sparse mixture-of-experts models save compute.

Token merging for SAMDIS5K, ThinObject-5K, COIFT, HRSOD, INbreast

StructSAM: structure- and spectrum-preserving token merging for Segment Anything models

Method. A training-free merge-unmerge scheme for the Segment Anything family: a feature-gradient token energy and grid flatness screening protect boundary and prompt regions, tokens in flat areas are merged and later recovered, with a bound on the induced spectral distortion.

Insight. Without fine-tuning, it cuts SAM encoder FLOPs by about 28.5% (ViT-B) and 21.9% (ViT-L) with largely unchanged mIoU on the boundary-sensitive DIS5K, ThinObject-5K, COIFT and HRSOD benchmarks. With Medical SAM on INbreast mammograms, Dice goes from 75.43 to 74.81 at 28.5% fewer FLOPs (74.72 at 41.8% with prompt-aware merging), ahead of ToMeSD, VidToMe and ALGM.

Impact. Cheaper promptable segmentation that keeps fine boundaries helps foundation segmentation models run where compute is limited, from medical imaging to robotic manipulation, where the paper also extends it to Efficient-SAM tracking.

Competition-based routingenwik8, text8, WikiText-103; SST, IMDB, BANKING77

CompeteSMoE: effective training of sparse mixture of experts via competition

Method. Route each input to the experts with the highest neural response, a competition mechanism shown to reach the same convergence rate as the optimal estimator, and train a light router to predict the competition outcome so the extra cost stays low.

Insight. With Switch Transformer and GLaM language models on enwik8, text8 and WikiText-103, CompeteSMoE beats SMoE, XMoE, StableMoE and fixed routing on every benchmark, and after fine-tuning it has the highest accuracy on SST-2, SST-5, IMDB and BANKING77 (for example 72.9% versus 69.2% for SMoE on BANKING77).

Impact. Tackling representation collapse makes sparse mixture-of-experts training, a way to grow model capacity without making the network deeper or wider, more effective at low extra cost.

The theory behind the applications. These results rest on a programme of mathematical guarantees: approximation of densities and conditional densities by mixtures (location-scale mixtures in Lebesgue spaces, mixtures of experts in Lebesgue spaces), convergence rates for parameter estimation in mixtures of experts (softmax-gated, Gaussian-gated, multinomial-logistic experts), non-asymptotic model selection and risk bounds (slope heuristic, joint rank and variable selection, risk bounds for mixture density estimation), consistent order selection (modified BIC), and the large-sample behaviour of Bayesian model evaluation (DIC and WBIC). The full list is on the Publications page, and the methods are runnable in the Software section.