Reshape a risk valley from a sharp point to a flat plateau and watch the limit law of the minimum empirical risk morph from a clean Gaussian into the skewed infimum of a Gaussian process, the over-specification regime where naive Gaussian inference and the bootstrap fail. Use the ▸ buttons beside each control (or the Run the experiments launchers) to auto-play; everything runs client-side.

The Asymptotic Law of the Minimum Empirical Risk

Fit a model by minimizing empirical risk and the smallest attainable risk, the minimum empirical risk (MER), is a random number. Its rescaled fluctuation √n (MER − min f) has a limit law you can use for inference: bootstrap confidence sets, goodness-of-fit tests, model selection. The catch is where the population risk is minimized. When the minimizer is a single point, the limit is a clean symmetric Gaussian and the bootstrap is valid. When the model is over-specified so the population argmin is a whole flat set S, the limit becomes the infimum of a Gaussian process over S: it detaches, shifts left, and skews, and naive Gaussian inference (and the standard bootstrap) break. Widen the valley and watch the histogram fall apart.

Empirical risk minimizationAsymptotic distributionOver-specificationGaussian-process infimum

Read the paper Full text

Minimum empirical risk: Gaussian limit vs Gaussian-process infimum

teal = population risk f · blue = one empirical risk f̂ₙ = f + n^−1/2G · teal band = argmin set S · amber = this draw's minimizer

histogram of √n (MER − min f) · teal curve = classical N(0,σ²)

Plateau width w 0.00 Correlation ℓ 0.10 Sample size n 60

The unknown is a hypothesis x ∈ [0,1]. The population risk f(x) (teal) is a valley whose bottom you reshape: at w = 0 it is a sharp parabola with a single minimizer x₀ = ½; as w grows the bottom flattens into a plateau, so the population argmin set S = [x₀ − w/2, x₀ + w/2] becomes an interval (teal band). Each draw is a fresh empirical risk f̂ₙ(x) = f(x) + n^−1/2 G(x) (blue), where G is a smooth mean-zero Gaussian field (a random-Fourier surrogate for the empirical process, correlation length ℓ, marginal variance σ²). We locate its minimizer (amber), read off the minimum empirical risk, and drop the rescaled blow-up √n (MER − min f) = inf_x [√n f(x) + G(x)] into the histogram. Singleton (w = 0): the minimizer localizes at x₀ and the statistic converges to G(x₀) ~ N(0, σ²), symmetric and centered, and the teal Gaussian overlay fits, so the CLT and the bootstrap are valid. Plateau (w > 0): the statistic becomes inf_x∈S G(x), the infimum of a Gaussian process over S, which is negatively shifted and left-skewed, so the histogram detaches from the Gaussian: this is the over-specification regime where naive Gaussian inference and the standard bootstrap fail. Raising n tightens the singleton toward the Gaussian but leaves the plateau's non-Gaussian law unchanged. (Honest toy: this is the i.i.d., continuous, √n baseline with a random-Fourier Gaussian-field stand-in for the empirical process and a grid argmin. The paper proves the general limit for non-i.i.d. data, discontinuous losses and losses indexed by non-Euclidean spaces, then builds consistent bootstrap confidence sets, penalized model-selection tests, and neural-network applications: none of that heavier machinery is in this 2D cartoon.)

Run the experiments

Every animation runs live in your browser. Click a button to run that experiment on the demo (it scrolls up and starts); drag any control to take over. Nothing is downloaded, it is generated on the fly.

Widen the plateau

Sweep the flat-bottom half-width from a sharp unique minimizer (w = 0) to a broad argmin set S, and watch the histogram of the rescaled minimum empirical risk detach from the Gaussian, shift left and skew.

Roughen the noise field

Change the correlation length of the Gaussian field driving the empirical risk: a rougher field gives the infimum over the plateau more chances to dip, deepening the left shift and skew.

Grow the sample size

Increase n and see the singleton case tighten toward the exact N(0, sigma^2), while the over-specified plateau keeps its non-Gaussian, skewed limit no matter how large n gets.

Singleton vs plateau

Toggle between the unique-minimizer (identifiable) case, whose histogram matches the symmetric Gaussian, and the flat-plateau (over-specified) case, whose limit is the skewed infimum of a Gaussian process where the bootstrap fails.

Accumulate fresh draws

Drop many fresh empirical-risk draws into the histogram one at a time to trace out the limiting law of the rescaled minimum empirical risk for the current valley shape.

The idea in three steps

Empirical risk minimization returns both an estimate and a number, the minimum empirical risk. That number fluctuates, and its limit law is the engine behind bootstrap confidence sets and goodness-of-fit tests, provided you know which regime you are in.

1 · Rescale

The √n blow-up

The gap between the minimum empirical risk and the population minimum is tiny, of order 1/n. Multiply by √n and it converges to a genuine random variable: inf_x[√n f(x) + G(x)], driven by a Gaussian process G from the CLT for the risk.

2 · Identify

One point, one Gaussian

If the population risk has a unique minimizer, the √n term pins the infimum to that point and the limit is just G there: a symmetric N(0,σ²). Here the CLT and the bootstrap deliver valid confidence sets and tests.

3 · Over-specify

A flat set, a skewed law

When the argmin is a whole set S (an over-specified or non-identifiable model) the limit is the infimum of the Gaussian process over S: shifted, skewed, non-Gaussian. Naive Gaussian intervals and the standard bootstrap are inconsistent, exactly the failure the general theory characterizes.

For the general limit theorems under non-i.i.d. data, discontinuous and non-Euclidean-indexed losses, and the consistent bootstrap confidence sets, penalized model-selection tests and neural-network applications, see On the Asymptotic Distribution of the Minimum Empirical Risk (Westerhout, TrungTin Nguyen, Guo & H. D. Nguyen, ICML 2024, PMLR Vol. 235).