Interactive demo · Probability and Statistical Inference · Chapter 12

Stein's paradox: shrinking k unrelated means towards 0 beats estimating each one separately

Observe Xj ~ N(θj, 1) independently for j = 1, …, k and lose the total squared error Σ(θ̂j − θj)². The obvious estimator θ̂ = X has risk k whatever θ is. The James-Stein estimator multiplies every coordinate by the same factor 1 − (k − 2)/‖X‖² (cut at 0 in its positive-part form (12.12)). Panel (a) shows that for k ≥ 3 its risk is below k at every ‖θ‖, panel (b) shows one simulated data set coordinate by coordinate, and panel (c) shows how the gain grows with the dimension.

Dimension k

Length of the mean vector ‖θ‖

Direction of θ

Simulated data sets R

Try these

Exact risks: James-Stein k − (k − 2)² E[1/(k − 2 + 2K)] with K ~ Poisson(‖θ‖²/2); the positive part by the formula of Filling the gap 12.17 (a Poisson mixture of chi-square tail probabilities). For k ≤ 2 the shrinkage constant (k − 2)+ is 0 and both estimators equal X. The simulation uses the seeded generator (seed in the readout).

In the lecture notes