Interactive demo · Nonparametric Statistical Inference · Chapter 7

James–Stein, the linear oracle and Pinsker's bound

For Zi ~ N(θi, σn²), i = 1, …, n, the best estimator of the form bZ uses b* = ‖θ‖²/(nσn² + ‖θ‖²), which needs the unknown ‖θ‖. The James–Stein estimator (1 − (n − 2)σn²/ΣZi²)Z estimates b* from the data and loses at most 2σn² (Theorem 7.47). Panel (a) shows the exact risks per coordinate against ‖θ‖²/(nσn²). Panel (b) takes σn² = σ²/n and the ball ‖θ‖ ≤ c: the worst-case James–Stein risk falls to Pinsker's minimax risk σ²c²/(σ² + c²) (Theorems 7.28 and 7.48).

Dimension n in panel (a)

Positive-part James–Stein (simulated)

Ball radius c in panel (b)

Noise level σ in panel (b)

Try these

Exact risks use (7.44): R = nσn² − (n − 2)²σn² E[1/(n − 2 + 2K)], K ~ Poisson(‖θ‖²/(2σn²)), summed to many standard deviations of K. The risk depends on θ only through ‖θ‖, so the simulation puts all of θ in the first coordinate. Replicate r uses its own seeded generator (seed 1 + 7919 r).

In the lecture notes