Interactive demo · Probability and Statistical Inference · Chapter 9

Asymptotic normality and efficiency of the MLE

Under regularity conditions √n(θ̂n − θ) is approximately N(0, 1/I(θ)), and no well-behaved estimator does better: 1/I(θ) is the Cramér–Rao bound. Each preset pits the MLE (blue) against a natural competitor (pink) on R simulated samples of size n. The top panel shows the two sampling distributions of √n(estimate − truth) with the Normal limits (gold: variance 1/I; dashed pink: the competitor's limit); the bottom panel shows n times the mean squared error against n on log axes. The MLE's curve settles on the bound, the competitor's on a higher level, and the ratio of the two levels is the asymptotic relative efficiency. The last preset breaks the rules: for the Uniform(0, θ) endpoint there is no Fisher information and the MLE converges at rate 1/n, faster than any "efficient" estimator.

Model and competitor

Sample size n

Simulated samples R

Try these

Normal: N(0, 1), mean against median. Cauchy: location 0, scale 1; the MLE is computed by Newton's method from the median, with a weighted-mean (EM) step whenever Newton's step would lower the likelihood. Gamma: shape α = 2 and scale β = 1 both unknown, the shape estimated by maximum likelihood (Newton on the profile equation log α − ψ(α) = log X̄ − mean log X) or by moments, X̄2/σ̂2. Exercise 9.6: N(0.5, 1) data, ψ = P(X > 0) = Φ(0.5), estimated by Φ(X̄) or by the fraction of positive observations. Uniform(0, 1): the maximum against 2X̄. Each sample size uses its own stream of the seeded generator, so the histogram is the point of the lower curve at the chosen n.

In the lecture notes