Interactive demo · Probability and Statistical Inference · Chapter 5

The delta method: a transformed mean, exactly and approximately

If Xn is approximately N(μ, σ2/n) and g is differentiable, the delta method replaces g by its tangent line at μ and concludes that g(Xn) is approximately N(g(μ), g′(μ)2σ2/n). For exponential, Bernoulli and Poisson samples the law of Xn is known exactly (gamma, binomial, Poisson), so the law of g(Xn) is too, and the approximation can be checked without simulation. The top panel shows g, its tangent at μ, and the density of Xn beneath: the approximation is good when g is nearly straight over the range where Xn lives. The middle panel compares the distribution functions, the bottom panel the Kolmogorov distance against n. When g′(μ) = 0 the first-order approximation collapses to a point, and the second-order limit (a scaled χ21) takes over.

Population

Parameter

Transformation g

Sample size n

Try these

Exponential with mean μ: Xn ~ Gamma(n, scale μ/n), and the moments of g(Xn) have closed forms (a mean or variance can be infinite, e.g. E(1/X1) = ∞). Bernoulli and Poisson: nX̄n is binomial or Poisson and everything is a finite sum. Transformations that are undefined at Xn = 0 (log, 1/x) are switched off for the discrete laws, which put mass there; x(1 − x) and arcsin√x are offered for the Bernoulli only. Kolmogorov distance: sup over y of |P(g(Xn) ≤ y) − approximation|.

In the lecture notes