Interactive demo · Probability and Statistical Inference · Chapter 6

Does a 95% interval for a proportion cover 95% of the time?

After S successes in n trials, three intervals for the success probability p are built from p̂ = S/n: the Normal-based Wald interval p̂ ± zα/2√(p̂(1 − p̂)/n), the Hoeffding interval p̂ ± √(log(2/α)/(2n)), and the Wilson interval (all p whose own standard error makes p̂ unsurprising). Because S takes only n + 1 values, the probability that an interval traps p is a finite sum of binomial probabilities, computed here exactly. The upper panel shows this coverage against p or against n; the lower panel shows 100 simulated polls at the current n and p.

Upper panel: coverage against

Sample size n

True proportion p

Level 1 − α

Intervals shown

Simulated polls (lower panel)

Try these

Coverage is Pp(a < p < b) for the open interval (a, b), summed exactly over the binomial distribution of S. "Lowest coverage" is the minimum over the grid p = 0.001, 0.002, …, 0.999. The expected width counts only the part of the interval inside [0, 1]. Polls are drawn with the seeded generator (seed in the readout): poll r uses n fresh uniforms u and counts u < p.

In the lecture notes