Interactive demo · First-year statistics · Proportions

Counting successes: exact and approximate methods for proportions

With yes/no data the count X can only take whole values 0, 1, …, n, so tests and intervals for a proportion cannot hit their nominal levels exactly. Top: the distribution of X when H0: p = p0 is true, and the exact test's rejection region (pink). Middle: the actual false-alarm rate (size) of the exact test and of the normal-approximation z-test, for every n from 5 to 100. Bottom: how often the 95% Wald interval p̂ ± 1.96√(p̂(1 − p̂)/n) really contains p. Nothing is simulated: every number is an exact binomial sum.

Number of trials (n)

Null value p₀ (test of H0: p = p₀ against p > p₀)

Significance level α

Scoreboard

Try these

Rule of thumb: the normal approximation is reasonable with at least about 5 successes and 5 failures. The pink bands in the bottom panel are where that fails, and that is where the Wald interval's coverage collapses.

In the lecture notes