After S successes in n trials, three intervals for the success probability p are built from p̂ = S/n: the Normal-based Wald interval p̂ ± zα/2√(p̂(1 − p̂)/n), the Hoeffding interval p̂ ± √(log(2/α)/(2n)), and the Wilson interval (all p whose own standard error makes p̂ unsurprising). Because S takes only n + 1 values, the probability that an interval traps p is a finite sum of binomial probabilities, computed here exactly. The upper panel shows this coverage against p or against n; the lower panel shows 100 simulated polls at the current n and p.
Coverage is Pp(a < p < b) for the open interval (a, b), summed exactly over the binomial distribution of S. "Lowest coverage" is the minimum over the grid p = 0.001, 0.002, …, 0.999. The expected width counts only the part of the interval inside [0, 1]. Polls are drawn with the seeded generator (seed in the readout): poll r uses n fresh uniforms u and counts u < p.
code/ch06_models_inference_learning.py (binom_interval, binom_coverage, coverage_curve, polls; Figure 6.S4).