With a Beta(a, b) prior and s successes in n Bernoulli trials, the posterior is Beta(a + s, b + n − s), and its equal-tailed 95% interval has posterior probability 95% for every data set. Whether it traps a fixed true proportion p 95% of the time is a different question. Top left: the posterior interval updated one observation at a time on a simulated data set. Top right: the interval for every possible outcome s = 0, …, n, with the binomial probabilities of the outcomes below; the coverage at p is the total probability of the teal (trapping) outcomes. Bottom: that coverage for every p.
Intervals are the equal-tailed posterior intervals, with Beta quantiles found by bisection on the regularised incomplete beta function. The coverage is Pp(LS < p < US) with S ~ Binomial(n, p), an exact finite sum; "lowest coverage" is its minimum over p = 0.001, 0.002, …, 0.999. The prior average is Σs m(s) P(Ls < p < Us | s) with m the beta-binomial distribution of S. Observation k of the data set is 1 when the k-th uniform of the seeded generator is below p.
code/ch11_bayesian_inference.py (qbeta, credible_ends, credible_coverage, prior_avg_coverage; Figure 11.S6).