You observe k heads in N tosses. The frequentist asks how the estimate k/N would vary over repeated data sets: the bootstrap approximates that sampling distribution by resampling the data (top left). The Bayesian reports the posterior of θ instead (top right); for a weak prior the two look alike, so the bootstrap is a “poor man's posterior”. The lower panel is the exact probability that each 95% interval contains the true θ, as θ varies: the Wald interval p ± 1.96√(p(1−p)/N) can fall far below 95%, and is degenerate when k = 0 or N. A confidence level describes the procedure over repeated data, not the probability that θ lies in the one interval you computed.
The bootstrap resamples the N observed tosses with replacement B times and records the proportion of heads each time. The posterior uses the uniform Beta(1,1) prior, so it is Beta(k+1, N−k+1); its draws come from the inverse cdf. Coverage is exact (a sum over the binomial distribution of k), not simulated. Wald uses the interval (p, p) when k = 0 or N. Jeffreys is the central credible interval of Beta(k+½, N−k+½).
code/ch04_statistics.py (bootstrap_mle_bernoulli, wald_interval, wilson_interval, jeffreys_interval, bernoulli_coverage).