Near its maximum the log-likelihood looks like a parabola whose curvature is the Fisher information In(θ̂). Cutting that parabola at −z2/2 gives the Wald interval θ̂ ± z/√In(θ̂) (gold). The same recipe applied to ψ = g(θ), a logit or a log, and mapped back gives a second, asymmetric interval (pink); the true log-likelihood cut at the same height gives the likelihood-ratio interval (teal). All three are justified by the same large-sample theory, yet they differ, and only an exact calculation says how often each one traps the truth. The top panel shows one simulated sample; the middle panel the exact coverage of the two Wald intervals for every value of the parameter, at your n; the bottom panel the coverage against n.
Wald interval: θ̂ ± z ŝe with ŝe = 1/√(n I(θ̂)), Theorem 9.19. Transformed: g(θ̂) ± z|g′(θ̂)| ŝe mapped back by g−1 (the delta method, Theorem 9.24), with g = logit for p and log otherwise. When θ̂ = 0 (or 1) the first interval is a single point and the second is undefined: both count as misses. Coverage is computed exactly: a binomial or Poisson sum for the discrete models; for the Exponential and the Normal scale the interval is built from a pivot, so its coverage is a Gamma or chi-square probability that does not depend on the parameter.
code/ch09_parametric_inference.py (wald_interval, wald_coverage, lr_interval).