Interactive demo · Probabilistic Machine Learning · Chapter 4

The beta-binomial model: prior, posterior, predictive

A coin has unknown heads probability θ. A Beta(a, b) prior combined with N1 heads and N0 tails gives the posterior Beta(a + N1, b + N0) (4.114): the prior counts simply add to the observed counts. The top panel shows the prior, the (rescaled) likelihood and the posterior with its central and highest-density intervals. The bottom panel predicts the number of heads in M further tosses, once by integrating θ out (the beta-binomial, 4.135) and once by plugging in the point estimate, which is too confident when data are scarce. A mixture-of-betas prior (4.144) is also available.

Prior

Prior pseudo-counts a, b (single beta)

Observed heads N1 and tails N0

Future tosses M

Try these

The mixture prior is 0.5 Beta(20, 20) + 0.5 Beta(30, 10) (a fair coin or one biased towards heads). The posterior of a mixture is a mixture with weights proportional to prior weight times marginal likelihood (4.147). The plug-in prediction uses the MAP estimate (equal to the MLE under a uniform prior; with a single beta prior it needs a, b ≥ 1 to exist). Intervals: 95% central, and 95% highest posterior density found by searching for the narrowest interval (single beta with a, b > 1 only).

In the lecture notes