Interactive demo · Probability and Statistical Inference · Chapter 11

Is "no information" the same in every scale? Flat priors and Jeffreys' prior

A rule for choosing a prior can be applied to the parameter θ (a proportion p or a Poisson rate λ) or to a transformed parameter φ = g(θ), such as the log-odds. The two answers describe the same beliefs only if carrying one to the other scale with the Jacobian |g′| gives the other. Left: priors on the φ scale. Middle: priors on the θ scale. Right: the resulting posteriors for θ. Blue: the rule applied to θ; red: the rule applied to φ. For the flat rule the colours disagree; for Jeffreys' rule, f ∝ √I, computed in each scale from the Fisher information of that scale, they coincide.

Model

Rule for the prior

Second parametrisation φ = g(θ)

Number of observations n

Successes s

Try these

Each prior curve is rescaled to have area 1 over the window shown, because several of them (flat on the log-odds, flat on λ, Jeffreys for λ) are improper. Carrying a density from one scale to the other multiplies it by the Jacobian |g′(θ)| or its inverse. The red Jeffreys prior uses the Fisher information of one observation in the φ scale, I(φ) = E[(∂ log f/∂φ)2], with the score found by a central difference and the expectation summed exactly over the outcomes; it does not use the chain rule. Posteriors are exact: every prior here is a power of p and 1 − p (Beta posteriors) or of λ (Gamma posteriors). Intervals are equal-tailed 95%.

In the lecture notes