A prior that looks vague on the weights of a logistic model can be a strong, extreme prior on the probabilities. Left: the implied prior of σ(α) when the offset has a N(0, ω) prior. Right: the distribution of the fraction of labels equal to 1 when D binary features (each equal to 1 with probability p) have independent N(0, s2) weights.
Left: exact density N(logit s; 0, ω) / (s(1 − s)); the dashed curves are ω = 10 (red) and ω = 1.5 (blue). Right: 2000 prior draws of the weights, each used to label 100 random feature vectors; the seeded generator is the same on every visit. With "1/√D" the weight standard deviation is s/√D.
code/ch15_generalized_linear_models.py (Figure 15.5).