Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 15

Priors on the logit scale

A prior that looks vague on the weights of a logistic model can be a strong, extreme prior on the probabilities. Left: the implied prior of σ(α) when the offset has a N(0, ω) prior. Right: the distribution of the fraction of labels equal to 1 when D binary features (each equal to 1 with probability p) have independent N(0, s2) weights.

Offset prior standard deviation √ω

Number of features D

Weight prior standard deviation s

Probability that a feature equals 1

Weight scaling

Try these

Left: exact density N(logit s; 0, ω) / (s(1 − s)); the dashed curves are ω = 10 (red) and ω = 1.5 (blue). Right: 2000 prior draws of the weights, each used to label 100 random feature vectors; the seeded generator is the same on every visit. With "1/√D" the weight standard deviation is s/√D.

In the lecture notes