Interactive demo · Probabilistic Machine Learning: Advanced Topics · Chapter 23

Scalar bijections and the densities they induce

Elementwise, coupling and autoregressive flows all rest on a strictly increasing map h of one variable. If u is standard normal, x = h(u) has density φ(u) / h′(u) at u = h−1(x): where the map is steep the density is thin, where it is flat the density piles up. Pick a family, move its parameters, and compare the formula (blue) with a histogram of 20 000 transformed draws (grey). Let h′ go negative and the map folds: the formula, which assumes a single preimage, then loses mass while the histogram keeps the truth.

Family

Parameters

Try these

Formulas: affine au + b with a = elog a; non-linear squared (23.9) au + b + c/(1 + (du + e)2), invertible iff a > 0.6495 |c| d; rational-quadratic spline with knots at −3, −1.5, 0, 1.5, 3, bin heights 6 × softmax of the four log heights, one common interior knot derivative, identity outside [−3, 3]; sum of squares (23.11) with h′ = (a0 + a1u)2 + (b0 + b1u)2 and h(0) = 0. The inverse is found by 80 bisection steps on [−30, 30]; the mass is a Riemann sum over [−8, 8] with 1 600 cells.

In the lecture notes