Interactive demo · Foundations of Deep Learning · Chapter 4

Decision theory for regression: the Minkowski loss

Given a predictive density p(t|x), which single number f should we predict? Under the loss |f − t|q the best choice minimises the expected loss Gq(f) = E|f − t|q. For q = 2 it is the conditional mean and for q = 1 the median. As q → 0 the minimiser tends to the minimiser of E[ln|f − t|] (gold), which coincides with the mode for a symmetric unimodal density but not in general: try the skewed and the bimodal densities.

Conditional density p(t|x)

Exponent q (log scale)

Try these

Densities: Gaussian N(1, 0.8²); Gamma with shape 2 and scale 1 (mean 2, mode 1); the mixture 0.3 N(−1, 0.2²) + 0.7 N(1.5, 0.7²). The expected losses are computed on a grid of 1201 values of t (the trapezoidal rule, with the cell around t = f integrated exactly), the same algorithm as lq_risk_grid in the Python code. The middle panel rescales each curve to the range [0, 1] over the plotted window, so curves for different q can be compared.

In the lecture notes