Interactive demo · Foundations of Deep Learning · Chapter 5

Decision theory: the loss matrix, the reject option and the ROC curve

A screening test measures x. People without the disease (class C1, "normal") and with it (C2, "cancer") have different densities p(x | Ck). Once the posterior p(C2 | x) is known, every decision is a threshold on it: say "cancer" when p(C2 | x) ≥ τ. The threshold τ = 1/2 makes the fewest mistakes; a missed cancer that costs L21 times as much as a false alarm moves it to τ = 1/(1 + L21); refusing to decide where the larger posterior is below θ buys a lower error on the rest. Sweeping τ from 1 to 0 traces the ROC curve, which does not depend on how common the disease is.

Separation of the classes d

Prevalence p(C₂)

Loss of a missed cancer L₂₁ (false alarm costs 1)

Threshold τ on p(C₂ | x)

Reject option θ

Scoreboard

Try these

In the lecture notes