Interactive demo · Probability and Statistical Inference · Chapter 10

The Neyman–Pearson lemma: no test beats the likelihood ratio

To test a simple null H0: θ = θ0 against a simple alternative H1: θ = θ1, the Neyman–Pearson lemma says: reject when the likelihood ratio L(θ1)/L(θ0) is large, with the cut-off chosen to give size α, and no other test of size α has more power. Here we simulate many samples under each hypothesis, compute the log likelihood ratio and a natural competitor (a median, a mean or a mean absolute value), and draw the ROC curve of each test: power against size as the critical value moves. The likelihood ratio curve is never below the competitor's.

Family

Sample size n

Alternative θ1

Size α

Simulated samples per hypothesis

likelihood ratio (NP)  competitor  exact NP

Try these

Location families: H0 has centre 0 and H1 centre θ1 (Normal with σ = 1, Laplace with density e−|x − θ|/2, standard Cauchy). Scale family: Xi ~ N(0, σ2) with σ = 1 against σ = σ1. Each statistic rejects for large values. At size α the critical value is the order statistic of the simulated H0 values with exactly ⌊αR⌋ values above it, so both tests have the same simulated size; the power is the fraction of H1 samples above it. Where the most powerful test has a closed form (Normal mean: reject for large x̄; Normal scale: Σxi2 ~ χ2n) its exact ROC curve is drawn in white. Seeded generator: the H0 samples use the first Rn draws and the H1 samples the next Rn.

In the lecture notes