Interactive demo · First-year statistics · Two-sample tests

Two samples: pooled or Welch, paired or not?

Two questions about comparing two groups, each answered by running thousands of studies and collecting their P-values (bottom panels; pink = rejected at 5%). Pooled or Welch: here the two groups have the same true mean, so H0 is true and a valid 5% test should reject in about 5% of studies with a flat P-value histogram; the pooled test assumes equal spreads, Welch's test does not. Paired or not: each person's sleep is measured before and after a change (true increase 0.425 h); analysing each person's difference removes the person-to-person variation.

Question

Group sizes (n₁, n₂)

Spread ratio σ₂/σ₁

Correlation of a person's two nights (ρ)

People (n)

Run

Scoreboard

Try these

In the lecture notes