Interactive demo · Probability and Statistical Inference · Chapter 15

The two-sample Kolmogorov-Smirnov test

When Y is binary and Z continuous, Y and Z are independent exactly when the distribution of Z is the same in both groups. The Kolmogorov-Smirnov test compares the two empirical distribution functions through their largest vertical gap D. Under the null hypothesis, and for continuous data, the distribution of D does not depend on the common distribution at all: the pooled ordering of the two samples is a uniformly random path, and the exact p-value is a count of paths. Theorem 15.12 approximates it by Kolmogorov's limit H. The top panel shows the two empirical cdfs and D; the bottom panel the exact null distribution of the scaled statistic √(n1n2/(n1 + n2)) D against H.

Data

Sizes n1, n2

Shift of sample 2

Spread of sample 2 / sample 1

Level α

Permutations B

Try these

D = max over x of |F̂1(x) − F̂2(x)|. Exact: P(D ≥ observed) when the data are continuous (no ties), by summing over lattice paths. Asymptotic: 1 − H(t) at the scaled statistic t, from (15.14). Permutation: the pooled values are shuffled B times and split into groups of sizes n1 and n2; (1 + #{D* ≥ D})/(B + 1) is valid with ties, which the exact formula ignores. Simulated data: sample 1 is N(0, 1), sample 2 is N(shift, spread2), from seeded normal draws. The calcium data are from Exercise 15.7; the coronary risk data (coris) compare 160 men with and 302 without coronary heart disease. Welch's t-test compares only the means.

In the lecture notes