Lecture notes · First-year statistics · Advanced

Estimation and Hypothesis Testing: Advanced Notes

The same topics as the beginner notes, now built up from the axioms of probability. Every formula is stated as a result and proved, so that you can see why each confidence interval and test works, and exactly which assumptions it needs.

How to read these notes

Prerequisites. Sets and functions, sums and integrals, derivatives, and for one proof (Theorem 0.9) orthogonal matrices. Everything about probability is developed from the axioms in Part 0.

Proofs are hidden. Read the statement first, decide whether you believe it (try a small case, or run a demonstration), then open the proof. The button below opens or closes all of them at once. After most results a ▶ Demo box opens an interactive demonstration already set to show that result; the full list is in Demonstrations.

Honest foundations. A few deep results are used without proof and are labelled (F1)–(F5) in Section 0.1: they are proved in a second-year probability course. Everything else is proved here from them and from the axioms. Results marked Going further are optional.

Main references

  1. D. P. Kroese and J. C. C. Chan, Statistical Modeling and Computation, Springer (2014): Chapters 1–6.
  2. G. Casella and R. L. Berger, Statistical Inference, 2nd edition, Duxbury (2002): Chapters 5–9, for the proofs.
  3. J. L. Devore and K. N. Berk, Modern Mathematical Statistics with Applications, 2nd edition, Springer (2012): Chapters 6–10.
  4. P. Lafaye de Micheaux, R. Drouilhet and B. Liquet, The R Software, Springer (2013): Chapter 13, for the R code.

Scope: estimation and hypothesis testing for a first-year course. The text, proofs, examples and exercises are my own.

Proofs start hidden: click any “Proof” bar to open one.

0. Foundations

0.1 Probability from the axioms

Axioms (Kolmogorov)

A probability \(P\) assigns to each event \(A\) of a sample space \(\Omega\) a number with (A1) \(P(A)\ge 0\); (A2) \(P(\Omega)=1\); (A3) if \(A_1,A_2,\dots\) are pairwise disjoint, \(P\big(\bigcup_i A_i\big)\)\({}=\sum_i P(A_i)\). Events \(A,B\) are independent if \(P(A\cap B)=P(A)P(B)\).

Proposition 0.1

(a) \(P(A^c)=1-P(A)\). (b) If \(A\subseteq B\) then \(P(A)\le P(B)\). (c) \(P(A\cup B)\)\({}=P(A)+P(B)-P(A\cap B)\)\({}\le P(A)+P(B)\).

Proof

Finite additivity first. Taking every \(A_i=\varnothing\) in (A3) gives \(P(\varnothing)=\sum_i P(\varnothing)\), which is impossible for a finite number unless \(P(\varnothing)=0\). So (A3) also holds for finitely many disjoint events: pad the list with empty sets.

(a) \(A\) and \(A^c\) are disjoint with union \(\Omega\), so by (A3) and (A2), \(P(A)+P(A^c)=1\). (b) \(B=A\cup(B\cap A^c)\) is a disjoint union, so \(P(B)=P(A)+P(B\cap A^c)\ge P(A)\) by (A1). (c) \(A\cup B=A\cup(B\cap A^c)\) and \(B=(B\cap A)\cup(B\cap A^c)\), both disjoint; subtract the two applications of (A3). The inequality uses \(P(A\cap B)\ge0\). ∎

A random variable \(X\) is a numerical function of the outcome. Its distribution is described by the cumulative distribution function \(F(x)=P(X\le x)\), and by a probability mass function \(p(x)\) (discrete case) or a density \(f(x)\) with \(P(a\lt X\le b)=\int_a^b f\) (continuous case). The expectation is \(E(X)=\sum_x x\,p(x)\) or \(\int x f(x)\,dx\), and \(E\,g(X)=\sum g(x)p(x)\) or \(\int g(x)f(x)\,dx\). Random variables are independent if events defined by them are independent. When \(X,Y\) have a joint density, they are independent exactly when it factorises, \(f_{X,Y}(x,y)=f_X(x)f_Y(y)\), and likewise for \(n\) variables.

Foundational results used without proof

  • (F1) Uniqueness of moment generating functions. If \(M_X(t)=E(e^{tX})\) and \(M_Y(t)\) are finite and equal for all \(t\) near 0, then \(X\) and \(Y\) have the same distribution.
  • (F2) Change of variables. If \(\mathbf Y=A\mathbf Z\) with \(A\) an invertible matrix, then \(f_{\mathbf Y}(\mathbf y)\)\({}=f_{\mathbf Z}(A^{-1}\mathbf y)/|\det A|\).
  • (F3) Continuity theorem. If \(M_{X_n}(t)\to M_X(t)\) for all \(t\) near 0, then \(P(X_n\le x)\to P(X\le x)\) at every \(x\) where the limit CDF is continuous (convergence in distribution).
  • (F4) Slutsky's theorem. If \(X_n\to X\) in distribution and \(Y_n\to c\) in probability (a constant), then \(X_n Y_n\to cX\) and \(X_n+Y_n\to X+c\) in distribution; and \(g(Y_n)\to g(c)\) in probability for continuous \(g\).
  • (F5) Exchange of derivative and integral under the regularity conditions stated in Theorem 2.4 (used only there).

0.2 Expectation and variance

Theorem 0.2 (linearity and variance rules)

For constants \(a,b\) and random variables \(X,Y\) with finite variances:

\[ E(aX+bY)=aE(X)+bE(Y) \]\[ \mathrm{Var}(X)=E(X^2)-(EX)^2 \]\[ \mathrm{Var}(aX+b)=a^2\mathrm{Var}(X) \] \[ \mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)+2\,\mathrm{Cov}(X,Y) \]\[ \mathrm{Cov}(X,Y)=E\big[(X-EX)(Y-EY)\big]. \]

If \(X,Y\) are independent, \(E(XY)=E(X)E(Y)\), hence \(\mathrm{Cov}(X,Y)=0\) and variances add.

Proof

Linearity (continuous case; the discrete case replaces integrals by sums): \(E(aX+bY)\)\({}=\iint (ax+by)f_{X,Y}(x,y)\,dx\,dy \)\({}= a\int x f_X(x)\,dx+b\int y f_Y(y)\,dy\), integrating out the other variable to get each marginal density.

Variance formula: with \(\mu=EX\), \(\mathrm{Var}(X)\)\({}=E(X-\mu)^2\)\({}=E(X^2)-2\mu E(X)+\mu^2\)\({}=E(X^2)-\mu^2\). Scaling: \(aX+b-E(aX+b)=a(X-\mu)\), so the variance is \(a^2E(X-\mu)^2\).

Sum: expand \(\big[(X-EX)+(Y-EY)\big]^2\) and take expectations term by term.

Independence: \(E(XY)\)\({}=\iint xy\,f_X(x)f_Y(y)\,dx\,dy\)\({}=\big(\int x f_X\big)\big(\int y f_Y\big)\). Then \(\mathrm{Cov}(X,Y)=E(XY)-E(X)E(Y)=0\) (expand the product in the definition). ∎

Corollary 0.3 (the sample mean)

If \(X_1,\dots,X_n\) are independent with common mean \(\mu\) and variance \(\sigma^2\), then \(\bar X=\frac1n\sum_{i=1}^n X_i\) has \(E(\bar X)=\mu\) and \(\mathrm{Var}(\bar X)=\sigma^2/n\).

Proof

By linearity \(E(\bar X)=\frac1n\sum E(X_i)=\mu\). By independence the covariances vanish, so \(\mathrm{Var}(\sum X_i)\)\({}=\sum\mathrm{Var}(X_i)\)\({}=n\sigma^2\), and scaling by \(1/n\) multiplies the variance by \(1/n^2\). Note that normality is not needed. ∎

0.3 Inequalities and the law of large numbers

Theorem 0.4 (Markov and Chebyshev)

If \(Y\ge 0\) and \(a\gt0\), then \(P(Y\ge a)\le E(Y)/a\). Consequently, for any \(X\) with mean \(\mu\) and variance \(\sigma^2\), and any \(\varepsilon\gt0\), \(P(|X-\mu|\ge \varepsilon)\)\({}\le \sigma^2/\varepsilon^2\).

Proof

Let \(I\) be the indicator of \(\{Y\ge a\}\). Then \(aI\le Y\) always (if \(I=1\), \(Y\ge a\); if \(I=0\), \(Y\ge 0\)). Taking expectations, \(aP(Y\ge a)\le E(Y)\). For Chebyshev apply Markov to \(Y=(X-\mu)^2\) and \(a=\varepsilon^2\), noting \(\{|X-\mu|\)\({}\ge\varepsilon\}\)\({}=\{(X-\mu)^2\)\({}\ge\varepsilon^2\}\). ∎

Theorem 0.5 (weak law of large numbers)

For iid \(X_i\) with mean \(\mu\) and finite variance, \(P(|\bar X_n-\mu|\ge\varepsilon)\)\({}\to0\) for every \(\varepsilon\gt0\): \(\bar X_n\to\mu\) in probability.

Proof

By Chebyshev and Corollary 0.3, \(P(|\bar X_n-\mu|\ge\varepsilon)\)\({}\le \dfrac{\sigma^2}{n\varepsilon^2}\)\({}\to0\). ∎

▶ Demo: Averages on trial With normal data and \(n=100\), the averages crowd around \(\mu\): none of 2000 falls outside \(\mu\pm\sigma/2\), well inside Chebyshev's bound \(4/n\).

0.4 Moment generating functions and the normal family

\(Z\sim\mathcal N(0,1)\) has density \(\varphi(z)\)\({}=\frac{1}{\sqrt{2\pi}}e^{-z^2/2}\) (that it integrates to 1 is the Gaussian integral \(\int e^{-z^2/2}dz=\sqrt{2\pi}\)). We write \(X\sim\mathcal N(\mu,\sigma^2)\) (with \(\sigma\gt0\)) when \(X\) has the same distribution as \(\mu+\sigma Z\), and \(\Phi\) for the CDF of \(Z\). The upper quantile \(z_\alpha\) is defined by \(P(Z\gt z_\alpha)=\alpha\); for example \(z_{0.05}=1.645\), \(z_{0.025}=1.960\).

Theorem 0.6 (MGFs and sums of independent normals)

(a) \(M_X(t)=e^{\mu t+\sigma^2t^2/2}\) for \(X\sim\mathcal N(\mu,\sigma^2)\). (b) If \(X,Y\) are independent then \(M_{X+Y}=M_XM_Y\). (c) If \(X_i\sim\mathcal N(\mu_i,\sigma_i^2)\) are independent, then \(\sum a_iX_i\sim\mathcal N\big(\sum a_i\mu_i,\ \sum a_i^2\sigma_i^2\big)\). In particular \(\bar X\sim\mathcal N(\mu,\sigma^2/n)\) for an iid normal sample.

Proof

(a) Complete the square: \(tz-\tfrac{z^2}{2}\)\({}=-\tfrac{(z-t)^2}{2}+\tfrac{t^2}{2}\), so \(E(e^{tZ})\)\({}=e^{t^2/2}\int\varphi(z-t)\,dz\)\({}=e^{t^2/2}\). Then \(E(e^{t(\mu+\sigma Z)})\)\({}=e^{\mu t}E(e^{(\sigma t)Z})\)\({}=e^{\mu t+\sigma^2t^2/2}\).

(b) \(e^{t(X+Y)}=e^{tX}e^{tY}\) and independence gives \(E(e^{tX}e^{tY})=E(e^{tX})E(e^{tY})\) (Theorem 0.2 applied to functions of independent variables).

(c) By (a) and (b), \(M_{\sum a_iX_i}(t)\)\({}=\prod_i e^{\mu_i a_i t+\sigma_i^2a_i^2t^2/2}\)\({}=\exp\big(t\sum a_i\mu_i+\tfrac{t^2}{2}\sum a_i^2\sigma_i^2\big)\), the MGF of the stated normal. By uniqueness (F1) that is the distribution. ∎

0.5 The central limit theorem

Theorem 0.7 (CLT)

If \(X_1,X_2,\dots\) are iid with mean \(\mu\) and variance \(\sigma^2\in(0,\infty)\), then for every \(z\), \(P\!\left(\dfrac{\bar X_n-\mu}{\sigma/\sqrt n}\le z\right)\)\({}\to\Phi(z)\).

Proof sketch (assuming the MGF exists near 0)

Let \(Y_i=(X_i-\mu)/\sigma\), so \(E Y_i=0\), \(E Y_i^2=1\), and \(Z_n\)\({}=\frac{\bar X_n-\mu}{\sigma/\sqrt n}\)\({}=\frac{1}{\sqrt n}\sum Y_i\). By independence, \(M_{Z_n}(t)\)\({}=\big[M_Y(t/\sqrt n)\big]^n\). Taylor's theorem gives \(M_Y(s)\)\({}=1+s\,E Y+\tfrac{s^2}{2}E Y^2+o(s^2)\)\({}=1+\tfrac{s^2}{2}+o(s^2)\). Hence

\[ M_{Z_n}(t)=\Big(1+\frac{t^2}{2n}+o(1/n)\Big)^n\longrightarrow e^{t^2/2}, \]

using \((1+c/n+o(1/n))^n\to e^c\). This is the MGF of \(\mathcal N(0,1)\), and the continuity theorem (F3) converts MGF convergence into convergence of the CDFs. The general proof (without an MGF) uses characteristic functions in the same way. ∎

▶ Demo: Averages on trial: the central limit theorem Skewed waiting times averaged over \(n=1\), \(n=5\) and \(n=30\): the standardised averages turn into the bell curve. Lumpy yes/no data need more: \(n=10\) against \(n=100\).

0.6 The chi-squared and t distributions

Definitions

If \(Z_1,\dots,Z_k\) are iid \(\mathcal N(0,1)\), then \(V=\sum_{j=1}^kZ_j^2\) has the chi-squared distribution with \(k\) degrees of freedom, \(V\sim\chi^2_k\). If \(Z\sim\mathcal N(0,1)\) and \(V\sim\chi^2_k\) are independent, then \(T=Z/\sqrt{V/k}\) has Student's t distribution with \(k\) degrees of freedom, \(T\sim t_k\).

Theorem 0.8 (properties)

(a) \(M_V(t)=(1-2t)^{-k/2}\) for \(t\lt\tfrac12\); \(E(V)=k\), \(\mathrm{Var}(V)=2k\). (b) If \(V_1\sim\chi^2_{a}\) and \(V_2\sim\chi^2_{b}\) are independent, \(V_1+V_2\sim\chi^2_{a+b}\). (c) \(t_k\) is symmetric about 0, and \(t_k\to\mathcal N(0,1)\) in distribution as \(k\to\infty\).

Proof

(a) For one term, \(E(e^{tZ^2})\)\({}=\int\frac{1}{\sqrt{2\pi}}e^{-(1-2t)z^2/2}dz\)\({}=(1-2t)^{-1/2}\), by substituting \(u=\sqrt{1-2t}\,z\). Independence multiplies \(k\) such MGFs. Differentiating, \(M'(0)=k\) and \(M''(0)=k(k+2)\), so \(\mathrm{Var}(V)=k(k+2)-k^2=2k\).

(b) \(M_{V_1+V_2}\)\({}=(1-2t)^{-a/2}(1-2t)^{-b/2}\)\({}=(1-2t)^{-(a+b)/2}\); apply (F1).

(c) \(Z\) and \(-Z\) have the same distribution and both are independent of \(V\), so the pairs \((Z,V)\) and \((-Z,V)\) have the same joint distribution; hence \(T\) and \(-T\) have the same distribution. As \(k\to\infty\), \(V/k\) is an average of \(k\) iid \(Z_j^2\) with mean 1, so \(V/k\to1\) in probability by the law of large numbers, and \(T=Z\cdot(V/k)^{-1/2}\to Z\) by Slutsky (F4). ∎

0.7 The normal-sample theorem

This single result is behind every exact interval and test for normal data in these notes.

Theorem 0.9 (sample mean and variance of a normal sample)

Let \(X_1,\dots,X_n\) be iid \(\mathcal N(\mu,\sigma^2)\), \(n\ge2\), and \(S^2\)\({}=\frac{1}{n-1}\sum (X_i-\bar X)^2\). Then

  1. \(\bar X\sim\mathcal N(\mu,\sigma^2/n)\);
  2. \(\dfrac{(n-1)S^2}{\sigma^2}\sim\chi^2_{n-1}\);
  3. \(\bar X\) and \(S^2\) are independent;
  4. \(\dfrac{\bar X-\mu}{S/\sqrt n}\sim t_{n-1}\).
Proof

Standardise: \(Z_i=(X_i-\mu)/\sigma\) are iid \(\mathcal N(0,1)\), with joint density \((2\pi)^{-n/2}\exp(-\tfrac12\|\mathbf z\|^2)\). Then \(\bar X=\mu+\sigma\bar Z\) and \((n-1)S^2/\sigma^2\)\({}=\sum(Z_i-\bar Z)^2\), so it suffices to work with the \(Z_i\).

An orthogonal rotation. The unit vector \(\mathbf u\)\({}=(1/\sqrt n,\dots,1/\sqrt n)\) can be extended (Gram–Schmidt) to an orthonormal basis of \(\mathbb R^n\). Let \(A\) be the orthogonal matrix with these basis vectors as rows, first row \(\mathbf u\), and put \(\mathbf Y=A\mathbf Z\). Because \(A^{-1}=A^{\mathsf T}\), \(|\det A|=1\) and \(\|A^{\mathsf T}\mathbf y\|\)\({}=\|\mathbf y\|\), (F2) gives \(f_{\mathbf Y}(\mathbf y)\)\({}=(2\pi)^{-n/2}\exp(-\tfrac12\|\mathbf y\|^2)\). This factorises, so \(Y_1,\dots,Y_n\) are again iid \(\mathcal N(0,1)\).

Reading off the pieces. \(Y_1\)\({}=\mathbf u\cdot\mathbf Z\)\({}=\sqrt n\,\bar Z\), and \(\sum Y_i^2\)\({}=\|\mathbf Y\|^2\)\({}=\|\mathbf Z\|^2\)\({}=\sum Z_i^2\). Since \(\sum(Z_i-\bar Z)^2\)\({}=\sum Z_i^2-n\bar Z^2\),

\[ \sum_{i=1}^n (Z_i-\bar Z)^2=\sum_{i=1}^n Y_i^2-Y_1^2=\sum_{i=2}^n Y_i^2 . \]

(i) \(\bar Z\)\({}=Y_1/\sqrt n\sim\mathcal N(0,1/n)\), so \(\bar X\sim\mathcal N(\mu,\sigma^2/n)\). (ii) The right-hand side is a sum of \(n-1\) iid squared standard normals: \(\chi^2_{n-1}\) by definition. (iii) \(\bar Z\) depends only on \(Y_1\) and \(\sum(Z_i-\bar Z)^2\) only on \(Y_2,\dots,Y_n\), which are independent of \(Y_1\). (iv) Write

\[ \frac{\bar X-\mu}{S/\sqrt n}=\frac{(\bar X-\mu)/(\sigma/\sqrt n)}{\sqrt{\dfrac{(n-1)S^2/\sigma^2}{n-1}}}, \]

a standard normal divided by the square root of an independent \(\chi^2_{n-1}\) over its degrees of freedom: \(t_{n-1}\) by definition. ∎

Why the \(n-1\)?

The proof shows it geometrically: the \(n\) deviations \(Z_i-\bar Z\) are forced to sum to zero, so they live in an \((n-1)\)-dimensional subspace, and their squared length is a sum of only \(n-1\) independent squared normals. One dimension was used up by estimating the mean.

▶ Demo: Why Student's t? Inside the normal-sample theorem For normal data \(\bar X\) and \(S\) are unrelated (part iii) and \(T\) has heavier tails than the bell curve (part iv); for skewed data they move together, and for uniform data they are uncorrelated yet dependent.

1. Estimators and their properties

Definitions

A statistical model is a family of distributions \(\{P_\theta:\theta\in\Theta\}\) for the data \(\mathbf X=(X_1,\dots,X_n)\). A statistic is any function of \(\mathbf X\) that does not involve \(\theta\). An estimator of \(\theta\) is a statistic \(\hat\Theta=g(\mathbf X)\) used to guess \(\theta\); its value \(\hat\theta=g(\mathbf x)\) on observed data is the estimate. The distribution of \(\hat\Theta\) under \(P_\theta\) is its sampling distribution.

\(\hat\Theta\) is unbiased if \(E_\theta(\hat\Theta)=\theta\) for every \(\theta\); its bias is \(E_\theta(\hat\Theta)-\theta\); its standard error is \(\sqrt{\mathrm{Var}_\theta(\hat\Theta)}\); its mean squared error is \(\mathrm{MSE}\)\({}=E_\theta(\hat\Theta-\theta)^2\). A sequence \(\hat\Theta_n\) is consistent if \(\hat\Theta_n\to\theta\) in probability for every \(\theta\).

Theorem 1.1 (bias–variance decomposition)

\[ \mathrm{MSE}(\hat\Theta)=\mathrm{Var}(\hat\Theta)+\mathrm{bias}(\hat\Theta)^2 . \]
Proof

Let \(m=E(\hat\Theta)\). Then \(\hat\Theta-\theta\)\({}=(\hat\Theta-m)+(m-\theta)\), and

\[ E(\hat\Theta-\theta)^2=E(\hat\Theta-m)^2+2(m-\theta)E(\hat\Theta-m)+(m-\theta)^2 . \]

The middle term is zero because \(E(\hat\Theta-m)=0\). ∎

Theorem 1.2 (MSE to zero implies consistency)

If \(\mathrm{MSE}(\hat\Theta_n)\to0\) for every \(\theta\), then \(\hat\Theta_n\) is consistent. In particular an unbiased estimator whose variance tends to 0 is consistent.

Proof

Markov's inequality applied to \((\hat\Theta_n-\theta)^2\): \(P(|\hat\Theta_n-\theta|\ge\varepsilon)\)\({}\le \mathrm{MSE}(\hat\Theta_n)/\varepsilon^2\)\({}\to0\). ∎

Theorem 1.3 (the standard estimators)

For iid data with mean \(\mu\) and variance \(\sigma^2\): (a) \(\bar X\) is unbiased for \(\mu\) with standard error \(\sigma/\sqrt n\), and consistent. (b) \(S^2\) is unbiased for \(\sigma^2\), whereas \(\hat\sigma^2\)\({}=\frac1n\sum(X_i-\bar X)^2\) has \(E(\hat\sigma^2)\)\({}=\frac{n-1}{n}\sigma^2\). (c) If \(X\sim\mathrm{Bin}(n,p)\), \(\hat P=X/n\) is unbiased for \(p\) with standard error \(\sqrt{p(1-p)/n}\). (d) For normal data, \(\mathrm{Var}(S^2)=2\sigma^4/(n-1)\).

Proof

(a) Corollary 0.3 and Theorem 1.2.

(b) First the algebraic identity \(\sum(X_i-\bar X)^2\)\({}=\sum X_i^2-n\bar X^2\) (expand the square and use \(\sum X_i=n\bar X\)). By Theorem 0.2, \(E(X_i^2)=\sigma^2+\mu^2\) and \(E(\bar X^2)\)\({}=\mathrm{Var}(\bar X)+\mu^2\)\({}=\sigma^2/n+\mu^2\). Therefore

\[ E\sum(X_i-\bar X)^2=n(\sigma^2+\mu^2)-n(\sigma^2/n+\mu^2)=(n-1)\sigma^2 . \]

Dividing by \(n-1\) gives an unbiased estimator; dividing by \(n\) gives mean \(\frac{n-1}{n}\sigma^2\).

(c) Write \(X=\sum_{i=1}^n B_i\) with \(B_i\) iid Bernoulli(\(p\)): \(E B_i=p\), \(E B_i^2=p\), \(\mathrm{Var}(B_i)=p-p^2\). Then \(\hat P=\bar B\) and Corollary 0.3 applies.

(d) By Theorem 0.9(ii), \(S^2=\frac{\sigma^2}{n-1}V\) with \(V\sim\chi^2_{n-1}\), and \(\mathrm{Var}(V)=2(n-1)\) by Theorem 0.8, so \(\mathrm{Var}(S^2)\)\({}=\frac{\sigma^4}{(n-1)^2}\cdot2(n-1)\). ∎

Example: is unbiased always better?

For normal data, compare \(S^2\) with \(\hat\sigma^2=\frac{n-1}{n}S^2\). Using Theorems 1.1 and 1.3, \(\mathrm{MSE}(S^2)\)\({}=\frac{2\sigma^4}{n-1}\), while \(\mathrm{MSE}(\hat\sigma^2)\)\({}=\big(\tfrac{n-1}{n}\big)^2\frac{2\sigma^4}{n-1}+\frac{\sigma^4}{n^2}\)\({}=\frac{(2n-1)\sigma^4}{n^2}\), which is smaller for every \(n\ge2\) (since \((2n-1)(n-1)\lt 2n^2\)). A little bias can buy a larger reduction in variance. So \(S^2\) is not chosen for its MSE. It is a convention: \(S^2\) is unbiased, and it gives the pivots of Section 3 their standard \(t_{n-1}\) and \(\chi^2_{n-1}\) forms. (Using \(\hat\sigma^2\) instead only rescales those pivots and gives exactly the same intervals.)

▶ Demo: Estimators on trial At \(n=5\), dividing by \(n\) lands 20% low on average, yet its MSE is the smaller one; at \(n=100\) the difference fades. Mean against median shows Theorem 1.3(a).

2. Finding estimators

2.1 Method of moments

The \(k\)-th population moment \(E_\theta(X^k)\) is a function of \(\theta\); the \(k\)-th sample moment \(\frac1n\sum X_i^k\) estimates it consistently by the law of large numbers (Theorem 0.5, when \(E X^{2k}\lt\infty\)). The method of moments sets as many sample moments equal to population moments as there are parameters, and solves.

Example 2.1

For \(\mathcal N(\mu,\sigma^2)\): \(E X=\mu\), \(E X^2=\sigma^2+\mu^2\). Solving \(\bar x=\mu\), \(\frac1n\sum x_i^2=\sigma^2+\mu^2\) gives \(\hat\mu=\bar x\) and \(\hat\sigma^2\)\({}=\frac1n\sum x_i^2-\bar x^2\)\({}=\frac1n\sum(x_i-\bar x)^2\).

2.2 Maximum likelihood

Definition

For observed data \(\mathbf x\), the likelihood is \(L(\theta)\)\({}=\prod_{i=1}^n f_\theta(x_i)\) (density or mass function, by independence). A maximum likelihood estimate (MLE) is a value \(\hat\theta\) that maximises \(L\), equivalently the log-likelihood \(\ell=\ln L\) (since \(\ln\) is strictly increasing).

Theorem 2.2 (MLE for a proportion)

If \(X\sim\mathrm{Bin}(n,p)\) and \(x\) is observed, the MLE is \(\hat p=x/n\).

Proof

\(L(p)=\binom nx p^x(1-p)^{n-x}\). If \(0\lt x\lt n\), \(\ell(p)=c+x\ln p+(n-x)\ln(1-p)\) on \((0,1)\), with \(\ell'(p)=\frac xp-\frac{n-x}{1-p}\). Setting \(\ell'=0\) gives \(x(1-p)=(n-x)p\), i.e. \(p=x/n\). Since \(\ell''(p)\)\({}=-\frac{x}{p^2}-\frac{n-x}{(1-p)^2}\)\({}\lt0\) everywhere, \(\ell\) is strictly concave and the stationary point is the global maximum on \((0,1)\); at the endpoints \(L(0)=L(1)=0\), which is smaller. If \(x=0\), \(L(p)=(1-p)^n\) is decreasing, maximised at \(p=0=x/n\); if \(x=n\), \(L(p)=p^n\) is increasing, maximised at \(p=1=x/n\). ∎

Theorem 2.3 (MLE for the normal model)

For an iid \(\mathcal N(\mu,\sigma^2)\) sample with \(\sum(x_i-\bar x)^2\gt0\), the MLE is \(\hat\mu=\bar x\), \(\hat\sigma^2\)\({}=\frac1n\sum(x_i-\bar x)^2\).

Proof

\(\ell(\mu,\sigma^2)\)\({}=-\frac n2\ln(2\pi)-\frac n2\ln\sigma^2-\frac{1}{2\sigma^2}\sum(x_i-\mu)^2\). Step 1: for any fixed \(\sigma^2\), maximising over \(\mu\) means minimising \(g(\mu)=\sum(x_i-\mu)^2\); \(g'(\mu)=-2\sum(x_i-\mu)=0\) gives \(\mu=\bar x\) and \(g''=2n\gt0\), so \(\mu=\bar x\) regardless of \(\sigma^2\). Step 2: with \(D=\sum(x_i-\bar x)^2\) and \(v=\sigma^2\), maximise \(h(v)=-\frac n2\ln v-\frac{D}{2v}\). Then \(h'(v)=-\frac{n}{2v}+\frac{D}{2v^2}\), which is positive for \(v\lt D/n\) and negative for \(v\gt D/n\), so \(v=D/n\) is the global maximum. ∎

So the MLE of \(\sigma^2\) is the biased \(\hat\sigma^2\). Maximum likelihood is a general recipe, not a guarantee of unbiasedness; its strength is large-sample behaviour (consistency and asymptotic efficiency, proved in later courses).

▶ Demo: Maximum likelihood: climbing the likelihood Draw samples one at a time: the top of \(\ell(p)\) always sits at \(x/n\) (\(n=10\), \(n=100\)), and the curvature at the top gives the standard error.

2.3 Going further: how small can the variance be?

Theorem 2.4 (Cramér–Rao lower bound)

Let \(X_1,\dots,X_n\) be iid with density \(f_\theta\), and suppose differentiation in \(\theta\) may be taken inside the integrals below (F5) and the support does not depend on \(\theta\). Define the Fisher information \(I(\theta)\)\({}=E_\theta\big[\big(\tfrac{\partial}{\partial\theta}\ln f_\theta(X)\big)^2\big]\), and assume \(0\lt I(\theta)\lt\infty\). Then every unbiased estimator \(\hat\Theta\) of \(\theta\) satisfies

\[ \mathrm{Var}_\theta(\hat\Theta)\ \ge\ \frac{1}{n\,I(\theta)} . \]
Proof

Let \(U\)\({}=\frac{\partial}{\partial\theta}\ln\prod f_\theta(X_i)\)\({}=\sum_i \frac{\partial}{\partial\theta}\ln f_\theta(X_i)\), the score. For one observation, \(E\big[\frac{\partial}{\partial\theta}\ln f_\theta(X)\big]\)\({}=\int \frac{\partial_\theta f_\theta}{f_\theta}f_\theta\,dx\)\({}=\frac{d}{d\theta}\int f_\theta\,dx\)\({}=\frac{d}{d\theta}1\)\({}=0\). Hence \(E(U)=0\) and, by independence, \(\mathrm{Var}(U)=nI(\theta)\).

Unbiasedness says \(\int g(\mathbf x)\prod f_\theta(x_i)\,d\mathbf x\)\({}=\theta\). Differentiating, \(\int g(\mathbf x)\,U(\mathbf x)\prod f_\theta(x_i)\,d\mathbf x\)\({}=1\), i.e. \(E(\hat\Theta U)=1\), so \(\mathrm{Cov}(\hat\Theta,U)=1\) because \(E U=0\). The Cauchy–Schwarz inequality \(\mathrm{Cov}(\hat\Theta,U)^2\)\({}\le\mathrm{Var}(\hat\Theta)\mathrm{Var}(U)\) gives \(1\)\({}\le\mathrm{Var}(\hat\Theta)\,nI(\theta)\). ∎

For \(\mathcal N(\mu,\sigma^2)\) with \(\sigma\) known, \(\frac{\partial}{\partial\mu}\ln f\)\({}=\frac{x-\mu}{\sigma^2}\), so \(I(\mu)=1/\sigma^2\) and the bound is \(\sigma^2/n=\mathrm{Var}(\bar X)\). No unbiased estimator of a normal mean beats \(\bar X\). So the sample median, which is also unbiased for normal data (by symmetry), cannot have a smaller variance than \(\bar X\); in fact its variance is asymptotically \(\pi/2\approx1.57\) times larger, a result proved in later courses. The estimators demonstration shows the gap.

▶ Demo: Maximum likelihood and the Cramér–Rao bound The normal mean attains the bound exactly; the exponential rate misses it at \(n=5\); the uniform maximum beats the \(1/n\) variance rate because the theorem does not apply.

3. Confidence intervals

Definition 3.1

Let \(T_1\)\({}=g_1(\mathbf X)\)\({}\le T_2\)\({}=g_2(\mathbf X)\) be statistics. The random interval \((T_1,T_2)\) is a \(100(1-\alpha)\%\) confidence interval for \(\theta\) if \(P_\theta(T_1\lt\theta\lt T_2)\)\({}\ge1-\alpha\) for every \(\theta\in\Theta\). It is exact if equality holds. An interval whose coverage probability only tends to \(1-\alpha\) as \(n\to\infty\) is called an approximate \(100(1-\alpha)\%\) interval.

Theorem 3.2 (the frequency meaning of "95%")

Repeat a study independently \(m\) times with an exact \(100(1-\alpha)\%\) interval, and let \(C_m\) be the number of intervals that contain \(\theta\). Then \(C_m\sim\mathrm{Bin}(m,1-\alpha)\), and \(C_m/m\to1-\alpha\) in probability.

Proof

Let \(I_j\) indicate that interval \(j\) covers \(\theta\). The studies are independent, so the \(I_j\) are iid Bernoulli with success probability \(P_\theta(T_1\lt\theta\lt T_2)\)\({}=1-\alpha\). Their sum is binomial by definition, and the law of large numbers (Theorem 0.5) gives the limit. ∎

For example, 100 independent 95% intervals cover the truth on average 95 times with standard deviation \(\sqrt{100\times0.95\times0.05}\)\({}=2.18\); seeing 93 or fewer has probability 0.23, so a run of 93 in the confidence-interval demonstration is unremarkable. Note what the theorem does not say: once the data are observed, \((t_1,t_2)\) is a fixed interval and \(\theta\) is a fixed number, so the statement \(t_1\lt\theta\lt t_2\) is simply true or false: its "probability" is 0 or 1 (we do not know which), never 0.95.

Theorem 3.3 (the pivot method)

Suppose \(Q=q(\mathbf X,\theta)\) has a distribution that does not depend on \(\theta\) (a pivot), with \(P(a\lt Q\lt b)=1-\alpha\). If, for every data set, \(\{\theta: a\)\({}\lt q(\mathbf x,\theta)\)\({}\lt b\}\) is an interval \((t_1(\mathbf x),t_2(\mathbf x))\), then \((T_1,T_2)\) is an exact \(100(1-\alpha)\%\) confidence interval.

Proof

The events \(\{a\lt q(\mathbf X,\theta)\lt b\}\) and \(\{T_1\lt\theta\lt T_2\}\) are the same event by construction, so they have the same probability, \(1-\alpha\), whatever the value of \(\theta\). ∎

3.1 The mean of normal data

Theorem 3.4

For an iid \(\mathcal N(\mu,\sigma^2)\) sample, with \(t_{\alpha/2,n-1}\) the upper \(\alpha/2\) quantile of \(t_{n-1}\):

\[ \text{(a) } \sigma \text{ known: } \bar X\pm z_{\alpha/2}\frac{\sigma}{\sqrt n} \]\[ \text{(b) } \sigma \text{ unknown: } \bar X\pm t_{\alpha/2,n-1}\frac{S}{\sqrt n} \]

are exact \(100(1-\alpha)\%\) confidence intervals for \(\mu\).

Proof

(b) By Theorem 0.9(iv), \(Q\)\({}=\frac{\bar X-\mu}{S/\sqrt n}\sim t_{n-1}\) whatever \(\mu,\sigma\): a pivot. By symmetry (Theorem 0.8c), \(P(-t_{\alpha/2,n-1}\lt Q\lt t_{\alpha/2,n-1})\)\({}=1-\alpha\). Multiply the inequalities by \(S/\sqrt n\gt0\) and rearrange:

\[ -t\,\tfrac{S}{\sqrt n}\lt \bar X-\mu\lt t\,\tfrac{S}{\sqrt n}\iff \bar X-t\,\tfrac{S}{\sqrt n}\lt\mu\lt\bar X+t\,\tfrac{S}{\sqrt n}. \]

Apply Theorem 3.3. (a) is identical with the pivot \(\frac{\bar X-\mu}{\sigma/\sqrt n}\sim\mathcal N(0,1)\) from Theorem 0.6. ∎

Example: body temperature, and what goes wrong with 1.96 when n is small

With \(n=130\), \(\bar x=36.81\), \(s=0.41\): \(t_{0.025,129}=1.979\), giving \(36.81\)\({}\pm1.979\times0.41/\sqrt{130}\)\({}=(36.74,\ 36.88)\) °C. If instead we used 1.96 with \(S\), the true coverage is \(P(|T|\lt1.96)\) for \(T\sim t_{n-1}\): 0.948 at \(n=130\), but only \(0.878\) at \(n=5\). The heavier tails of \(t_4\) are exactly the extra uncertainty from estimating \(\sigma\).

▶ Demo: Confidence intervals: does the method catch the truth? With \(n=5\) the normal multiplier catches the truth only about 88% of the time (try it); Student's t repairs it (try it). The reason: \(T\) has heavy tails.

3.2 The mean of non-normal data (large samples)

Theorem 3.5

For iid data with finite variance \(\sigma^2\gt0\) (any shape), \(\bar X\pm z_{\alpha/2}S/\sqrt n\) is an approximate \(100(1-\alpha)\%\) confidence interval for \(\mu\).

Proof

Write \(\frac{\bar X-\mu}{S/\sqrt n}\)\({}=\frac{\bar X-\mu}{\sigma/\sqrt n}\cdot\frac{\sigma}{S}\). The first factor tends to \(\mathcal N(0,1)\) by the CLT. For the second, \(S^2\)\({}=\frac{n}{n-1}\big(\frac1n\sum X_i^2-\bar X^2\big)\)\({}\to(\sigma^2+\mu^2)-\mu^2\)\({}=\sigma^2\) in probability, by the law of large numbers applied to \(X_i\) and to \(X_i^2\) (assuming \(E X^4\lt\infty\) so that Theorem 0.5 applies to \(X_i^2\); the result holds without it, using a stronger law of large numbers) and continuity (F4). Hence \(\sigma/S\to1\), and by Slutsky the ratio tends to \(\mathcal N(0,1)\). So \(P(-z_{\alpha/2}\lt\cdot\lt z_{\alpha/2})\)\({}\to1-\alpha\); rearrange as in Theorem 3.4. ∎

Sample size. The half-width ("margin of error") is \(E=z_{\alpha/2}\sigma/\sqrt n\). Solving for \(n\) gives \(n=(z_{\alpha/2}\sigma/E)^2\), rounded up; for \(\sigma\approx0.41\) and \(E=0.05\), \(n=258.3\to259\).

3.3 The variance of normal data

Theorem 3.6

Let \(q_1,q_2\) be the lower and upper \(\alpha/2\) quantiles of \(\chi^2_{n-1}\). For normal data, \(\Big(\dfrac{(n-1)S^2}{q_2},\ \dfrac{(n-1)S^2}{q_1}\Big)\) is an exact \(100(1-\alpha)\%\) confidence interval for \(\sigma^2\); square roots of the endpoints give one for \(\sigma\).

Proof

\(Q\)\({}=(n-1)S^2/\sigma^2\sim\chi^2_{n-1}\) by Theorem 0.9(ii), a pivot, and \(P(q_1\lt Q\lt q_2)=1-\alpha\). Since all quantities are positive, \(q_1\)\({}\lt\frac{(n-1)S^2}{\sigma^2}\)\({}\lt q_2\)\({}\iff\frac{(n-1)S^2}{q_2}\)\({}\lt\sigma^2\)\({}\lt\frac{(n-1)S^2}{q_1}\). The square root is strictly increasing, so the event for \(\sigma\) is the same event. ∎

Body temperature: \(q_1=99.45\), \(q_2=162.33\) on 129 df give \((0.134,\,0.218)\) for \(\sigma^2\) and \((0.37,\,0.47)\) °C for \(\sigma\). Unlike Theorem 3.5, there is no CLT rescue here: the result depends on normality even for large \(n\), because \(\mathrm{Var}(S^2)\) depends on the fourth moment of the data.

▶ Demo: Why Student's t? Skewed data with \(n=30\): the t interval still covers about 93% (approximately right, as Theorem 3.5 promises), but the variance interval covers only about 72% (Theorem 3.6 needs normality).

3.4 Two independent normal samples

Theorem 3.7 (pooled interval, equal variances)

Let \(X_1,\dots,X_{n_1}\) iid \(\mathcal N(\mu_X,\sigma^2)\) and \(Y_1,\dots,Y_{n_2}\) iid \(\mathcal N(\mu_Y,\sigma^2)\) be independent samples with a common \(\sigma^2\), and \(S_p^2\)\({}=\frac{(n_1-1)S_X^2+(n_2-1)S_Y^2}{n_1+n_2-2}\). Then

\[ \frac{(\bar X-\bar Y)-(\mu_X-\mu_Y)}{S_p\sqrt{1/n_1+1/n_2}}\sim t_{n_1+n_2-2}, \]

so \(\bar X-\bar Y\)\({}\pm t_{\alpha/2,n_1+n_2-2}\,S_p\sqrt{1/n_1+1/n_2}\) is an exact \(100(1-\alpha)\%\) interval for \(\mu_X-\mu_Y\).

Proof

Numerator. \(\bar X-\bar Y\) is a linear combination of independent normals, so by Theorem 0.6 it is \(\mathcal N\big(\mu_X-\mu_Y,\ \sigma^2(1/n_1+1/n_2)\big)\); standardising gives \(Z\sim\mathcal N(0,1)\).

Denominator. By Theorem 0.9(ii) applied to each sample, \((n_1-1)S_X^2/\sigma^2\sim\chi^2_{n_1-1}\) and \((n_2-1)S_Y^2/\sigma^2\sim\chi^2_{n_2-1}\), independent because the samples are. By Theorem 0.8(b), \(V\)\({}=(n_1+n_2-2)S_p^2/\sigma^2\sim\chi^2_{n_1+n_2-2}\).

Independence. \(\bar X\) is independent of \(S_X^2\) (Theorem 0.9iii) and of the whole \(Y\)-sample; likewise for \(\bar Y\). So \(Z\) (a function of \(\bar X,\bar Y\)) is independent of \(V\) (a function of \(S_X^2,S_Y^2\)).

The ratio equals \(Z/\sqrt{V/(n_1+n_2-2)}\), since the unknown \(\sigma\) cancels: \(t_{n_1+n_2-2}\) by definition. The interval follows as in Theorem 3.4. ∎

Theorem 3.8 (Welch interval, unequal variances: approximate)

Without assuming \(\sigma_X=\sigma_Y\), use \(\bar X-\bar Y\)\({}\pm t_{\alpha/2,\nu}\sqrt{S_X^2/n_1+S_Y^2/n_2}\) with

\[ \nu=\frac{\big(S_X^2/n_1+S_Y^2/n_2\big)^2}{\dfrac{(S_X^2/n_1)^2}{n_1-1}+\dfrac{(S_Y^2/n_2)^2}{n_2-1}} \]\[ \min(n_1,n_2)-1\ \le\ \nu\ \le\ n_1+n_2-2 . \]
Derivation (Satterthwaite's moment matching)

No exact pivot is known here (this is the Behrens–Fisher problem), so we approximate. Let \(W=S_X^2/n_1+S_Y^2/n_2\), the estimated variance of \(\bar X-\bar Y\). If \(W\) behaved like \(E(W)\cdot\chi^2_\nu/\nu\) for some \(\nu\), the studentised difference would be exactly \(t_\nu\) by the argument of Theorem 3.7. Choose \(\nu\) so that this model has the right mean and variance. The model has \(\mathrm{Var}(W)\)\({}=E(W)^2\cdot 2\nu/\nu^2\)\({}=2E(W)^2/\nu\), so \(\nu=2E(W)^2/\mathrm{Var}(W)\).

By Theorem 1.3, \(E(W)=\sigma_X^2/n_1+\sigma_Y^2/n_2\), and by Theorem 1.3(d) and independence, \(\mathrm{Var}(W)\)\({}=\frac{2\sigma_X^4}{n_1^2(n_1-1)}+\frac{2\sigma_Y^4}{n_2^2(n_2-1)}\). Substituting and replacing the unknown variances by their estimates gives the formula for \(\nu\).

Bounds. Put \(a=S_X^2/n_1\), \(b=S_Y^2/n_2\), \(k_1=n_1-1\), \(k_2=n_2-1\), so \(\nu=(a+b)^2/(a^2/k_1+b^2/k_2)\). Upper: by Cauchy–Schwarz, \((a+b)^2\)\({}=\big(\sqrt{k_1}\tfrac{a}{\sqrt{k_1}}+\sqrt{k_2}\tfrac{b}{\sqrt{k_2}}\big)^2\)\({}\le(k_1+k_2)\big(\tfrac{a^2}{k_1}+\tfrac{b^2}{k_2}\big)\). Lower: with \(k=\min(k_1,k_2)\), \(\tfrac{a^2}{k_1}+\tfrac{b^2}{k_2}\)\({}\le\tfrac{a^2+b^2}{k}\)\({}\le\tfrac{(a+b)^2}{k}\). ∎

The bounds explain two rules of thumb: using \(\min(n_1,n_2)-1\) degrees of freedom by hand is conservative (wider interval), and Welch can never claim more degrees of freedom than the pooled method. Women versus men in the temperature data: difference \(0.16\), SE \(0.070\), \(\nu\approx127.7\) from these rounded summaries (127.5 from the full data), interval \((0.02,\ 0.30)\) °C.

▶ Demo: Two samples: pooled or Welch? With \(H_0\) true, the pooled test rejects about 21% of the time when the smaller group has three times the spread, and under 1% when the larger group does; Welch stays close to 5%.

3.5 Proportions

Theorem 3.9 (Wald intervals)

If \(X\sim\mathrm{Bin}(n,p)\) with \(0\lt p\lt1\), then \(\hat P\)\({}\pm z_{\alpha/2}\sqrt{\hat P(1-\hat P)/n}\) is an approximate \(100(1-\alpha)\%\) interval for \(p\). For independent \(X\sim\mathrm{Bin}(n_1,p_X)\), \(Y\sim\mathrm{Bin}(n_2,p_Y)\), \(\hat P_X-\hat P_Y\)\({}\pm z_{\alpha/2}\sqrt{\hat P_X(1-\hat P_X)/n_1+\hat P_Y(1-\hat P_Y)/n_2}\) is an approximate interval for \(p_X-p_Y\).

Proof

\(\hat P\) is the mean of \(n\) iid Bernoulli(\(p\)) variables (Theorem 1.3c), so by the CLT \(\frac{\hat P-p}{\sqrt{p(1-p)/n}}\)\({}\to\mathcal N(0,1)\). By the law of large numbers \(\hat P\to p\), so \(\sqrt{p(1-p)}/\sqrt{\hat P(1-\hat P)}\)\({}\to1\) by continuity (F4), and Slutsky lets us replace \(p\) by \(\hat P\) in the standard error. Rearranging gives the interval. For two samples, \(\hat P_X-\hat P_Y\) has variance \(p_X(1-p_X)/n_1+p_Y(1-p_Y)/n_2\) by independence; applying the CLT to each sample and combining the two independent normal limits gives an approximately normal difference, and the same Slutsky step applies. ∎

The approximation is reasonable when there are at least about 5 successes and 5 failures in each sample; for smaller counts, exact binomial methods are preferred. Examples: 64 of 80 seeds gives \((0.71,\ 0.89)\); 45/150 versus 30/150 gives \((0.003,\ 0.197)\) for the difference.

▶ Demo: Counting successes The exact coverage of the 95% Wald interval with \(n=20\) swings between about 81% and 96% for \(0.1\le p\le0.9\) and collapses near 0 and 1; with \(n=100\) it settles. The approximation behind it: averages of yes/no data.

4. The logic of tests

Definitions

A hypothesis test of \(H_0:\theta\in\Theta_0\) against \(H_1:\theta\in\Theta_1\) is a rule that rejects \(H_0\) when the data fall in a rejection region \(R\). Its power function is \(\pi(\theta)\)\({}=P_\theta(\mathbf X\in R)\). The test has level \(\alpha\) if \(\pi(\theta)\le\alpha\) for all \(\theta\in\Theta_0\); its size is \(\sup_{\theta\in\Theta_0}\pi(\theta)\). A Type I error is rejecting a true \(H_0\) (probability \(\pi(\theta)\), \(\theta\in\Theta_0\)); a Type II error is not rejecting a false \(H_0\) (probability \(\beta(\theta)=1-\pi(\theta)\), \(\theta\in\Theta_1\)).

If the test rejects for large values of a statistic \(T\), the P-value of observed data is \(p(\mathbf x)\)\({}=\sup_{\theta\in\Theta_0}P_\theta\big(T(\mathbf X)\ge T(\mathbf x)\big)\): the largest probability, under \(H_0\), of a result at least as extreme as the one observed. (For left-sided tests reverse the inequality; for a two-sided test based on a symmetric \(T\), use \(|T|\), which gives \(2P(T\ge|t|)\).)

Theorem 4.1 (P-values control the Type I error)

Let \(H_0:\theta=\theta_0\) be simple. (a) If \(T\) has a continuous CDF under \(H_0\), then \(p(\mathbf X)\sim\mathrm{Uniform}(0,1)\) under \(H_0\), so the rule "reject if \(p\le\alpha\)" has size exactly \(\alpha\). (b) In general (for example a discrete \(T\)), \(P_{\theta_0}(p(\mathbf X)\le\alpha)\)\({}\le\alpha\) for every \(\alpha\in(0,1)\).

Proof

Let \(G(t)=P_{\theta_0}(T\ge t)\), a non-increasing function, so that \(p(\mathbf X)=G(T)\).

(b) Fix \(\alpha\), and let \(c=\inf\{t: G(t)\le\alpha\}\). Because \(G\) is non-increasing, \(\{G(T)\le\alpha\}\) is either \(\{T\ge c\}\) or \(\{T\gt c\}\). In the first case \(G(c)\le\alpha\) (the infimum is attained), so \(P(T\ge c)=G(c)\le\alpha\). In the second case, the events \(\{T\ge t\}\) increase to \(\{T\gt c\}\) as \(t\downarrow c\), so \(P(T\gt c)\)\({}=\lim_{t\downarrow c}G(t)\) (continuity of \(P\) along increasing events, a consequence of (A3)), and this limit is \(\le\alpha\) because \(G(t)\le\alpha\) for every \(t\gt c\). Either way \(P_{\theta_0}(p\le\alpha)\le\alpha\).

(a) If the CDF of \(T\) is continuous, \(G\) is continuous, so by the intermediate value theorem there is a \(c\) with \(G(c)=\alpha\), and then \(P(T\ge c)=\alpha\) exactly and \(\{G(T)\le\alpha\}=\{T\ge c\}\) up to an event of probability zero. Hence \(P(p\le\alpha)=\alpha\) for every \(\alpha\): the CDF of \(p(\mathbf X)\) is that of a Uniform(0,1) variable. ∎

This is why, in the hypothesis-testing demonstration, the histogram of P-values is flat when \(H_0\) is true (and piles up near 0 when it is false). It also shows what a P-value is not: it is a random variable computed assuming \(H_0\), not the probability that \(H_0\) is true.

▶ Demo: Counting successes Part (b) in action: a discrete statistic cannot use its full \(\alpha\). The exact binomial test for 18 tasters has size 0.048, and with 10 tasters only 0.011.

Theorem 4.2 (tests and confidence intervals are two views of one object)

(a) If \(C(\mathbf X)\) is a \(100(1-\alpha)\%\) confidence set for \(\theta\), then for each \(\theta_0\) the test "reject \(H_0:\theta=\theta_0\) iff \(\theta_0\notin C(\mathbf X)\)" has level \(\alpha\). (b) Conversely, if for each \(\theta_0\) a level-\(\alpha\) test has acceptance region \(A(\theta_0)\), then \(C(\mathbf x)\)\({}=\{\theta_0:\mathbf x\in A(\theta_0)\}\) is a \(100(1-\alpha)\%\) confidence set.

Proof

(a) \(P_{\theta_0}(\text{reject})\)\({}=P_{\theta_0}(\theta_0\notin C(\mathbf X))\)\({}=1-P_{\theta_0}(\theta_0\in C(\mathbf X))\)\({}\le\alpha\). (b) For any true \(\theta\), \(\theta\in C(\mathbf X)\)\({}\iff\mathbf X\in A(\theta)\), whose probability is \(1-P_\theta(\text{reject})\)\({}\ge1-\alpha\). ∎

For the \(t\) procedures: \(|\bar x-\mu_0|\)\({}\lt t_{\alpha/2,n-1}s/\sqrt n\) is the statement "\(\mu_0\) is inside the t interval" and also "\(|t|\lt t_{\alpha/2,n-1}\)". So 37 °C lying outside \((36.74,36.88)\) is a rejection at the 5% level.

▶ Demo: Tests and intervals: two views of one object Drag \(\mu_0\) across the interval: the P-value is above \(\alpha\) exactly inside it, and equals \(\alpha\) at its endpoints.

5. The one-sample t-test and power

Theorem 5.1 (exact size)

For iid normal data and \(T=\frac{\bar X-\mu_0}{S/\sqrt n}\), the tests "reject \(H_0:\mu=\mu_0\) if \(|T|\gt t_{\alpha/2,n-1}\)" (two-sided), "if \(T\gt t_{\alpha,n-1}\)" (\(H_1:\mu\gt\mu_0\)) and "if \(T\lt-t_{\alpha,n-1}\)" (\(H_1:\mu\lt\mu_0\)) all have size exactly \(\alpha\), and each is equivalent to "P-value \(\le\alpha\)" (up to an event of probability 0).

Proof

Under \(H_0\), \(T\sim t_{n-1}\) (Theorem 0.9iv with \(\mu=\mu_0\)). By the definition of the quantiles and symmetry, \(P(|T|\gt t_{\alpha/2,n-1})\)\({}=2\cdot\frac{\alpha}{2}\)\({}=\alpha\), and similarly for the one-sided rules. The equivalence with the P-value is Theorem 4.1(a): \(T\) has a continuous distribution, and \(G(t)\)\({}\le\alpha\)\({}\iff t\)\({}\ge t_{\alpha,n-1}\) since \(G\) is strictly decreasing. The rule "\(T\gt t_{\alpha,n-1}\)" differs from this only when \(T\) equals the critical value exactly, an event of probability 0. ∎

Body temperature: \(t\)\({}=\frac{36.81-37}{0.41/\sqrt{130}}\)\({}=-5.28\), two-sided \(p\approx5\times10^{-7}\) from the rounded summaries (the full data give \(t=-5.46\), \(p\approx2\times10^{-7}\)).

Theorem 5.2 (power and sample size, \(\sigma\) known)

For the one-sided \(z\)-test that rejects \(H_0:\mu=\mu_0\) in favour of \(H_1:\mu\gt\mu_0\) when \(Z\)\({}=\frac{\bar X-\mu_0}{\sigma/\sqrt n}\)\({}\gt z_\alpha\), with standardised effect \(d=(\mu-\mu_0)/\sigma\), the power is

\[ \pi(\mu)=\Phi\big(\sqrt n\,d-z_\alpha\big), \]

and, when \(d\gt0\), power at least \(1-\beta\) requires

\[ n\ \ge\ \Big(\frac{z_\alpha+z_\beta}{d}\Big)^2 . \]
Proof

Under the true mean \(\mu\), \(Z\)\({}=\frac{\bar X-\mu}{\sigma/\sqrt n}+\frac{\mu-\mu_0}{\sigma/\sqrt n}\)\({}=Z_0+\sqrt n\,d\) with \(Z_0\sim\mathcal N(0,1)\) (Theorem 0.6). So \(\pi(\mu)\)\({}=P(Z_0\gt z_\alpha-\sqrt n d)\)\({}=1-\Phi(z_\alpha-\sqrt n d)\)\({}=\Phi(\sqrt n d-z_\alpha)\), using the symmetry \(1-\Phi(x)=\Phi(-x)\). Power at least \(1-\beta=\Phi(z_\beta)\) means \(\sqrt n d-z_\alpha\ge z_\beta\), because \(\Phi\) is increasing; divide by \(d\gt0\) and square. ∎

Consequences, all visible in the formula: power equals \(\alpha\) at \(d=0\), increases with \(d\), increases with \(n\) when \(d\gt0\), and decreases as \(\alpha\) is made smaller. For \(d=0.5\), \(\alpha=0.05\), 80% power: \(n\)\({}\ge\big((1.645+0.842)/0.5\big)^2\)\({}=24.7\), so 25 if \(\sigma\) were known. With \(\sigma\) estimated, the exact power of the \(t\)-test uses the non-central \(t\) distribution \(\frac{Z_0+\sqrt n d}{\sqrt{V/(n-1)}}\), and the answer rises to \(n=27\) (R's power.t.test gives 26.1). The price of not knowing \(\sigma\) is two observations.

▶ Demo: Power and sample size: two bell curves \(d=0.5\), \(n=25\) gives about 80%; halving \(d\) needs four times the people; \(\alpha=0.01\) costs power. The same power appears as a rejection rate in thousands of simulated t-tests.

6. Going further: why these tests are optimal

Theorem 6.1 (Neyman–Pearson lemma)

To test a simple \(H_0:f=f_0\) against a simple \(H_1:f=f_1\), consider the likelihood-ratio test that rejects when \(f_1(\mathbf x)\gt k f_0(\mathbf x)\), with \(k\ge0\) chosen so that its size is \(\alpha\). Every other test of level \(\alpha\) has power no greater than this one.

Proof

Describe a test by \(\phi(\mathbf x)\in\{0,1\}\) (1 = reject). Let \(\phi^*\) be the likelihood-ratio test and \(\phi\) any test with \(E_0\phi\le\alpha=E_0\phi^*\). For every \(\mathbf x\),

\[ \big(\phi^*(\mathbf x)-\phi(\mathbf x)\big)\big(f_1(\mathbf x)-kf_0(\mathbf x)\big)\ \ge\ 0, \]

because when \(f_1\gt kf_0\), \(\phi^*=1\ge\phi\), and when \(f_1\le kf_0\), \(\phi^*=0\le\phi\). Integrate over \(\mathbf x\):

\[ E_1\phi^*-E_1\phi\ \ge\ k\,\big(E_0\phi^*-E_0\phi\big)\ \ge\ 0 . \]

So the power of \(\phi^*\) is at least the power of \(\phi\). ∎

Application: the one-sided z-test is the best possible

For iid \(\mathcal N(\mu,\sigma^2)\) with \(\sigma\) known, test \(\mu=\mu_0\) against a single \(\mu_1\gt\mu_0\). The likelihood ratio is

\[ \frac{f_1(\mathbf x)}{f_0(\mathbf x)}=\exp\!\Big(\frac{\mu_1-\mu_0}{\sigma^2}\sum x_i-\frac{n(\mu_1^2-\mu_0^2)}{2\sigma^2}\Big), \]

an increasing function of \(\bar x\). So "ratio \(\gt k\)" is "\(\bar x\gt c\)", and size \(\alpha\) forces \(c=\mu_0+z_\alpha\sigma/\sqrt n\): the z-test. The rejection region does not depend on which \(\mu_1\gt\mu_0\) we picked, so the z-test is simultaneously most powerful against every \(\mu_1\gt\mu_0\): it is uniformly most powerful.

▶ Demo: Power and sample size The teal power curve of the one-sided z-test is the highest possible at every \(d\gt0\): no test with the same false-alarm rate can lift it.

7. Tests for one proportion

Theorem 7.1

Let \(X\sim\mathrm{Bin}(n,p)\), \(H_0:p=p_0\), \(H_1:p\gt p_0\). (a) The exact test with P-value \(P_{p_0}(X\ge x)\) has level \(\alpha\) (usually size strictly below \(\alpha\)). (b) The approximate test based on \(Z\)\({}=\frac{\hat P-p_0}{\sqrt{p_0(1-p_0)/n}}\) has size tending to \(\alpha\) as \(n\to\infty\).

Proof

(a) Theorem 4.1(b) with \(T=X\). The size is \(P_{p_0}(X\ge c)\) for the cut-off \(c\) found there, and because \(X\) takes only the values \(0,\dots,n\), this usually cannot equal \(\alpha\) exactly. (b) Under \(H_0\), \(Z\to\mathcal N(0,1)\) by the CLT (Theorem 1.3c gives the mean and variance of \(\hat P\)). No Slutsky step is needed: under \(H_0\) the standard error is known, which is why the test uses \(p_0\) rather than \(\hat P\). ∎

Example: a discrete test cannot hit 5% exactly

14 of 18 tasters prefer a new recipe; \(H_0:p=0.5\). Exact: \(P(X\ge14)=0.015\). Approximate: \(z=2.36\), \(p=0.009\). At level 0.05 the exact rejection region is \(X\ge13\), whose size is \(P(X\ge13)=0.048\), because \(P(X\ge12)=0.119\) is too large: no region of the form \(X\ge c\) has size exactly 0.05. Similarly, the seed rule "reject \(p=0.7\) if \(X\le10\) out of 20" has size \(0.048\), and power \(0.59\) at \(p=0.5\) and \(0.87\) at \(p=0.4\).

▶ Demo: Counting successes: exact and approximate methods The taste test as above; with 10 tasters the exact test can use only 1.1% while the z-test oversteps to 5.5%; for rare events the z-test's size is 7.3%.

8. Two samples: pooled, Welch and paired

By the duality theorem (4.2), each interval in Section 3.4 gives a test of \(H_0:\mu_X=\mu_Y\): the pooled statistic \(\frac{\bar X-\bar Y}{S_p\sqrt{1/n_1+1/n_2}}\) is exactly \(t_{n_1+n_2-2}\) under \(H_0\) when the variances are equal (Theorem 3.7), and the Welch statistic is approximately \(t_\nu\) without that assumption (Theorem 3.8).

Proposition 8.1 (equal sample sizes)

If \(n_1=n_2=n\), the pooled and Welch statistics are identical; they differ only in their degrees of freedom.

Proof

With \(n_1=n_2=n\), \(S_p^2=\frac{S_X^2+S_Y^2}{2}\), so \(S_p^2\big(\frac1n+\frac1n\big)\)\({}=\frac{S_X^2+S_Y^2}{n}\)\({}=\frac{S_X^2}{n}+\frac{S_Y^2}{n}\). The denominators agree. ∎

Women versus men (65 each): both statistics equal \(t=2.28\), on 127.7 and 128 df, \(p=0.024\). When the sample sizes are very unequal and the variances differ, the pooled test's size can be far from \(\alpha\), which is why Welch is the safer default (Welch, 1947).

▶ Demo: Two samples: pooled or Welch? Equal sizes make the two statistics identical (try it); unequal sizes with unequal spreads make the pooled test fail (try it).

Theorem 8.2 (why pairing helps)

Let \((X_i,Y_i)\) be measured on the same subject, \(D_i=X_i-Y_i\), with \(\mathrm{Var}(X)=\sigma_X^2\), \(\mathrm{Var}(Y)=\sigma_Y^2\) and correlation \(\rho\). Then \(\mathrm{Var}(D)\)\({}=\sigma_X^2+\sigma_Y^2-2\rho\sigma_X\sigma_Y\). If the \(D_i\) are iid normal, \(\frac{\bar D-\mu_D}{S_D/\sqrt n}\sim t_{n-1}\), and the paired \(t\)-test is the one-sample test applied to the \(D_i\).

Proof

Theorem 0.2 with \(Y\) replaced by \(-Y\): \(\mathrm{Var}(X-Y)\)\({}=\mathrm{Var}(X)+\mathrm{Var}(Y)-2\mathrm{Cov}(X,Y)\), and \(\mathrm{Cov}(X,Y)\)\({}=\rho\sigma_X\sigma_Y\) by the definition of correlation. The distributional statement is Theorem 0.9(iv) for the sample \(D_1,\dots,D_n\). ∎

Example: sleep before and after a routine change (8 people)

The sample standard deviations are 0.493 h (before) and 0.512 h (after), with correlation \(r=0.85\) between a person's two nights. The variance of the differences is \(0.076\), against \(0.493^2+0.512^2\approx0.505\) if the samples were independent: pairing removes about 85% of the noise. Paired test: mean difference \(0.425\), \(t=4.35\) on 7 df, \(p=0.0034\). Ignoring the pairing (Welch): \(t=1.69\), \(p=0.11\).

▶ Demo: Two samples: paired or not? With correlation 0.85 and 8 people the paired test has about 96% power, against about 28% when the pairing is ignored; with no correlation pairing gains nothing.

9. Two proportions

Theorem 9.1 (pooled z-test)

For independent \(X\sim\mathrm{Bin}(n_1,p_X)\), \(Y\sim\mathrm{Bin}(n_2,p_Y)\) and \(H_0:p_X=p_Y\), let \(\hat P=\frac{X+Y}{n_1+n_2}\). Under \(H_0\), as \(n_1,n_2\to\infty\),

\[ Z=\frac{\hat P_X-\hat P_Y}{\sqrt{\hat P(1-\hat P)\big(\frac1{n_1}+\frac1{n_2}\big)}}\ \longrightarrow\ \mathcal N(0,1). \]
Proof

Under \(H_0\) both samples share one \(p\). By Theorem 1.3(c) and independence, \(\hat P_X-\hat P_Y\) has mean 0 and variance \(p(1-p)\big(\frac1{n_1}+\frac1{n_2}\big)\), and the CLT (applied to each sample) makes its standardised version approximately \(\mathcal N(0,1)\). Under \(H_0\), \(X+Y\sim\mathrm{Bin}(n_1+n_2,p)\), so \(\hat P\to p\) in probability, and Slutsky lets us replace \(p\) by \(\hat P\). Using all \(n_1+n_2\) observations to estimate the common \(p\) is what the null hypothesis permits; the confidence interval (Theorem 3.9) cannot assume \(p_X=p_Y\), so it uses the separate estimates. ∎

Reminder emails: 45/150 versus 30/150, \(\hat p=0.25\), \(z=2.00\), one-sided \(p=0.023\).

10. Exercises

  1. \(S\) is biased. Show that \(E(S)\le\sigma\), with equality only if \(S\) is constant. (Hint: \(\mathrm{Var}(S)\ge0\).)

    Solution

    \(0\)\({}\le\mathrm{Var}(S)\)\({}=E(S^2)-E(S)^2\)\({}=\sigma^2-E(S)^2\), using \(E(S^2)=\sigma^2\) (Theorem 1.3b). So \(E(S)\le\sigma\), with equality iff \(\mathrm{Var}(S)=0\). For normal data with \(n=5\), \(E(S)\approx0.940\,\sigma\). Unbiasedness is not preserved by non-linear transformations.

  2. Two estimators for a uniform. Let \(X_i\) be iid Uniform\((0,\theta)\). Show that the method-of-moments estimator is \(2\bar X\) and the MLE is \(M=\max_i X_i\). Show \(\mathrm{MSE}(2\bar X)\)\({}=\frac{\theta^2}{3n}\) and \(\mathrm{MSE}(M)\)\({}=\frac{2\theta^2}{(n+1)(n+2)}\). Which is better?

    Solution

    \(E X=\theta/2\), so \(\bar x=\theta/2\) gives \(2\bar X\). \(L(\theta)=\theta^{-n}\) if \(\theta\ge\max x_i\) and 0 otherwise; it is decreasing on \([\max x_i,\infty)\), so \(\hat\theta=M\) (calculus fails here: the maximum is at a boundary). \(2\bar X\) is unbiased with variance \(4\cdot\frac{\theta^2/12}{n}\)\({}=\frac{\theta^2}{3n}\). \(P(M\le m)=(m/\theta)^n\), so \(M\) has density \(nm^{n-1}/\theta^n\), \(E M=\frac{n}{n+1}\theta\), \(E M^2=\frac{n}{n+2}\theta^2\); then \(\mathrm{Var}(M)\)\({}=\frac{n\theta^2}{(n+1)^2(n+2)}\), bias\(^2=\frac{\theta^2}{(n+1)^2}\), and Theorem 1.1 gives \(\frac{2\theta^2}{(n+1)(n+2)}\). For \(n\ge3\) this is smaller than \(\frac{\theta^2}{3n}\) (they are equal at \(n=1,2\)), and it shrinks like \(1/n^2\) rather than \(1/n\): the biased MLE wins. (Even the unbiased version \(\frac{n+1}{n}M\) has variance \(\frac{\theta^2}{n(n+2)}\), below the value \(\theta^2/n\) that Theorem 2.4 would give with \(I(\theta)=1/\theta^2\). There is no contradiction: the support \((0,\theta)\) depends on \(\theta\), so Theorem 2.4 does not apply.)

    ▶ Check it in a demo: the uniform maximum in the likelihood demonstration

  3. An exponential rate. For iid Exponential(\(\lambda\)) data (density \(\lambda e^{-\lambda x}\), mean \(1/\lambda\)), find the MLE of \(\lambda\). Given that \(\sum X_i\) has the Gamma(\(n,\lambda\)) distribution, for which \(E\big(1/\sum X_i\big)\)\({}=\lambda/(n-1)\) when \(n\ge2\), is the MLE unbiased? Correct it.

    Solution

    \(\ell(\lambda)\)\({}=n\ln\lambda-\lambda\sum x_i\), \(\ell'=n/\lambda-\sum x_i=0\) gives \(\hat\lambda=1/\bar x\), and \(\ell''=-n/\lambda^2\lt0\). \(E(1/\bar X)\)\({}=nE(1/\sum X_i)\)\({}=\frac{n}{n-1}\lambda\), biased upward. The estimator \(\frac{n-1}{n\bar X}\)\({}=\frac{n-1}{\sum X_i}\) is unbiased. (Simulation check: \(n=5\), \(\lambda=2\) gives an average of \(1/\bar X\) of about 2.50 = \(\frac54\cdot2\).)

    ▶ Check it in a demo: the exponential rate in the likelihood demonstration

  4. Two-sided sample size. Adapt Theorem 5.2 to the two-sided z-test (ignore the negligible probability of rejecting in the wrong direction). How many observations give 80% power at \(d=0.5\), \(\alpha=0.05\)?

    Solution

    Rejection on the correct side has probability \(\Phi(\sqrt n|d|-z_{\alpha/2})\), so \(n\)\({}\ge\big((z_{\alpha/2}+z_\beta)/d\big)^2\)\({}=\big((1.960+0.842)/0.5\big)^2\)\({}=31.4\): 32 observations with \(\sigma\) known (the exact \(t\)-test needs 34).

    ▶ Check it in a demo: the two-sided test in the power demonstration

  5. Seven steps, filled in. A manufacturer claims a mean battery life of 10 h; \(n=25\), \(\bar x=9.6\), \(s=0.9\). Test against \(\mu\lt10\), stating which theorem justifies each step.

    Solution

    Model: iid \(\mathcal N(\mu,\sigma^2)\). Hypotheses: \(H_0:\mu=10\), \(H_1:\mu\lt10\). Statistic: \(T=\frac{\bar X-10}{S/5}\), which is \(t_{24}\) under \(H_0\) by Theorem 0.9(iv). Observed \(t=-0.4/0.18=-2.22\); left-sided P-value \(P(T_{24}\le-2.22)=0.018\). By Theorems 4.1 and 5.1, rejecting when \(p\le0.05\) has size 0.05, so reject \(H_0\): moderate evidence that the mean life is below 10 h. The 95% interval \((9.23,\,9.97)\) excludes 10, consistent with Theorem 4.2 (whose two-sided test also rejects).

    ▶ Check it in a demo: the battery data in the test–interval demonstration

  6. Uniform P-values, by simulation. In R, simulate 10,000 samples of size 10 from \(\mathcal N(0,1)\), compute the two-sided P-value of t.test(x, mu = 0) for each, and plot a histogram. Which theorem predicts the shape? What changes if you use mu = 0.5?

    Solution

    p <- replicate(1e4, t.test(rnorm(10), mu = 0)$p.value); hist(p) gives a flat histogram (Theorem 4.1a), with about 5% of values below 0.05. With mu = 0.5 the null is false (the true mean is 0, so \(d=-0.5\)), and the P-values pile up near 0; the fraction below 0.05 estimates the power (about 0.29 for this \(n\) and \(d\)).

    ▶ Check it in a demo: flat P-values in the hypothesis-testing demonstration

Demonstrations for these notes

Each demonstration runs in your browser and is tied to the results it illustrates; the ▶ Demo boxes above open them already set to the relevant case. Simulations use a fixed seed, so a class sees the same pictures.

DemonstrationWhat you can seeResults
Averages on trial: the law of large numbers and the central limit theoremAverages gather at the mean; standardised, they become the bell curve for any population shapeCor. 0.3, Thm 0.4, 0.5, 0.7, 3.5
Why Student's t? Inside the normal-sample theorem\(\bar X\) and \(S\) independent, the chi-squared and t pivots, and how each fails without normalityThm 0.8, 0.9, 3.4, 3.5, 3.6
Estimators on trial: bias, spread and the n − 1Sampling distributions of competing estimators: bias, standard error and MSEThm 1.1, 1.3, Example, Thm 2.4
Maximum likelihood: climbing the likelihoodThe log-likelihood of each sample, its top and curvature, and the MLE's spread against the Cramér–Rao boundThm 2.2, 2.3, 2.4, Ex. 2, 3
Confidence intervals: does the method catch the truth?A hundred intervals at a time: coverage, width, and the failure of 1.96 for small nDef. 3.1, Thm 3.2, 3.3, 3.4
Two samples: pooled or Welch, paired or not?When the pooled test breaks, why Welch holds, and how much pairing gainsThm 3.7, 3.8, Prop. 8.1, Thm 8.2
Counting successes: exact and approximate methods for proportionsExact sizes of binomial tests and the exact coverage of the Wald intervalThm 3.9, 4.1(b), 7.1
A hypothesis test on trial: P-values, errors and powerThousands of t-tests: false alarms at rate \(\alpha\), flat P-values, and simulated powerThm 4.1, 5.1, 5.2, Ex. 6
Tests and intervals: two views of one objectThe P-value as a function of \(\mu_0\): the values not rejected form the intervalThm 4.2, 3.4, 5.1, Ex. 5
Power and sample size: two bell curvesThe areas \(\alpha\), \(\beta\) and power, the power curve, and the sample size for a target powerThm 5.2, 6.1, Ex. 4

References

  1. Kroese, D. P. and Chan, J. C. C. (2014). Statistical Modeling and Computation. Springer, New York. doi:10.1007/978-1-4614-8775-3.
  2. Casella, G. and Berger, R. L. (2002). Statistical Inference, 2nd edition. Duxbury, Pacific Grove, CA.
  3. Devore, J. L. and Berk, K. N. (2012). Modern Mathematical Statistics with Applications, 2nd edition. Springer, New York. doi:10.1007/978-1-4614-0391-3.
  4. Lafaye de Micheaux, P., Drouilhet, R. and Liquet, B. (2013). The R Software: Fundamentals of Programming and Statistical Analysis. Springer, New York. doi:10.1007/978-1-4614-9020-3.
  5. Neyman, J. and Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society A, 231, 289–337. doi:10.1098/rsta.1933.0009.
  6. Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2(6), 110–114. doi:10.2307/3002019.
  7. Welch, B. L. (1947). The generalization of "Student's" problem when several different population variances are involved. Biometrika, 34(1–2), 28–35. doi:10.1093/biomet/34.1-2.28.
  8. Shoemaker, A. L. (1996). What's normal? Temperature, gender, and heart rate. Journal of Statistics Education, 4(2).

The sleep, seed, taste-test, reminder-email and battery examples are illustrative numbers constructed for these notes. All numerical results were computed in R.