Interactive demo · Probability and Statistical Inference · Chapter 24

Importance sampling: why the proposal needs heavy tails

To estimate I = ∫ h f, draw from a proposal g instead of f and average the terms h(X)f(X)/g(X). The estimate is unbiased for every g, but its variance is ∫ h²f²/g − I², which is small when g looks like |h|f and infinite when g has lighter tails than f where h f lives. Top left: the target (gold) and the proposal (blue), with the first draws of one run as ticks whose height is their weight. Top right: that run's estimate as N grows. Bottom: the estimates of many independent runs.

Problem

Proposal g

Location μ

Scale s

Draws per run N

Threshold c in P(Z > c)

Try these

For P(Z > c) and ∫12 φ the estimate is the plain average of h f/g; for the posterior mean of Example 24.7 (flat prior, two observations with t3 errors) it is self-normalised, Σ wiθi / Σ wi with w = L/g. The effective sample size is (Σwi)²/Σwi², computed from the terms h f/g for the first two problems (it equals N for the optimal g*) and from w for the third. The exact standard deviation is √(V/N), with V found by Simpson's rule; whether it is finite is decided from the tails. Each run uses its own seeded stream; t3 draws are Z/√(χ²3/3) and Cauchy draws tan(π(U − ½)).

In the lecture notes