Start a Markov chain in one state and follow the distribution of Xn: it is the row vector μn = μ0Pn. For an irreducible, aperiodic finite chain it converges to the stationary distribution π, whatever the start, and the distance dn = maxA|μn(A) − π(A)| falls geometrically. The bottom panel shows why, in the way the convergence theorem is proved: run a second copy of the chain started from π, move the two independently until they meet, and then together. After they meet they have the same distribution, so dn can never exceed the probability that they have not met by time n. A periodic chain never forgets where it started, and the two copies may never meet; a reducible chain converges, but to a limit that depends on the start.
Classes are the sets of states that reach each other; a class that cannot be left is closed (its states are recurrent in a finite chain), the others are transient. The period of a class is the gcd of the lengths of its closed paths. Target: π when the chain has a single closed class (then π is unique); otherwise the limit of μn from the chosen start. Coupling: X starts at the chosen state, Y is drawn from the target, both move independently by inverse-cdf draws until X = Y. Laziness replaces P by εI + (1 − ε)P: same stationary distributions, but every state becomes aperiodic. Random numbers are seeded.
code/ch23_stochastic_processes.py (classify, stationary, tv_curve, demo_markov).