One affine autoregressive layer sets x1 = μ1 + eα1u1 and x2 = μ2(x1) + eα2(x1)u2, so every conditional p(x2 | x1) is Gaussian, however flexible the networks μ2 and α2 are. The cross below has two-humped conditionals, so one layer fails; stacking layers and reversing the order of the variables between them fixes it. The flows were trained in Python by maximum likelihood; your browser runs them: the density by the parallel inverse pass, new samples by the sequential forward pass.
The cross is an equal mixture of two zero-mean Gaussians with standard deviations 1.5 along the diagonals and 0.25 across them. Each layer's conditioner is a network with two hidden layers of 32 tanh units; the order of the variables is reversed after every layer. Log-likelihoods are averages over the same 2 000 test draws, in nats; the true density scores −2.350 on them (its entropy is 2.346 nats). The red curve on the right is the exact conditional of the cross, a mixture of N(±0.946 x1, 0.3492).
code/ch23_normalizing_flows.py (class AffineAR2D, get_maf, export_json); trained weights in data/ch23-maf-cross.json.