Each point \(i\) turns distances into neighbour probabilities \(p_{j\mid i}\) with a bandwidth \(\sigma_i\) chosen so that the perplexity \(2^{H(P_i)}\) equals the value you set: roughly the number of neighbours it pays attention to. The embedding \(\mathbf z_i\) is then moved by gradient descent on \(\mathrm{KL}(P\,\|\,Q)\), with \(q_{ij}\propto(1+\|\mathbf z_i-\mathbf z_j\|^2)^{-1}\) (t-SNE) or \(q_{ij}\propto e^{-\|\mathbf z_i-\mathbf z_j\|^2}\) (Gaussian symmetric SNE). Small perplexity shatters clusters into fragments; large perplexity merges them; the Gaussian kernel squeezes distant groups together because its tails vanish.
The optimiser is gradient descent with momentum (0.5, then 0.8 after iteration 100), step size 100, adaptive gains, early exaggeration 4 for the first 100 iterations and a deterministic start from the first two principal components scaled to standard deviation 0.01, so every run is reproducible. Data: three Gaussian clusters (150 points), 300 UCI digits, a 300-point Swiss roll.
code/ch20_dimensionality_reduction.py (sne_conditional_p, tsne_grad, tsne).