Interactive demo · Probabilistic Machine Learning · Chapter 19

Semi-supervised learning from a few labels

A handful of points are labeled (large discs with a white ring, red or blue); the other 300 are not (small dots). A supervised classifier sees only the labeled ones. Self-training labels the confident unlabeled points with its own predictions and refits; entropy minimisation adds the mean prediction entropy on the unlabeled points to the loss; label propagation spreads the labels along a similarity graph. Change the number of labels, the labeled set, the threshold, the entropy weight or the graph width, and compare with the supervised-only fit on the same labels.

Data

Method

Labeled points per class nl

Which labeled points

Confidence threshold τ

Entropy weight β

Graph width σ

Unlabeled points

Try these

Data: two moons (noise 0.1) with RBF features on a fixed 7 by 5 grid of centres, or two elongated Gaussian clusters (separation 1.6) with a linear model; both simulated, 300 unlabeled training points and 400 test points. Classifiers are logistic regressions with a ridge penalty (λ = 0.1); self-training runs 10 rounds; the entropy objective is minimised by 3000 gradient steps from the supervised solution; label propagation uses Gaussian weights exp(−d2/(2σ2)) and the harmonic solution, extended to the test points by kernel averaging. Accuracy on the unlabeled points excludes the labeled ones.

In the lecture notes