The distortion of the best K-means solution falls with every extra cluster, so it cannot choose K by itself (the elbow is hard to see). The silhouette coefficient of a point compares its mean distance to its own cluster, a, with the smallest mean distance to another cluster, b: s = (b − a) / max(a, b). The BIC of a Gaussian mixture (full covariances) penalises extra components and has a minimum near the number of blobs. The data are five blobs, but the three small ones on the left sit close together, so the silhouette prefers K = 3 while BIC prefers K = 5. Move the slider and compare.
Data: 2000 simulated points from five Gaussian blobs (a stand-in for the book's blob data). For each K the clustering is the best of 30 K-means++ runs (seeds fixed). The silhouette is the standard one (smallest mean distance over the other clusters). BIC is −2 log-likelihood + (number of parameters) log N for a full-covariance mixture, fitted by EM in Python (the book's (21.21) is −1/2 of this, so there larger is better); it is stored with the page data.
code/ch21_clustering.py (silhouette, gmm_em, k_selection_curves).