Skillversitas

K-Means Clustering

K-means is the standard unsupervised clustering algorithm: given a number k, it partitions the data into k groups by iteratively moving group centers toward the mean of their members. There are no labels - the algorithm discovers structure. It is fast, simple, and surprisingly effective on data with compact, roughly spherical clusters, and its two real questions are how to choose k and how to read the result honestly.

The Algorithm

  1. Pick k initial centers (randomly, by default, then refined by restarting).
  2. Assign: every point joins the nearest center.
  3. Update: each center moves to the mean of its assigned points.
  4. Repeat assign-update until the centers stop moving (or a max iteration count is hit).

The objective being minimized is the within-cluster sum of squares - the total squared distance from each point to its center. K-means finds a local minimum of that objective, which is why scikit-learn runs the whole procedure multiple times from different starts (n_init) and keeps the best run.

Fitting K-Means

import numpy as np
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler

X, y_true = make_blobs(n_samples=500, centers=4, cluster_std=0.8, random_state=0)
scaler = StandardScaler().fit(X)
Xs = scaler.transform(X)

km = KMeans(n_clusters=4, n_init=10, random_state=0).fit(Xs)
print("inertia (within-cluster SS):", round(km.inertia_, 1))
print("cluster sizes:", np.bincount(km.labels_))
print("centers:\n", km.cluster_centers_.round(2))

Scaling before clustering is as mandatory as for KNN - k-means is a distance algorithm, and the largest-range feature would otherwise define the geometry. labels_ gives each point's cluster, cluster_centers_ the final means, and inertia_ the objective value. The cluster labels are arbitrary integers - "cluster 2" has no inherent meaning, only membership does.

Choosing k

from sklearn.metrics import silhouette_score

inertias, silhouettes = [], []
for k in range(2, 9):
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(Xs)
    inertias.append(km.inertia_)
    silhouettes.append(silhouette_score(Xs, km.labels_))

for k, i, s in zip(range(2, 9), inertias, silhouettes):
    print(f"k={k}  inertia={i:8.1f}  silhouette={s:.3f}")

Two diagnostics, used together. The elbow: plot inertia against k - it falls steeply while real clusters are being separated and flattens once k exceeds the true count; the bend is a candidate. The silhouette score: for each point, how much closer it is to its own cluster than to the nearest other cluster, averaged - higher (closer to 1) means tighter, better-separated clusters, and its peak across k is the more reliable of the two signals. In practice the data often suggests k from the domain ("we expect three customer segments"), and the diagnostics confirm rather than decide.

Honest Limitations

  • Spherical assumption: k-means wants compact, ball-shaped clusters. Elongated or crescent-shaped groups (the classic moons) get cut up; DBSCAN or a spectral method handles those shapes.
  • Equal-size bias: the objective prefers roughly balanced clusters; a tiny dense group plus a huge diffuse one is a bad fit.
  • No outliers model: extreme points drag centers and distort boundaries; robust scaling or a pre-filtering step helps.
  • Clusters are not categories: the algorithm found a partition that minimizes variance - whether that partition means anything requires checking it against the data (feature distributions per cluster, domain review).

Summary

Cluster with KMeans on scaled data, choose k with the silhouette score (the elbow as a sanity check), and always verify the result by inspecting what each cluster actually contains - mean feature values, histograms, domain labels. K-means is the right first clustering tool for compact, ball-like groups and the wrong one for elongated or overlapping shapes, where the choice of algorithm - not the choice of k - is the real decision.