Skillversitas

Hierarchical Clustering

Hierarchical Clustering builds a hierarchy of clusters step by step. The result is a dendrogram that shows the relationships between clusters, without needing to determine the number of clusters in advance.

Agglomerative vs Divisive

Type Approach Complexity
Agglomerative Bottom-up: starts from individual points More common
Divisive Top-down: starts from one large cluster Less common

How Agglomerative Works

  1. Treat each data point as its own cluster
  2. Find the two most similar clusters
  3. Merge them into a single cluster
  4. Repeat until only one cluster remains

Linkage Methods

Method Explanation Characteristics
Single Minimum distance between clusters Chain-like clusters
Complete Maximum distance between clusters Compact clusters
Average Average distance between clusters A compromise
Ward Minimizes total variance Spherical clusters 
from sklearn.cluster import AgglomerativeClustering
from sklearn.datasets import make_blobs

X, _ = make_blobs(n_samples=200, centers=4, random_state=42)

models = {}
for linkage in ['ward', 'complete', 'average', 'single']:
    model = AgglomerativeClustering(n_clusters=4, linkage=linkage)
    labels = model.fit_predict(X)
    n_clusters = len(set(labels))
    print(f"Linkage {linkage:10s} → Cluster: {n_clusters}, "
          f"Labels: {sorted(set(labels))}")

Dendrogram

A dendrogram is a visualization of the clustering hierarchy. The height of a join indicates the distance at which clusters were merged.

from scipy.cluster.hierarchy import dendrogram, linkage
from sklearn.datasets import make_blobs
import matplotlib.pyplot as plt

X, _ = make_blobs(n_samples=100, centers=4, random_state=42)

Z = linkage(X, method='ward')

plt.figure(figsize=(10, 5))
dendrogram(Z, truncate_mode='lastp', p=30, leaf_rotation=90)
plt.title('Dendrogram (Ward Linkage)')
plt.xlabel('Sample Index')
plt.ylabel('Distance')
plt.tight_layout()
plt.show()

Cutting the Dendrogram

To determine the number of clusters, cut the dendrogram at a certain distance:

from scipy.cluster.hierarchy import fcluster
from scipy.cluster.hierarchy import linkage

Z = linkage(X, method='ward')

# Cut at a specific distance
labels_2 = fcluster(Z, t=2, criterion='maxclust')
labels_3 = fcluster(Z, t=3, criterion='maxclust')
labels_4 = fcluster(Z, t=4, criterion='maxclust')

print(f"2 clusters: {len(set(labels_2))}")
print(f"3 clusters: {len(set(labels_3))}")
print(f"4 clusters: {len(set(labels_4))}")

Example: Customer Segmentation

import numpy as np
import pandas as pd
from sklearn.cluster import AgglomerativeClustering
from sklearn.preprocessing import StandardScaler
from scipy.cluster.hierarchy import dendrogram, linkage
import matplotlib.pyplot as plt

np.random.seed(42)
df = pd.DataFrame({
    'income': np.random.randint(20000, 150000, 100),
    'score': np.random.randint(1, 100, 100)
})

scaler = StandardScaler()
X_scaled = scaler.fit_transform(df)

# Dendrogram
Z = linkage(X_scaled, method='ward')
plt.figure(figsize=(10, 5))
dendrogram(Z, truncate_mode='lastp', p=20)
plt.title('Customer Dendrogram')
plt.show()

# Clustering
model = AgglomerativeClustering(n_clusters=3, linkage='ward')
df['cluster'] = model.fit_predict(X_scaled)
print(df.groupby('cluster').mean().round(0))

Pros and Cons

Pros Cons
No need to determine k in advance Slow on large data O(n³)
Informative dendrogram Cannot be updated (must recompute)
Flexible linkage Sensitive to noise
Good for hierarchical data Memory intensive

Comparison with K-Means

Aspect K-Means Hierarchical
Number of clusters Must be set in advance Can be read from the dendrogram
Speed Fast O(n) Slow O(n³)
Scalability Good for large data Not for very large data
Result Spherical clusters Flexible shapes

The next chapter covers DBSCAN, a density-based clustering algorithm that can find clusters of arbitrary shape.

Next: DBSCAN →