ROC Curve and AUC

The ROC (receiver operating characteristic) curve plots a model's detection power against its false alarm rate, sweeping the decision threshold from "flag everything" to "flag nothing." Its area - the AUC - is a single threshold-free number for how well the model ranks positives above negatives. It is the standard metric for comparing candidate models before any business threshold is chosen.

The Axes

TPR (sensitivity, recall): of the real positives, the fraction flagged.
FPR: of the real negatives, the fraction falsely flagged.

At threshold 0 (flag everything) the point is (1, 1); at threshold 1 (flag nothing) it is (0, 0). Sweeping between them traces the curve. A coin-flip model lands on the diagonal; a perfect model hugs the top-left corner and reaches (1, 0). Every point on the curve is an achievable operating point - the curve is the model's full menu of precision/recall trade-offs.

In Python

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_curve, auc
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt

X, y = make_classification(n_samples=1500, n_informative=5, random_state=9)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)
model = LogisticRegression(max_iter=2000).fit(Xtr, ytr)
proba = model.predict_proba(Xte)[:, 1]

fpr, tpr, thresholds = roc_curve(yte, proba)
score = auc(fpr, tpr)
print("AUC:", round(score, 4))

plt.plot(fpr, tpr, label=f"logistic (AUC={score:.3f})")
plt.plot([0, 1], [0, 1], "k--")
plt.xlabel("false positive rate")
plt.ylabel("true positive rate")
plt.legend()
plt.show()

Reading AUC

AUC has a clean probabilistic interpretation: pick one random positive and one random negative - the AUC is the probability the model gives the positive the higher score. 0.5 is chance, 1.0 is perfect separation, and real tabular models typically land between 0.7 and 0.95. Two models with AUC 0.90 vs 0.88 are nearly tied in ranking quality; differences below ~0.02 are usually within noise, so use confidence intervals (bootstrap or DeLong) before declaring a winner.

When ROC Misleads

ROC is insensitive to class balance: with 1% positives, a model can have FPR look tiny in absolute terms while flooding you with false alarms, and the AUC can look healthy while precision is poor. On strongly imbalanced data the precision-recall curve (and its area, average precision) is the honest view of the positive class - it zooms into the region that actually matters. Use ROC to compare models on balanced or moderately skewed data, and PR to choose the operating point when positives are rare.

Choosing a Point on the Curve

Pick the threshold where the curve's bend matches your cost ratio: each extra true positive found is worth R times each extra false alarm. On the thresholds array from roc_curve, find the point that maximizes TPR - lambda*FPR for your chosen lambda, or the point closest to the top-left corner for a symmetric cost. The model's job ends at the probabilities; the threshold is a business decision made on this curve.

Summary

The ROC curve is the model's complete menu of detection-vs-alarm trade-offs; AUC is its threshold-free ranking score with a clean random-pair interpretation. Compare models on balanced data with AUC, switch to precision-recall when positives are rare, and pick the final threshold from the curve using the real cost ratio.