Binary Classification Example

This chapter walks a complete binary classification loop on a small synthetic credit dataset: build the data, train, check what the model believes, make a decision on one customer, and see where the 0.5 threshold sits relative to the real risk. The mechanics are identical to any real tabular problem - churn, default, click, failure.

The Data

import numpy as np

rng = np.random.default_rng(7)
n = 2000
income = rng.normal(50_000, 15_000, n)
debt   = rng.gamma(2.0, 10_000, n)
years  = rng.integers(0, 20, n)

# ground-truth risk: high income and long history lower default odds
z = 0.00004 * income - 0.00010 * debt + 0.15 * years
p_default = 1 / (1 + np.exp(-z))
y = (rng.random(n) < p_default).astype(int)

X = np.c_[income, debt, years]
print("default rate:", round(y.mean(), 3))   # ~37%

Train

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score

Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)
scaler = StandardScaler().fit(Xtr)

model = LogisticRegression(max_iter=2000).fit(scaler.transform(Xtr), ytr)
print("test accuracy:", round(accuracy_score(yte, model.predict(scaler.transform(Xte))), 3))

What the Model Believes

names = ["income", "debt", "years"]
for name, w in zip(names, model.coef_[0]):
    print(f"{name:8s} coef={w:+.3f}  odds multiplier per unit: {np.exp(w):.3f}")

Expect a positive coefficient for income and years, negative for debt - the model recovered the direction of the ground-truth signal. The intercept (on the scaled space) encodes the base rate.

One Customer, One Decision

customer = np.array([[70_000, 15_000, 8]])   # income, debt, years
scaled = scaler.transform(customer)

p = model.predict_proba(scaled)[0, 1]
label = model.predict(scaled)[0]
print(f"estimated default probability: {p:.3f}")
print("label at 0.5 threshold:", label)

# the cost-aware view: reject only if expected loss says so
p_loss, n_loss = 0.8, 0.1    # cost ratio of a false reject vs a missed default
cutoff = p_loss / (p_loss + n_loss)
print("cost-aware cutoff:", round(cutoff, 3),
      "- decision:", "reject" if p > cutoff else "accept")

The default threshold of 0.5 answers "which class is more probable?"; the cost-aware cutoff answers "is the expected cost of accepting higher than rejecting?" - and in a lending business that is the real question, which is why the probability (not the label) is the output you should keep.

A Segment View

proba = model.predict_proba(scaler.transform(Xte))[:, 1]
for lo, hi in [(0, 0.2), (0.2, 0.4), (0.4, 0.6), (0.6, 0.8), (0.8, 1.0)]:
    mask = (proba >= lo) & (proba < hi)
    print(f"prob {lo:.1f}-{hi:.1f}: n={mask.sum():5d}  actual default rate={yte[mask].mean():.3f}")

If the model is calibrated, the actual default rate should climb roughly monotonically across the probability bins - the strongest single sanity check for a production scoring model.

Summary

The binary loop: build labeled data, scale on train, fit, and read the coefficients as odds multipliers. Keep the probability as your product output, pick thresholds from the cost of errors rather than from 0.5, and verify calibration with probability bins. That is the whole workflow for a scoring model.