Skillversitas

End-to-End ML Project

This chapter runs the complete workflow on one problem - predicting which telecom customers will cancel in the next 60 days - using everything the tutorial built: data handling, features, pipelines, cross-validation, tuning, final evaluation, and serialization. The dataset is synthetic, but the structure is exactly what a real tabular project looks like, and each section maps to the chapter that introduced the technique.

The Data and the Question

import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
n = 8000
df = pd.DataFrame({
    "tenure_months": rng.integers(1, 60, n),
    "monthly_charges": rng.normal(65, 15, n).clip(30, 120),
    "support_tickets": rng.poisson(1.3, n),
    "contract": rng.choice(["month", "year"], n, p=[0.6, 0.4]),
})
z = (-0.055 * df["tenure_months"] - 1.4 * df["support_tickets"]
     + 0.03 * (df["monthly_charges"] - 65)
     - 1.2 * (df["contract"] == "month"))
p = 1 / (1 + np.exp(-z))
df["churned"] = (rng.random(n) < p).astype(int)
print("churn rate:", round(df["churned"].mean(), 3))   # ~25%

The decision the model feeds is a win-back offer program: the offer costs $40 and saves an average of $600 in margin per retained customer, so the offer threshold will come from that cost ratio, not from 0.5.

Features and the Pipeline

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

num_pipe = Pipeline([("imputer", SimpleImputer(strategy="median")),
                     ("scaler", StandardScaler())])
cat_pipe = Pipeline([("ohe", OneHotEncoder(handle_unknown="ignore",
                                            sparse_output=False))])

pipe = Pipeline([
    ("pre", ColumnTransformer([
        ("num", num_pipe,
         ["tenure_months", "monthly_charges", "support_tickets"]),
        ("cat", cat_pipe, ["contract"]),
    ])),
    ("model", LogisticRegression(max_iter=3000)),
])

Split, Baseline, Final Score

from sklearn.model_selection import (train_test_split, StratifiedKFold,
                                     cross_val_score)
from sklearn.metrics import classification_report, roc_auc_score

X = df.drop(columns="churned")
y = df["churned"]
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3,
                                      random_state=0, stratify=y)

# development baseline: CV on the train set only
cv = cross_val_score(pipe, Xtr, ytr,
                     cv=StratifiedKFold(5, shuffle=True, random_state=0),
                     scoring="roc_auc")
print("CV AUC:", round(cv.mean(), 4), "+/-", round(cv.std(), 4))

# final fit and the single test-set evaluation
pipe.fit(Xtr, ytr)
proba = pipe.predict_proba(Xte)[:, 1]
print("test AUC:", round(roc_auc_score(yte, proba), 4))
print(classification_report(yte, (proba >= 0.5).astype(int)))

Tune, Then Score Once

from sklearn.model_selection import GridSearchCV

grid = {"model__C": [0.01, 0.1, 1.0, 10.0]}
search = GridSearchCV(pipe, grid, cv=5, scoring="roc_auc", n_jobs=-1)
search.fit(Xtr, ytr)
print(search.best_params_, round(search.best_score_, 4))

final = search.best_estimator_
proba = final.predict_proba(Xte)[:, 1]
print("final test AUC:", round(roc_auc_score(yte, proba), 4))   # the claim

Ship It: Serialize and Predict

import joblib

joblib.dump(final, "churn_model.joblib")
loaded = joblib.load("churn_model.joblib")

new_customer = pd.DataFrame([{
    "tenure_months": 3, "monthly_charges": 85,
    "support_tickets": 4, "contract": "month",
}])
p = loaded.predict_proba(new_customer)[0, 1]

cutoff = 40 / (40 + 600)        # from the offer economics
print(f"churn probability: {p:.3f} - send offer: {p >= cutoff}")

The deployable artifact is the pipeline, not the weights: loading the file reproduces imputation, encoding, scaling, and the model in a single call. A table of coefficients without its preprocessing chain is a number, not a system.

After Launch

  • Score monthly cohorts and track AUC plus probability-bin calibration; drift in the bins is the early warning that the data has moved.
  • Keep a shadow copy of the previous model for a month after each retrain so regressions are visible, not assumed.
  • Retrain on a schedule or when the drift monitor trips - a model is a component with an expiry date, not a result you achieve once.

Summary

The project - synthetic churn data, a ColumnTransformer pipeline, stratified CV for the development AUC, a small grid search on C, one final test-set score, joblib serialization, and a cost-ratio threshold - is the complete, repeatable template for tabular classification. Every step maps to a chapter in this tutorial; running the sequence in order, with the test set touched exactly once, is what turns a notebook full of results into a system someone can trust, audit, and maintain.