Supervised Learning Algorithms

Supervised learning trains a function from labeled examples: every training row pairs features X with a known target y. Classification predicts discrete classes (spam or not, digit 0–9), while regression predicts continuous values (price, demand). Python's scikit-learn exposes every major algorithm behind the same three verbs — fit, predict, score — so switching models is almost free.

Linear Models

Linear regression fits a weighted sum of features; logistic regression applies the same structure to a sigmoid that outputs class probabilities. Both are fast, interpretable baselines that often rival more complex models when features are well engineered.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y)

clf = LogisticRegression(max_iter=1000, C=1.0)  # C controls regularization strength
clf.fit(X_train, y_train)
pred = clf.predict(X_test)
print(accuracy_score(y_test, pred))
print(clf.predict_proba(X_test[:3]))  # probabilities, useful for threshold tuning

Trees and Ensembles

Decision trees split feature space by simple rules and capture nonlinear interactions automatically, though single trees overfit easily. Random forests average many trees trained on bootstrap samples and random feature subsets, and gradient boosting builds trees sequentially, each correcting the errors of the previous ones.

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

rf = RandomForestClassifier(n_estimators=300, max_depth=None, random_state=42)
scores = cross_val_score(rf, X_train, y_train, cv=5, scoring="accuracy")
print("CV accuracy: %.3f +/- %.3f" % (scores.mean(), scores.std()))

rf.fit(X_train, y_train)
importances = sorted(zip(X_train.columns, rf.feature_importances_),
                     key=lambda t: -t[1])[:5]
print(importances)

For regression the pattern is identical with RandomForestRegressor or GradientBoostingRegressor; the estimator changes, the interface does not.

Instance-Based Methods

k-Nearest Neighbors stores the training set and classifies a query by voting among its k closest points. It needs scaled features and can be slow at prediction time on large data, but it makes almost no modeling assumptions, which makes it a strong quick baseline.

from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

knn = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=7))
knn.fit(X_train, y_train)
print("k-NN accuracy:", knn.score(X_test, y_test))

Support Vector Machines

SVMs find the separating hyperplane with the widest margin; the kernel trick (especially RBF) lets them carve curved decision boundaries in high-dimensional space. They shine on small-to-medium datasets and text classification, at a higher training cost than trees.

from sklearn.svm import SVC

svm = make_pipeline(StandardScaler(), SVC(kernel="rbf", C=10, gamma="scale"))
svm.fit(X_train, y_train)
print("SVM accuracy:", svm.score(X_test, y_test))

Choosing an Algorithm

Start simple: a regularized linear model as baseline, then a random forest or gradient-boosting ensemble, which are the strongest all-rounders on tabular data. Use small k in k-NN for smooth boundaries and large k for stability, and keep class imbalance in mind by passing class_weight="balanced" where supported. Always evaluate with the same folds when comparing candidates.

Summary

Supervised learning covers classification and regression, unified in scikit-learn's fit/predict interface. Linear models give interpretable baselines, tree ensembles dominate tabular problems, k-NN offers assumption-free simplicity, and SVMs handle margin-sensitive problems. Benchmark several candidates with cross-validation on the same splits, then tune the winner.