Skillversitas

Decision Boundary

Because logistic regression predicts class 1 whenever the score z = w·x + b is positive, the boundary between its two classes is the flat surface where z = 0 - a hyperplane. In two dimensions that is a line; in three, a plane; in d dimensions, a (d-1)-dimensional slice through space. This geometry is the model's biggest strength (simple, easy to verify) and its biggest limitation (it can only draw straight cuts).

The Boundary in Two Features

With two features the boundary w0 + w1·x1 + w2·x2 = 0 can be solved for x2: x2 = -(w0 + w1·x1) / w2. One line, two half-planes, class 1 on the side where z > 0. The weights set the slope and the intercept sets where the line crosses; larger |w| pushes the boundary to decide more quickly (steeper probability transition).

Plotting the Boundary

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression

rng = np.random.default_rng(0)
X = rng.normal(size=(200, 2))
X[:100] += np.array([2, 2])
y = np.array([0] * 100 + [1] * 100)

model = LogisticRegression().fit(X, y)
w, b = model.coef_[0], model.intercept_[0]

xs = np.linspace(X[:, 0].min() - 1, X[:, 0].max() + 1, 100)
plt.scatter(X[:, 0], X[:, 1], c=y, cmap="bwr", alpha=0.4)
plt.plot(xs, -(w[0] * xs + b) / w[1], "k-", lw=2)   # the boundary line
plt.xlabel("feature 1")
plt.ylabel("feature 2")
plt.show()

Every point on the black line has predicted probability exactly 0.5; moving away from it in either direction increases the confidence toward one class or the other. The distance of a point from the boundary is proportional to |z|, so the probability is a squashed measure of signed distance.

What a Straight Line Cannot Do

Consider the XOR pattern: the positive class sits in two diagonal corners, and no straight line separates it from the negative class. The model will still train, but its boundary will be a compromise line and its accuracy will plateau near 50-75%. The standard fixes, in order of escalation:

  • Add engineered features (interactions, polynomials) so the boundary becomes a curve in the new space - this keeps the model interpretable.
  • Use a kernel method (SVM with an RBF kernel) that implicitly bends the boundary.
  • Switch models - trees, random forests, and boosting build axis-parallel or arbitrary partitions without any boundary shape restriction.

Margin and Confidence

Points close to the boundary are exactly the model's "unsure" cases: probability near 0.5, small |z|. Production systems often add a third band - predict class 1, predict class 0, or escalate to review when |z| is below a margin. That band is free with logistic regression, because the score already encodes the distance.

Summary

The decision boundary of logistic regression is the hyperplane w·x + b = 0: a straight cut whose orientation and position come straight from the fitted weights. If your data needs a curved cut, either engineer features so a straight cut works in the new space, or move to a model without the linearity constraint.