The Sigmoid Function

The sigmoid (logistic) function squashes any real number into the interval (0, 1), which makes its output readable as a probability. Its shape is the heart of the algorithm: very negative inputs map to nearly 0, very positive inputs to nearly 1, and the input 0 maps to exactly 0.5. The steeper the region around 0, the more decisively the model separates the two classes.

Definition and Properties

The function is 1 / (1 + e^(-z)). Three properties matter. First, the range is strictly between 0 and 1, so the output is a valid probability for any input. Second, it is symmetric: sigma(z) = 1 - sigma(-z), so the two classes are mirror images. Third, it saturates - for |z| beyond about 6 the output is essentially 0 or 1, and the gradient is nearly zero there, which is both a feature (confident predictions) and a training hazard (vanishing gradients for far-away points).

In Python

import numpy as np

def sigmoid(z):
    return 1.0 / (1.0 + np.exp(-z))

zs = np.linspace(-6, 6, 7)
for z, p in zip(zs, sigmoid(zs)):
    print(f"z = {z:+.1f}   p = {p:.4f}")

For numerical stability with large |z|, branch on the sign so the exponential never overflows:

def sigmoid_stable(z):
    out = np.empty_like(z, dtype=float)
    pos = z >= 0
    out[pos] = 1.0 / (1.0 + np.exp(-z[pos]))
    ez = np.exp(z[~pos])
    out[~pos] = ez / (1.0 + ez)
    return out

The Log-Odds Reading

The most useful interpretation is the inverse: z = log(p / (1 - p)), the log-odds (logit) of the event. Logistic regression is literally a model of the log-odds as a linear function of the inputs. A coefficient of 0.7 for a feature means: one more unit of that feature multiplies the odds of class 1 by e^0.7 ≈ 2.0, holding everything else fixed. That single fact powers the interpretation chapter later in this tutorial.

From Probability to Class

A decision needs a threshold: predict class 1 when p ≥ 0.5. The default threshold of 0.5 corresponds to "classify by whichever class has the higher probability," and it ignores the cost of being wrong. When false positives and false negatives cost different amounts - fraud detection, medical screening - you move the threshold to match the economics, and the ROC and precision-recall chapters show exactly how to pick it.

Summary

The sigmoid turns a linear score into a probability and gives the model its name: the inputs are linear in the log-odds. Compute it with care for large inputs, read coefficients as odds multipliers, and remember that 0.5 is a default, not a law - the right threshold depends on what mistakes cost.