A loan application lands on a desk. The reviewer wants a number — not a vague "looks risky," but a probability: this applicant has a 12 percent chance of default, that one a 78 percent chance. The inputs are ordinary measurements like income, debt, and credit history. The output has to be a probability, bounded firmly between 0 and 1. That gap, between a free-ranging weighted sum and a well-behaved probability, is exactly what logistic regression bridges.
This article explains why a straight line cannot output a probability, how the sigmoid function fixes that, and how to read the decision boundary that the model draws.
Why this calculation matters
Logistic regression is the default tool for binary classification across engineering and science: pass or fail, defective or sound, spam or legitimate, disease present or absent. It is the model people reach for first because it is fast to fit, hard to overfit, and — unlike many alternatives — easy to interpret. Each coefficient has a plain meaning in terms of how the odds shift.
It also matters as a foundation. A single artificial neuron with a sigmoid activation is, mathematically, a logistic regression. Understanding this one model well gives you a direct handle on how neural networks make decisions at every node. And because its output is a genuine probability rather than a raw score, logistic regression supports decisions that need a calibrated confidence — where the cost of a false positive and a false negative are not equal, and you want to set the threshold deliberately rather than accept a hard-coded 0.5.
The core method
Start with a linear score, often called the logit. With one predictor x it is just a weighted input plus a bias:
z = b0 + b1*x
That score z can be any real number, from large negative to large positive — which is the problem, because a probability cannot. The fix is the sigmoid, or logistic, function. It takes any real z and squeezes it smoothly into the open interval between 0 and 1:
p = 1 / (1 + e^(-z))
When z is large and positive, e^(-z) shrinks toward zero and p approaches 1. When z is large and negative, e^(-z) blows up and p approaches 0. When z is exactly 0, p is exactly 0.5. The curve is a smooth S: nearly flat at the extremes, steepest in the middle.
The point where p = 0.5 is the decision boundary. It is where z = 0, the dividing surface between the two predicted classes. For the one-variable model, set b0 + b1*x = 0 and solve for x. With more predictors, z = 0 defines a line, a plane, or a hyperplane through the feature space.
The coefficients are not fitted by least squares. They are chosen to maximise the likelihood of the observed labels — equivalently, to minimise the cross-entropy loss — and that optimisation is solved numerically, usually with gradient descent. The sign of each coefficient tells the story: a positive b1 means larger x pushes the prediction toward the positive class.
A worked example
Take a fitted model with intercept b0 = -4 and slope b1 = 2. The task is to predict the probability of the positive class at x = 3.
Step 1 — compute the linear score.
z = b0 + b1*x = -4 + 2*3 = -4 + 6 = 2
Step 2 — pass the score through the sigmoid.
p = 1 / (1 + e^(-z)) = 1 / (1 + e^(-2))
The value of e^(-2) is about 0.135, so:
p = 1 / (1 + 0.135) = 1 / 1.135 = 0.881
The model predicts an 88 percent probability that this case belongs to the positive class. With a standard 0.5 threshold, it is classified positive — and confidently so.
Step 3 — locate the decision boundary. The boundary sits where p = 0.5, which means z = 0:
b0 + b1*x = 0
-4 + 2*x = 0
x = 2
So any input above x = 2 is predicted positive, anything below it negative, and x = 3 sits one full unit into positive territory — consistent with the 88 percent figure. Notice the slope's role: b1 = 2 controls how sharply the probability swings as x crosses the boundary. A larger b1 makes the S-curve steeper and the model more decisive near x = 2.
Common mistakes
Treating the output score as a probability before the sigmoid. The linear score z is not a probability — it is unbounded and can be negative. Only after passing through the sigmoid does the value become a probability between 0 and 1. Skipping that step is a common slip when reading model internals.
Assuming the 0.5 threshold is sacred. Classifying at p = 0.5 is a convention, not a requirement. If a missed positive is far costlier than a false alarm, lower the threshold. Logistic regression hands you a probability precisely so you can choose the cutoff that fits the problem.
Interpreting coefficients as probabilities. A coefficient b1 acts on the log-odds, not on p directly. Because the sigmoid is nonlinear, the same one-unit change in x shifts the probability a lot near the boundary and very little out at the flat tails.
Fitting it like a linear regression. There is no closed-form least-squares solution. The coefficients come from maximising likelihood, solved iteratively. A learning rate that is too large can make the optimisation diverge instead of settle.
Forgetting that perfectly separable data breaks the fit. If a single feature splits the two classes cleanly with no overlap, the maximum-likelihood coefficients run off toward infinity. Regularisation, such as an L2 penalty, keeps them finite and the model stable.
Try the interactive NovaSolver calculator
Reading the sigmoid formula is one thing; watching a decision boundary form is another. The Logistic Regression (2D Binary Classifier) Simulator on NovaSolver fits a sigmoid model to two-dimensional data by gradient descent and shows the linear decision boundary it learns. You can adjust the learning rate, the number of iterations, and the L2 regularization strength, then read off the training accuracy, the final cross-entropy loss, and the fitted weight and bias — a direct, hands-on view of how the optimisation behaves.
Related calculators
- Linear Regression Simulator — the continuous-output cousin, where least squares fits a straight line instead of an S-curve.
- Regression Analysis & Curve Fitting — for fitting polynomial and other nonlinear trends to numeric data.
- Gaussian Naive Bayes Classifier Simulator — a contrasting probabilistic classifier that models each class with its own distribution.
Browse the full set in the math and statistics tools hub.
Closing note
Logistic regression earns its place by doing one thing cleanly: it converts a linear weighted sum into a probability you can act on. The sigmoid is the whole trick — an S-curve that bounds the output, with z = 0 marking the decision boundary. Compute the score, squeeze it through the sigmoid, and read the probability; then choose a threshold that respects the real costs of each kind of error. Master this model and you have not just a reliable classifier but also the basic unit from which neural networks are built.
Top comments (0)