How Logistic Regression Works: Beginner to Pro Guide

Anish Rane — Tue, 06 Oct 2026 03:45:00 +0000

TL;DR: Logistic regression made simple: sigmoid, probabilities and maximum likelihood, with a real diabetes prediction example anyone can follow.

Logistic regression is a supervised machine learning model which is built on the linear regression formula that which is why it is called a regression model. This model is mainly used for classification tasks where the goal is to predict the probability that an instance belongs to a given class or not. Logistic Regression is a statistical algorithm which analyses the relationship between two data factors

Types of Logistic Regression

Binomial Regression

Logistic regression is used for binary classification where we use sigmoid function that takes the input as an independent variable and produces a probability value between 0 and 1
Where we use sigmoid fuction as the activiation

Multinomial Regression

In multinomial regression there can be 3 or more possible unordered types of dependent variable such a ‘bike’, ‘sedan’, ‘SUV’, etc
Where we use the softmax fuction as the activation

Ordinal Regression

In an ordinal logistic regression there can be 3 or more possible ordered types of dependent variable such as “Low”, “Medium”, or “High”.
Here to we use the softmax function as the activation

But before you put your data through, logistic regression comes with a few ground rules:

Independent Observations : Each data point should stand on its own. Think of them like coworkers at a virtual meeting, no side chats or shared secrets.
Binary Dependent Variable : The outcome should be binary (0 or 1). If your target variable has more than two classes (say, cat/dog/hamster), it’s time to invite the softmax function to the party.
Linearity of Log Odds : There must be a linear relationship between the independent variables and the log-odds of the dependent variable. In plain English, the predictors should line up nicely when mapped to the odds of your outcome, not too wild, not too weird.
No Outliers, Please : Logistic regression is a bit of a neat freak. Outliers can throw off the model’s mojo, so make sure your dataset is clean and well-behaved.
Go Big or Go Home : A large sample size is key. With more data, the model learns better and performs more reliably. Small datasets can lead to shaky predictions and models that get cold feet.

If linear regression is the friendly neighborhood line of best fit, then logistic regression is its more decisive cousin who only deals in black and white or rather, zeros and ones. At the heart of logistic regression lies the sigmoid activation function, a smooth, S-shaped curve that takes your wild, wandering predictions and gently nudges them into binary clarity.

Unlike linear regression, which aims to predict continuous values by fitting a straight line through your data, binomial logistic regression is in the business of classification. It’s here to answer yes/no questions like: “Will this email be spam?” or “Is this customer likely to churn?” Instead of fitting a line, we’re fitting probabilities bounded between 0 and 1. You could say it’s regression with commitment issues it never promises more than 1.

consider a medical issue Suppose there is a person, with a blood sugar level of 195, and you do not know whether that person has diabetes or not. What would you do then? Would you classify him/her as a diabetic or as a non-diabetic?

It features a hard threshold decision boundary at a blood sugar level of 200, with clear red and blue markers and a step function line.

based on the boundary, you may be tempted to declare this person a diabetic, but can you really do that? This person’s sugar level (195 mg/dL) is very close to the threshold (200 mg/dL), below which people are declared as non-diabetic. It is, therefore, quite possible that this person was just a non-diabetic with a slightly high blood sugar level. After all, the data does have people with slightly high sugar levels (220 mg/dL), who are not diabetics. using a simple boundary decision method would not work in this case.

Sigmoid activation fuction

this a mathamticall fuction which maps the predicted value to a probabilities

this fuction maps any real value into another value between 0 & 1 this forms a S curve

in the logistic regression we use the concept of the threshold value whihc defins the probability of either 0 or 1 and value that

Plot with a sigmoid curve, which smoothly models the probability of diabetes based on blood sugar level. The curve is centered at 200, showing a gradual transition from 0 to 1 instead of a sharp threshold.

the logistic regression considers the probility of the output. now considering the probility the equation for logistic regression:

Where:

: Probability of the outcome (e.g., having diabetes) given the input (e.g., blood sugar level)
: Intercept term
: Coefficient (slope) for the predictor
: Base of the natural logarithm

the best fitting combination of & will be the one which maxixmizes the product mentioend below

A sigmoid curve showing diabetes probability based on blood sugar level.

the equation is called the likeliy hood function

by trying different values of and , you can manipulate the shape of the sigmoid curve. At some combination of and , the ‘likelihood’ will be maximised.

find the optimal values of and such that the likelihood function is maximized? The optimisation methods used to do that are maximum likelihood estimation, or MLE

Related guides you may like

💬 Over to you

Have you used logistic regression on a real project, or did you jump straight to tree models? What was the trickiest part to get your head around: the sigmoid, log-odds or maximum likelihood?

If this helped, hit ❤️ and follow for more plain-English ML and engineering guides. New posts go out every Tuesday and Friday, and this one is part of the Machine Learning Basics series.

Originally published on ML AI Insights Hub. Find more plain-English ML guides, worked engineering examples and free Python/Excel calculators at mlaiinsightshub.blog.

Hi DEV 👋 I'm Anish: a Mechanical Engineer Explaining ML & AI in Plain English

Anish Rane — Mon, 05 Oct 2026 03:45:00 +0000

Hi DEV community! I'm Anish Rane, a mechanical engineer from India who moved into machine learning. I recently finished an MSc in Machine Learning & AI at Liverpool John Moores University.

I started writing because most ML tutorials either drown you in maths or skip the why. I write the guides I wish I'd had when I switched from engineering to AI: plain English, worked examples, and code you can actually run.

What you'll find here

🧠 Machine Learning Basics: linear and logistic regression, regularization, activation functions. Each one is explained from first principles and comes with Python/scikit-learn examples.

🤖 Fine-Tuning LLMs: how to choose a base model, and when to use LoRA, QLoRA, prompt-tuning or RLHF, depending on your hardware and data.

⚙️ Powertrain Engineering: how diesel engines and epicyclic gears work, and why duty cycles decide engine life. This is classic mechanical engineering, with a data-driven angle.

Who it's for

Students and early-career engineers getting into ML
Mechanical/automotive engineers curious about AI
Anyone who wants the intuition before the equations

Free tools

I also build free Python and Excel engineering calculators. You can find them, along with every guide, at 👉 mlaiinsightshub.blog.

Let's connect

🌐 Portfolio: anishrane-cox.github.io/Portfolio
💻 GitHub: github.com/AnishRane-cox
🦋 Bluesky: @mlaiinsightshub.bsky.social

New posts go out every Tuesday and Friday. Follow along, and tell me in the comments: what ML or engineering topic should I explain next?