Binning a continuous variable throws information away. That is not a criticism, it is the definition — the question is how much, and whether what you get in return is worth it. The amount is calculable, and for the most common case it is startlingly large.
The cost, as a number
Take the most common form of binning: splitting a continuous predictor at its median into two groups. Jacob Cohen worked out the consequence in “The Cost of Dichotomization”, published in Applied Psychological Measurement in 1983. For a bivariate normal pair, dichotomising one variable at its median attenuates the correlation by a factor of sqrt(2/pi) ≈ 0.798.
That factor is exact, and it comes from the point-biserial correlation between a variable and a median split of a normally distributed partner. The consequence for a study is easier to feel in terms of sample size: statistical power for detecting a correlation scales with the square of the effect size, so an attenuation of 0.798 in the correlation corresponds to 0.798² ≈ 0.637 of the information. Median-splitting a normally distributed predictor costs roughly the same as discarding 36% of your rows.
The split point makes it worse, not better, if it is not the median. A split at the 80th percentile leaves one group with a fifth of the rows, and the attenuation grows as the split moves toward either tail. Royston, Altman and Sauerbrei laid out the full case against the practice in “Dichotomizing continuous predictors in multiple regression: a bad idea”, published in Statistics in Medicine in 2006, including the point that a data-chosen cutpoint inflates the apparent significance of the result because the cutpoint was selected using the outcome.
A worked continuous column
Consider an age column on a churn model, ages 18 to 80, where the true relationship with churn probability is smooth and monotone: about 6% at 20, rising to about 22% at 70. Bin it into three buckets — under 35, 35 to 55, over 55 — and ask what the model can still see.
Inside the middle bucket, every row is now identical to the model. A 36-year-old and a 54-year-old receive the same value. The relationship between age and churn within a 20-year span is not small; on the stated curve, moving from 36 to 54 is about a third of the total effect across the whole range. That third is now unrecoverable, because the information was destroyed before the model saw it.
import numpy as np, pandas as pd
from sklearn.metrics import roc_auc_score
rng = np.random.default_rng(0)
age = rng.uniform(18, 80, size=50_000)
p = 0.06 + (age - 20) * (0.22 - 0.06) / 50 # smooth, monotone
y = rng.binomial(1, np.clip(p, 0, 1))
binned = pd.cut(age, [17, 35, 55, 81], labels=[0, 1, 2]).astype(int)
print("auc, raw age: ", round(roc_auc_score(y, age), 4))
print("auc, 3 bins: ", round(roc_auc_score(y, binned), 4))
print("auc, median cut:", round(roc_auc_score(y, (age > np.median(age)).astype(int)), 4))
The ordering of those three numbers is the point, and it is forced by the mechanism rather than by the seed: raw beats three bins, which beats a median cut, because each step down coarsens a monotone relationship the metric was already able to use. Run it on your own column and the gaps will differ; the ordering will not, as long as the underlying relationship is monotone.
The boundary problem
Binning does not just lose resolution evenly. It concentrates the loss at the cut points and creates a discontinuity where none existed. A customer aged 34 years 11 months and one aged 35 years 1 month are assigned different categories and can receive materially different predictions, while the 36-to-54 pair receive identical ones. The model now has a step function where the world has a slope.
This has a practical consequence that outlives the modelling discussion: if the bin edges are visible in a decision — a pricing tier, an eligibility threshold — they become a target. People and systems arrange themselves just on the favourable side of a published boundary, and the distribution shifts around the edge in a way that a smooth function would not have invited.
Three cases where binning earns its place
- The relationship is genuinely non-monotone and you are fitting a linear model. If risk is high for the very young and the very old and low in the middle, a linear term for age is worse than useless — it fits a slope through a U. Bins let a linear model express the shape. A spline expresses it better and without the discontinuity, but bins are far easier to explain, which is not nothing.
- The measurement is not trustworthy at full resolution. Self-reported income, a sensor with drift, a field that is sometimes estimated and sometimes measured. Binning here is not discarding information; it is declining to model noise as if it were signal. The bins should be wide enough to swamp the measurement error.
- The output must be a small number of explainable bands. Credit scorecards are built this way deliberately. Weight-of-evidence binning trades accuracy for a model that a regulator, an adjudicator and a customer can all follow, and in that setting the trade is the requirement rather than a compromise.
Notice what is absent from that list: “the model handles it better”. Gradient boosted trees already bin internally — histogram-based implementations discretise each feature into a few hundred bins to make split-finding fast — and they choose the edges using the loss. Pre-binning a feature into three buckets before handing it to a boosted tree replaces its 255 loss-optimised bins with your three arbitrary ones. It is strictly worse.
If you are going to bin, bin properly
Choose edges from domain meaning where domain meaning exists — the legal drinking age, the retirement age, the point where a warranty expires. These are real discontinuities, and a bin edge that sits on one is describing the world rather than flattening it.
Where no such meaning exists, prefer quantile bins over equal-width bins. Equal-width bins on a skewed column put 95% of the rows in the first bucket and produce a feature that is nearly constant; quantile bins keep the counts balanced, which keeps every bin’s estimate reasonably precise. In scikit-learn that is KBinsDiscretizer(strategy="quantile"), and it must be fitted inside the cross-validation fold like any other transformer.
Never choose the cutpoint by searching for the one that maximises the association with the target and then reporting the resulting p-value as if the cutpoint had been fixed in advance. That is the specific error the Royston paper names, and it manufactures significance out of the search.
Finally: keep the raw column alongside the binned one and let the model decide, unless the reason for binning was explainability — in which case the raw column defeats the purpose. Everything about the resulting encoding, and how it interacts with the ordering of the bins, is the subject of ordinal versus nominal encoding, because a set of bins is an ordinal feature and encoding it as an unordered category throws away a second helping of the same information.
Top comments (0)