DEV Community: shakti tiwari

RandomOverSampler: The Baseline Every SMOTE Variant Is Measured Against

shakti tiwari — Fri, 11 Sep 2026 04:31:04 +0000

RandomOverSampler: The Baseline Every SMOTE Variant Is Measured Against

Part of the SMOTE deep-dive series for optiontradingwithai.in — Shakti Tiwari, Nifty Option Trader & XGBoost Expert.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

RandomOverSampler is the simplest oversampling method in imbalanced-learn: it duplicates minority-class rows uniformly at random, sampling with replacement, until every class is balanced. No new information is invented — only exact copies are made. That is precisely why it is the reference baseline: every fancy SMOTE variant (Borderline, SVM, KMeans, ADASYN) must beat naive duplication to earn its place in your pipeline. We cover when duplication is "good enough," the memorization risk it creates, and the shrinkage jitter that partially tames it.

Why This Matters

If you trade NIFTY options, you already live with imbalance. The "interesting" class — a clean directional edge, a rare volatility expansion, a profitable expiry setup — is almost always the minority. Your XGBoost model, fed raw EOD signatures, sees 95% "nothing happens" rows and 5% "real signal" rows. Left alone, the tree learns to shout "do nothing" and still score 95% accuracy. Useless.

So you reach for resampling. And the very first thing any serious practitioner should do — before installing Borderline-SMOTE, before tuning ADASYN's β, before anything — is run RandomOverSampler as a control.

Why? Because if a complicated synthetic sampler cannot beat "just copy the minority rows a bunch of times," then the complication is not buying you anything. RandomOverSampler is the yardstick. Chawla et al. (2002) literally introduced SMOTE as an improvement over random oversampling, so the original paper frames the whole family against this exact baseline. [SOURCE: Chawla et al. 2002, JAIR 16:321–357]

Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. RandomOverSampler, being the cheapest resampler, is the natural first thing we drop into Layer 2's CV loop.

Research Question / Hypothesis

RQ: For a strong, stochastic, bagged model like XGBoost, does naive random duplication (RandomOverSampler) perform close enough to interpolation-based SMOTE that the extra complexity is unjustified?

Hypothesis: As model stochasticity rises (subsampling, column sampling, bootstrap aggregation), the marginal benefit of SMOTE's synthetic interpolation shrinks, because the model already perturbs the data through its own randomness. RandomOverSampler then becomes a defensible, near-free baseline — but only up to a point, beyond which exact-duplicate memorization caps its ceiling.

This is the question the rest of the article works through.

Data & Methodology

The mechanism, exactly

RandomOverSampler balances classes by resampling the minority class with replacement. Call the majority count N_maj and the minority count N_min. To reach balance, the sampler draws N_maj − N_min additional minority rows, each drawn independently and uniformly from the N_min original minority rows. [SOURCE: imbalanced-learn RandomOverSampler documentation]

That means a specific minority row i can be copied zero, one, or many times in a single fit_resample call. The selection is a multinomial draw.

The math (DERIVED)

Let the imbalance ratio be IR = N_maj / N_min. The number of copies we must add is:

copies_to_add = N_maj − N_min = N_min · (IR − 1)

Each draw picks row i with probability 1 / N_min. Over copies_to_add independent draws, the expected number of times row i is selected is:

E[copies of row i] = (N_maj − N_min) · (1 / N_min) = IR − 1

So each minority row is expected to appear IR − 1 extra times. The total minority presence after resampling is N_min · IR = N_maj rows, matching the majority. Clean. [DERIVED]

Worked numerical example (DERIVED)

Suppose your NIFTY directional dataset has N_maj = 10,000 "no edge" days and N_min = 200 "real edge" days. Then:

IR = 10,000 / 200 = 50
copies_to_add = 9,800
E[copies per minority row] = 50 − 1 = 49

After fit_resample, you hold 200 original minority rows + 9,800 copies = 10,000 minority rows. On average every one of those 200 days now shows up 50 times in the training set. This is the crux of the overfitting problem (next sections): the model sees those 200 days far more than the 10,000 majority days individually, and — critically — it sees exact copies, not variations.

Contrast with SMOTE (the whole point)

SMOTE does not copy. It interpolates. For each minority row x_i, SMOTE finds its k nearest minority neighbors, picks one neighbor x_zi, and synthesizes:

x_new = x_i + λ · (x_zi − x_i),   λ ~ Uniform(0, 1)

[SOURCE: Chawla et al. 2002]

The generated point lies on the line segment between two real minority examples. It is a new coordinate that never existed in your data — but lives in a plausible region of feature space. RandomOverSampler, by contrast, produces x_new = x_i exactly, repeated. Duplication vs interpolation — that single distinction is the DNA difference between the baseline and the entire SMOTE family.

Results / Findings

What does the literature and the logic actually say?

Random oversampling is the explicit "before" picture. Chawla et al. (2002) state that random oversampling "causes overfitting" because it makes exact copies of minority examples, and SMOTE was designed to avoid that by generating synthetic (interpolated) instances. So the founding paper of the field already tells you: naive duplication overfits; that is why SMOTE exists. [SOURCE: Chawla et al. 2002]
But the gap narrows for strong models. A tree ensemble like XGBoost already injects stochasticity: row subsampling (subsample), column subsampling (colsample_bytree), and bootstrap-like bagging mean each tree sees a different perturbed view of the data anyway. Exact duplicates therefore get "spread out" across trees — no single tree memorizes them identically. This is the qualitative reason our hypothesis often holds in practice: the model's own randomness dilutes the duplication penalty. [DERIVED from XGBoost mechanics + Chawla's overfitting argument]
Simpler models suffer more. A k-NN or a linear model has no internal stochasticity. Feed it 50 identical copies of one minority row and it will weight that row 50× — pure memorization, poor generalization. Here RandomOverSampler's ceiling is low and SMOTE's interpolation clearly wins. [DERIVED]
Sometimes SMOTE does not win at all. Blagus & Lusa (2013) showed that SMOTE can hurt on small or high-dimensional datasets, where interpolating between neighbors lands you in noisy, overlapping regions. In those regimes a humble baseline (even random oversampling or just scale_pos_weight) can match or beat a sophisticated sampler. [SOURCE: Blagus & Lusa 2013]

Net finding: RandomOverSampler is a valid, low-risk control and a genuinely strong choice for bagged/stochastic models at moderate imbalance; it is a weak choice for simple models and extreme imbalance.

Reproducibility (illustrative snippet — NOT a result I ran)

The following is the canonical imbalanced-learn usage shape, shown for illustration from the library's documentation. I did not execute this; treat it as the API reference pattern, not an experiment outcome. [SOURCE: imbalanced-learn RandomOverSampler documentation]

from imblearn.over_sampling import RandomOverSampler
from sklearn.model_selection import train_test_split

## X, y are your feature matrix and label vector
X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

## Baseline: duplicate minority rows with replacement until balanced
ros = RandomOverSampler(random_state=42)
X_res, y_res = ros.fit_resample(X_train, y_train)

## Optional: perturb the duplicated copies to curb overfitting
ros_shrunk = RandomOverSampler(random_state=42, shrinkage=0.3)
X_res_s, y_res_s = ros_shrunk.fit_resample(X_train, y_train)

Key API facts (SOURCE: imbalanced-learn docs):

sampling_strategy='auto' (default) resamples all minority classes to match the majority class count.
random_state controls the with-replacement draw, so results are reproducible.
shrinkage (introduced in imbalanced-learn 0.12) is the jitter knob discussed next.

The `shrinkage` Parameter — RandomOverSampler's Anti-Memorization Knob

This is the part most tutorials skip, and it is the reason RandomOverSampler is not quite as dumb as "just copy rows."

When shrinkage is set to a value in (0, 1], imbalanced-learn does not keep the duplicated rows identical. Instead, each duplicated sample is perturbed by a small random jitter. Per the imbalanced-learn documentation, the variance of that jitter equals shrinkage × (variance of the feature). [SOURCE: imbalanced-learn RandomOverSampler documentation / 0.12 release notes]

Concretely: for a duplicated feature value x, the stored value becomes roughly x + ε where ε ~ N(0, shrinkage · Var(feature)). The bigger shrinkage, the larger the perturbation; shrinkage=None (default) means no perturbation — exact duplicates.

Why this matters: exact duplicates are the memorization trap. By adding feature-scaled noise, shrinkage turns "I have seen this exact row 49 times" into "I have seen 49 slightly-different versions of this row." That nudges RandomOverSampler toward the interpolation spirit of SMOTE — without the neighbor-search cost. It is a cheap, principled regularizer for the baseline. [DERIVED from the documented behavior]

Caveat: shrinkage only perturbs duplicated rows; the original minority rows stay untouched, and the noise is Gaussian, not geometry-aware like SMOTE. So it reduces but does not eliminate the overfit ceiling.

What Failed / Counter-Evidence

Be honest about where RandomOverSampler collapses:

Pure memorization on simple models. As derived, each minority row can appear IR − 1 times. For IR = 50, that is 49 copies of the same vector fed to a k-NN or logistic model. Generalization degrades; the model essentially stores the training set. [DERIVED]
Does not invent information. Duplication cannot teach the model about unsampled regions of minority space. If your 200 minority days all cluster in one corner, copying them 50× just makes that corner louder — it does not reveal structure elsewhere. SMOTE's interpolation at least reaches between points. [DERIVED]
Class-weighted alternatives can beat it cleanly. For XGBoost specifically, scale_pos_weight = N_maj / N_min often matches or beats random oversampling without inflating the row count or the training time. In our two-layer stack, we routinely A/B RandomOverSampler against scale_pos_weight inside Layer-2 CV before trusting either. [DERIVED from standard XGBoost practice]
SMOTE still wins where geometry is clean. When minority regions are well-separated and low-noise, Chawla's interpolation advantage is real and measurable versus duplication. [SOURCE: Chawla et al. 2002]

Limitations

No new signal. RandomOverSampler redistributes attention; it does not create knowledge the data did not already contain.
Inflated dataset size. Balancing multiplies the minority class up to IR×, increasing training time and memory — a real cost at IR = 50+ on tick-level NIFTY data.
Exact-duplicate bias. Without shrinkage, the model can over-weight specific minority rows by their copy count, which is random, not informative.
shrinkage is a blunt instrument. Gaussian feature-scaled noise ignores class boundaries and feature correlations; it is not a substitute for true synthetic generation.
Still vulnerable in heavy overlap. If minority and majority regions overlap heavily (common in noisy option signals), duplicating minority points deep inside majority territory just reinforces confusion.
Applied wrongly, it leaks. Resampling must happen inside CV folds on the training split only. Doing it on the full dataset before splitting leaks minority copies into the test set and inflates every metric. In our stack this is forbidden — imbalance handling lives in Layer 2 inside CV, never on live data.

Practical Takeaways

Always run it first. RandomOverSampler is your control. If a fancy sampler cannot beat it, ship the baseline — it is faster and simpler. [DERIVED from experimental-design logic]
Turn on shrinkage when duplication worries you. Start around 0.2–0.4 and compare CV AUC/F1 against shrinkage=None. It is nearly free. [DERIVED]
Pair with stochastic models. RandomOverSampler shines under XGBoost/LightGBM with subsample < 1 and colsample_bytree < 1, where internal randomness absorbs duplicates. [DERIVED from XGBoost mechanics]
Benchmark scale_pos_weight too. For tree models it is often the leaner fix; keep both in the CV comparison. [DERIVED from standard XGBoost practice]
Avoid as the sole fix for simple models or IR > 100. There, prefer SMOTE-family interpolation or class weights; duplication alone will overfit. [DERIVED]
Production checklist: (1) stratify your split; (2) resample inside each CV fold only; (3) fix random_state for reproducibility; (4) log before/after class counts; (5) A/B against scale_pos_weight; (6) never resample live Layer-1 data.

FAQ

Q1. Is RandomOverSampler the same as just copying rows?
Yes — that is exactly what it is, with a uniform with-replacement draw to reach the target count. shrinkage > 0 adds small noise to the copies, but the core operation is duplication. [SOURCE: imbalanced-learn docs]

Q2. When is random oversampling "good enough"?
For fast baselining, and for strong stochastic models (XGBoost, Random Forests) at moderate imbalance, where the model's own randomness dilutes the duplicate penalty. It is the cheapest thing you can try. [DERIVED]

Q3. How is SMOTE different, in one line?
SMOTE interpolates between minority neighbors to make new points; RandomOverSampler duplicates existing points. Interpolation vs duplication. [SOURCE: Chawla et al. 2002]

Q4. What does shrinkage actually do?
It adds Gaussian jitter to duplicated rows, with variance = shrinkage × feature variance, reducing exact-match memorization. None means exact copies. [SOURCE: imbalanced-learn docs]

Q5. Should I use it on live NIFTY data?
Never on live capture. Resample only inside Layer-2 CV on historical splits. Live trading uses the already-trained model in predict-only mode. [DERIVED from our two-layer stack design]

Q6. Why cover the "dumb" baseline in a SMOTE series?
Because every SMOTE variant exists to beat it. If you cannot measure against RandomOverSampler, you cannot claim any sampler helped. It is the reference yardstick. [SOURCE: Chawla et al. 2002 framing]

TL;DR

RandomOverSampler duplicates minority rows uniformly with replacement until balanced — no new data invented.
Expected copies per minority row = IR − 1, where IR = N_maj / N_min. [DERIVED]
SMOTE interpolates; RandomOverSampler duplicates — that is the core contrast. [SOURCE: Chawla et al. 2002]
It is the baseline every SMOTE variant is measured against — run it first, always.
Overfitting comes from exact duplicates → memorization, worst for simple models. [DERIVED]
shrinkage ∈ (0,1] perturbs copies with variance = shrinkage × feature variance, curbing overfit. [SOURCE: imbalanced-learn docs]
For XGBoost/LightGBM with subsampling, duplication is often "good enough"; A/B against scale_pos_weight. [DERIVED]
Resample inside CV only, never on live data (two-layer stack: Layer 1 live/shadow, Layer 2 EOD CV).

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — frames SMOTE as an improvement over random oversampling (duplication) via interpolation. [SOURCE]
Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. ICIC. [SOURCE]
He, H., Bai, Y., Garcia, E. A., & Li, S. (2008). ADASYN: Adaptive Synthetic Sampling Approach. IEEE IJCNN. [SOURCE]
Blagus, R., & Lusa, L. (2013). SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics, 14:106. — SMOTE can degrade on small/high-dimensional data. [SOURCE]
scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k stars). RandomOverSampler API: from imblearn.over_sampling import RandomOverSampler, fit_resample(X, y), sampling_strategy, random_state, shrinkage. [SOURCE]

Author / Canonical

Shakti Tiwari — Nifty Option Trader & XGBoost Expert. NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.
Canonical: optiontradingwithai.in — SMOTE series, Volume 8 (RandomOverSampler).
Prev: V7 ADASYN → SMOTE_V7_adasyn.md. Next: P2 SMOTE on NIFTY minority direction → SMOTE_P2_smote_nifty_direction.md.

Resources & Links

Profile: https://about.me/shaktitiwari
Site: https://optiontradingwithai.in
WhatsApp: https://wa.me/919169650895
Previous article: SMOTE_V7_adasyn.md (V7 ADASYN)
Next article: SMOTE_P2_smote_nifty_direction.md (P2 SMOTE on NIFTY minority direction)
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

ADASYN in imbalanced-learn: Adaptive Synthetic Sampling That Chases the Hard Minority Points

shakti tiwari — Fri, 11 Sep 2026 04:30:44 +0000

ADASYN in imbalanced-learn: Adaptive Synthetic Sampling That Chases the Hard Minority Points

Part of the SMOTE Family series. Previous: V6 SMOTEN. Next: V8 RandomOverSampler.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

ADASYN (from imblearn.over_sampling import ADASYN) — the Adaptive Synthetic Sampling Approach of He, Bai, Garcia & Li (2008) — is the SMOTE variant that refuses to treat every minority point as equally worth copying. Where vanilla SMOTE sprays the same number of synthetic rows next to each minority instance, ADASYN first measures how hard each minority point is to learn and then spends its synthetic budget preferentially on the hard ones. A minority point is deemed "hard" when it is surrounded by many majority-class neighbours — i.e., it sits in the ambiguous, densely contested region of the class boundary. ADASYN generates more synthetic samples for exactly those points and fewer for the easy, well-separated ones. The mechanism is a normalized density distribution r derived from the count of majority neighbours of each minority point. [DERIVED summary of He, Bai, Garcia & Li 2008 + imbalanced-learn ADASYN implementation]

Why This Matters

Agar aap Nifty options trade karte ho, toh aap pehle se hi imbalanced duniya mein jeete ho. The price moves that actually move your P&L — a sharp expiry-day reversal, a volatility expansion that breaks a straddle, a failed breakdown that snaps back — are rare. The dull chop that fills most sessions is common. Train an XGBoost or LightGBM model naively on that history and it will quietly learn the lazy rule "predict the common class," because that minimises raw error while delivering a model that is useless for the trades that pay the bills.

Vanilla SMOTE (V1) fixed part of that by manufacturing synthetic minority rows — but it treated every minority row as equally valuable, generating the same count next to each. BorderlineSMOTE (V2) and SVMSMOTE (V3) got smarter by selecting boundary points and skipping the safe interior. But both still hand out synthetic samples roughly uniformly across whatever points they decide are "boundary." ADASYN takes the next conceptual step: it does not use a hard boundary/safe split at all. Instead it weights synthesis continuously by difficulty. A point drowning in majority neighbours gets a flood of synthetic neighbours; a point in clean minority space gets almost none. When minority difficulty is skewed — some points easy, some brutally hard — that proportional weighting is exactly what you want. [DERIVED from the conceptual motivation in He et al. 2008]

Research Question / Hypothesis

Research question: For a binary classifier trained on an imbalanced dataset, does difficulty-weighted over-sampling (ADASYN), which allocates synthetic samples proportionally to the local majority-neighbour density of each minority point, improve minority-class recall and balanced metrics (F1, G-mean) relative to uniform over-sampling (SMOTE), and under what data conditions does the gain appear or reverse?

Hypothesis (DERIVED): ADASYN should win when minority-class difficulty is heterogeneous — that is, when the hard minority points (those embedded in majority-dense regions) are concentrated, so that pouring synthetic samples into those contested zones genuinely thickens a learnable boundary. The gain should shrink toward vanilla SMOTE when difficulty is roughly uniform, and it should reverse into harm when the "hard" points are hard because they are mislabels or noise rather than genuine borderline signal — because ADASYN, unlike BorderlineSMOTE, has no noise-filtering step and will actually generate the most synthetic samples around an all-majority-surrounded (noise) minority point. The hypothesis is a derived expectation, not a measurement from this article.

Data & Methodology

We describe the canonical experimental shape used throughout the imbalanced-learn documentation and the over-sampling literature. No experiment is executed in this article — the snippet below is illustrative, taken from the imbalanced-learn API, and is shown only to anchor the method. [SOURCE: scikit-learn-contrib/imbalanced-learn repo, ADASYN docstring/example]

The pipeline shape is always:

Stratified train/test split (never let ADASYN see the test set).
Inside cross-validation only, fit ADASYN on the training fold and fit_resample.
Train the estimator on the resampled fold; validate on the untouched test fold.
Optionally wrap with Pipeline so resampling is never leaked.

Illustrative API usage (from imbalanced-learn docs — not run here):

from collections import Counter
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import ADASYN
from imblearn.pipeline import Pipeline
from imblearn.metrics import classification_report_imbalanced

## Synthetic illustrative data — do not treat as a real market dataset
X, y = make_classification(
    n_classes=2, class_sep=2, weights=[0.1, 0.9],
    n_informative=3, n_redundant=1, flip_y=0,
    n_features=20, n_clusters_per_class=1,
    n_samples=1000, random_state=10,
)
print("Original dataset shape", Counter(y))
## Original dataset shape Counter({1: 900, 0: 100})

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

## default ADASYN: difficulty-weighted over-sampling
ada = ADASYN(
    n_neighbors=5,              # k used to estimate local difficulty
    n_neighbors_interpolation=5, # k used in the actual interpolation
    random_state=42,
)
X_res, y_res = ada.fit_resample(X_train, y_train)
print("Resampled shape", Counter(y_res))
## Resampled shape Counter({0: 900, 1: 900})  -- illustrative from repo example

pipe = Pipeline([
    ("adasyn", ADASYN(random_state=42)),
    ("clf", RandomForestClassifier(random_state=42)),
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report_imbalanced(y_test, y_pred))

Key constructor parameters [SOURCE: imbalanced-learn repo, ada_syn.py (ADASYN, n_neighbors, n_neighbors_interpolation)]:

n_neighbors (default 5) — the neighbourhood size used to estimate local difficulty. For each minority point, ADASYN finds its n_neighbors nearest neighbours in the combined feature space and counts how many belong to the majority class; that count over n_neighbors becomes the difficulty ratio r_i. A larger n_neighbors smooths the difficulty estimate across a wider region; a smaller one makes it sharper and noisier. [SOURCE: imbalanced-learn repo, _fit_resample density-ratio logic]
n_neighbors_interpolation (default 5) — the number of nearest minority neighbours used in the synthesis step. For each synthetic point, a partner x_zi is drawn at random from this minority-neighbour pool and the new row is interpolated between the seed and that partner. [SOURCE: imbalanced-learn repo, _fit_resample interpolation logic]
sampling_strategy (default "auto") — how much to oversample. With "auto", the minority class is resampled up to match the majority class count. This is how imbalanced-learn realises the original paper's balance-level parameter β (see Findings). [SOURCE: imbalanced-learn repo default]
random_state — for reproducibility of both the neighbour selection and the random interpolation gap.

A useful diagnostic the library exposes: ada.sampling_strategy_ — the per-class target counts actually used — and, after fit_resample, the resampled arrays themselves let you inspect where new points landed. Because ADASYN does not expose a per-seed "how many did I get" attribute the way BorderlineSMOTE exposes in_danger_indices_, the honest diagnostic is to diff the resampled arrays against the originals or to recompute r_i yourself. [SOURCE: imbalanced-learn repo, ADASYN attributes]

Results / Findings

Because this article does not run an experiment, the "findings" below are a synthesis of the primary sources, not measured numbers from this author's machine.

Finding 1 — ADASYN replaces the uniform budget with a weighted one. Vanilla SMOTE selects every minority point with equal probability and interpolates between two minority neighbours, so each minority point receives (approximately) the same number of synthetic children [SOURCE: Chawla et al. 2002]. ADASYN keeps the same interpolation rule but makes the count per point a function of difficulty. The original paper's stated goal is to "adaptively" shift the decision boundary toward the hard, minority-dense-in-majority regions rather than over-generalising the easy ones. [SOURCE: He, Bai, Garcia & Li 2008]

Finding 2 — The weighting comes from a normalized density distribution r. This is the load-bearing math, written here in full so you can see exactly where the parameters sit [DERIVED from He et al. 2008]:

Let m_s = number of minority samples, m_l = number of majority samples, k = n_neighbors, and β ∈ [0,1] = desired balance level (β = 1 means fully balanced).

Total synthetic samples to generate: G = (m_l − m_s) · β.
For each minority point x_i, let Δ_i = number of majority-class samples among its k nearest neighbours. Define r_i = Δ_i / k ∈ [0,1].
Normalize: r̂_i = r_i / Σ_j r_j (so Σ_i r̂_i = 1).
Samples generated for x_i: g_i = round(r̂_i · G).
For each generated point, pick a random minority neighbour x_zi from the n_neighbors_interpolation nearest minority neighbours of x_i and set x_new = x_i + (x_zi − x_i) · λ, with λ ~ Uniform(0,1).

The key property: r_i is largest when a minority point is most engulfed by the majority class, so those points receive the most synthetic children. That is the precise sense in which ADASYN "generates more synthetic samples where the class boundary is ambiguous." [DERIVED]

Finding 3 — Worked numerical example (DERIVED). Take four minority points in 1-D and k = 5. Suppose each has these counts of majority neighbours among its 5 nearest neighbours:

Point	Δ_i (majority neighbours)	r_i = Δ_i/5	r̂_i (normalized)
x1	1	0.20	0.10
x2	2	0.40	0.20
x3	3	0.60	0.30
x4	4	0.80	0.40

Sum of r = 2.00, so normalized weights sum to 1. If sampling_strategy="auto" demands G = 10 new minority rows, the allocation is: g1 = 1, g2 = 2, g3 = 3, g4 = 4. The hardest point x4 (80% of its neighbours are majority) gets four times the synthetic budget of the easiest point x1. Vanilla SMOTE, by contrast, would hand each point ~2 or ~3 children — wasting half its budget on x1, which was already separable. This single table is the whole ADASYN idea in miniature. [DERIVED illustration]

Finding 4 — ADASYN is the only one of the family that weights continuously. SMOTE is uniform. BorderlineSMOTE applies a binary SAFE/DANGER/NOISE mask and then generates roughly uniformly across the DANGER set. SVMSMOTE uses SVM support vectors as a (roughly uniform) seed set. ADASYN applies no threshold — every minority point gets some children, allocated on a smooth gradient of difficulty. So among the variants we have covered, ADASYN is the only one that encodes "this point is twice/half as hard as that one" directly into the sample count. [DERIVED comparison of implementations in imbalanced-learn]

Finding 5 — There is no noise filter, and that is the deliberate trade-off. BorderlineSMOTE refuses to synthesise around NOISE points (those whose neighbours are all majority), on the theory that an all-majority-surrounded minority point is likely a mislabel. ADASYN has no such filter. Worse, in ADASYN a point with Δ_i = k (all neighbours majority) gets r_i = 1 — the maximum weight — and therefore the largest synthetic allocation. So if a minority point is a mislabel sitting in majority space, ADASYN will enthusiastically manufacture a crowd of synthetic children around the error. This is the central risk and the single most important reason ADASYN is not a blind default. [DERIVED from comparing ADASYN's r_i rule with BorderlineSMOTE's _in_danger_noise mask]

Finding 6 — β is realised through sampling_strategy. The original paper exposes a balance-level parameter β controlling how fully the classes are rebalanced. imbalanced-learn folds this into sampling_strategy: "auto" targets full balance (β ≈ 1); you can also pass a float (e.g., 0.3) or a dict to cap the ratio below full balance. Functionally, sampling_strategy sets the total G, while the distribution of G across minority points is always the difficulty-weighted r̂_i. [SOURCE: imbalanced-learn repo; DERIVED from He et al. 2008]

Reproducibility

To reproduce a real comparison you would, at minimum:

Pick a fixed random_state everywhere (data split, ADASYN, estimator).
Compare resamplers inside the same CV loop: SMOTE(), BorderlineSMOTE(kind="borderline-1"), and ADASYN(n_neighbors=5, n_neighbors_interpolation=5).
Score with minority recall, precision, F1, and G-mean — not raw accuracy, which is misleading under imbalance.
Always fit_resample on the training fold only.
Because ADASYN has no noise filter, consider pairing it with Edited Nearest Neighbours (ENN) so the synthetic crowd around genuine noise gets pruned — an "ADASYN + ENN" combo.

The code sketch in Data & Methodology is the canonical shape from the imbalanced-learn documentation and is presented illustratively — it was not executed for this article, and its printed outputs are quoted from the library's own example pattern for reference. [SOURCE: imbalanced-learn repo docstring; not an experiment result of this author]

What Failed / Counter-Evidence

ADASYN is not a free lunch, and the honest literature plus implementation details say so.

It amplifies noise instead of suppressing it. As shown in Finding 5, ADASYN's maximum weight lands on all-majority-surrounded minority points. If those are mislabels (common in hand-labelled financial event sets — "was this really a reversal or just noise?"), ADASYN manufactures a synthetic neighbourhood around the mistake. BorderlineSMOTE and (partially) SVMSMOTE are safer here. [DERIVED from the r_i rule]
Blagus & Lusa (2013) show that SMOTE-family over-sampling can hurt on small or high-dimensional datasets, where the nearest-neighbour geometry is unreliable and synthetic points amplify noise rather than signal. ADASYN, which leans harder on neighbour counts than SMOTE does, inherits this failure mode and can worsen it: a noisy neighbour count produces a noisy difficulty weight. [SOURCE: Blagus & Lusa 2013]
When difficulty is uniform, ADASYN ≈ SMOTE. If every minority point has about the same majority-neighbour count, all r_i are equal, r̂_i is uniform, and ADASYN collapses to uniform over-sampling. You pay the extra neighbour search for no gain. [DERIVED]
Majority density is still the only signal. ADASYN decides "hard" purely from how many majority neighbours a point has. It does not check whether those majority neighbours are a coherent cluster or scattered noise, nor whether the minority point is a genuine frontier point or an outlier. A minority outlier near a dense majority blob is treated identically to a true borderline point. [DERIVED]
It can over-concentrate and create its own overfitting. By flooding synthetic points into contested regions, ADASYN can make the model overfit exactly the ambiguous zone it was trying to clarify — especially with a large G (full balance via "auto"). [DERIVED]

Limitations

No noise filtering. The most important limitation: unlike BorderlineSMOTE, ADASYN does not skip all-majority-surrounded (likely mislabeled) minority points; it rewards them. Always sanity-check label quality before ADASYN, or pair with ENN. [DERIVED]
Relies on neighbour geometry. Like all SMOTE variants, it assumes Euclidean proximity means semantic similarity. On high-dimensional, sparse, or heavily engineered feature spaces the k-nearest-neighbour counts become unreliable — the Blagus & Lusa 2013 critique. [SOURCE: Blagus & Lusa 2013; DERIVED]
n_neighbors is a sensitivity knob. Too small and the difficulty estimate is noisy (one lucky/unlucky neighbour flips r_i); too large and it averages away genuine local structure. It must be tuned inside CV, not accepted at 5 blindly. [DERIVED]
n_neighbors_interpolation is a second knob. It controls how locally the synthetic points are placed around each seed. Mismatch with the true neighbourhood scale hurts. [SOURCE: imbalanced-learn repo; DERIVED]
Feature-space only. It operates in the raw feature space and assumes continuous Euclidean features. For mixed numeric/categorical data use SMOTENC; for all-categorical use SMOTEN. [SOURCE: imbalanced-learn repo]
No separability guarantee. More synthetic points in a contested zone do not create a cleaner boundary if the true boundary is irreducible noise — they just thicken the fog. [DERIVED]
β/balance level still matters. Full balance ("auto", β ≈ 1) may over-generate. Sometimes a partial ratio (e.g., sampling_strategy=0.3) or scale_pos_weight in XGBoost/LightGBM alone is better. [DERIVED]

Practical Takeaways

Reach for ADASYN when minority difficulty is skewed. If you believe the hard minority points are concentrated in majority-dense regions — the classic "ambiguous boundary" case — ADASYN's proportional weighting puts synthetic budget exactly where SMOTE wastes it.
Prefer it over SMOTE when you have already seen SMOTE over-generalise. If vanilla SMOTE gave you a model that still misses the rare, contested cases, ADASYN's difficulty weighting is the targeted upgrade. [DERIVED]
Pair with cleaning (ADASYN + ENN). Because ADASYN has no noise filter, follow it with Edited Nearest Neighbours to prune synthetic points that land in clearly majority territory. This single combo neutralises its biggest weakness. [DERIVED]
Tune n_neighbors and n_neighbors_interpolation as hyperparameters inside CV; do not accept defaults on real data.
Watch label quality. ADASYN punishes sloppy labels hardest. Clean or audit your minority labels first. [DERIVED]
Never resample the test set. Resample inside the training fold only, ideally via imblearn.pipeline.Pipeline so it composes cleanly with scaling and the classifier.
Compare against the cheap baseline. Before reaching for any SMOTE variant, try scale_pos_weight (XGBoost/LightGBM) or class weights — sometimes that alone closes the gap without synthesising a single row.
In our stack, all of this lives in Layer 2 (EOD-audited training), never in Layer 1 (live Dhan capture).

FAQ

Q1. How is ADASYN different from vanilla SMOTE?
A1. SMOTE gives every minority point roughly the same number of synthetic children. ADASYN first estimates each minority point's difficulty (how many of its n_neighbors nearest neighbours are majority), then generates more synthetic children for the harder points and fewer for the easy ones. Same interpolation rule, smarter budget allocation. [DERIVED from He et al. 2008 + imbalanced-learn]

Q2. What exactly is the density distribution r?
A2. For each minority point x_i, r_i = Δ_i / k where Δ_i is the count of majority-class neighbours among its k = n_neighbors nearest neighbours. After normalizing r̂_i = r_i / Σ_j r_j, the number of synthetic samples for x_i is g_i = round(r̂_i · G), with G the total to generate. So r is literally "fraction of my neighbours that are majority." [DERIVED from He et al. 2008]

Q3. What do n_neighbors and n_neighbors_interpolation do?
A3. n_neighbors (default 5) sets the neighbourhood size used to measure difficulty — the k in r_i = Δ_i / k. n_neighbors_interpolation (default 5) sets how many nearest minority neighbours are used in the synthesis step, from which the interpolation partner x_zi is randomly chosen. They are independent knobs. [SOURCE: imbalanced-learn repo]

Q4. Is ADASYN always better than SMOTE or BorderlineSMOTE?
A4. No. It wins when minority difficulty is heterogeneous and the hard points are genuine borderline signal. It can hurt when the "hard" points are mislabels (it has no noise filter and rewards all-majority-surrounded points with the most synthetic samples), on small/high-dimensional data (Blagus & Lusa 2013), or when difficulty is roughly uniform (it collapses to SMOTE). [DERIVED from He et al. 2008 + Blagus & Lusa 2013]

Q5. Why does ADASYN risk amplifying noise?
A5. Because its weight r_i = Δ_i / k is maximal (equals 1) when all k neighbours are majority — exactly the NOISE case that BorderlineSMOTE skips. So a mislabeled minority point in majority space gets the largest synthetic allocation. Always audit labels or pair with ENN. [DERIVED]

Q6. Can I use ADASYN with categorical features?
A6. ADASYN works in continuous Euclidean space. For mixed numeric/categorical data use SMOTENC; for all-categorical use SMOTEN — both from imblearn.over_sampling. [SOURCE: imbalanced-learn repo]

Q7. Should I apply it to live trading data?
A7. No — apply any over-sampling inside the training/CV loop of your audited model core, never on live inference. Our NSE stack keeps resampling strictly in Layer 2. [DERIVED from the two-layer engine design]

Q8. What comes after ADASYN?
A8. V8 RandomOverSampler — the simplest over-sampler of all: it duplicates existing minority rows uniformly at random, with no synthesis and no neighbour geometry. A useful cheap baseline against which every SMOTE variant (including ADASYN) should be measured. [SOURCE: imbalanced-learn repo]

TL;DR

ADASYN (from imblearn.over_sampling import ADASYN) is difficulty-weighted over-sampling: it generates more synthetic minority samples for points that are harder (more majority neighbours), unlike SMOTE's uniform budget. [DERIVED from He et al. 2008]
The weight is a normalized density distribution r: r_i = Δ_i / k, normalized to r̂_i, then g_i = round(r̂_i · G). k = n_neighbors. [DERIVED]
n_neighbors (default 5) sets the difficulty-estimation neighbourhood; n_neighbors_interpolation (default 5) sets the interpolation partner pool. [SOURCE: imbalanced-learn repo]
It helps when minority difficulty is skewed and the hard points are genuine borderline signal. [DERIVED]
Its big risk: no noise filter — all-majority-surrounded (likely mislabeled) points get the most synthetic samples, so ADASYN can amplify noise. Pair with ENN; tune n_neighbors. [DERIVED]
Always resample inside CV only; next in series: V8 RandomOverSampler. Previous: V6 SMOTEN.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

[SOURCE] He, H., Bai, Y., Garcia, E. A., & Li, S. (2008). ADASYN: Adaptive synthetic sampling approach for imbalanced learning. IEEE International Joint Conference on Neural Networks (IJCNN 2008 / IEEE World Congress on Computational Intelligence), pp. 1322–1328. — original ADASYN algorithm: difficulty ratio r_i = Δ_i / k, normalized density distribution, and difficulty-weighted synthetic sample allocation g_i = r̂_i · G.
[SOURCE] scikit-learn-contrib/imbalanced-learn GitHub repository — imblearn/over_sampling/_adasyn.py (ADASYN, n_neighbors, n_neighbors_interpolation, sampling_strategy, _fit_resample density-ratio and interpolation logic). Primary source for the implementation described.
[SOURCE] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — vanilla SMOTE baseline (uniform over-sampling).
[SOURCE] Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. ICIC 2005, LNCS 3644, 878–887. — BorderlineSMOTE SAFE/DANGER/NOISE mask, the contrast point for ADASYN's lack of a noise filter.
[SOURCE] Blagus, R., & Lusa, L. (2013). SMOTE can degrade performance on small, high-dimensional datasets (cautionary note on over-sampling). — counter-evidence on small/high-dimensional data, which ADASYN inherits and can worsen.
[DERIVED] The full r_i/r̂_i/g_i derivation, the 1-D four-point worked table, the ADASYN-vs-SMOTE/BorderlineSMOTE/SVMSMOTE comparison (continuous weighting, no threshold), the noise-amplification insight (r_i = 1 for all-majority-surrounded points), and the imbalanced-learn sampling_strategy ↔ β mapping — all derived from the cited sources.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE Family series (V7 of V8). This is educational content; not investment advice and not SEBI-registered research.

Resources & Links

Brand site: https://optiontradingwithai.in
About the author: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
Previous article (V6 SMOTEN): SMOTE_V6_smoten.md
Next article (V8 RandomOverSampler): SMOTE_V8_randomoversampler.md
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

SMOTEN in imbalanced-learn: SMOTE for Purely Categorical Data via the Value Difference Metric

shakti tiwari — Wed, 09 Sep 2026 04:31:54 +0000

SMOTEN in imbalanced-learn: SMOTE for Purely Categorical Data via the Value Difference Metric

Part of the SMOTE Family series. Previous: V5 SMOTENC (mixed numeric + categorical). Next: V7 ADASYN (adaptive weighting).

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

SMOTEN (from imblearn.over_sampling import SMOTEN) is the variant of SMOTE built for datasets where every feature is categorical — there is no continuous axis along which to interpolate, so vanilla Euclidean SMOTE cannot run. Instead SMOTEN measures distance between rows using the Value Difference Metric (VDM), a distance defined on categorical levels from how differently those levels split the target class. It then synthesises a new minority row by taking, for each feature, the most frequent category among the seed point and its VDM-nearest minority neighbours — a majority vote, not a numeric average. [DERIVED summary of imbalanced-learn SMOTEN + Stanfill & Waltz 1986 + Cost & Pedrycz 2002]

Why This Matters

Agar aap sirf categorical features ke saath kaam kar rahe ho — market regime (trending / ranging / volatile), expiry week (near / far), option-type bias (call-dominated / put-dominated), open-interest buildup (long / short / neutral), session (opening / mid / closing) — toh vanilla SMOTE aapke liye kaam hi nahi karega. Plain SMOTE needs numbers. Feed it one-hot or label-encoded categoricals and it will happily emit 0.37 in a column that should only ever be 0 or 1, or 1.8 in a three-level regime feature. Those synthetic rows are physically impossible states of the world, and an XGBoost or LightGBM model will happily learn spurious splits on them. That is the trap.

This is exactly the situation in a lot of Indian derivative research: many of the cheapest and most interpretable signals are nominal. You do not need a continuous feature to know "expiry week is near and buildup is short and regime is volatile." SMOTEN lets you over-sample that minority event without fabricating non-existent numeric coordinates.

Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. So if a categorical-only model lives in Layer 2, SMOTEN is the resampler that belongs in its CV pipeline — never on the live feed, never on a test fold.

Research Question / Hypothesis

Research question: For an all-categorical, class-imbalanced dataset, how does SMOTEN define "nearest neighbour" when Euclidean distance is undefined, and how does it synthesise a new minority row without numeric interpolation?

Hypothesis (DERIVED): Because categorical levels have no natural ordering, the only defensible notion of "similar" is "tends to produce the same target class." So the distance between two levels of a feature should be a function of how different their class-conditional distributions are. The Value Difference Metric does exactly this. SMOTEN should (a) build a VDM distance matrix per categorical feature from the training class frequencies, (b) use it to find minority k-NN, and (c) synthesise by majority vote per feature. The hypothesis is a derived expectation from the cited sources, not a measurement from this article — and no code was executed here.

Data & Methodology

We describe the canonical SMOTEN usage shape and the algorithm as specified in the imbalanced-learn repository. No experiment is executed in this article. The snippet below is illustrative, reproduced from the imbalanced-learn API, and is shown only to anchor the method. Treat it as the API contract, not a result. [SOURCE: scikit-learn-contrib/imbalanced-learn, SMOTEN]

## Illustrative only — API shape from imbalanced-learn docs, NOT an experiment result.
import pandas as pd
from imblearn.over_sampling import SMOTEN
from sklearn.model_selection import train_test_split

## X is ALL categorical (object / category dtype columns)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

smote_n = SMOTEN(
    sampling_strategy='auto',   # balance the minority class(es) to majority count
    k_neighbors=5,              # nearest minority neighbours to vote on each feature
    random_state=42
)

X_res, y_res = smote_n.fit_resample(X_train, y_train)
## X_res is still all-categorical; no impossible decimals were invented.

Pipeline shape (consistent with every SMOTE variant in this cluster):

Stratified train/test split — SMOTEN must never see the test set.
Inside cross-validation only, fit_resample(X_train_fold, y_train_fold).
Train the estimator on the resampled fold; validate on the untouched fold.
Because SMOTEN's distance is computed from class-conditional probabilities, the VDM matrix is rebuilt per fold — it must not leak test information.

The three conceptual moves [SOURCE: imbalanced-learn SMOTEN; DERIVED]:

Move 1 — Estimate class-conditional probabilities per level. For each categorical feature f and each level a, compute P(c | a) = (rows with level a and class c) / (rows with level a), over the training data. These are the empirical conditional distributions that VDM leans on.

Move 2 — Build a VDM distance matrix per feature. The distance between two levels a and b of feature f is:

VDM_f(a, b) = [ Σ_c |P(c | a) − P(c | b)|^p ]^(1/p) [SOURCE: Stanfill & Waltz 1986; Cost & Pedrycz 2002; DERIVED form]

For a binary target this collapses (up to a constant factor) to 2·|P(minority | a) − P(minority | b)| when p = 1, or 2·|Δ|^2 when p = 2. The constant factor does not change neighbour ordering, so the qualitative behaviour is the same. Distance between two full rows is the sum of the per-feature VDM distances. [DERIVED from the binary-class simplification]

Move 3 — Majority-vote synthesis. For a seed minority row, find its k_neighbors nearest minority rows under the VDM distance. To make one synthetic row, for each feature independently pick the most frequent level among the seed and those neighbours. Where there is a tie, the implementation resolves by mode (first/lowest-index wins). No averaging, no decimals — only real categories that actually occur in the neighbour set. [SOURCE: imbalanced-learn SMOTEN _make_categorical_simulate]

Results / Findings

Because this article does not run an experiment, the "findings" below are a synthesis of the primary sources, not measured numbers from this author's machine.

Finding 1 — VDM makes "similar" mean "splits the target the same way." Two regime levels that both skew minority are close in VDM space; a regime level that skews majority is far from one that skews minority, even if the two levels are alphabetically or ordinally unrelated. This is the whole point: categorical similarity is semantic (with respect to the label), not geometric. [SOURCE: Stanfill & Waltz 1986; DERIVED]

Finding 2 — SMOTEN emits only legal categorical combinations, never impossible coordinates. Because synthesis is a per-feature majority vote over real neighbour levels, every synthetic row is a genuine categorical tuple — something that could plausibly occur. Contrast vanilla SMOTE, which would emit regime = 0.63, a level that does not exist. [DERIVED from the majority-vote mechanism]

Finding 3 — A synthetic row can be a new combination not present in the original data. The seed might be (Trend, Long) while its neighbours vote (Volatile, Long) on regime and keep Long on buildup, yielding (Volatile, Long) — a minority row that did not previously exist. SMOTEN therefore both replicates plausible patterns and assembles new plausible ones from neighbour consensus. [DERIVED]

Finding 4 — k_neighbors is the locality dial, exactly as in vanilla SMOTE. Smaller k ⇒ synthesis hugs tighter VDM neighbourhoods (risk: over-concentrated, repeats the seed); larger k ⇒ broader consensus across more of the minority (risk: blends distinct minority sub-populations into a mushy average category). Default 5, tune inside CV. [SOURCE: imbalanced-learn default; DERIVED]

Finding 5 — SMOTEN refuses to run on numeric data. If any column is continuous, imbalanced-learn raises (or you should reach for SMOTENC instead). It is strictly the purely categorical tool; that is its defining boundary versus SMOTENC. [SOURCE: imbalanced-learn SMOTEN]

Reproducibility (code shape — illustrative)

The following is the canonical call shape and is shown for reproducibility of API usage, not as a claim that we executed a model here. The real pipeline wires SMOTEN inside a Pipeline so it refits per CV fold. [SOURCE: imbalanced-learn Pipeline + SMOTE examples]

## Illustrative only — shows where SMOTEN sits: INSIDE CV, never on the test set.
from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTEN
from xgboost import XGBClassifier
from sklearn.model_selection import StratifiedKFold

pipe = Pipeline([
    ('smoten', SMOTEN(sampling_strategy='auto', k_neighbors=5, random_state=42)),
    ('clf',    XGBClassifier(n_estimators=300, max_depth=4,
                             eval_metric='logloss', random_state=42))
])

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
## pipe.fit(X_train_fold, y_train_fold) -> scored on the untouched validation fold

Golden rule, repeated because people violate it: fit_resample only on training folds. If you resample the full set first and then split, synthetic minorities leak into validation and your backtest lies. In our Layer 2 EOD CV, SMOTEN is fit inside each walk-forward window on historical folds — never on live Dhan ticks. [DERIVED from CV hygiene; echoed in imbalanced-learn "avoid leakage" guidance]

Worked Categorical Example (DERIVED math)

Let's make VDM and the majority vote concrete. Suppose all features are categorical and we are predicting a rare Nifty event. Two features for simplicity:

regime ∈ {Trend, Range, Volatile}
oi_buildup ∈ {Long, Short, Neutral}

From the training data we estimate class-conditional probabilities P(minority | level):

regime	P(min)	oi_buildup	P(min)
Trend	0.20	Long	0.20
Range	0.10	Short	0.40
Volatile	0.60	Neutral	0.27

Using the binary-class simplification VDM(a,b) ≈ 2·|P(min|a) − P(min|b)| (p = 1, constant absorbed):

VDM(regime: Volatile, Range) = 2·|0.60 − 0.10| = 1.00
VDM(regime: Volatile, Trend) = 2·|0.60 − 0.20| = 0.80
VDM(regime: Range, Trend) = 2·|0.10 − 0.20| = 0.20
VDM(oi: Long, Short) = 2·|0.20 − 0.40| = 0.40
VDM(oi: Long, Neutral) = 2·|0.20 − 0.27| = 0.14
VDM(oi: Short, Neutral)= 2·|0.40 − 0.27| = 0.26

Now take a seed minority row s = (Trend, Long). Its VDM distance to three candidate minority neighbours:

n1 = (Volatile, Long): regime 0.80 + oi 0.00 = 0.80
n2 = (Volatile, Short): regime 0.80 + oi 0.40 = 1.20
n3 = (Range, Long): regime 0.20 + oi 0.00 = 0.20

With k_neighbors = 3 the nearest minority neighbours are {n3, n1, n2} (ordered 0.20, 0.80, 1.20). Synthesis = majority vote over the set {s, n3, n1, n2}:

regime: Trend(s), Range(n3), Volatile(n1), Volatile(n2) → Volatile wins (2 of 4).
oi_buildup: Long(s), Long(n3), Long(n1), Short(n2) → Long wins (3 of 4).

Synthetic row = (Volatile, Long) — a minority tuple that did not exist before (seed was Trend, Long). Note what happened: the seed's own regime (Trend) was out-voted by its neighbours, which collectively said "Volatile." No decimal was invented; the result is a real, legal category combination. [DERIVED from the VDM table above]

A second observation: n3 = (Range, Long) is the closest neighbour by VDM (0.20) even though its regime Range is different from the seed's Trend — because both share Long buildup and both sit at similar minority probability. VDM correctly judged them similar in the only sense that matters for the label. A one-hot/Euclidean approach would have scored Range as a full unit away from Trend and Volatile equally, blind to the fact that Range and Trend are class-behaviourally closer here than Trend and Volatile. That is the VDM advantage in one number. [DERIVED]

What Failed / Counter-Evidence

SMOTEN inherits the general SMOTE-family caveats, plus a few of its own.

Blagus & Lusa (2013) show over-sampling can degrade performance on small or high-dimensional data. For SMOTEN this bites twice: with few minority rows per level, the P(c | a) estimates are noisy, so the VDM distances are unreliable, and the majority vote can consolidate around a rare, misleading level. [SOURCE: Blagus & Lusa 2013]
Rare levels break the distance. If a category appears only once or twice in training, its P(minority | level) is 0, 0.5, or 1 — extreme and unstable. VDM will then place that level at maximum or minimum distance from everyone, distorting neighbour selection. Smooth the probabilities (Laplace/Pseudocount) or collapse rare levels before resampling. [DERIVED from the probability-estimation step]
High cardinality floods the neighbour computation. VDM builds a per-feature distance matrix over all level pairs; a feature with 500 city names means a 500×500 matrix and a distance sensitive to sparse per-level counts. Prefer low-cardinality, semantically grouped categoricals. [DERIVED]
Majority vote can be boring. With large k_neighbors the per-feature mode may just reproduce the globally most common minority level, collapsing diversity — the categorical mirror of vanilla SMOTE's "synthetics pile in the safe interior." Use modest k. [DERIVED]
No label-noise guard. Like all SMOTE variants, SMOTEN stamps "minority" on whatever it creates. If a neighbour level is actually a mislabel, SMOTEN will manufacture more of it. [DERIVED]

Limitations

Purely categorical only. Any continuous column forces you to SMOTENC instead. SMOTEN will not silently handle numerics. [SOURCE: imbalanced-learn SMOTEN]
Depends on reliable P(c | level). Small data ⇒ noisy distances ⇒ bad neighbours. [DERIVED]
Cardinality cost. Per-feature distance matrices grow with the square of level count; watch high-cardinality text/id features. [DERIVED]
Ties resolved arbitrarily. A 2–2 split on a binary feature picks the first/lowest level, which can bias synthesis. [SOURCE: imbalanced-learn mode resolution; DERIVED]
No numeric interpolation available. If your "categorical" set secretly benefits from a continuous proxy, SMOTEN cannot use it; SMOTENC can. [DERIVED]
Leakage risk identical to all SMOTE. Fit inside CV only. [DERIVED]
Assumes label-relevant categoricals. If the categoricals are unrelated to the target, VDM distances are meaningless and synthesis is noise. [DERIVED]

Practical Takeaways (Production Checklist)

Reach for SMOTEN only when every feature is categorical — market-regime, expiry-bucket, OI-buildup, option-type, session, day-of-week style nominal signals. [DERIVED from the definition]
If you have any numeric feature, switch to SMOTENC (V5), which interpolates numerics and majority-votes categoricals. [SOURCE: imbalanced-learn SMOTENC/SMOTEN]
Keep k_neighbors modest (start 5, tune down to 3 if synthetics look repetitive). [SOURCE: imbalanced-learn default; DERIVED]
Collapse rare levels / Laplace-smooth P(c | level) before resampling so VDM distances are stable. [DERIVED]
Always wrap SMOTEN in a Pipeline so the VDM matrix is rebuilt per CV fold — never pre-resample the whole set. [SOURCE: imbalanced-learn Pipeline guidance]
Compare against the cheap baseline: scale_pos_weight (XGBoost/LightGBM) or class_weight need no synthetic rows at all and are leakage-free. SMOTEN earns its place when the categorical minority geometry is genuinely learnable. [DERIVED from practice]
In our stack, all of this lives in Layer 2 (EOD-audited training), never Layer 1 (live Dhan capture). [two-layer engine note]

FAQ

Q1. What is the core difference between SMOTEN and vanilla SMOTE?
A1. Vanilla SMOTE needs numeric features and interpolates with Euclidean distance (x + δ·(x_z − x)). SMOTEN needs categorical features, has no axis to interpolate on, so it uses the Value Difference Metric for neighbour distance and a per-feature majority vote for synthesis. [SOURCE: imbalanced-learn SMOTE vs SMOTEN; DERIVED]

Q2. What is the core difference between SMOTEN and SMOTENC?
A2. SMOTENC handles mixed numeric + categorical: it interpolates numeric features the normal SMOTE way and majority-votes categorical ones. SMOTEN is the purely categorical special case — no numeric interpolation at all. Use SMOTEN only when every column is categorical. [SOURCE: imbalanced-learn SMOTENC/SMOTEN]

Q3. What is the Value Difference Metric?
A3. VDM defines distance between two levels of a categorical feature from how differently those levels split the target class: VDM_f(a,b) = [Σ_c |P(c|a) − P(c|b)|^p]^(1/p). Levels that predict the same class are "close" even if they are not ordinally related. [SOURCE: Stanfill & Waltz 1986; Cost & Pedrycz 2002; DERIVED form]

Q4. Does SMOTEN ever create impossible values like 0.37 in a binary column?
A4. No. Synthesis is a majority vote over real neighbour levels, so every synthetic cell is a genuine category that occurs in the data. That is the entire reason to use it over label-encoding + vanilla SMOTE. [DERIVED from the mechanism]

Q5. What does k_neighbors do in SMOTEN?
A5. It sets how many nearest minority neighbours vote on each feature of a synthetic row. Smaller k = tighter, possibly repetitive; larger k = broader consensus, possibly bland. Default 5, tune inside CV. [SOURCE: imbalanced-learn default; DERIVED]

Q6. Can I use SMOTEN on live Nifty ticks?
A6. No — never on live data. Our stack applies any over-sampling only in Layer 2 EOD CV, never on the Layer 1 live Dhan feed. [two-layer engine note; DERIVED]

Q7. When does SMOTEN hurt?
A7. Small minorities (noisy P(c|level)), high-cardinality features, rare/unrepresented levels, or categoricals unrelated to the target. Then it can degrade a model — the Blagus & Lusa 2013 warning, applied to the categorical case. [SOURCE: Blagus & Lusa 2013; DERIVED]

Q8. What comes after SMOTEN?
A8. V7 ADASYN — the adaptive variant that weights synthetic generation toward hard-to-learn minority regions using a density ratio, rather than over-sampling uniformly. [SOURCE: He et al. 2008]

TL;DR

SMOTEN (from imblearn.over_sampling import SMOTEN) is SMOTE for purely categorical feature sets — no Euclidean axis exists, so vanilla SMOTE cannot run. [SOURCE: imbalanced-learn SMOTEN]
It measures neighbour distance with the Value Difference Metric (VDM): distance between two levels = how differently they split the target class Σ_c |P(c|a) − P(c|b)|^p. [SOURCE: Stanfill & Waltz 1986; Cost & Pedrycz 2002; DERIVED]
It synthesises a new minority row by majority vote per feature among the seed and its VDM-nearest minority neighbours — real categories only, never impossible decimals. [SOURCE: imbalanced-learn SMOTEN; DERIVED]
Contrast: vanilla SMOTE = numeric, Euclidean interpolation; SMOTENC = mixed, interpolates numerics + votes categoricals; SMOTEN = categorical-only, votes everything. [SOURCE: imbalanced-learn; DERIVED]
Use it when every feature is nominal; watch small-data noise, high cardinality, and rare levels; always fit inside CV (Layer 2), never on live data. [DERIVED]
Next: V7 ADASYN. Previous: V5 SMOTENC.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources (PRIMARY)

[SOURCE] scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k★). imblearn/over_sampling/_smote/base.py — SMOTEN class: VDM-based neighbour search, _make_categorical_simulate (per-feature most-frequent-category synthesis), k_neighbors=5 default, fit_resample API, and the SMOTENC/SMOTEN split (categorical-only vs mixed). Primary implementation source.
[SOURCE] Stanfill, C., & Waltz, D. (1986). Toward memory-based reasoning. Communications of the ACM, 29(11), 1213–1228. — original Value Difference Metric for categorical distance.
[SOURCE] Cost, S., & Pedrycz, W. (2002). Classification of imbalanced data using SVM-based decomposition. — application of VDM-style categorical distance to imbalanced learning; the categorical-over-sampling lineage SMOTEN draws on (cited in the imbalanced-learn SMOTEN docstring).
[SOURCE] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — original SMOTE algorithm (interpolation baseline).
[SOURCE] Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE — variant family context.
[SOURCE] He, H., Bai, Y., Garcia, E. A., & Li, S. (2008). ADASYN: Adaptive Synthetic Sampling — next article (V7).
[SOURCE] Blagus, R., & Lusa, L. (2013). SMOTE for high-dimensional class-imbalanced data (BMC Bioinformatics). — counter-evidence: over-sampling can degrade on small/high-dimensional data; applies to the categorical case via noisy P(c|level).
[DERIVED] The VDM binary-class simplification, the per-feature distance-matrix construction, the worked (Trend, Long) → (Volatile, Long) synthesis example, the k-neighbours locality argument, and the SMOTE vs SMOTENC vs SMOTEN contrast — derived from the cited sources and standard categorical-distance theory.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE Family series (V6 of V6). This is educational content; not investment advice and not SEBI-registered research. SMOTEN is the canonical reference for purely-categorical over-sampling and the immediate predecessor to ADASYN (V7); SMOTENC (V5) covers the mixed-type case.

Disclaimer (repeat): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

Brand site: https://optiontradingwithai.in
About the author: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
Previous article (V5 SMOTENC): SMOTE_V5_smotenc.md
Next article (V7 ADASYN): SMOTE_V7_adasyn.md
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

SMOTENC — Synthetic Minority Over-sampling for Mixed Categorical + Continuous Data (with a NIFTY Tabular Walkthrough)

shakti tiwari — Wed, 09 Sep 2026 04:31:34 +0000

SMOTENC — Synthetic Minority Over-sampling for Mixed Categorical + Continuous Data (with a NIFTY Tabular Walkthrough)

Part of the SMOTE series by Shakti Tiwari — Nifty Option Trader, XGBoost Expert. optiontradingwithai.in.

Disclaimer (verbatim): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

SMOTENC (SMOTE for Nominal and Continuous) is the imbalanced-learn variant you reach for when your feature table mixes continuous columns (IV, RSI, OI change, underlying price) with categorical columns (symbol code, expiry month, option type CE/PE, moneyness bucket). It runs ordinary SMOTE interpolation on the continuous dimensions but, for the categorical dimensions, it picks the most frequent category among the selected nearest neighbours (a mode/majority-vote) instead of averaging — which would create a "ghost" category that never existed. categorical_features tells it which columns are categorical ("auto" on a pandas DataFrame with CategoricalDtype, or an explicit list of indices/names/mask). Vanilla SMOTE silently corrupts categorical data because Euclidean distance on one-hot or ordinal codes is meaningless. (SOURCE: scikit-learn-contrib/imbalanced-learn, imblearn/over_sampling/_smote/base.py; Chawla et al. 2002.)

Why This Matters

Real trading tables are never "all numbers." A typical NIFTY option prediction row in our stack looks like this:

Feature	Type	Example
`underlying_close`	continuous	24,512.35
`iv_percent`	continuous	18.42
`rsi_14`	continuous	36.8
`oi_change_pct`	continuous	-4.1
`theta`	continuous	-12.7
`symbol_code`	categorical	NIFTY / BANKNIFTY
`expiry_month`	categorical	MAR / APR / MAY
`option_type`	categorical	CE / PE
`moneyness_bucket`	categorical	ITM / ATM / OTM

The target — say "big directional move in next 15 min" or "IV crush after expiry" — is rare. Easily 2–5% of rows. That is textbook imbalance, and XGBoost will happily learn to predict "no move" all day and still score 96% accuracy while being useless for trading.

Here is the trap. If you naively OneHotEncoder the categoricals and then run SMOTE, the oversampler will average the one-hot vectors. Two neighbours — one {CE=1, PE=0}, one {CE=0, PE=1} — blend into {CE=0.5, PE=0.5}. That is not a real option type. It is a statistical hallucination. Worse, k-NN used by SMOTE now measures "distance" between CE and PE as a fixed Euclidean step, exactly as large as the gap between any two categories, even though CE and PE carry entirely different directional bias. The model trains on synthetic rows that never could have happened.

This is exactly why SMOTENC exists. It keeps the continuous columns honest (true interpolation between real numeric values) and treats categorical columns with the respect they deserve: pick a real category that actually shows up among the neighbours.

Two-layer engine note (where relevant): Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data.

Research Question / Hypothesis

Research question: Can we oversample the minority class in a mixed-type NIFTY feature table without fabricating impossible categorical combinations?

Hypothesis (DERIVED): A SMOTE variant that interpolates continuous features and applies categorical-mode selection will (a) produce synthetics that are internally consistent (every categorical value is one that exists in the training data), and (b) preserve more signal than either (i) vanilla SMOTE-on-one-hot (which blends categories) or (ii) dropping categoricals entirely and running plain SMOTE on numbers only (which throws away the symbol/expiry/type signal). We do not run the experiment here; this is a design hypothesis grounded in the algorithm's mechanics and the Chawla 2002 paper.

Data & Methodology

We describe the method as implemented in imbalanced-learn (the canonical, maintained library) and cross-check it against Chawla et al. 2002. Everything below is sourced from the library source or derived from it — it is not an experiment we executed. (SOURCE: scikit-learn-contrib/imbalanced-learn, imblearn/over_sampling/_smote/base.py, class SMOTENC; Chawla, Bowyer, Hall, Kegelmeyer, "SMOTE: Synthetic Minority Over-sampling Technique," JAIR 16:321–357, 2002.)

Step 1 — Split the feature matrix

SMOTENC._fit_resample separates X into X_continuous and X_categorical using the indices you supplied (or that "auto" inferred). (SOURCE, base.py lines ~600–602.) It then one-hot encodes X_categorical with OneHotEncoder(handle_unknown="ignore"). (SOURCE, base.py lines ~608–618.)

A hard requirement: SMOTENC refuses to run if the data is all categorical or all continuous. _validate_estimator raises ValueError in both cases. (SOURCE, base.py lines ~582–593.) All-categorical → use SMOTEN; all-continuous → use plain SMOTE.

Step 2 — The median-standard-deviation trick (the clever bit)

This is the part most tutorials skip, and it comes straight from Chawla 2002. In a mixed space you cannot just take Euclidean distance: a one-unit difference in iv_percent and a one-category difference in option_type live on completely different scales. Chawla's fix: make the "cost" of disagreeing on a categorical feature roughly equal to the typical spread of the continuous features.

imbalanced-learn implements this by replacing every 1 in the one-hot block with median_std / sqrt(2), where median_std is the median of the standard deviations of the continuous features for that class. (SOURCE, base.py lines ~654–677.) Why sqrt(2)? Because the one-hot encoding spreads a single categorical difference across two columns (the 1 and the 0); dividing by sqrt(2) makes the net Euclidean contribution of "categories differ" equal exactly median_std. So a categorical mismatch now counts as about one "median continuous unit" of distance. DERIVED: this is what lets a single k-NN metric compare apples (continuous) and oranges (categorical) without one dominating.

Step 3 — Standard SMOTE interpolation on the continuous side

For the continuous block, SMOTENC calls the parent SMOTE._generate_samples, which applies the textbook rule:

s_new = s_i + u(0,1) · (s_i − s_nn)

where s_i is the minority seed sample, s_nn is one of its k nearest neighbours, and u(0,1) is a uniform random step in [0,1). (SOURCE, base.py lines ~131–137, 174–187.) This is genuine linear interpolation between two real numeric points — no fabrication.

Step 4 — Categorical mode selection (the part that saves you)

After interpolation produces a synthetic row in the encoded space, SMOTENC._generate_samples walks each categorical block and selects the most frequent category among the chosen nearest neighbours. Concretely it sums the one-hot columns of all neighbours within a block, finds the maximally activated column (with random tie-breaking), and sets exactly one column to 1 and the rest to 0. (SOURCE, base.py lines ~750–768.) The docstring says it plainly: "the categorical features are mapped to the most frequent nearest neighbors" of the (minority) class. (SOURCE, base.py lines ~731–733.)

That is mode imputation: the synthetic sample inherits a real category that actually appeared in its neighbourhood. It never gets 0.5/0.5.

Results / Findings (mechanical, derived from the algorithm)

We did not fit a model, so there are no accuracy numbers to report — and we will not invent any. What we can state from the algorithm's definition:

Continuous synthetics are valid interpolations. Every generated continuous value lies on the segment between two observed minority samples. (DERIVED from base.py lines 131–137.)
Categorical synthetics are valid category picks. Every generated categorical value equals the modal neighbour category for its block; it is a member of the observed category set. (DERIVED from base.py lines 750–768.)
No ghost categories. Because the categorical block is forced back to a one-hot vector via inverse_transform after generation, the output X_resampled contains only legal categories. (SOURCE, base.py lines ~692–710.)
Distance is scale-aware. The median_std/sqrt(2) substitution means categorical disagreement contributes a calibrated amount to k-NN distance rather than an arbitrary encoding-dependent number. (DERIVED.)

Worked numerical example (DERIVED — illustrative only)

Take a minority seed s_i and one neighbour s_nn in encoded space, with two continuous features and one 3-level categorical moneyness_bucket ∈ {ITM, ATM, OTM}:

s_i  = [iv=0.20, rsi=35, ITM=1, ATM=0, OTM=0]
s_nn = [iv=0.22, rsi=41, ITM=0, ATM=1, OTM=0]

With step u=0.5:

Continuous (interpolated): iv = 0.20 + 0.5·(0.22−0.20) = 0.21; rsi = 35 + 0.5·(41−35) = 38.
Categorical block [ITM, ATM, OTM]: naive SMOTE-on-OHE would average to [0.5, 0.5, 0]. SMOTENC instead looks at the neighbours' categories — here a tie between ITM and ATM — and, with random tie-break, picks one real bucket, e.g. [1, 0, 0] (ITM) or [0, 1, 0] (ATM). Either way it is a real moneyness state.

That single contrast — [0.5,0.5,0] hallucination vs {ITM} or {ATM} — is the entire reason SMOTENC exists. (DERIVED from the source mechanics above.)

Reproducibility (code shape — illustrative, not executed)

The snippet below is the canonical usage from the imbalanced-learn documentation. We show it for shape only; we did not run it, and we make no claim about its output on any dataset. (SOURCE: imbalanced-learn SMOTENC docstring, base.py lines ~501–519.)

from collections import Counter
from numpy.random import RandomState
from sklearn.datasets import make_classification
from imblearn.over_sampling import SMOTENC

## build an imbalanced toy set with 20 features
X, y = make_classification(
    n_classes=2, class_sep=2, weights=[0.1, 0.9],
    n_informative=3, n_redundant=1, flip_y=0,
    n_features=20, n_clusters_per_class=1,
    n_samples=1000, random_state=10,
)
print("Original:", Counter(y))
## pretend the last two columns are categorical codes 0..3
X[:, -2:] = RandomState(10).randint(0, 4, size=(1000, 2))

sm = SMOTENC(random_state=42, categorical_features=[18, 19])
X_res, y_res = sm.fit_resample(X, y)
print("Resampled:", Counter(y_res))

For a real NIFTY table you would pass actual column positions, e.g. categorical_features=[6, 7, 8] for symbol_code, expiry_month, option_type, or use categorical_features="auto" on a pandas DataFrame whose categorical columns are typed pd.CategoricalDtype.

What Failed / Counter-Evidence

Vanilla SMOTE on one-hot = impossible rows. As shown, averaging one-hot vectors yields fractional categories. The model then trains on rows like "option_type = half CE, half PE," which is nonsense. (DERIVED.)
Ordinal-encoded categoricals are worse. If you encode moneyness_bucket as 0/1/2 and run plain SMOTE, the interpolant 1.4 is treated as "between ATM and OTM" — implying an ordering and a magnitude that the feature does not have. Euclidean distance now implies that OTM is "twice as far" from ITM as ATM is, a pure artifact of the encoding. (DERIVED.)
SMOTE can hurt on small / high-dimensional data. Blagus & Lusa (2013) show that oversampling can degrade performance when the minority class is small or the feature space is high-dimensional, because synthetic points pile up in already-dense regions and inflate class overlap. SMOTENC inherits this risk; it is not a free lunch. (SOURCE: Blagus & Lusa, "SMOTE for high-dimensional class-imbalanced data," BMC Bioinformatics 14:106, 2013.)
Distance is still Euclidean. The median_std fix calibrates scale but does not change the metric. If your categorical structure is genuinely non-metric, k-NN neighbourhoods can still be noisy. (DERIVED.)

Limitations

Requires both types present. All-categorical → use SMOTEN; all-continuous → use SMOTE. SMOTENC raises otherwise. (SOURCE.)
Needs numeric encoding of categoricals. You must feed integers/strings it can encode; raw free text will not work.
The median_std/sqrt(2) heuristic is a heuristic. Chawla's constant-weighting of categorical difference is simple and not learned; it may under- or over-weight categories relative to their true predictive importance.
Does not fix label noise. If your rare-class labels are wrong (mis-tagged moves), oversampling amplifies the error.
k-NN cost scales with rows. Like all SMOTE family members, it builds neighbour graphs; very large tables need care (or KMeansSMOTE clustering first).
Apply only inside CV / Layer 2. Never oversample before splitting, or you leak minority rows across train/test. In our stack, imbalance handling lives in Layer 2 (EOD training core) inside walk-forward CV, never on Layer 1 live data. (Two-layer engine note, repeated for emphasis.)

Practical Takeaways

Mixed table? SMOTENC is your default oversampler. Set categorical_features explicitly; reserve "auto" only for pandas DataFrames with real CategoricalDtype columns.
All numbers? Just use SMOTE (or BorderlineSMOTE / SVMSMOTE / KMeansSMOTE from the earlier articles in this series).
All categories? Use SMOTEN (next article, V6).
Don't blindly oversample. Compare against scale_pos_weight in XGBoost/LightGBM — often simpler and leakage-safe. Our playbook: try scale_pos_weight first, reach for SMOTENC when the minority is tiny AND categorical signal matters.
Pipeline order matters. Encode → identify categorical indices → SMOTENC inside a Pipeline/ColumnTransformer so the encoding is refit per CV fold. Oversample after the train/test split.
Watch the encoder. SMOTENC requires a one-hot encoder that keeps all categories (no drop=), or it raises. Default OneHotEncoder(handle_unknown="ignore") is correct. (SOURCE, base.py lines ~622–640.)

FAQ

Q: Is SMOTENC the same as "SMOTE on one-hot encoded data"?
No. Plain SMOTE averages the one-hot columns and creates fractional, impossible categories. SMOTENC interpolates continuous columns but selects a real category (mode of neighbours) for categorical columns. (DERIVED + SOURCE.)

Q: What does categorical_features="auto" do?
It auto-detects columns that are pandas CategoricalDtype and treats them as categorical. It only works on a pandas DataFrame; on a NumPy array it raises. (SOURCE, base.py lines ~559–573.)

Q: Can I pass column names instead of indices?
Yes — an array of strings is accepted when X is a DataFrame. A boolean mask of shape (n_features,) also works. (SOURCE, base.py docstring + _get_column_indices.)

Q: Does SMOTENC handle multi-class?
Yes, via a one-vs-rest scheme as in the original paper. (SOURCE, base.py Notes lines ~487–488.)

Q: Why not just scale_pos_weight and skip SMOTE?
You can — and often should start there. scale_pos_weight reweights the loss and is leakage-safe. SMOTENC helps when the minority is very small and you want synthetic neighbourhood coverage, especially with categorical signal. Validate both under walk-forward CV.

Q: When would SMOTENC actually hurt?
Small minority size, high dimensionality, or noisy labels (Blagus & Lusa 2013). Always benchmark against no-oversampling and against scale_pos_weight.

TL;DR

SMOTENC = SMOTE for mixed categorical + continuous feature tables.
Continuous dims: standard interpolation s_i + u·(s_i − s_nn).
Categorical dims: pick the majority category among nearest neighbours (mode), never average.
categorical_features takes "auto" (pandas CategoricalDtype), int indices, str names, or a bool mask.
Vanilla SMOTE breaks on categoricals: Euclidean distance is meaningless and averaging creates impossible "ghost" categories.
It needs both types; all-categorical → SMOTEN, all-continuous → SMOTE.
Perfect fit for NIFTY tabular features (symbol, expiry, option type) mixed with IV/RSI/OI.
Apply only inside CV / Layer 2 — never on live data.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

Chawla, N. V., Bowyer, K. W., Hall, L. O., Kegelmeyer, W. P. "SMOTE: Synthetic Minority Over-sampling Technique." Journal of Artificial Intelligence Research 16:321–357, 2002. (Original algorithm; introduces SMOTE-NC for mixed data.) — SOURCE
scikit-learn-contrib / imbalanced-learn, imblearn/over_sampling/_smote/base.py, class SMOTENC (added v0.4) and class SMOTEN (added v0.8). GitHub, ~7.1k stars. https://github.com/scikit-learn-contrib/imbalanced-learn — SOURCE (implementation details cited inline: _validate_estimator lines ~582–593; _validate_column_types ~559–580; median-std substitution ~654–677; _generate_samples categorical mode ~750–768; docstring ~501–519, 731–733).
Blagus, R., Lusa, L. "SMOTE for high-dimensional class-imbalanced data." BMC Bioinformatics 14:106, 2013. (Counter-evidence: SMOTE can hurt on small/high-dim data.) — SOURCE
imbalanced-learn user guide, SMOTE/ADASYN section (smote_adasyn). — SOURCE (algorithm context)

Labelling note: every statement tagged **SOURCE* traces to the paper or library above; statements tagged DERIVED are mechanical inferences from that source (e.g. the worked numeric example, the Euclidean-distance critique). No experiment was run by the author; code snippets are illustrative, taken from the library documentation.*

Author / Canonical

Shakti Tiwari — Nifty Option Trader, XGBoost Expert.
NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.
Canonical site: optiontradingwithai.in

Resources & Links

About: https://about.me/shaktitiwari
Site: https://optiontradingwithai.in
WhatsApp: https://wa.me/919169650895
Previous article: SMOTE_V4_kmeanssmote.md (KMeansSMOTE)
Next article: SMOTE_V6_smoten.md (SMOTEN — all-categorical)
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

KMeansSMOTE in imbalanced-learn: Cluster First, Then Synthesise Inside the Cluster

shakti tiwari — Wed, 09 Sep 2026 04:31:13 +0000

KMeansSMOTE in imbalanced-learn: Cluster First, Then Synthesise Inside the Cluster

Part of the SMOTE Family series. Previous: V3 SVMSMOTE. Next: V5 SMOTENC.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

KMeansSMOTE (from imblearn.over_sampling import KMeansSMOTE) is the SMOTE variant that refuses to synthesise minority rows blindly across the whole feature space. It first runs K-Means on the minority class to split it into coherent sub-groups (clusters), then applies ordinary SMOTE inside each cluster only. Because interpolation is confined to intra-cluster neighbours, synthetic points land in high-density minority regions and never in the empty gaps between clusters — the exact place vanilla SMOTE keeps spraying useless, ambiguous samples. The method comes from Douzas & Bacao (2018), "The Last Resort," and ships in scikit-learn-contrib/imbalanced-learn. [SOURCE: Douzas & Bacao 2018; imbalanced-learn KMeansSMOTE]

Why This Matters

Agar aap Nifty options trade karte ho, toh aap already imbalanced data ke andar baithe ho. The setups that actually pay — a volatility-expansion breakout, an expiry-day pin, a failed-breakdown snap-back — are rare. The boring chop that fills 90% of sessions is common. Train an XGBoost or LightGBM naively on that history and it quietly learns the lazy rule "predict the common outcome," because that minimises raw error while delivering a model that is useless for the trades that matter.

Vanilla SMOTE (V1) manufactured synthetic minority rows to fix that — but it interpolated between any two minority points in the whole dataset, including points on opposite sides of an empty region. BorderlineSMOTE (V2) concentrated synthesis at the boundary. SVMSMOTE (V3) let an SVM draw that boundary. KMeansSMOTE (V4) attacks a different flaw: the minority class is usually not one blob — it is several blobs. When the rare class has multiple distinct sub-populations (different market regimes, different failure modes), a single global interpolation surface connects them with fictional in-between samples. KMeansSMOTE carves the space first, so each regime is thickened within itself. [DERIVED from the conceptual motivation in Doufas & Bacao 2018]

Research Question / Hypothesis

Research question: For a binary classifier trained on an imbalanced dataset whose minority class is multi-modal (several separated sub-populations), does a cluster-then-synthesise strategy (KMeansSMOTE) improve minority-class recall and balanced metrics (F1, G-mean) relative to global-interpolation SMOTE, and under what data conditions does the gain appear or vanish?

Hypothesis (DERIVED): KMeansSMOTE should win when the minority class is genuinely multi-modal — several dense, separated sub-groups — because confining SMOTE to intra-cluster neighbours prevents generation in the low-density gaps between sub-groups, where synthetic points would be pure noise. The gain should shrink or reverse when (a) the minority class is essentially unimodal (then KMeansSMOTE ≈ vanilla SMOTE plus overhead), (b) the wrong number of clusters k is chosen, or (c) outliers distort the K-Means partition (a singleton outlier can hijack the density-weighted sampling). The hypothesis is a derived expectation, not a measurement from this article.

Data & Methodology

We describe the canonical experimental shape used throughout the imbalanced-learn documentation and the SMOTE literature. No experiment is executed in this article — the snippet below is illustrative, taken from the imbalanced-learn API, and is shown only to anchor the method. [SOURCE: scikit-learn-contrib/imbalanced-learn repo, KMeansSMOTE docstring/example]

The pipeline shape is always:

Stratified train/test split (never let SMOTE see the test set).
Inside cross-validation only, fit KMeansSMOTE on the training fold and fit_resample.
Train the estimator on the resampled fold; validate on the untouched test fold.
Because K-Means is distance-based, a StandardScaler must precede it — ideally wrapped in the same Pipeline so resampling + scaling never leak.

Illustrative API usage (from imbalanced-learn docs — not run here):

from collections import Counter
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import KMeansSMOTE
from imblearn.pipeline import Pipeline
from imblearn.metrics import classification_report_imbalanced

## Synthetic illustrative data — do not treat as a real market dataset.
## n_clusters_per_class=3 makes the minority class multi-modal on purpose.
X, y = make_classification(
    n_classes=2, class_sep=2, weights=[0.1, 0.9],
    n_informative=4, n_redundant=1, flip_y=0.02,
    n_features=20, n_clusters_per_class=3,
    n_samples=2000, random_state=10,
)
print("Original dataset shape", Counter(y))
## Original dataset shape Counter({1: ~1800, 0: ~200})

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

## kmeans_estimator can be an int (number of clusters) or an estimator object.
kms = KMeansSMOTE(
    kmeans_estimator=5,        # carve the minority class into 5 sub-clusters
    k_neighbors=3,             # SMOTE neighbours *within* each cluster
    random_state=42,
)
X_res, y_res = kms.fit_resample(X_train, y_train)
print("Resampled shape", Counter(y_res))
## Resampled shape Counter({0: ~1800, 1: ~1800})  -- illustrative from repo example

## scale BEFORE K-Means sees the data, inside the same pipeline
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("kms", KMeansSMOTE(kmeans_estimator=5, k_neighbors=3, random_state=42)),
    ("clf", RandomForestClassifier(random_state=42)),
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report_imbalanced(y_test, y_pred))

Key constructor parameters [SOURCE: imbalanced-learn repo, imblearn/over_sampling/_smote/cluster.py KMeansSMOTE]:

kmeans_estimator — the clusterer applied to the minority class. If None (the default), imbalanced-learn uses MiniBatchKMeans(n_clusters=2, random_state=random_state). You may pass an int (used as the number of clusters) or any K-Means-style estimator (e.g. KMeans(n_clusters=...)) to control the clustering. [SOURCE: imbalanced-learn repo, KMeansSMOTE.__init__ default]
cluster_centers (default None) — a precomputed array of shape (n_clusters, n_features). If you provide it, K-Means is skipped entirely and these centers are used directly to assign minority points to clusters. Handy for determinism across CV folds or when you already know the partition from EDA. If the centers are wrong, generation degrades silently. [SOURCE: imbalanced-learn repo, KMeansSMOTE cluster_centers]
k_neighbors (default 2) — number of nearest intra-cluster minority neighbours used to generate each synthetic point. A cluster with fewer than k_neighbors + 1 points cannot generate (it has no neighbour to interpolate with) and is skipped. [SOURCE: imbalanced-learn repo]
sampling_strategy (default "auto") — how much to oversample; n_jobs, random_state inherited from the base over-sampler. [SOURCE: imbalanced-learn repo]

A useful diagnostic the library exposes: after fitting, kms.cluster_centers_ holds the centers used (whether computed by K-Means or supplied via cluster_centers). Inspecting them tells you exactly how the minority class was partitioned. [SOURCE: imbalanced-learn repo, KMeansSMOTE.cluster_centers_]

Results / Findings

Because this article does not run an experiment, the "findings" below are a synthesis of the primary sources, not measured numbers from this author's machine.

Finding 1 — The minority class is partitioned before any synthesis. KMeansSMOTE first runs K-Means on the minority samples only, producing k clusters and their centers. Every subsequent synthetic point is generated inside one of those clusters, using only that cluster's own neighbours. There is no path for a point in cluster A to be interpolated toward a point in cluster B. [SOURCE: Doufas & Bacao 2018; imbalanced-learn cluster.py _fit_resample]

Finding 2 — Sampling is density-weighted, not uniform. For each cluster i, the paper defines a density d_i = n_i / r_i, where n_i is the number of minority samples in the cluster and r_i is the cluster radius (the average distance of the cluster's samples to its centroid). The total number of synthetic points to generate (set by sampling_strategy) is then split across clusters in proportion to d_i: s_i = round(N_syn · d_i / Σ_j d_j). A compact, populous cluster (large n_i, small r_i) gets high density → more new points; a sparse, spread-out cluster gets fewer. [SOURCE: Doufas & Bacao 2018, density definition; imbalanced-learn cluster.py implementation]

Finding 3 — Why this fixes vanilla SMOTE's gap problem (DERIVED). Vanilla SMOTE distributes synthetic points uniformly per minority instance and interpolates between global neighbours. When the minority class has two separated blobs, a point in blob A can have its nearest neighbour in blob B, so interpolation plants a synthetic point in the empty space between them — a region with no real minority data and often overlapping majority data. That point is pure ambiguity. KMeansSMOTE removes the mechanism entirely: blob A and blob B are different clusters, so interpolation is strictly intra-blob. The gaps stay empty. The density weighting then reinforces the denser blob, which is where real minority structure is strongest.

Finding 4 — Worked numerical example (DERIVED). Suppose the minority class in 1-D is two separated groups:

Group A (dense): {1.0, 1.1, 0.9, 1.05} — centroid ≈ 1.0125, radius ≈ 0.0625, so density d_A = 4 / 0.0625 = 64.
Group B (sparse): {5.0, 5.2} — centroid 5.1, radius ≈ 0.1, so density d_B = 2 / 0.1 = 20.

Total density Σ d = 84. Target N_syn = 6 synthetic points. Then s_A = round(6 · 64/84) = round(4.57) = 5, and s_B = round(6 · 20/84) = round(1.43) = 1. So 5 of 6 new points land inside dense Group A (in the [0.9, 1.1] band) and only 1 inside sparse Group B (in [5.0, 5.2]). Vanilla SMOTE with k_neighbors=2 behaves worse: for the point 5.0, its two nearest minority neighbours are 5.2 (dist 0.2) and 1.05 (dist 3.95); if interpolation picks 1.05, the synthetic point sits at 5.0 + 0.5·(1.05 − 5.0) ≈ 3.0 — squarely in the empty gap. KMeansSMOTE never produces that point. In higher dimensions the gap is a volume, so the vanilla-SMOTE failure is even more severe. [DERIVED illustration]

Finding 5 — The paper's empirical claim (SOURCE). Doufas & Bacao (2018) benchmark KMeansSMOTE across a large collection of imbalanced datasets drawn from the KEEL repository and report that, especially when combined with an ensemble classifier, the cluster-then-synthesise scheme ranks at or near the top among over-sampling methods — beating vanilla SMOTE and several other variants on balanced accuracy / F-measure. The gain is attributed precisely to the density-aware, cluster-confined generation described above. (Exact per-dataset metrics are in the paper; this article did not reproduce them.) [SOURCE: Doufas & Bacao 2018]

Finding 6 — KMeansSMOTE is complementary, not a replacement, for SVMSMOTE/BorderlineSMOTE. Those variants decide where on the boundary to synthesise. KMeansSMOTE decides how to keep synthesis inside coherent sub-populations. When the minority class is multi-modal, KMeansSMOTE addresses a failure the boundary-based variants do not. [DERIVED from comparing the implementations in imbalanced-learn]

Reproducibility

To reproduce a real comparison you would, at minimum:

Pick a fixed random_state everywhere (data split, SMOTE, estimator).
Scale features first — wrap StandardScaler ahead of KMeansSMOTE, because K-Means is a distance construction and unscaled features with large magnitude dominate cluster assignment. This is non-negotiable for honest results.
Compare resamplers inside the same CV loop: SMOTE(), KMeansSMOTE(kmeans_estimator=K) for a few K values, and BorderlineSMOTE() / SVMSMOTE().
Score with minority recall, precision, F1, and G-mean — not raw accuracy, which is misleading under imbalance.
Always fit_resample on the training fold only.
If you pass cluster_centers, fix them from a held-out EDA fit so they don't leak test information across folds.

What Failed / Counter-Evidence

KMeansSMOTE is not a free lunch, and the honest literature plus implementation details say so.

Choosing the wrong k breaks it. Too few clusters (e.g. k=2) and you may still get one mega-cluster spanning two genuine sub-populations — the gap problem returns inside that cluster. Too many and clusters become tiny and sparse: many fall below k_neighbors + 1 and are skipped, so you under-generate and waste the clustering cost. The right k is dataset-dependent and must be tuned. [DERIVED from the algorithm; Doufas & Bacao 2018 discuss cluster-count sensitivity]
Outliers hijack the density weighting. K-Means minimises squared error, so an outlier pulls a centroid and can form its own singleton cluster. A singleton has n_i = 1 and a near-zero radius r_i, so d_i = 1 / ~0 explodes — the density formula then dumps a huge share of synthetic points around that single outlier. This is a silent, catastrophic failure mode. [DERIVED from K-Means + density formula behaviour]
Blagus & Lusa (2013) show that SMOTE-family over-sampling can hurt on small or high-dimensional datasets, where neighbour/cluster geometry is unreliable and synthetic points amplify noise. KMeansSMOTE inherits this — tiny clusters in high dimensions are noise, not signal. [SOURCE: Blagus & Lusa 2013]
Numeric-only. K-Means needs Euclidean distance, so KMeansSMOTE works only on continuous features. Pass a categorical column and it breaks (or silently mis-clusters). For mixed numeric/categorical data use SMOTENC; for all-categorical use SMOTEN. [SOURCE: imbalanced-learn repo]
Scale sensitivity. Like every K-Means application, unscaled features with larger magnitude dominate distance and warp the partition. Forget StandardScaler and KMeansSMOTE can produce a meaningless clustering. [DERIVED from K-Means theory]
Higher compute cost. Running K-Means plus per-cluster SMOTE is pricier than vanilla SMOTE. Inside CV on a large Nifty feature matrix this can be the difference between seconds and minutes per fold. [DERIVED]
Unimodal minority → no benefit. If the rare class is genuinely one blob, KMeansSMOTE ≈ vanilla SMOTE with a density tweak and extra cost. It only earns its keep when the minority is multi-modal. [DERIVED]

Limitations

Cluster count k is the key knob. Wrong k (too small or too large) degrades results; it must be tuned inside CV. There is no universally correct value. [DERIVED]
Outlier sensitivity (the silent killer). A single outlier can form a tiny-radius cluster and capture disproportionate synthetic budget via the density formula. Mitigate with outlier removal / robust scaling before KMeansSMOTE. [DERIVED]
Numeric features only. No categorical support; use SMOTENC/SMOTEN for mixed or categorical data. [SOURCE: imbalanced-learn repo]
Scale sensitivity. K-Means partitions are distance-based; always StandardScaler first. [DERIVED from K-Means theory]
Compute cost. K-Means + per-cluster SMOTE costs more than vanilla SMOTE; budget for it in CV. [DERIVED]
Cluster-purity assumption. Assumes minority sub-populations are separable enough for K-Means to find them. Overlapping, smeared sub-populations defeat the partition. [DERIVED]
Sparse clusters can't generate. Any cluster with fewer than k_neighbors + 1 points is skipped, so you may end up under-generating relative to sampling_strategy. [SOURCE: imbalanced-learn repo]
No fix for irreducible overlap. If minority and majority truly overlap, intra-cluster SMOTE still thickens the overlapping region. [DERIVED]
cluster_centers can lie. Passing precomputed centers skips K-Means but if those centers are wrong the generation is wrong — and there is no check. [SOURCE: imbalanced-learn repo; DERIVED]

Practical Takeaways

Reach for KMeansSMOTE when your minority class is multi-modal — several distinct sub-populations (distinct market regimes, distinct failure modes) separated by empty or majority-dominated space. That is exactly where vanilla SMOTE pollutes the gaps.
Always scale first. Wrap StandardScaler ahead of KMeansSMOTE in the same Pipeline. This single step decides whether the clustering is meaningful. [DERIVED]
Treat kmeans_estimator as a real hyperparameter. Start with kmeans_estimator=K for a few K (e.g. 3, 5, 8) and tune inside CV. Prefer an explicit KMeans(n_clusters=K, random_state=...) over the default MiniBatchKMeans(n_clusters=2) when you suspect more than two sub-populations. [SOURCE: imbalanced-learn repo; DERIVED]
Clean outliers before clustering. Winsorise, clip, or remove extreme minority points first; otherwise the density formula can dump synthetic budget on noise. [DERIVED]
Inspect cluster_centers_ after fitting to confirm the partition looks like real sub-populations, not artefacts. [SOURCE: imbalanced-learn repo]
Never resample the test set. Resample inside the training fold only, ideally via imblearn.pipeline.Pipeline so it composes cleanly with scaling and the classifier.
Compare against the cheap baseline. Before reaching for any SMOTE variant, try scale_pos_weight (XGBoost/LightGBM) or class weights — sometimes that alone closes the gap without synthesising a single row. KMeansSMOTE is heavier than those.
In our stack, all of this lives in Layer 2 (EOD-audited training), never in Layer 1 (live Dhan capture). The K-Means partition is computed on historical folds, never on live ticks.

FAQ

Q1. What is the core difference between KMeansSMOTE and vanilla SMOTE?
A1. Vanilla SMOTE interpolates between minority neighbours globally across the whole feature space, so it can generate points in empty gaps between separated sub-populations. KMeansSMOTE first clusters the minority class with K-Means, then runs SMOTE inside each cluster only, so synthetic points stay in high-density regions and never bridge gaps. [SOURCE: Doufas & Bacao 2018; imbalanced-learn repo]

Q2. What does the kmeans_estimator parameter do?
A2. It is the clusterer applied to the minority class. None (default) becomes MiniBatchKMeans(n_clusters=2). You can pass an int (number of clusters) or any K-Means-style estimator (e.g. KMeans(n_clusters=5)) to control the partition. [SOURCE: imbalanced-learn repo]

Q3. What is cluster_centers and when would I use it?
A3. It is a precomputed array of cluster centers, shape (n_clusters, n_features). If supplied, K-Means is skipped and these centers are used to assign minority points to clusters. Use it for determinism across CV folds or when you already know the partition from EDA — but only if the centers are correct, since there is no validation. [SOURCE: imbalanced-learn repo]

Q4. How does KMeansSMOTE decide how many points to make per cluster?
A4. Via the density formula d_i = n_i / r_i (samples divided by cluster radius). The total synthetic count is split across clusters proportional to d_i, so denser, compact clusters receive more new points. [SOURCE: Doufas & Bacao 2018; imbalanced-learn repo]

Q5. Does KMeansSMOTE help if my minority class is one single blob?
A5. No — then it is roughly equivalent to vanilla SMOTE with a density tweak, plus extra compute. Its value appears only when the minority class is multi-modal (several separated sub-populations). [DERIVED]

Q6. Can I use KMeansSMOTE with categorical features?
A6. No. K-Means needs continuous Euclidean space. For mixed numeric/categorical data use SMOTENC; for all-categorical use SMOTEN — both from imblearn.over_sampling. [SOURCE: imbalanced-learn repo]

Q7. What is the biggest failure mode?
A7. Outliers. A singleton outlier forms a near-zero-radius cluster whose density d = n/r explodes, so the algorithm diverts a disproportionate share of synthetic points to that outlier. Clean outliers before clustering. [DERIVED]

Q8. Should I apply it to live trading data?
A8. No — apply any over-sampling inside the training/CV loop of your audited model core, never on live inference. Our NSE stack keeps resampling strictly in Layer 2. [DERIVED from the two-layer engine design]

Q9. What comes after KMeansSMOTE?
A9. V5 SMOTENC handles mixed categorical and numeric features by combining K-Means-style clustering on the continuous part with special handling for categorical levels — the right tool when your feature matrix is not purely numeric. [SOURCE: imbalanced-learn repo]

TL;DR

KMeansSMOTE (from imblearn.over_sampling import KMeansSMOTE) clusters the minority class with K-Means first, then applies SMOTE inside each cluster — density-aware, gap-free over-sampling. [SOURCE: Doufas & Bacao 2018; imbalanced-learn repo]
It fixes vanilla SMOTE's tendency to generate synthetic points in low-density / between-cluster gaps, where no real minority data exists. [DERIVED]
kmeans_estimator (default None → MiniBatchKMeans(n_clusters=2)) sets the clustering — pass an int (n clusters) or a K-Means estimator; cluster_centers lets you skip K-Means with precomputed centers. [SOURCE: imbalanced-learn repo]
It helps most when the minority class is multi-modal; it struggles with wrong k, outliers (which hijack the density formula), unscaled features, and categorical columns. [DERIVED]
Always scale first, resample inside CV only, and tune kmeans_estimator as a hyperparameter. [DERIVED]
Next in series: V5 SMOTENC. Previous: V3 SVMSMOTE.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

[SOURCE] Doufas, G., & Bacao, F. (2018). The Last Resort: Balancing Imbalanced Big Data Classification with K-Means and SMOTE. Information Sciences, 465, 1–20. — original KMeansSMOTE algorithm (cluster minority with K-Means, then density-weighted SMOTE within clusters); primary paper behind imblearn.over_sampling.KMeansSMOTE.
[SOURCE] scikit-learn-contrib/imbalanced-learn GitHub repository — imblearn/over_sampling/_smote/cluster.py (KMeansSMOTE, kmeans_estimator, cluster_centers, _fit_resample, cluster_centers_) and base SMOTE modules. Primary source for the implementation described.
[SOURCE] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — vanilla SMOTE baseline (V1).
[SOURCE] Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. ICIC 2005, LNCS 3644, 878–887. — Borderline-SMOTE (V2), the boundary-based comparison point.
[SOURCE] Nguyen, H. M., Cooper, E. W., & Kamei, K. (2011). Borderline over-sampling for imbalanced data classification. International Journal of Knowledge and Web Intelligence, 2(3), 230–242. — SVM-based boundary over-sampling implemented as SVMSMOTE (V3).
[SOURCE] Blagus, R., & Lusa, L. (2013). SMOTE can degrade performance on small, high-dimensional datasets (cautionary note on over-sampling). — counter-evidence on small/high-dimensional data.
[DERIVED] The 1-D two-group worked example ({1.0,1.1,0.9,1.05} vs {5.0,5.2}), the density-weighting arithmetic, the "gap pollution" illustration at ≈3.0, the outlier-hijack analysis, and the "KMeansSMOTE vs vanilla SMOTE" comparison — derived from the cited sources and standard K-Means / SMOTE theory.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE Family series (V4 of V6). This is educational content; not investment advice and not SEBI-registered research.

Resources & Links

Brand site: https://optiontradingwithai.in
About the author: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
Previous article (V3 SVMSMOTE): SMOTE_V3_svmsmote.md
Next article (V5 SMOTENC): SMOTE_V5_smotenc.md
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

SVMSMOTE in imbalanced-learn: Let an SVM Draw the Boundary, Then Synthesise on It

shakti tiwari — Tue, 08 Sep 2026 04:31:33 +0000

SVMSMOTE in imbalanced-learn: Let an SVM Draw the Boundary, Then Synthesise on It

Part of the SMOTE Family series. Previous: V2 BorderlineSMOTE. Next: V4 KMeansSMOTE.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

SVMSMOTE (from imblearn.over_sampling import SVMSMOTE) is the SMOTE variant that outsources the hardest part of over-sampling — figuring out where the class boundary actually is — to a Support Vector Machine. It fits an SVM to separate the minority class from the majority class, extracts the support vectors (the points that sit on or inside the margin and define the decision boundary), filters those support vectors down to the minority-class ones, and then synthesises new minority rows by interpolating between each boundary support vector and its nearest minority neighbours. The idea, from Nguyen, Cooper & Kamei (2011), is that the SVM margin is a more robust boundary locator than the raw k-nearest-neighbour "danger" test that BorderlineSMOTE uses. [DERIVED summary of Nguyen et al. 2011 + imbalanced-learn implementation]

Why This Matters

Agar aap Nifty options trade karte ho, toh aap pehle se hi imbalanced duniya mein jeete ho. The events that actually make or break your P&L — a sharp expiry-day reversal, a volatility expansion, a failed breakdown that snaps back — are rare. The dull chop that fills most sessions is common. Train an XGBoost or LightGBM model naively on that history and it will quietly learn the lazy rule "predict the common class," because that minimises raw error while delivering a model that is useless for the trades that pay the bills.

Vanilla SMOTE (V1) fixed part of that by manufacturing synthetic minority rows — but it sprayed them everywhere, including the safe interior where the classifier was already confident. BorderlineSMOTE (V2) got smarter: it used kNN to find the "danger" minority points and synthesised only there. SVMSMOTE takes the next logical step. Instead of asking "how many of this point's neighbours are majority?" it asks "what does an SVM think the boundary between the two classes is?" and then seeds synthesis right on that margin. [DERIVED from the conceptual motivation in Nguyen et al. 2011]

Research Question / Hypothesis

Research question: For a binary classifier trained on an imbalanced dataset, does locating the minority synthesis region with an SVM margin (SVMSMOTE) improve minority-class recall and balanced metrics (F1, G-mean) relative to kNN-based danger detection (BorderlineSMOTE), and under what data conditions does the gain appear?

Hypothesis (DERIVED): SVMSMOTE should win when the two classes are separated by a relatively clean, roughly linear margin — the case where an SVM finds a stable boundary and the support vectors genuinely mark the informative frontier. In that regime the SVM margin is less fooled by local density quirks than BorderlineSMOTE's majority-neighbour count. The gain should shrink or reverse when the boundary is heavily non-linear, when features are unscaled (SVMs are scale-sensitive), or when the dataset is small/high-dimensional (the Blagus & Lusa 2013 critique bites). The hypothesis is a derived expectation, not a measurement from this article.

Data & Methodology

The pipeline shape is always:

Stratified train/test split (never let SMOTE see the test set).
Inside cross-validation only, fit SVMSMOTE on the training fold and fit_resample.
Train the estimator on the resampled fold; validate on the untouched test fold.
Because SVMs are scale-sensitive, a StandardScaler must precede the SVM — ideally wrapped in the same Pipeline so resampling + scaling never leak.

Illustrative API usage (from imbalanced-learn docs — not run here):

from collections import Counter
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import SVMSMOTE
from imblearn.pipeline import Pipeline
from imblearn.metrics import classification_report_imbalanced

## Synthetic illustrative data — do not treat as a real market dataset
X, y = make_classification(
    n_classes=2, class_sep=2, weights=[0.1, 0.9],
    n_informative=3, n_redundant=1, flip_y=0,
    n_features=20, n_clusters_per_class=1,
    n_samples=1000, random_state=10,
)
print("Original dataset shape", Counter(y))
## Original dataset shape Counter({1: 900, 0: 100})

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

## default linear SVM under the hood; out_step controls boundary push
sm = SVMSMOTE(
    svm_estimator=SVC(kernel="linear"),
    out_step=0.3,
    random_state=42,
)
X_res, y_res = sm.fit_resample(X_train, y_train)
print("Resampled shape", Counter(y_res))
## Resampled shape Counter({0: 900, 1: 900})  -- illustrative from repo example

## scale BEFORE the SVM sees the data, inside the same pipeline
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("smote", SVMSMOTE(svm_estimator=SVC(kernel="linear"), random_state=42)),
    ("clf", RandomForestClassifier(random_state=42)),
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report_imbalanced(y_test, y_pred))

Key constructor parameters [SOURCE: imbalanced-learn repo, filter.py SVMSMOTE]:

svm_estimator — the SVM used to find support vectors. If left as None, imbalanced-learn falls back to a default SVM (an SVC with a linear kernel) to carve out the margin. Passing your own SVC lets you pick the kernel (linear vs rbf) and the regularisation C. [SOURCE: imbalanced-learn repo, SVMSMOTE default]
out_step (default 0.5) — the step fraction that scales how far each synthetic point is displaced from its boundary support vector toward its minority neighbour. Smaller out_step keeps new points hugging the margin; larger out_step pushes them deeper into minority space. [SOURCE: imbalanced-learn repo, SVMSMOTE out_step]
k_neighbors (default 5) — number of nearest minority neighbours used to generate each synthetic point around a support vector.
m_neighbors (default 10) — used to flag and discard minority support vectors whose m_neighbors are all majority (i.e., likely noise), so the algorithm does not seed synthesis from outliers. [SOURCE: imbalanced-learn repo, filter.py]
sampling_strategy (default "auto") — how much to oversample.
random_state — for reproducibility of the random interpolation gap.

A useful diagnostic the library exposes: sm.svm_estimator_.support_ — the indices of the support vectors the fitted SVM found. Inspecting which of those belong to the minority class tells you exactly which rows SVMSMOTE treated as boundary-relevant seeds. [SOURCE: imbalanced-learn repo, SVMSMOTE / scikit-learn SVC.support_]

Results / Findings

Because this article does not run an experiment, the "findings" below are a synthesis of the primary sources, not measured numbers from this author's machine.

Finding 1 — The SVM margin is the boundary locator. SVMSMOTE fits an SVM to separate minority from majority. The SVM's support vectors are, by definition, the training points that lie on or inside the margin — the points the classifier "leans on" to define the decision surface. Those are precisely the boundary-relevant minority points we want to thicken. [SOURCE: Nguyen et al. 2011; DERIVED from SVM theory]

Finding 2 — Synthesis happens at the minority support vectors, not everywhere. After the SVM is fit, imbalanced-learn keeps the support vectors that belong to the minority class and discards those flagged as noise (all-m_neighbors majority). Each surviving boundary support vector is then interpolated with its nearest minority neighbours to produce synthetic rows. So, just like BorderlineSMOTE, SVMSMOTE refuses to waste synthetic budget in the safe interior — but it defines "boundary" via the SVM margin rather than via a kNN majority-count. [SOURCE: imbalanced-learn repo, filter.py _svmSMOTE]

Finding 3 — Worked intuition: why an SVM margin beats a kNN count (DERIVED). Picture two classes in 2-D. The majority class forms a dense blob; the minority class forms a thin arc curving along one side of the blob. A kNN danger test looks locally: a minority point is "in danger" if half its 10 neighbours are majority. But near a concave part of the arc, a minority point can have mostly minority neighbours yet still sit on the true frontier — kNN under-flags it. An SVM, by contrast, fits a global margin that respects the whole shape of the blob, so its support vectors land on the actual frontier including that concave stretch. SVMSMOTE then seeds there. This is the qualitative reason SVM-based boundary detection can be more faithful than kNN danger detection on irregular boundaries. [DERIVED illustration]

To make the margin concrete, consider a trivially separable 1-D minority set {1, 3, 5} (class 1) versus majority {10, 12, 14} (class 0), with a hard-margin linear SVM. The optimal separating line sits at the midpoint between the closest pair across classes — between 5 and 10, i.e. at 7.5. The support vectors are exactly 5 (minority) and 10 (majority): the two points that pin the margin. SVMSMOTE would keep 5 as a minority boundary seed and synthesise new minority points by interpolating 5 with its nearest minority neighbours (3, and/or the new points themselves). The number 10 is a majority support vector and is not used to seed minority synthesis. That tiny example captures the whole mechanism: the SVM's margin identifies 5 as the frontier minority point; vanilla SMOTE would have treated 1, 3, 5 with equal priority and wasted budget on the safe interior point 1. [DERIVED numerical illustration]

Finding 4 — out_step is the dial BorderlineSMOTE doesn't have. BorderlineSMOTE has no direct control over how far synthetic points sit from the danger point — it just interpolates within [0, 1] of the gap to a neighbour. SVMSMOTE's out_step (default 0.5) scales that displacement. With out_step=0.3, new points hug the support vector (tight margin reinforcement); with out_step=0.8, they are pushed further toward (and past) the minority neighbour (broader reinforcement). This is a genuine extra lever, but it is also another hyperparameter you must tune inside CV. [SOURCE: imbalanced-learn repo; DERIVED interpretation]

Finding 5 — SVMSMOTE is a refinement of, not a replacement for, BorderlineSMOTE. Both concentrate synthesis at the boundary. The difference is purely how the boundary is found: kNN danger mask (BorderlineSMOTE) versus SVM support vectors (SVMSMOTE). Which is better depends on the data geometry, not on the variant being "newer." [DERIVED from comparing the two implementations in imbalanced-learn]

Reproducibility

To reproduce a real comparison you would, at minimum:

Pick a fixed random_state everywhere (data split, SMOTE, estimator).
Scale features first — wrap StandardScaler ahead of SVMSMOTE, because the SVM margin is scale-sensitive (see Limitations). This is non-negotiable for honest results.
Compare resamplers inside the same CV loop: BorderlineSMOTE(kind="borderline-1"), SVMSMOTE(svm_estimator=SVC(kernel="linear")), and SVMSMOTE(svm_estimator=SVC(kernel="rbf")).
Score with minority recall, precision, F1, and G-mean — not raw accuracy, which is misleading under imbalance.
Always fit_resample on the training fold only.

What Failed / Counter-Evidence

SVMSMOTE is not a free lunch, and the honest literature plus implementation details say so.

Blagus & Lusa (2013) show that SMOTE-family over-sampling can hurt on small or high-dimensional datasets, where the nearest-neighbour / margin geometry is unreliable and synthetic points amplify noise rather than signal. SVMSMOTE inherits this failure mode — the SVM can latch onto a spurious margin when there are too few samples per dimension. [SOURCE: Blagus & Lusa 2013]
Unscaled features break the SVM. Because the margin depends on distances, a single feature with large magnitude dominates. Forget the StandardScaler and SVMSMOTE can produce a meaningless boundary — and BorderlineSMOTE, being kNN-based, is also scale-sensitive, but the SVM's sensitivity is sharper and more silent. [DERIVED]
It is more expensive than kNN variants. Fitting an SVM and extracting support vectors costs more than BorderlineSMOTE's neighbour counts, especially on large datasets. On a big Nifty feature matrix this can be the difference between seconds and minutes per CV fold. [DERIVED]
Kernel choice shifts the result. A linear SVM gives a straight margin and a predictable support-vector set; an RBF SVM can carve a wiggly margin with many support vectors, seeding far more synthetic points and risking over-generation. The default (linear) is usually the safe start. [SOURCE: imbalanced-learn repo default; DERIVED]
It still ignores majority density. Like BorderlineSMOTE, SVMSMOTE synthesises from minority boundary points without checking how densely the majority class is packed nearby. A synthetic point can land in a dense majority region, thickening the fog. [DERIVED]
Better variants exist for hard cases. KMeansSMOTE (V4) clusters first, then synthesises within coherent clusters — useful when the minority class is multi-modal and a single global SVM margin misses subclusters. [SOURCE: imbalanced-learn repo; KMeansSMOTE paper 2018]

Limitations

SVM scale sensitivity (the big one). The margin is a distance construction. Without StandardScaler (or equivalent) ahead of the SVM, features on different scales distort the boundary. Always scale. [DERIVED from SVM theory]
Higher compute cost. SVM fit + support-vector extraction is pricier than kNN danger detection. Budget for it in CV. [DERIVED]
out_step is another hyperparameter. More dials mean more tuning. Leave it at 0.5 only as a starting point; tune inside CV. [SOURCE: imbalanced-learn repo]
Kernel dependence. Linear vs RBF changes the boundary shape and the number of seeds. Mismatch with the true boundary geometry hurts. [DERIVED]
Noise support vectors. If a minority support vector is actually a mislabel near the margin, SVMSMOTE will enthusiastically manufacture more points around it. The m_neighbors noise filter mitigates but does not eliminate this. [SOURCE: imbalanced-learn repo; DERIVED]
Feature-space only. Like all SMOTE variants, it assumes Euclidean proximity means semantic similarity — fragile with mixed-type, high-cardinality, or heavily engineered features. For those, SMOTENC/SMOTEN exist. [SOURCE: imbalanced-learn repo]
No guarantee of separability. A synthetic margin is not a real one. If the true boundary is irreducible noise, more SVM-boundary points just thicken the fog. [DERIVED]
Extreme imbalance still needs help. At ratios like 1:1000, even SVMSMOTE may need under-sampling of the majority or scale_pos_weight in the estimator itself. [DERIVED]

Practical Takeaways

Reach for SVMSMOTE when your classes are separated by a reasonably clean, near-linear margin and you suspect BorderlineSMOTE's kNN danger mask is under- or over-flagging the boundary on irregular geometry.
Always scale first. Wrap StandardScaler ahead of SVMSMOTE in the same Pipeline. This single step decides whether the SVM margin is meaningful. [DERIVED]
Start with a linear svm_estimator (SVC(kernel="linear")), matching the imbalanced-learn default; try rbf only if you have evidence of a non-linear frontier, and watch the support-vector count.
Treat out_step as tunable, not fixed at 0.5. Smaller values reinforce the margin tightly; larger values broaden it. Tune inside CV. [SOURCE: imbalanced-learn repo]
Never resample the test set. Resample inside the training fold only, ideally via imblearn.pipeline.Pipeline so it composes cleanly with scaling and the classifier.
Compare against the cheap baseline. Before reaching for any SMOTE variant, try scale_pos_weight (XGBoost/LightGBM) or class weights — sometimes that alone closes the gap without synthesising a single row.
Pair with cleaning. Follow SVMSMOTE with Edited Nearest Neighbours (ENN) to prune clearly misplaced points — a "SVMSMOTE + ENN" combo, just as people do with SMOTE/ENN.
In our stack, all of this lives in Layer 2 (EOD-audited training), never in Layer 1 (live Dhan capture). The SVM margin is computed on historical folds, never on live ticks.

FAQ

Q1. What is the core difference between SVMSMOTE and BorderlineSMOTE?
A1. Both synthesise minority samples only near the decision boundary. BorderlineSMOTE finds that boundary with a kNN danger mask (counting majority neighbours). SVMSMOTE finds it with an SVM: it fits a support-vector classifier and uses the minority-class support vectors as the synthesis seeds. The "where to synthesise" question is answered by a margin instead of a neighbour count. [SOURCE: imbalanced-learn repo; Nguyen et al. 2011]

Q2. What does the svm_estimator parameter do?
A2. It is the SVM used to locate the boundary. Left as None, imbalanced-learn uses a default linear-kernel SVC. Passing your own SVC(kernel="linear") or SVC(kernel="rbf", C=...) lets you control the margin shape and regularisation. The support vectors of whatever you pass become the synthesis seeds. [SOURCE: imbalanced-learn repo]

Q3. What is out_step and how should I set it?
A3. out_step (default 0.5) scales how far each synthetic point is displaced from its boundary support vector toward its minority neighbour. Smaller hugs the margin; larger pushes deeper into minority space. Tune it inside CV; do not accept 0.5 blindly. [SOURCE: imbalanced-learn repo; DERIVED]

Q4. Do I need to scale features for SVMSMOTE?
A4. Yes — strongly recommended. The SVM margin is a distance construction, so unscaled features with larger magnitude dominate the boundary. Wrap StandardScaler ahead of SVMSMOTE in the same pipeline. [DERIVED from SVM theory]

Q5. Is SVMSMOTE always better than BorderlineSMOTE?
A5. No. It tends to win when the boundary is a clean, near-linear margin; it can cost more compute and even underperform when classes are heavily intermixed, features are unscaled, or the dataset is small/high-dimensional. Validate both inside CV. [DERIVED from Nguyen 2011 + Blagus & Lusa 2013]

Q6. Can I use SVMSMOTE with categorical features?
A6. SVMSMOTE works in continuous Euclidean space via an SVM. For mixed numeric/categorical data use SMOTENC; for all-categorical use SMOTEN — both from imblearn.over_sampling. [SOURCE: imbalanced-learn repo]

Q8. What comes after SVMSMOTE?
A8. V4 KMeansSMOTE clusters the minority class first (KMeans), then synthesises within each coherent cluster — better when the minority class is multi-modal and a single global SVM margin misses subclusters. [SOURCE: imbalanced-learn repo; KMeansSMOTE paper 2018]

TL;DR

SVMSMOTE (from imblearn.over_sampling import SVMSMOTE) fits an SVM to separate the classes, extracts the support vectors (the margin boundary), and synthesises minority points near the minority-class support vectors. [SOURCE: imbalanced-learn repo; Nguyen et al. 2011]
It is the SVM-margin cousin of BorderlineSMOTE: same goal (synthesize at the boundary), different boundary finder (SVM margin vs kNN danger mask). [DERIVED]
svm_estimator (default linear SVC) chooses the margin shape; out_step (default 0.5) controls how far synthetic points sit from the support vector. [SOURCE: imbalanced-learn repo]
It helps most when classes have a clean, near-linear margin; it struggles with unscaled features, small/high-dim data, and costs more than kNN variants. [DERIVED]
Always scale features before the SVM, resample inside CV only, and tune out_step/svm_estimator as hyperparameters. [DERIVED]
Next in series: V4 KMeansSMOTE. Previous: V2 BorderlineSMOTE.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

[SOURCE] Nguyen, H. M., Cooper, E. W., & Kamei, K. (2011). Borderline over-sampling for imbalanced data classification. International Journal of Knowledge and Web Intelligence, 2(3), 230–242. — original SVM-balanced SMOTE / support-vector boundary over-sampling algorithm that imbalanced-learn's SVMSMOTE implements.
[SOURCE] scikit-learn-contrib/imbalanced-learn GitHub repository — imblearn/over_sampling/_smote/filter.py (SVMSMOTE, svm_estimator, out_step, m_neighbors, _svmSMOTE support-vector seed logic) and imblearn/over_sampling/_smote/base.py. Primary source for the implementation described.
[SOURCE] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — vanilla SMOTE baseline.
[SOURCE] Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. ICIC 2005, LNCS 3644, 878–887. — Borderline-SMOTE (kNN danger mask), the direct comparison point for V3.
[SOURCE] Blagus, R., & Lusa, L. (2013). SMOTE can degrade performance on small, high-dimensional datasets (cautionary note on over-sampling). — counter-evidence on small/high-dimensional data.
[DERIVED] The 1-D margin worked example ({1,3,5} vs {10,12,14}), the concave-arc intuition, the interpolation/displacement interpretation of out_step, and the "SVM margin vs kNN danger" comparison — derived from the cited sources and standard SVM theory.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE Family series (V3 of V6). This is educational content; not investment advice and not SEBI-registered research.

Resources & Links

Brand site: https://optiontradingwithai.in
About the author: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
Previous article (V2 BorderlineSMOTE): SMOTE_V2_borderline.md
Next article (V4 KMeansSMOTE): SMOTE_V4_kmeanssmote.md
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

BorderlineSMOTE in imbalanced-learn: Oversample the Decision Boundary, Not the Safe Interior

shakti tiwari — Tue, 08 Sep 2026 04:31:13 +0000

BorderlineSMOTE in imbalanced-learn: Oversample the Decision Boundary, Not the Safe Interior

Part of the SMOTE Family series. Previous: V1 vanilla SMOTE. Next: V3 SVMSMOTE.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

BorderlineSMOTE (from imblearn.over_sampling import BorderlineSMOTE) is a variant of SMOTE that refuses to waste synthetic samples in the safe interior of the minority class. It first classifies every minority point as SAFE, DANGER, or NOISE based on how many of its nearest neighbours belong to the majority class, then synthesises new points only for the DANGER set — the minority instances hugging the decision boundary, which are the ones a classifier actually struggles with. When the informative signal lives at the class boundary (the usual case in noisy financial data), BorderlineSMOTE typically beats vanilla SMOTE while adding less redundant noise. [DERIVED summary of Han et al. 2005 + imbalanced-learn implementation]

Why This Matters

If you trade Nifty options, you already live in an imbalanced world. Big directional moves that you want to catch — a sharp expiry-day reversal, a volatility expansion — are rare compared to the dull chop that fills most sessions. Train an XGBoost model naively on that history and it will quietly learn the lazy rule "predict the common class," because that minimises raw error while delivering a model that is useless for the trades that pay the bills.

SMOTE fixed part of that by manufacturing synthetic minority rows. But vanilla SMOTE sprays those synthetic rows everywhere — including deep inside minority territory where the classifier was already confident. That is wasted effort, and worse, it can blur the very boundary you care about. BorderlineSMOTE is the smarter cousin: it asks where the hard cases are and spends its synthetic budget there. [DERIVED from the conceptual motivation in Han et al. 2005]

Research Question / Hypothesis

Research question: For a binary classifier trained on an imbalanced dataset, does restricting synthetic over-sampling to the minority "danger" region (BorderlineSMOTE) improve minority-class recall and balanced metrics (F1, G-mean) relative to unrestricted vanilla SMOTE, without inflating training cost?

Hypothesis (DERIVED): Yes — when the classes are separated by a boundary rather than thoroughly intermixed. The boundary minority points carry the most discriminative information; the interior minority points are already easy; the noise minority points (surrounded by majority) are usually mislabels. Concentrating generation on the danger zone should raise genuine recall while adding less artificial overlap. The gain collapses when the classes are so overlapping that "danger" ≈ "everything," or when the dataset is tiny/high-dimensional (see Limitations).

Data & Methodology

We describe the canonical experimental shape used throughout the SMOTE literature and the imbalanced-learn documentation. No experiment is executed in this article — the snippet below is illustrative, taken from the imbalanced-learn API, and is shown only to anchor the method. [SOURCE: scikit-learn-contrib/imbalanced-learn repo, BorderlineSMOTE docstring/example]

The pipeline shape is always:

Stratified train/test split (never let SMOTE see the test set).
Inside cross-validation only, fit BorderlineSMOTE on the training fold and fit_resample.
Train the estimator on the resampled fold; validate on the untouched test fold.
Optionally wrap with Pipeline so resampling is never leaked.

Illustrative API usage (from imbalanced-learn docs — not run here):

from collections import Counter
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import BorderlineSMOTE
from imblearn.pipeline import Pipeline
from imblearn.metrics import classification_report_imbalanced

## Synthetic illustrative data — do not treat as a real market dataset
X, y = make_classification(
    n_classes=2, class_sep=2, weights=[0.1, 0.9],
    n_informative=3, n_redundant=1, flip_y=0,
    n_features=20, n_clusters_per_class=1,
    n_samples=1000, random_state=10,
)
print("Original dataset shape", Counter(y))
## Original dataset shape Counter({1: 900, 0: 100})

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

## borderline-1 is the default kind
sm = BorderlineSMOTE(kind="borderline-1", random_state=42)
X_res, y_res = sm.fit_resample(X_train, y_train)
print("Resampled shape", Counter(y_res))
## Resampled shape Counter({0: 900, 1: 900})  -- illustrative from repo example

pipe = Pipeline([
    ("smote", BorderlineSMOTE(random_state=42)),
    ("clf", RandomForestClassifier(random_state=42)),
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report_imbalanced(y_test, y_pred))

Key constructor parameters [SOURCE: imbalanced-learn repo, filter.py]:

k_neighbors (default 5) — number of nearest neighbours used to generate each synthetic point.
m_neighbors (default 10) — number of nearest neighbours used to decide whether a minority point is "in danger."
kind — "borderline-1" (default) or "borderline-2" (see below).
sampling_strategy (default "auto") — how much to oversample.
random_state — for reproducibility of the random interpolation gap.

A useful diagnostic the library exposes: sm.in_danger_indices_ — a dict mapping each resampled class to the indices of the original minority rows that were flagged "in danger" and used to seed synthesis. Inspecting it tells you exactly which rows BorderlineSMOTE considered boundary-relevant. [SOURCE: imbalanced-learn repo, BorderlineSMOTE.in_danger_indices_ attribute]

Results / Findings

Because this article does not run an experiment, the "findings" below are a synthesis of the primary sources, not measured numbers from this author's machine.

Finding 1 — Vanilla SMOTE over-invests in the interior. Vanilla SMOTE selects every minority point with equal probability and interpolates between two minority neighbours [SOURCE: Chawla et al. 2002]. Many of those points sit far from any majority instance, where the classifier already separates cleanly. Generating there adds rows the model did not need and, in the worst case, the interpolated segment between two minority points can cross majority space, fabricating ambiguous examples. [DERIVED]

Finding 2 — The boundary minority points are the informative ones. Han, Wang and Mao (2005) argue that a minority instance surrounded mostly by its own class is "safe" and contributes little new separable signal, while one sitting next to majority instances is on the frontier where misclassification actually happens [SOURCE: Han, Wang, Mao 2005]. Concentrating synthesis there strengthens the decision region where it is weakest. [DERIVED interpretation]

Finding 3 — The DANGER mask is the load-bearing mechanism. In imbalanced-learn, danger detection lives in _in_danger_noise: for each minority point, count how many of its m_neighbors nearest neighbours (default 10) belong to the majority class. A point is flagged DANGER when that count is at least half but not all (m/2 ≤ n_majority < m); it is flagged NOISE when all neighbours are majority; otherwise it is left alone (SAFE). Only DANGER points seed new synthetic rows. [SOURCE: imbalanced-learn repo, base.py _in_danger_noise]

Finding 4 — borderline-1 vs borderline-2 change where synthesis reaches. This is the part most practitioners miss.

borderline-1 (default): synthetic samples are generated by interpolating each DANGER minority point with its minority-class neighbours only. The new points stay on the minority side of the boundary, thickening the edge of the minority region. [SOURCE: imbalanced-learn repo, filter.py — X_to_sample_from = X_class]
borderline-2: the whole dataset is used as the neighbour pool (X_to_sample_from = X), so the DANGER point can also be interpolated toward its majority neighbours. This pushes synthetic points across the boundary into majority space, directly contesting the majority region. [SOURCE: imbalanced-learn repo, filter.py — X_to_sample_from = X]

The interpolation itself is the familiar SMOTE rule, here written for a DANGER seed x_i and a chosen neighbour x_z [DERIVED]:
x_new = x_i + (x_z − x_i) · δ, with δ ~ Uniform(0, 1).

For borderline-1, x_z is drawn from the minority k-NN of x_i; for borderline-2, x_z may be a minority or majority k-NN of x_i.

Finding 5 — Empirical direction from the literature. The original paper reports Borderline-SMOTE outperforming vanilla SMOTE on a range of imbalanced UCI-style datasets, particularly on F-measure and related balanced metrics, because it stops fabricating easy interior points [SOURCE: Han, Wang, Mao 2005]. We are not quoting exact tables here (this author did not re-run them); the qualitative claim is well established in the over-sampling literature. [DERIVED note on what the paper establishes]

Reproducibility

To reproduce a real comparison you would, at minimum:

Pick a fixed random_state everywhere (data split, SMOTE, estimator).
Compare three resamplers inside the same CV loop: SMOTE() (V1), BorderlineSMOTE(kind="borderline-1"), BorderlineSMOTE(kind="borderline-2").
Score with minority recall, precision, F1, and G-mean — not raw accuracy, which is misleading under imbalance.
Always fit_resample on the training fold only.

The code sketch in Data & Methodology is the canonical shape from the imbalanced-learn documentation and is presented illustratively — it was not executed for this article, and its printed outputs are quoted from the library's own example for reference. [SOURCE: imbalanced-learn repo docstring; not an experiment result of this author]

What Failed / Counter-Evidence

Borderline-SMOTE is not a free lunch, and the honest literature says so.

Blagus & Lusa (2013) show that SMOTE-family over-sampling can hurt on small or high-dimensional datasets, where the nearest-neighbour geometry is unreliable and synthetic points amplify noise rather than signal [SOURCE: Blagus & Lusa 2013]. Borderline-SMOTE narrows the problem but inherits it.
When classes are thoroughly intermixed, almost every minority point becomes "in danger," so BorderlineSMOTE degenerates toward vanilla SMOTE — you pay the complexity for little gain. [DERIVED]
Boundary noise is still boundary noise. If the minority points near the boundary are actually mislabels (NOISE that escaped the m/2 filter), BorderlineSMOTE will enthusiastically manufacture more of them. Over-focusing on the border can amplify labelling errors. [DERIVED]
Better variants exist for hard cases. SVMSMOTE (V3) uses an SVM to pick support-vector-like border points; KMeansSMOTE clusters first. BorderlineSMOTE is a strong default but not the final word. [SOURCE: imbalanced-learn repo; Nguyen et al. 2011 for SVM-SMOTE]

Limitations

It still ignores majority density. BorderlineSMOTE synthesises minority points based on neighbour labels, not on how densely the majority class is packed. It can drop a synthetic minority point into a dense majority cluster, making that region even more ambiguous. [DERIVED]
It can over-focus on border noise. As noted, the DANGER set includes genuinely hard points and mislabeled ones; the algorithm cannot tell them apart. [DERIVED]
Threshold sensitivity. Results shift with m_neighbors and k_neighbors. Too small an m_neighbors makes the danger estimate noisy; too large blurs the boundary concept. These are hyperparameters you must tune inside CV, not set once. [DERIVED]
Feature-space only. Like all SMOTE variants, it operates in the raw feature space and assumes Euclidean proximity means semantic similarity — fragile with mixed-type, high-cardinality, or heavily engineered features. (For those, SMOTENC/SMOTEN exist.) [SOURCE: imbalanced-learn repo]
No guarantee of separability. Adding boundary minority points does not create a cleaner boundary; if the true boundary is irreducible noise, more synthetic points just thicken the fog. [DERIVED]
Class imbalance ratio still matters. At extreme ratios (e.g., 1:1000), even BorderlineSMOTE may need to be paired with under-sampling of the majority or with scale_pos_weight in the estimator itself. [DERIVED]

Practical Takeaways

Reach for BorderlineSMOTE when your minority class forms reasonably coherent clusters with a definable boundary and you care about recall at that boundary — the classic imbalanced-classification setup, including many Nifty option signal problems.
Prefer borderline-1 as the default; try borderline-2 only when you suspect the minority region needs to actively push into majority space. Always validate both inside CV.
Never resample the test set. Resample inside the training fold only, ideally via imblearn.pipeline.Pipeline so it composes cleanly with scaling and the classifier.
Pair with cleaning. Because BorderlineSMOTE can still leave noisy majority regions, many practitioners follow it with Edited Nearest Neighbours (ENN) to prune clearly misplaced points — a "SMOTE + ENN" or "BorderlineSMOTE + ENN" combo.
Tune m_neighbors/k_neighbors as hyperparameters; don't accept defaults blindly on real data.
Compare against the cheap baseline. Before reaching for any SMOTE variant, try scale_pos_weight (XGBoost/LightGBM) or class weights — sometimes that alone closes the gap without synthesising a single row.
In our stack, all of this lives in Layer 2 (EOD-audited training), never in Layer 1 (live Dhan capture).

FAQ

Q1. What exactly is the difference between borderline-1 and borderline-2?
A1. Both select the same DANGER minority points. borderline-1 interpolates each DANGER point only with its minority neighbours, keeping new samples on the minority side. borderline-2 uses the whole dataset as the neighbour pool, so a DANGER point can also be interpolated toward majority neighbours, pushing synthetic points across the boundary. [SOURCE: imbalanced-learn repo]

Q2. How does BorderlineSMOTE decide a point is "in danger"?
A2. For each minority point it counts majority-class neighbours among its m_neighbors (default 10) nearest neighbours. If at least half but not all are majority → DANGER (used for synthesis). If all are majority → NOISE (skipped). Otherwise → SAFE (skipped). [SOURCE: imbalanced-learn repo, _in_danger_noise]

Q3. Is BorderlineSMOTE always better than vanilla SMOTE?
A3. No. It wins when the informative minority signal sits at a boundary and the interior is already separable. When classes are heavily intermixed or the dataset is tiny/high-dimensional, it can degenerate toward vanilla SMOTE or even hurt. [DERIVED from Han 2005 + Blagus & Lusa 2013]

Q4. Can I use BorderlineSMOTE with categorical features?
A4. BorderlineSMOTE works in continuous Euclidean space. For mixed numeric/categorical data, use SMOTENC; for all-categorical, use SMOTEN — both from imblearn.over_sampling. [SOURCE: imbalanced-learn repo]

Q5. Should I apply it to my live trading data?
A5. No — apply any over-sampling inside the training/CV loop of your audited model core, never on live inference. Our NSE stack keeps resampling strictly in Layer 2. [DERIVED from the two-layer engine design]

Q6. What comes after BorderlineSMOTE?
A6. V3 SVMSMOTE uses an SVM to locate border-support points, and KMeansSMOTE clusters before synthesising. They refine the "where to synthesise" question further. [SOURCE: imbalanced-learn repo; Nguyen et al. 2011]

TL;DR

BorderlineSMOTE synthesises minority samples only near the decision boundary, not in the safe interior. [DERIVED]
It tags each minority point SAFE / DANGER / NOISE by counting majority neighbours; only DANGER seeds new rows. [SOURCE: imbalanced-learn repo]
borderline-1 (default) keeps new points on the minority side; borderline-2 can push them into majority space. [SOURCE: imbalanced-learn repo]
It usually beats vanilla SMOTE when the boundary carries the signal, but ignores majority density and can amplify border noise. [DERIVED]
Always resample inside CV only; tune m_neighbors/k_neighbors; consider pairing with ENN. [DERIVED]
Next in series: V3 SVMSMOTE. Previous: V1 vanilla SMOTE.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

[SOURCE] Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. Advances in Intelligent Computing (ICIC 2005), Lecture Notes in Computer Science, vol. 3644, pp. 878–887. Springer. — original Borderline-SMOTE algorithm, SAFE/BORDERLINE/NOISE classification, borderline-1 vs borderline-2.
[SOURCE] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — vanilla SMOTE baseline.
[SOURCE] scikit-learn-contrib/imbalanced-learn GitHub repository — imblearn/over_sampling/_smote/filter.py (BorderlineSMOTE, m_neighbors, kind, in_danger_indices_) and imblearn/over_sampling/_smote/base.py (_in_danger_noise danger/noise mask logic). Primary source for the implementation described.
[SOURCE] Blagus, R., & Lusa, L. (2013). A note on the estimation of the optimal SMOTE oversampling ratio. / SMOTE can degrade performance on small, high-dimensional datasets. — counter-evidence.
[SOURCE] Nguyen, H. M., Cooper, E. W., & Kamei, K. (2011). Borderline over-sampling for imbalanced data classification. — basis for SVM-SMOTE (V3).
[DERIVED] Interpolation formula x_new = x_i + (x_z − x_i)·δ and the "where to synthesise" interpretation, derived from the cited sources.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE Family series (V2 of V6). This is educational content; not investment advice and not SEBI-registered research.

Resources & Links

Brand site: https://optiontradingwithai.in
About the author: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
Previous article (V1 vanilla SMOTE): SMOTE_V1_smote.md
Next article (V3 SVMSMOTE): SMOTE_V3_svmsmote.md
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

Vanilla SMOTE in imbalanced-learn: The kNN Interpolation Engine Behind Balanced XGBoost

shakti tiwari — Tue, 08 Sep 2026 04:30:53 +0000

Vanilla SMOTE in imbalanced-learn: The kNN Interpolation Engine Behind Balanced XGBoost

Shakti Tiwari — Nifty Option Trader, XGBoost Expert

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

This is article V1 in the SMOTE cluster. PREV: S1 — The SMOTE Algorithm (intuition). NEXT: V2 — BorderlineSMOTE.

Quick Answer

Vanilla SMOTE from imbalanced-learn fixes class imbalance by synthesizing new minority-class samples as random convex combinations between each minority point and one of its k nearest minority neighbors — instead of blindly duplicating rows like random oversampling does (SOURCE: Chawla et al. 2002, JAIR 16:321–357). In imbalanced-learn the defaults are sampling_strategy='auto' (balance every minority class up to the majority count), k_neighbors=5, and an optional random_state for reproducibility (SOURCE: scikit-learn-contrib/imbalanced-learn, ~7.1k★). It helps most when the minority class forms a connected, dense region; it can hurt on tiny, high-dimensional, or noisy sets where invented points land in the wrong territory.

Why This Matters

Class imbalance is the silent killer of every Nifty option signal model. Your rare-event labels — a sharp expiry-day reversal, an OTM strangle that actually pays — might be 1–3% of rows. Train XGBoost on that raw and the tree will happily ignore the minority entirely because predicting "no move" already nets a 97% accuracy (DERIVED from the definition of accuracy on a 97/3 split). That 97% is a lie you can trade against at your own peril.

This is precisely why our resampling discipline lives in Layer 2 of the engine. Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. You never synthesize imaginary ticks on the live feed — that would be fraud dressed as feature engineering.

Understanding vanilla SMOTE — the plain SMOTE class, no borderline logic, no adaptive weighting — is the foundation. Every variant (BorderlineSMOTE, SVMSMOTE, ADASYN, KMeansSMOTE) is a mutation of this same interpolation core. If you don't understand the base, the variants are just incantations.

Research Question / Hypothesis

RQ: What exactly distinguishes vanilla SMOTE from random oversampling, how does its kNN interpolation behave under the default sampling_strategy='auto' / k_neighbors=5 configuration, and under what data-geometry conditions does it improve a classifier versus degrade it?

Hypothesis (DERIVED): Because SMOTE injects new points along minority–minority segments rather than replaying identical rows, it should (a) reduce the overfitting that duplication causes and (b) smooth the decision boundary — provided the minority manifold is locally coherent. When that manifold is sparse or the features are noisy/high-dimensional, the synthetic points will be unreliable and can poison the boundary instead.

Data & Methodology

We do not run code in this article. The snippet below is illustrative only, reproduced from the canonical imbalanced-learn usage shape — it is NOT an experiment result (SOURCE: scikit-learn-contrib/imbalanced-learn documentation). Treat it as the API contract, nothing more.

## Illustrative only — API shape from imbalanced-learn docs, not an experiment result.
from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y)

smote = SMOTE(
    sampling_strategy='auto',   # balance all minority classes to majority count
    k_neighbors=5,              # nearest minority neighbors to interpolate between
    random_state=42             # reproducibility
)

X_res, y_res = smote.fit_resample(X_train, y_train)

The methodology we describe is the algorithmic specification as given by Chawla et al. (2002) and as implemented in the imbalanced-learn repository (SOURCE for both). We break the vanilla procedure into its atomic steps and then derive the geometry of one synthetic point by hand in the next section.

The four atomic steps (SOURCE: Chawla 2002; SOURCE: imbalanced-learn SMOTE)

Isolate the minority set. For each minority class C (when sampling_strategy='auto', every class smaller than the majority), take its instances T_C.
Set the target count. sampling_strategy decides how many synthetic samples to make. With 'auto', the target equals the majority class size, so the number of new points for class C is N_majority − N_C (DERIVED).
Find k nearest minority neighbors. For each minority point x_i, compute Euclidean distances to all other points of the same class and keep the k_neighbors closest. Default k_neighbors=5 (SOURCE: imbalanced-learn default).
Interpolate. For each x_i, draw a random neighbor x_zi from its k-NN set, draw δ ∼ Uniform(0,1), and emit: x_new = x_i + δ · (x_zi − x_i) (SOURCE: Chawla 2002, Equation for synthetic sample generation).

Repeat step 4 until the required count of synthetics is reached. Note what SMOTE does not do: it never duplicates an existing row. Every emitted point is strictly a new coordinate on a segment between two real minority points (DERIVED from the convex-combination formula with δ∈[0,1]; at δ=0 it equals x_i and at δ=1 it equals x_zi, but δ is drawn continuously so exact endpoints are measure-zero events).

Results / Findings

Below are the concrete behavioral findings, each anchored to a primary source or a derivation from the formula above.

F1 — SMOTE differs from random oversampling in mechanism, not just magnitude. Random oversampling copies existing minority rows until balance. SMOTE manufactures points in the convex hull of minority pairs (SOURCE: Chawla 2002 motivation section; DERIVED: copying ⇒ identical rows in training set ⇒ a tree can split on a memorized row, while interpolation ⇒ novel points).

F2 — The default sampling_strategy='auto' fully balances all minorities to the majority count. In imbalanced-learn, 'auto' is documented to resample all classes except the majority to the majority's cardinality (SOURCE: imbalanced-learn user guide). If you have a 1000/50 split, 'auto' creates 950 synthetic minors. You can also pass a float (e.g. 0.2 ⇒ minority becomes 20% of majority) or a dict (per-class overrides) — but the smart default is full balance (SOURCE: imbalanced-learn API).

F3 — k_neighbors=5 sets the locality of invention. Smaller k ⇒ synthetics hug tighter neighborhoods (risk: over-local, can create tiny clusters); larger k ⇒ synthetics spread toward farther cousins (risk: blend across potentially distinct sub-manifolds). The choice is a bias–variance dial on the minority geometry (DERIVED from k-NN smoothing theory).

F4 — random_state makes the synthetics reproducible, not the model. Same random_state ⇒ same synthetic coordinates across runs; the downstream classifier still has its own seed. This is essential for the walk-forward CV in Layer 2 so that two backtests of the same window are comparable (DERIVED from determinism of fit_resample under fixed seed; SOURCE: imbalanced-learn design).

F5 — SMOTE operates in raw feature space with Euclidean distance. It has no notion of feature scale, categorical vs numeric, or label noise. That is both its strength (simplicity, speed) and its trap (DERIVED from the algorithm using raw coordinates).

Reproducibility (code shape — illustrative)

As stated up top, the following is the canonical imbalanced-learn call shape and is shown for reproducibility of API usage, not as a claim that we executed a model here. The pipeline pattern below is what we actually wire into Layer 2 CV (SOURCE: imbalanced-learn Pipeline + SMOTE examples).

## Illustrative only — shows where SMOTE sits: INSIDE CV, never on the test set.
from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTE
from xgboost import XGBClassifier
from sklearn.model_selection import StratifiedKFold

pipe = Pipeline([
    ('smote', SMOTE(sampling_strategy='auto', k_neighbors=5, random_state=42)),
    ('clf',   XGBClassifier(n_estimators=300, max_depth=4,
                            eval_metric='logloss', random_state=42))
])

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
## pipe.fit(X_train_fold, y_train_fold)  ->  scored on the untouched validation fold

The golden rule, repeated because people violate it constantly: fit_resample must be called only on training folds, never on validation or test data, or you leak synthetic minorities into your evaluation and your backtest lies (DERIVED from CV hygiene; echoed in every imbalanced-learn tutorial — here SOURCE: imbalanced-learn "avoid leakage" guidance).

Worked 2-D Interpolation Example (DERIVED math)

Let's make the formula concrete in two dimensions so you see what SMOTE emits. Suppose one minority point is A = (2, 3) and its randomly chosen 5-NN neighbor is B = (4, 7). The segment AB has direction B − A = (2, 4).

SMOTE draws δ ∼ Uniform(0,1) and computes A + δ·(B−A). Three independent draws:

δ	x = 2 + δ·2	y = 3 + δ·4	Synthetic point
0.25	2.50	4.00	(2.50, 4.00)
0.50	3.00	5.00	(3.00, 5.00) ← exact midpoint
0.80	3.60	6.20	(3.60, 6.20)

Every synthetic lies on the line segment between A and B, never beyond it, because δ∈[0,1] (DERIVED). Now scale this: with k_neighbors=5, each minority point gets 5 candidate neighbors, so the set of possible segments is rich, and SMOTE samples different neighbors and different δ to fill the minority region with a cloud of plausible in-between points rather than 5 copies of A. This is the entire conceptual difference from random oversampling, which would simply append (2,3) five more times (DERIVED comparison).

A second minority point C = (5, 1) with neighbor D = (6, 4) (D−C = (1,3)) and δ=0.6 yields (5.6, 2.8). Notice the synthetic cloud now spans two distinct minority sub-regions, and a classifier trained on the augmented set learns a boundary that interpolates between real minors instead of over-memorizing them (DERIVED; consistent with Chawla 2002's stated intent — SOURCE).

What Failed / Counter-Evidence

SMOTE is not a free lunch, and the literature is blunt about it.

C1 — Blagus & Lusa (2013) show SMOTE can degrade performance on small or high-dimensional datasets. When the minority class has few samples or the feature space is wide, the k-NN neighborhoods become unreliable — synthetic points get placed in regions that don't reflect the true minority distribution, and classifiers trained on them perform worse than on the raw imbalanced data (SOURCE: Blagus & Lusa 2013). This is the single most important caveat for options data, where rare events are genuinely sparse.

C2 — Synthetic points can cross the decision boundary into majority territory. If a minority point sits near the majority region, its neighbor-based interpolation can emit a point that actually belongs to the majority class. SMOTE has no label-check; it just stamps "minority" on whatever it creates (DERIVED from the algorithm lacking any label-consistency test; SOURCE-adjacent: this limitation motivates BorderlineSMOTE and SVMSMOTE).

C3 — Random oversampling sometimes matches or beats SMOTE. On datasets where the minority class is already well-separated, simply duplicating can be sufficient, and the added complexity of SMOTE buys nothing (SOURCE: multiple empirical comparisons in the imbalanced-learning literature; DERIVED: more synthetic noise without benefit when the boundary is already clear).

Limitations

Scale blindness. Euclidean distance on unstandardized features lets large-magnitude columns dominate neighbor selection. Always scale inside the CV pipeline (DERIVED from distance geometry).
No categorical handling. Vanilla SMOTE interpolates numerically; mixing it on one-hot or label-encoded categoricals produces impossible hybrids (e.g. 0.4 in a binary column). Use SMOTENC/SMOTEN for mixed types (SOURCE: imbalanced-learn SMOTENC/SMOTEN).
Curse of dimensionality. In high-D spaces, "nearest" neighbors are often far, so synthetic points are wild (DERIVED from concentration of distances; SOURCE-adjacent: Blagus & Lusa 2013).
Leakage risk. Applied outside CV, it contaminates evaluation (see Reproducibility).
Assumes connected manifold. Discrete, multi-modal minorities with gaps violate the interpolation assumption (DERIVED).

Practical Takeaways (Production Checklist)

Use SMOTE(sampling_strategy='auto', k_neighbors=5, random_state=42) as the default baseline; tune k_neighbors only after baseline CV (SOURCE: imbalanced-learn defaults; DERIVED tuning order).
Always wrap SMOTE in a Pipeline so it fits per-fold, never on the full set (SOURCE: imbalanced-learn Pipeline guidance).
Scale features before SMOTE inside the same pipeline (DERIVED).
Compare against scale_pos_weight in XGBoost — often cheaper and leakage-free; SMOTE earns its place when the minority geometry is genuinely learnable (DERIVED from practice; consistent with imbalanced-learn recommendations).
For Nifty option rare-event labels: start with scale_pos_weight, add vanilla SMOTE only inside Layer 2 CV, and watch for boundary-crossing degradation on sparse expiries (DERIVED from our two-layer discipline).
If vanilla SMOTE underperforms, the next article (V2 BorderlineSMOTE) shows the targeted variant that only synthesizes near the decision border.

FAQ

Q: Is vanilla SMOTE the same as random oversampling?
No. Random oversampling duplicates existing minority rows; SMOTE creates new points interpolated between minority neighbors (SOURCE: Chawla 2002; DERIVED comparison).

Q: What is the default sampling_strategy?
'auto', which balances every minority class up to the majority count (SOURCE: imbalanced-learn).

Q: What does k_neighbors do?
It sets how many nearest minority neighbors each point can interpolate with; default 5 (SOURCE: imbalanced-learn default).

Q: Does random_state make my model reproducible?
It makes the synthetic samples reproducible. The classifier needs its own seed (DERIVED).

Q: Can I use SMOTE on live Nifty ticks?
No — never on live data. Our stack applies it only in Layer 2 EOD CV (two-layer engine note above; DERIVED from our discipline).

Q: When does SMOTE hurt?
Small/high-dimensional/noisy minorities where neighbors are unreliable (SOURCE: Blagus & Lusa 2013).

Q: What comes after vanilla SMOTE?
BorderlineSMOTE (V2), then SVMSMOTE, ADASYN, KMeansSMOTE — the variant family covered in later articles.

TL;DR

Vanilla SMOTE synthesizes minority samples as convex combinations x_i + δ·(x_zi − x_i) between a point and one of its k_neighbors (default 5) minority neighbors, balancing classes via sampling_strategy='auto'. Unlike random oversampling (which duplicates), it invents plausible in-between points — great on dense, connected minorities, risky on sparse/high-dimensional/noisy ones. Always run it inside CV, never on live or test data.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources (PRIMARY)

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research (JAIR), 16:321–357. — original algorithm, interpolation formula, motivation vs random oversampling.
scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k★). imblearn.over_sampling.SMOTE — default parameters (sampling_strategy='auto', k_neighbors=5), fit_resample API, Pipeline/leakage guidance, SMOTENC/SMOTEN for categoricals.
Blagus, R., Lusa, L. (2013). SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. — evidence SMOTE can degrade on small/high-dimensional data.
Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE — previewed variant (V2).
He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN — adaptive variant.
Nguyen, H.M., Cooper, E.W., Kamei, K. (2011). SVM-SMOTE — SVM-margin variant.
KMeansSMOTE (2018, last-resort paper) — clustering-guided variant.

Author / Canonical

Disclaimer (repeat): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

Brand site: https://optiontradingwithai.in
About: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
PREV article: S1 — The SMOTE Algorithm (intuition)
NEXT article: V2 — BorderlineSMOTE
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

SMOTE random_state & Reproducibility: Why Seeded Synthetic Samples Matter for Auditable XGBoost Backtests

shakti tiwari — Mon, 07 Sep 2026 04:31:20 +0000

SMOTE `random_state` & Reproducibility: Why Seeded Synthetic Samples Matter for Auditable XGBoost Backtests

Shakti Tiwari — Nifty Option Trader, XGBoost Expert

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

This is article S4 in the SMOTE cluster. PREV: S3 — sampling_strategy. NEXT: V3 — SVMSMOTE.

Quick Answer

SMOTE is not deterministic by default. For every synthetic minority sample it draws a random neighbor from the k-NN set and a random interpolation factor λ ∼ Uniform(0,1) — both pulled from a pseudo-random number generator (SOURCE: Chawla et al. 2002, JAIR 16:321–357; SOURCE: scikit-learn-contrib/imbalanced-learn, ~7.1k★). If you do not pass random_state, the two draws come from the global NumPy RNG and change every run. Pass random_state=<int> and fit_resample emits the exact same synthetic coordinates each time. That reproducibility is what makes a backtest auditable — and our Layer-2 training core depends on it.

Why This Matters

Here is the uncomfortable truth about options trading research: a backtest you cannot reproduce is a backtest you cannot trust. Aur bhai, agar kal ki run mein 18% return dikh raha tha aur aaj 11% — and you changed nothing — then your pipeline has a hidden non-determinism, and your "edge" might just be noise wearing a tuxedo.

Class imbalance is everywhere in Nifty option signals. Your rare-event label — a sharp expiry-day reversal, an OTM strangle that actually pays, a VIX-spike continuation — might be 1–3% of rows. To teach XGBoost that minority, we often synthesize minority samples with SMOTE inside the training fold. That synthesis is random. If the random draw is not pinned, two backtests of the same historical window produce different synthetic training sets, different models, and different reported metrics. Now tell me: which number do you show the client? Which number do you defend in an audit?

This is precisely why our resampling discipline lives in Layer 2 of the engine. Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. Within that audited core, every fit_resample call is seeded so that a reviewer can reconstruct the exact training set that produced a given model and a given walk-forward metric. Reproducibility is not a nice-to-have here; it is the whole point of "audited."

Understanding random_state — what it controls, what it does not control, and how it tangles with the global NumPy/sklearn RandomState — is the difference between a research notebook that lies silently and one that can be replayed verbatim. Let's pull it apart.

Research Question / Hypothesis

RQ: What exactly does SMOTE's random_state parameter govern? How does it interact with the global NumPy/sklearn RandomState? And why is a fixed seed necessary — but not sufficient — for an auditable, reproducible backtest?

Hypothesis (DERIVED): Because _make_samples consumes two RNG draws per synthetic point — (a) a random neighbor index and (b) a random λ ∈ [0,1] — the entire synthetic set is a deterministic function of (X_minority, k_neighbors, sampling_strategy, random_state). Seeding it makes the synthetic set reproducible. But the model needs its own seed, and the global RNG can still leak into SMOTE if random_state=None, so a fixed SMOTE seed alone does not guarantee a reproducible end-to-end pipeline.

Data & Methodology

We do not run code in this article. The snippets below are illustrative only, reproduced from the canonical imbalanced-learn usage shape — they are NOT experiment results (SOURCE: scikit-learn-contrib/imbalanced-learn documentation). Treat them as the API contract, nothing more.

## Illustrative only — API shape from imbalanced-learn docs, not an experiment result.
from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y)

smote = SMOTE(
    sampling_strategy='auto',   # set by the PREV article (S3)
    k_neighbors=5,
    random_state=42             # <-- the subject of THIS article
)

X_res, y_res = smote.fit_resample(X_train, y_train)

The methodology we describe is the algorithmic specification as given by Chawla et al. (2002) and as implemented in the imbalanced-learn repository (SOURCE for both). We then derive, from the interpolation formula and from sklearn's documented check_random_state semantics, exactly which RNG draws SMOTE makes and how a seed pins them.

Where the randomness actually lives (SOURCE: Chawla 2002; SOURCE: imbalanced-learn `BaseSMOTE._make_samples`)

Recall the synthetic-sample formula from the algorithm article (S1):

x_new = x_i + λ · (x_zi − x_i),   λ ~ U(0,1)

For each minority point x_i that must be "grown," SMOTE must choose:

A random neighbor x_zi — drawn from the k_neighbors nearest minority neighbors of x_i. This is one RNG draw (an integer index in [0, k_neighbors)).
A random interpolation factor λ — drawn uniformly from [0,1]. This is a second RNG draw.

Both draws are taken from a numpy.random.RandomState object that SMOTE obtains through sklearn.utils.check_random_state(random_state) (SOURCE: scikit-learn check_random_state contract; SOURCE: imbalanced-learn uses this). Therefore the sequence of neighbor picks and λ values — and hence every synthetic coordinate — is fully determined by the seed fed into that RandomState. With random_state=None, the object is the global np.random singleton, which is mutated by every random call in your program (DERIVED from check_random_state's documented behaviour for the None case).

Results / Findings

Each finding is anchored to a primary source or derived from the formula / RNG contract above.

F1 — Without random_state, the synthetic set differs every run. Because neighbor index and λ are RNG draws, two consecutive fit_resample calls with random_state=None (and no intervening global seed reset) produce different synthetic points (DERIVED from the two-draw mechanism). Concretely, the same minority point can be paired with a different neighbor and a different λ, landing at a different coordinate (SOURCE: imbalanced-learn design; DERIVED).

F2 — With random_state=<int>, the synthetic set is bit-for-bit reproducible. Passing an integer creates a fresh np.random.RandomState(seed) that is independent of the global stream, so repeated fit_resample calls emit identical synthetics (SOURCE: check_random_state int-branch; DERIVED). This is the property an audited backtest needs: same data + same seed ⇒ same training set ⇒ same model ⇒ same metric.

F3 — random_state=None silently couples SMOTE to the global NumPy RNG. When random_state=None, check_random_state returns the global np.random singleton (SOURCE: sklearn contract). That means np.random.seed(7) somewhere else in your script shifts SMOTE's draws, and other np.random calls (shuffles, dropouts, initializations) consume from the same stream and displace SMOTE's draws. This is the single most common "why did my synthetic set move?" bug (DERIVED from global-singleton sharing).

F4 — Passing an int random_state decouples SMOTE from the global RNG. Once you pass 42, SMOTE builds its own RandomState(42) and ignores the global np.random state entirely (DERIVED from check_random_state int-branch). So np.random.seed(999) no longer moves SMOTE's synthetics — but it does still affect anything else using the global stream. Mixing the two mental models is where teams get confused (DERIVED).

F5 — random_state reproduces the synthetics, not the model. The classifier (XGBoost/LightGBM/RF) has its own seed. A seeded SMOTE gives you a reproducible training set; the model trained on it still needs its own random_state/seed for a reproducible model (DERIVED; SOURCE-adjacent: every imbalanced-learn SMOTE+Pipeline example seeds both SMOTE and the estimator).

F6 — Reproducibility ≠ validity. A perfectly reproducible synthetic set can still be a bad synthetic set (e.g. boundary-crossing points on sparse expiries). Seeding guarantees you can replay the mistake, not that the mistake is small (DERIVED; consistent with Blagus & Lusa 2013's caution that SMOTE can hurt on small/high-dimensional data — SOURCE).

Reproducibility (code shape — illustrative)

As stated up top, the following is the canonical imbalanced-learn call shape and is shown for reproducibility of API usage, not as a claim that we executed a model here. The pattern below is what we actually wire into Layer-2 walk-forward CV (SOURCE: imbalanced-learn Pipeline + SMOTE examples).

## Illustrative only — shows where random_state sits: on BOTH SMOTE and the estimator, INSIDE CV.
from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTE
from xgboost import XGBClassifier
from sklearn.model_selection import StratifiedKFold

SEED = 42   # ONE canonical seed, stored in config, version-controlled.

pipe = Pipeline([
    ('smote', SMOTE(sampling_strategy='auto', k_neighbors=5, random_state=SEED)),
    ('clf',   XGBClassifier(n_estimators=300, max_depth=4,
                            eval_metric='logloss', random_state=SEED))
])

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=SEED)
## pipe.fit(X_train_fold, y_train_fold) -> scored on the untouched validation fold

The golden rule, repeated because people violate it constantly: seed everything that touches the random stream — SMOTE, the splitter, and the estimator — and store that seed in version control next to the data snapshot. A backtest is only auditable if a reviewer can re-run it and land on the same number (DERIVED from CV hygiene; echoed in imbalanced-learn "avoid leakage" guidance — SOURCE).

Worked Example: Same Data, Two Seeds, Two Synthetic Sets (DERIVED math)

Let's make the seed-dependence concrete in two dimensions. Take one minority point A = (2, 3). Its k_neighbors=2 minority neighbors are B = (4, 7) and C = (1, 5), so the candidate segment set is AB and AC.

SMOTE makes two draws per synthetic: (neighbor index ∈ {0,1}, λ ∼ U(0,1)).

Run 1 — random_state=42: Suppose the RNG yields (neighbor=0 → B, λ=0.25).

x_new = A + 0.25·(B − A) = (2,3) + 0.25·(2,4) = (2.50, 4.00)

Run 2 — random_state=7: Suppose the RNG yields (neighbor=1 → C, λ=0.80).

x_new = A + 0.80·(C − A) = (2,3) + 0.80·(−1,2) = (1.20, 4.60)

Two different seeds ⇒ two different neighbors ⇒ two different λ ⇒ two synthetic points that are not even on the same segment. Multiply this by hundreds of minority points and you get entirely different synthetic clouds from the same real minority set (DERIVED). This is exactly why an unseeded SMOTE poisons backtest comparability: you are not comparing models, you are comparing random draws.

Key nuance (DERIVED): the real minority points A, B, C never change — only the invented ones do. So unseeded SMOTE doesn't alter your ground truth; it alters the training fiction you built on top of it, and that fiction is what your model memorizes.

What Failed / Counter-Evidence

Seeding SMOTE is necessary, but several things still break end-to-end reproducibility even when random_state is set. Be honest about these:

C1 — Global np.random leakage via random_state=None. If any colleague "cleans up" the code and removes random_state (thinking the global np.random.seed() call covers it), SMOTE rejoins the global stream and the synthetic set drifts again (DERIVED from F3/F4). We have seen this exact regression in shared notebooks (SOURCE-adjacent: a well-known failure mode in imbalanced-learn GitHub issues).

C2 — The estimator's own non-determinism dominates. Even with a seeded SMOTE, XGBoost on GPU or with certain histogram threading can return slightly different trees across runs; LightGBM's feature-parallel splits can, too. So "I seeded SMOTE, why isn't the metric identical?" — because the model seed or hardware threading moved (DERIVED from XGBoost/LightGBM known non-determinism on parallel/GPU paths). SMOTE seeding is necessary but not sufficient (DERIVED from F5).

C3 — Library version drift changes the RNG stream. The exact sequence of draws produced by a given integer seed depends on the implementation of the sampler and the RNG backend. A different imbalanced-learn / scikit-learn / numpy version can emit a different synthetic set for the same random_state (DERIVED from version-sensitive RNG implementations; SOURCE-adjacent: sklearn's documented RNG/Generator migration notes). Pin your versions — reproducibility includes the software, not just the seed (DERIVED engineering principle).

C4 — Different seed per fold hides instability. Some teams reseed SMOTE per CV fold "to get more data." That means each fold trains on a different synthetic cloud, so fold metrics are not comparable across reruns and the variance of the synthetic step is masked (DERIVED). For auditability you want the same seed every fold, every run.

C5 — Pickling SMOTE without recording the seed. If you joblib.dump a fitted SMOTE and later fit_resample a fresh instance without the seed, you get a new synthetic set (DERIVED from F2). Store the seed alongside the artifact.

Limitations

Seeding controls randomness, not correctness. A reproducible bad synthetic set is still bad (see F6; SOURCE: Blagus & Lusa 2013 on SMOTE's limits on small/high-dimensional data).
random_state=None is the default trap. If you instantiate SMOTE() with no seed, you opted into global-stream coupling (DERIVED from defaults; SOURCE: imbalanced-learn default random_state=None).
Global-singleton sharing is invisible. Nothing in the API warns you that random_state=None shares np.random with every other random call (DERIVED from check_random_state semantics).
Parallelism can reintroduce non-determinism in the downstream model, even when SMOTE is perfectly seeded (see C2).
Version sensitivity. Same seed, different library version ⇒ different synthetics (C3).
Assumes you actually re-run on the same X/y snapshot. Reproducibility of synthetics is moot if the input data itself changed between runs (DERIVED — data versioning is part of the audit).

Practical Takeaways (Production Checklist)

Always pass random_state=<int> to SMOTE. Never leave it None in a research or production pipeline (SOURCE: imbalanced-learn API; DERIVED from F1/F3).
Pick ONE canonical seed (we use 42 as a documented default) and store it in config, version-controlled next to the data snapshot (DERIVED from audit discipline).
Seed the estimator too. XGBoost random_state, the StratifiedKFold random_state, and SMOTE's — all three (DERIVED from F5/C2).
Wrap SMOTE in a Pipeline so it fits per-fold, never on the full set, and the same seed applies consistently inside CV (SOURCE: imbalanced-learn Pipeline guidance).
Don't rely on np.random.seed() to control SMOTE — pass the int. Global seeding is fragile once any other random call exists (DERIVED from F3/F4).
Pin library versions (imbalanced-learn, scikit-learn, numpy, xgboost) in your environment lockfile; reproducibility includes the software (DERIVED from C3).
For Nifty option rare-event labels: apply SMOTE only inside Layer-2 EOD walk-forward CV, seed it, and compare against a scale_pos_weight baseline that is inherently seed-free (DERIVED from our two-layer discipline). When the minority is genuinely sparse, scale_pos_weight is often the more auditable choice.
Log the seed with every backtest run. An auditable run is (data hash, code hash, library versions, seed) → metric. Miss any one and you can't replay it (DERIVED).

FAQ

Q: Does SMOTE need random_state to work?

It works without it — but produces a different synthetic set every run. For any reproducible research or backtest, set it (SOURCE: imbalanced-learn; DERIVED from F1).

Q: What does random_state actually control in SMOTE?

The random neighbor index and the random λ ∈ [0,1] used in x_i + λ·(x_zi − x_i) — i.e. which synthetic points get made (SOURCE: Chawla 2002; DERIVED from _make_samples).

Q: If I set np.random.seed(42), is SMOTE reproducible?

Only if random_state=None — and even then, any other np.random call shifts SMOTE's draws. Pass random_state=42 to SMOTE directly; that decouples it from the global stream (DERIVED from F3/F4).

Q: Does seeding SMOTE make my XGBoost model reproducible?

No. It makes the training set reproducible. The model needs its own seed (F5).

Q: Why does reproducibility matter for trading backtests?

Because an auditable backtest must be replayable: same data + same seed + same code + same versions ⇒ same number. Otherwise you can't defend or compare results (DERIVED from our Layer-2 audit discipline).

Q: Can two seeds give totally different synthetic clouds?

Yes — different neighbor picks and λ values put synthetic points on different segments (see Worked Example). Same real minority, different fiction (DERIVED).

Q: Is random_state the same across SMOTE variants (SVMSMOTE, Borderline, ADASYN)?

Yes, they all inherit the same check_random_state contract, so the seeding principle is identical; only the selection of points differs (SOURCE: imbalanced-learn; DERIVED). The NEXT article (V3 SVMSMOTE) covers that variant.

Q: Can I use SMOTE on live Nifty ticks?

No — never on live data. Our stack applies it only in Layer-2 EOD CV (two-layer engine note above).

TL;DR

SMOTE draws a random neighbor and a random λ∼U(0,1) per synthetic sample, so without random_state the synthetic set changes every run. Pass random_state=<int> to make it bit-for-bit reproducible — essential for auditable Layer-2 backtests. But note: random_state=None silently couples SMOTE to the global NumPy RNG, an int seed decouples it, and the model still needs its own seed. Seeding is necessary, not sufficient: pin library versions, seed the splitter and estimator too, and log the seed with every run. Source: Chawla et al. 2002 + imbalanced-learn.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources (PRIMARY)

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research (JAIR), 16:321–357. — original algorithm; interpolation formula x_new = x_i + λ·(x_zi − x_i) with λ drawn randomly; motivation vs random oversampling.
scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k★). imblearn.over_sampling.SMOTE — random_state parameter (default None), fit_resample API, BaseSMOTE._make_samples neighbour/λ draws, Pipeline/leakage guidance, SVMSMOTE/BorderlineSMOTE/ADASYN/KMeansSMOTE variants sharing the same check_random_state contract.
scikit-learn sklearn.utils.check_random_state contract — None ⇒ global np.random singleton; int ⇒ np.random.RandomState(int); RandomState instance ⇒ used as-is. (SOURCE: scikit-learn API docs; DERIVED behaviour described above.)
Blagus, R., Lusa, L. (2013). SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. — evidence SMOTE can degrade on small/high-dimensional data (reproducibility does not imply validity; F6/C1 context).
Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE — boundary-aware variant (previewed; NEXT family).
He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN — adaptive variant sharing the same seeding contract.
Nguyen, H.M., Cooper, E.W., Kamei, K. (2011). SVM-SMOTE — SVM-margin variant (NEXT: V3).
KMeansSMOTE (2018, last-resort paper) — clustering-guided variant.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE engineering cluster, sibling to the XGBoost cluster. This article (S4, random_state & reproducibility) is the canonical reference for seeding SMOTE and for understanding its interaction with the global NumPy/sklearn RandomState inside an auditable Layer-2 backtest. The PREV article (S3, sampling_strategy) covers how many synthetics to make; the NEXT article (V3, SVMSMOTE) covers which points the variant synthesizes.

Disclaimer (repeat): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

Brand site: https://optiontradingwithai.in
About: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
PREV article: S3 — sampling_strategy
NEXT article: V3 — SVMSMOTE
Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF

SMOTE `sampling_strategy`: How to Set the Oversampling Ratio Without Drowning Your Majority Class

shakti tiwari — Mon, 07 Sep 2026 04:30:59 +0000

SMOTE `sampling_strategy`: How to Set the Oversampling Ratio Without Drowning Your Majority Class

A practical guide for Nifty option traders and XGBoost practitioners — by Shakti Tiwari

Quick Answer

SMOTE's sampling_strategy decides how much of the minority class gets synthesized. A string like 'not majority' (the older 'auto') forces a 1:1 balance; a float such as 0.5 makes the minority half the size of the majority; a dict lets you pin exact per-class counts; and 'minority'/'all' control which classes get touched. For rare-event models — like Nifty's rare big-move days — a forced 1:1 usually over-injects synthetic noise and hurts the real signal, so a moderate ratio (1:2 or 1:3) typically wins. [SOURCE: imbalanced-learn docs; DERIVED]

Disclaimer (verbatim): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Why This Matters

Class imbalance is the silent killer of trading models. In Nifty option data, the class you actually care about — a genuinely big directional move that makes a straddle/strangle payoff — shows up maybe 3–8% of the time. The "nothing happened" days dominate. [DERIVED from typical Nifty daily-range distributions] If you hand that raw distribution to XGBoost, the tree will happily learn to predict "no move" on every row and still score 94% accuracy. Useless. You've built a coin that's right because the market is boring, not because you're smart.

SMOTE (Synthetic Minority Over-sampling Technique) fixes this by manufacturing new minority samples in feature space instead of copying existing rows (that would just overfit). [SOURCE: Chawla et al. 2002] But here's the trap most tutorials skip: SMOTE has a dial, and leaving it at the default is often wrong. That dial is sampling_strategy.

The default behaviour in imbalanced-learn oversamples the minority class until it matches the majority class — a perfect 1:1 ratio. That sounds fair. It's usually not. When the true prior is 5% minority, forcing 50% minority teaches your model a world that doesn't exist, and it starts crying wolf on quiet days. For an option seller, that's a margin-eating disaster. [DERIVED]

This article is the third in our SMOTE deep-dive series. We already covered k_neighbors (S2). Here we tear apart sampling_strategy — every option, the math, and how to pick a ratio that survives walk-forward, not just a tidy notebook. [SOURCE: imbalanced-learn docs]

Research Question / Hypothesis

Research question: For a binary Nifty "big-move vs no-big-move" classifier, what sampling_strategy value maximizes out-of-sample recall and F1 without inflating false positives?

Hypothesis: A moderate oversampling ratio (minority-to-majority ≈ 0.3–0.5, i.e. 1:3 to 1:2) will beat forced 1:1 ('not majority'), because 1:1 (a) drowns the majority signal, (b) stacks synthetic density in borderline regions where classes overlap, and (c) shifts the effective class prior far from reality. [HYPOTHESIS; DERIVED]

Note: this is a hypothesis framed for walk-forward testing inside our engine. The numeric examples below are derived illustrations, not experiment results — I did not execute code in writing this piece. The canonical snippet in the Reproducibility section is taken from the imbalanced-learn documentation shape and is shown for structure only. [SOURCE: imbalanced-learn docs]

Data & Methodology (the Nifty angle)

Picture a realistic daily feature set for Nifty50:

Features (X): previous-day range, ATM implied volatility, PCR (put-call ratio), VIX, overnight gap, RSI(14), distance-to-spot of nearest strike, session volume z-score.
Target (y): 1 = big move day (|Nifty close − open| > 1.5× trailing 20-day ATR); 0 = quiet day.

A typical year gives roughly 9500 quiet days (class 0, the majority) and 500 big-move days (class 1, the minority) across the historical window we study. That's a ~5% prior. [DERIVED illustration, not a measured dataset]

Two-layer engine note (must-read for reproducibility): Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. SMOTE only ever touches the training fold; the validation fold stays raw so we measure honest performance on the true distribution. [ARCHITECTURE NOTE — internal stack]

This matters enormously for sampling_strategy: the ratio you choose is a hyperparameter tuned by cross-validation, not a one-time setting. More on that in Practical Takeaways.

The `sampling_strategy` Parameter: Every Option Explained

sampling_strategy accepts four shapes: str, float, dict, or callable. Let's go through each. [SOURCE: imbalanced-learn docs]

1. String options

'not majority' — resample every class except the majority class up to the majority's count. For a binary problem this means minority → majority size, i.e. 1:1 balance. [SOURCE: imbalanced-learn docs]
'auto' — historically the default; for over-sampling it is equivalent to 'not majority'. Deprecated since imbalanced-learn 0.9+ in favour of 'not majority'. If you see 'auto' in old code, it means 1:1. [SOURCE: imbalanced-learn docs]
'minority' — resample only the single minority class (to majority size). In binary classification this yields the same 1:1 result as 'not majority', but in multiclass it differs: 'minority' balances only the smallest class, while 'not majority' balances all minority classes. [SOURCE: imbalanced-learn docs; DERIVED]
'all' — resample all classes. In binary this again collapses to 1:1; it's more meaningful in multiclass setups. [SOURCE: imbalanced-learn docs]
'not minority' — resample everything except the minority. This is primarily an under-sampling notion; you'll rarely use it with SMOTE over-sampling. [SOURCE: imbalanced-learn docs]

Takeaway: the string options are coarse. They're either "balance everything to 1:1" or "balance a subset to 1:1." They give you no control over the degree of oversampling. That's why the float and dict forms exist.

2. Float ratio (the fine dial)

When sampling_strategy is a float between 0 and 1, it specifies the desired ratio of minority samples to majority samples after resampling:

α = N_minority(resampled) / N_majority(original)

So:

0.5 → minority becomes half the majority (1:2). [SOURCE: imbalanced-learn docs; DERIVED]
0.33 → minority becomes a third of the majority (1:3).
1.0 → 1:1 (same as 'not majority').
0.1 → minority is just 10% of majority (gentle nudge).

Critical constraint: the float form is only valid for binary classification. For multiclass you must use a dict (next). [SOURCE: imbalanced-learn docs]

3. Dict (per-class targets — full control)

A dict maps each class label to the exact number of samples you want after resampling:

sampling_strategy = {1: 2000}   # make the big-move class (label 1) have 2000 samples
sampling_strategy = {0: 9500, 1: 3000}  # explicit, both classes

The keys are class labels; the values are desired counts, not ratios. [SOURCE: imbalanced-learn docs] This is the most explicit form and the one I recommend once you've grid-searched a good target count. It also works for multiclass imbalance, where you might want class A at 4000 and class B at 2500 independently.

4. Callable (advanced)

A callable receives the y array and must return a dict of target counts. Useful for programmatic, data-dependent logic (e.g. "oversample each minority class to 60% of the largest"). [SOURCE: imbalanced-learn docs] Most traders never need this; the float and dict cover 99% of cases.

Results / Findings (worked numerical example)

Let's make the math concrete with our Nifty illustration: N_majority = 9500, N_minority = 500. Watch how the synthetic count explodes as the ratio approaches 1:1. [DERIVED — illustrative arithmetic, not measured]

`sampling_strategy`	Minority after	Synthetic added	Total rows	Ratio
`'not majority'` (1:1)	9500	9000	19000	1:1
`0.5`	4750	4250	14250	1:2
`0.33`	3135	2635	12635	1:3
`0.2`	1900	1400	11400	1:5
`{1: 2000}`	2000	1500	11500	~1:4.75

[DERIVED from α = N_min/N_maj; rounding applied]

Two things jump out:

1:1 adds 9000 synthetic points — 18× the real minority count. You are now trusting generated data more than real data. Chawla et al. (2002) originally demonstrated SMOTE by oversampling the minority to about 2× its original size in their experiments — not 18×. Pushing far past that dilutes the genuine minority geometry. [SOURCE: Chawla et al. 2002; DERIVED]
A 1:2 or 1:3 ratio adds a fraction of that synthetic volume while still giving the learner enough minority mass to find a boundary. You get the benefit (the tree actually splits on the rare class) without the cost (a synthetic fog covering half your feature space).

Why 1:1 oversampling often hurts

It drowns the majority signal. At 1:1 the model's training prior is 50/50, but reality is 5/95. The classifier is implicitly biased toward crying "big move!" — raising false positives. In options, that means legging into trades on quiet days and bleeding theta. [DERIVED from class-prior shift]
It over-packs the borderline region. SMOTE creates each synthetic point on the line segment between two real minority points (plus small Gaussian jitter on the k_neighbors). At 18× density, those segments tile the entire convex hull of the minority region — including zones that overlap the majority class. The boundary gets fuzzy and overfit. [SOURCE: Chawla et al. 2002; DERIVED]
It wastes the very signal that makes the majority useful. The quiet days aren't noise; they carry the "normal regime" shape. Flattening their relative weight teaches the model to under-weight the regime it will actually live in 95% of the time. [DERIVED]

Why moderate ratios (1:2, 1:3) are usually better

They preserve skew awareness. At 1:3 the model still "knows" big moves are rare, so it doesn't casually flag quiet days.
Less synthetic density = less boundary overfit. Fewer interpolated points means each synthetic sample carries more marginal information. [DERIVED]
They're closer to the original SMOTE intent. Chawla's experiments oversampled modestly; the technique was never designed for 19× inflation. [SOURCE: Chawla et al. 2002]
Independent critiques agree caution is warranted. Blagus & Lusa (2013) showed SMOTE can degrade performance on small-sample and high-dimensional datasets — exactly the regime where aggressive oversampling amplifies noise. A gentler ratio is a cheaper, safer first move. [SOURCE: Blagus & Lusa 2013]

How to choose (decision flow)

Start moderate, not maximal. For Nifty big-move models, begin at sampling_strategy = 0.3 to 0.5 (1:3 to 1:2). [DERIVED recommendation]
Grid-search the ratio inside CV. Try {0.1, 0.2, 0.3, 0.5, 1.0} and rank by PR-AUC / F1, never raw accuracy (accuracy rewards the lazy "always quiet" model). [DERIVED]
Consider scale_pos_weight as an alternative. XGBoost natively reweights the minority gradient via scale_pos_weight = N_majority/N_minority (~19 here). SMOTE changes the data; scale_pos_weight changes the loss. They're not identical — SMOTE adds synthetic feature-space points (helping non-tree models and boundary geometry), while scale_pos_weight only reweights (cleaner, no synthetic leakage risk). Many production stacks use scale_pos_weight as the primary lever and SMOTE only when the minority is catastrophically small. [DERIVED; SOURCE: XGBoost docs convention]
Lock the winner as a dict once CV settles, e.g. {1: 3000}, so the count is explicit and reproducible across retrains. [DERIVED]
For multiclass (e.g. up-move / down-move / flat), skip the float — use a dict like {0: 6000, 1: 4000, 2: 5000}. [SOURCE: imbalanced-learn docs]

Reproducibility (code shape — illustrative only)

The snippet below follows the imbalanced-learn API exactly. It is reproduced from the library's documented usage pattern and is illustrative — I did not execute it, and no experiment results are claimed from it. [SOURCE: imbalanced-learn docs; NOT RUN]

from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline
from sklearn.model_selection import StratifiedKFold
from xgboost import XGBClassifier

## Illustrative only — not executed in this article.
## SMOTE must live INSIDE the CV pipeline so the validation fold stays raw.
smote = SMOTE(
    sampling_strategy=0.5,   # minority -> half of majority (1:2)
    k_neighbors=5,           # see S2: k_neighbors
    random_state=42          # see S4: random_state
)

pipe = Pipeline([
    ("smote", smote),
    ("xgb", XGBClassifier(n_estimators=300, max_depth=4,
                          scale_pos_weight=1.0))  # or tune jointly
])

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
## scores = cross_val_score(pipe, X, y, cv=cv, scoring="f1")

Key points that keep this honest:

fit_resample(X_train, y_train) is the canonical call; here it's wrapped in a Pipeline so it only fires on training folds. [SOURCE: imbalanced-learn docs]
sampling_strategy=0.5 is the float form (binary only). [SOURCE: imbalanced-learn docs]
k_neighbors and random_state are the neighbouring SMOTE dials (S2 and S4 of this series). [SOURCE: imbalanced-learn docs]

What Failed / Counter-Evidence

Being honest about where this advice bends:

1:1 is not always wrong. On very large datasets with strong regularization (shallow max_depth, high min_child_weight), a 1:1 balance can be fine because the model can't overfit the synthetic fog. The "1:1 hurts" claim is a default expectation, not a law. [DERIVED caveat]
SMOTE before the train/test split = leakage. If you resample the whole dataset and then split, synthetic points leak into the test set and your metrics lie. Always wrap SMOTE in the CV pipeline (as above). This is the #1 mistake I see in trading notebooks. [SOURCE: imbalanced-learn user guide; DERIVED]
SMOTE can hurt on small/high-dimensional data. Blagus & Lusa (2013) document cases where SMOTE degrades generalization — usually when the minority is tiny and features are noisy. In that regime, prefer scale_pos_weight or ADASYN/Borderline-SMOTE over vanilla SMOTE. [SOURCE: Blagus & Lusa 2013]
Borderline failure mode. When minority and majority heavily overlap, vanilla SMOTE will synthesize points inside the majority region, creating label-noise. That's the motivation for Borderline-SMOTE (Han, Wang & Mao 2005) and ADASYN (He et al. 2008), which focus synthesis near the decision boundary or on hard minority regions. [SOURCE: Han et al. 2005; He et al. 2008]

Limitations

Float form is binary-only. For multiclass Nifty regimes you must use a dict. [SOURCE: imbalanced-learn docs]
Dict requires you to know target counts up front; that's fine once CV has spoken, awkward as a first guess.
SMOTE assumes a meaningful Euclidean feature space. Raw one-hot categoricals or high-cardinality IDs break it. Use SMOTENC (mixed categorical/numeric) or SMOTEN (pure categorical) from the same library when features aren't continuous. [SOURCE: imbalanced-learn docs]
It does not by itself encode cost asymmetry. A false "big move" and a missed big move may cost very differently in options; pair SMOTE with scale_pos_weight or a custom objective for true cost-awareness. [DERIVED]
'auto' is deprecated. New code should write 'not majority', not 'auto'. [SOURCE: imbalanced-learn docs]

Practical Takeaways

✅ Default 'auto'/1:1 is a starting point, not a destination — tune the ratio.
✅ For Nifty big-move models, start at 0.3–0.5 (1:3 to 1:2) and CV-grid from there.
✅ Use the float form for binary problems; use a dict for multiclass or once you've locked a target count.
✅ Wrap SMOTE in a Pipeline inside StratifiedKFold — never resample before splitting.
✅ Rank candidates by F1 / PR-AUC, not accuracy.
✅ Keep SMOTE in Layer 2 (EOD training), inside CV — never on Dhan live WebSocket data.
✅ When the minority is tiny or features noisy, lean on scale_pos_weight or Borderline-SMOTE/ADASYN instead of vanilla SMOTE.
✅ Prefer explicit dict targets in production so retrains are reproducible.

FAQ

Q: Is 'auto' the same as 1:1?
Yes, for over-sampling 'auto' equalled 'not majority', which forces the minority to match the majority (1:1). It's now deprecated; write 'not majority'. [SOURCE: imbalanced-learn docs]

Q: How is the float ratio computed exactly?
α = (minority count after resampling) ÷ (majority count). So 0.5 means minority ends up at half the majority's size. Float is binary-only. [SOURCE: imbalanced-learn docs; DERIVED]

Q: SMOTE vs scale_pos_weight — which should I use?
SMOTE adds synthetic data points (helps boundary geometry, useful for non-tree models); scale_pos_weight reweights the loss (cleaner, no synthetic leakage). They're complementary; many stacks use scale_pos_weight as primary and SMOTE when the minority is extremely scarce. [DERIVED]

Q: Can I use a dict for multiclass imbalance?
Yes — that's the recommended path. Map each class label to its desired post-resample count, e.g. {0: 6000, 1: 4000, 2: 5000}. [SOURCE: imbalanced-learn docs]

Q: What ratio do you use for Nifty big-move days?
We start at 0.3–0.5 (1:3 to 1:2) and let walk-forward CV pick the winner by F1. We've rarely seen 1:1 win on this target. [DERIVED recommendation]

Q: Does SMOTE run on live data?
No. In our two-layer NSE stack, SMOTE lives only in Layer 2 (EOD-audited training) inside cross-validation. Layer 1 (Dhan WebSocket) is shadow/predict-only and never resampled. [ARCHITECTURE NOTE]

TL;DR

sampling_strategy controls how much minority data SMOTE creates.
Strings ('not majority', deprecated 'auto', 'minority', 'all') give coarse 1:1 balancing.
Float (e.g. 0.5) = minority becomes that fraction of the majority (1:2); binary only.
Dict = exact per-class target counts; best for multiclass and production locking.
Forced 1:1 usually hurts Nifty big-move models (drowns majority, overfits borderline).
Moderate 1:2 / 1:3 ratios typically win — start there, CV-grid by F1/PR-AUC.
SMOTE stays in Layer 2 inside CV, never on live data.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources (primary)

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16:321–357. [SOURCE]
imbalanced-learn documentation — imblearn.over_sampling.SMOTE, sampling_strategy parameter semantics (str/float/dict/callable, 'not majority' replacing deprecated 'auto', binary-only float). scikit-learn-contrib/imbalanced-learn. [SOURCE]
Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. [SOURCE]
He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning. [SOURCE]
Blagus, R., Lusa, L. (2013). Evaluation of SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. [SOURCE]
imbalanced-learn — SMOTENC / SMOTEN (categorical handling) and SVMSMOTE / KMeansSMOTE implementations. [SOURCE]

Author / Canonical

Written by Shakti Tiwari — Nifty Option Trader, XGBoost Expert.
Brand site: optiontradingwithai.in · Profile: about.me/shaktitiwari

Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

🌐 Brand site: https://optiontradingwithai.in
👤 Author profile: https://about.me/shaktitiwari
💬 WhatsApp: https://wa.me/919169650895
📚 Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF
⬅️ Previous article (S2): SMOTE_S2_k_neighbors.md
➡️ Next article (S4): SMOTE_S4_random_state.md

SMOTE `k_neighbors` Parameter: How 5 Little Neighbors Decide If Your Synthetic Samples Help or Hurt

shakti tiwari — Mon, 07 Sep 2026 04:30:38 +0000

SMOTE `k_neighbors` Parameter: How 5 Little Neighbors Decide If Your Synthetic Samples Help or Hurt

Part of the SMOTE deep-dive cluster for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in).

Disclaimer (verbatim): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Two-layer engine note: Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data.

Quick Answer

The k_neighbors argument of SMOTE controls how many minority-class neighbors each synthetic sample is allowed to interpolate between. The default is 5 (SOURCE: scikit-learn-contrib/imbalanced-learn, default value in SMOTE; Chawla et al. 2002 uses "k nearest neighbors" without fixing a canonical number, the library chose 5). Too small and SMOTE collapses toward duplication — the new points pile onto thin line segments and stop covering new territory. Too large and it reaches across the feature space, dragging in distant neighbors so the synthetic points land in sparse, ambiguous regions that blur the class boundary. The right value tracks the local dimensionality of your minority manifold, and you tune it inside cross-validation, never on held-out data.

Why This Matters

Imbalanced datasets are the daily reality for anyone trading Nifty options. A profitable breakout, a genuine reversal, or a directional signal is rare relative to the thousands of "nothing happens" candles. Train XGBoost naively on that and the model quietly learns to always predict "no trade" — it scores 95% accuracy and is completely useless. (DERIVED: this is the textbook minority-class neglect problem; the accuracy paradox is well documented in the imbalance literature.)

SMOTE is the most-used fix, and k_neighbors is the knob most people leave at its default without understanding. But here is the uncomfortable truth: SMOTE is not automatically safe. Blagus & Lusa (2013) showed that on small, high-dimensional datasets SMOTE can degrade classifier performance. The reason it can backfire usually traces back to how the synthetic points are built — and k_neighbors is the parameter that decides the geometry of every single one of them.

If you are running SMOTE in a pipeline (as you should be, inside CV), a bad k_neighbors means you are spending compute to manufacture noise. A good one means you are genuinely thickening the minority manifold in a way your gradient-boosted model can learn. So this is not a cosmetic setting. It is the difference between an augmented dataset and a corrupted one.

Research Question / Hypothesis

Research question: How does the choice of k_neighbors in SMOTE trade off between two failure modes — under-diversification (synthetic samples collapse toward duplication) and over-generalization (synthetic samples are placed in sparse, ambiguous regions far from the true minority manifold)?

Hypothesis (DERIVED): There exists a problem-dependent sweet spot. When the minority class is locally dense and low-dimensional, a smaller k_neighbors (e.g. 3–5) is sufficient and safer because the nearest neighbors already lie on the manifold. When the minority class is sparse or sits in a high-dimensional feature space where nearest-neighbor distances grow fast (the curse of dimensionality), a slightly larger k_neighbors is needed to give SMOTE enough candidates — but pushing it too far pulls in genuinely distant points and hurts. The optimal value is therefore a function of local dimensionality, not a universal constant.

Data & Methodology

We reason from two primary sources and the library's own implementation, not from a fresh experiment we ran (we are explicit: no code was executed for this article; every snippet below is illustrative, taken from the canonical imbalanced-learn API shape).

Primary sources:

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). "SMOTE: Synthetic Minority Over-sampling Technique." Journal of Artificial Intelligence Research 16:321–357. (SOURCE: original algorithm definition.)
scikit-learn-contrib/imbalanced-learn GitHub repository (~7.1k★). (SOURCE: the reference implementation of SMOTE.k_neighbors.)

The mechanism (DERIVED from Chawla 2002 + imbalanced-learn source):

SMOTE operates on the minority class only. For each minority sample x_i:

Find its k_neighbors nearest neighbors within the minority class (Euclidean distance in the original feature space, by default).
Randomly pick one neighbor x_zi from that pool.
Generate x_new = x_i + (x_zi − x_i) · δ, where δ ~ Uniform(0, 1).

So k_neighbors literally sets the size of the candidate pool each point interpolates against. The library builds a kNN index (a NearestNeighbors object, backed by a BallTree or KDTree depending on the metric) over the minority points, queried with n_neighbors = k_neighbors + 1 so the point itself is excluded from its own neighbor list. (DERIVED: this is how imbalanced-learn wires the parameter internally; the "+1" is to skip the self-match.)

That single number therefore controls both the diversity of the interpolation and the maximum reach of any synthetic point. Hold that thought — it is the whole story.

Results / Findings

Finding 1 — Small `k_neighbors` collapses toward duplication

Take k_neighbors = 1. Each minority point has exactly one neighbor to interpolate with. Every synthetic point for x_i lies on the single line segment between x_i and that lone neighbor. When minority points cluster tightly, the lone nearest neighbor is often the same point for many rows, so the synthesizer keeps drawing on the same few line segments. The result is a synthetic cloud that is geometrically thin — it sits on a 1-D skeleton rather than filling a 2-D (or higher-D) region. Functionally it resembles duplication: you are adding volume without adding information about the manifold's shape.

Even at k_neighbors = 2 or 3, if the minority is sparse, the nearest couple of neighbors may still be very close together. Interpolating between near-identical points yields near-duplicates. This is why people sometimes report SMOTE "doing nothing" — it is not that SMOTE failed; the k_neighbors was too small to escape the local clump. (DERIVED.)

Finding 2 — Large `k_neighbors` over-reaches into ambiguous space

Now push to k_neighbors = 15 or 20 on a minority set that is genuinely scattered. The 15th or 20th nearest neighbor can be far from x_i — possibly across a gap that separates the minority cluster from the majority region. When x_zi is that distant point, the synthetic sample x_i + (x_zi − x_i)·δ lands somewhere in the middle of that gap. That middle is precisely the ambiguous zone the classifier should not be confidently trained on, because it is likely near the true decision boundary or even inside majority territory. You have manufactured a misleading label.

This is the classic SMOTE pitfall: synthetic points that cross into the majority's space create false confidence and widen the boundary incorrectly. (DERIVED; consistent with the rationale behind Borderline-SMOTE (Han, Wang, Mao 2005), which was invented precisely because vanilla SMOTE generates noisy points near the border.)

Finding 3 — Local dimensionality governs the sweet spot

Here is the subtle part. The distance from x_i to its k-th neighbor does not grow linearly with k — it grows with the intrinsic dimensionality of the local minority manifold. In a genuinely 2-D, well-sampled blob, the 5th neighbor is still close; in a 50-dimensional options feature space, even the 5th neighbor can already be surprisingly far because volume expands explosively with dimension (the curse of dimensionality). (DERIVED: standard concentration-of-measure argument.)

So the "safe" k_neighbors shrinks as local dimensionality rises. This is exactly why Blagus & Lusa (2013) found SMOTE can hurt on small/high-dimensional data: at high dimension the neighbor pool fills with distant points almost immediately, so any k_neighbors large enough to avoid duplication is also large enough to inject noise. The remedy there is not cranking k_neighbors — it is reducing dimensionality first (PCA, feature selection) or switching to a border-aware variant.

Worked 2-D Geometric Intuition

Let the minority class be five points in 2-D:

P1 = (0, 0)
P2 = (1, 0)
P3 = (0, 1)
P4 = (1, 1)
P5 = (2, 2) ← a stray outlier, far from the tight unit square

Focus on P1 = (0,0).

Case A — k_neighbors = 1. The only neighbor is P2 = (1,0) (distance 1). Every synthetic point for P1 is (δ, 0) with δ ∈ (0,1). All of them lie on the x-axis segment between P1 and P2. They never explore toward P3, P4, or the interior of the unit square. The synthetic manifold is a 1-D line, not the 2-D region the real minority occupies. If many minority rows behave like this, the augmentation is a thin shell — essentially duplication along one axis. (DERIVED.)

Case B — k_neighbors = 5 (default). The neighbors of P1 are P2 (1.00), P3 (1.00), P4 (1.41), P5 (2.83). The pool now includes the stray outlier P5. If the random draw selects P5, the synthetic point is (2.83·δ, 2.83·δ). At δ = 0.5 that is (1.41, 1.41) — far outside the tight unit-square cluster, landing in open space near the P5 outlier. This is over-generalization: SMOTE reached across a 2.83-unit gap to a stray point and manufactured a sample in a sparse, ambiguous region that the original minority data did not actually populate. (DERIVED.)

The lesson (DERIVED): With this geometry, k_neighbors around 3–4 would have been better — P1's three closest neighbors (P2, P3, P4) all sit in the dense unit square, so interpolation stays on-manifold and still has variety. k=1 is too thin; k=5 already drags in the outlier. No single default is right; the data's shape decides.

Finding 4 — The connection to the kNN index

k_neighbors is not just a math parameter; it is a query size against an index. Internally SMOTE fits a NearestNeighbors structure (BallTree/KDTree) and issues a k-NN query per minority point. Two consequences follow:

Cost scales with k. Larger k_neighbors means a larger neighbor list per point and more interpolation candidates — modestly more compute, usually negligible, but real on very large minority sets.
Index quality bounds the result. The neighbors returned are only as meaningful as the distance metric and the feature scaling. If your features are on wildly different scales (e.g. an options "strike distance" feature in thousands vs. an RSI in 0–100), Euclidean distance is dominated by the large-scale feature, so the "nearest" neighbors are nearest in the wrong sense. Standardize features before SMOTE. (DERIVED; standard preprocessing guidance, reinforced by imbalanced-learn's own pipeline examples.)

Reproducibility (code shape — illustrative only)

We did not run this. The snippet below shows the canonical imbalanced-learn usage shape so you can reproduce the experiment yourself. Treat it as illustrative, sourced from the library's documented API — not as an experiment result from us.

## ILLUSTRATIVE ONLY — not executed. Standard imbalanced-learn API shape.
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
from xgboost import XGBClassifier
from sklearn.preprocessing import StandardScaler

## Always scale BEFORE SMOTE (SMOTE uses Euclidean distance).
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("smote", SMOTE(random_state=42, k_neighbors=5)),  # <-- the knob
    ("clf", XGBClassifier(random_state=42, eval_metric="logloss")),
])

## Tune k_neighbors INSIDE CV — never on the test set.
param_grid = {"smote__k_neighbors": [3, 5, 7, 9, 11]}
search = GridSearchCV(pipe, param_grid,
                      scoring="f1",   # or average_precision / roc_auc
                      cv=5, n_jobs=-1)
search.fit(X_train, y_train)

Key points reproduced from the docs (SOURCE: imbalanced-learn): SMOTE is imported from imblearn.over_sampling; it is fit via fit_resample(X, y); k_neighbors accepts an int (number of neighbors) or, in newer versions, an estimator object inheriting from KNeighborsMixin used to find neighbors. The int form is what 99% of users pass.

What Failed / Counter-Evidence

The hypothesis that "bigger k is always better for sparse classes" fails once you cross into the ambiguous zone (Finding 2). Empirical literature supports caution: Blagus & Lusa (2013) report cases where SMOTE worsens results, and the border-noise mechanism is the leading explanation. Borderline-SMOTE (Han et al. 2005) and ADASYN (He et al. 2008) were both invented to fix exactly the failure mode that vanilla SMOTE with a careless k_neighbors produces — generating points in the wrong place. So if grid search keeps preferring very large k_neighbors, suspect that your minority is too sparse or too high-dimensional for vanilla SMOTE at all, and consider a border-aware variant or dimensionality reduction instead.

Limitations

We did not execute code. All geometry here is derived analytically; the numbers are hand-computed illustrations, not measured from a run. Validation on your own data is mandatory.
k_neighbors only affects where points are drawn, not how many. The count of synthetic samples is controlled by sampling_strategy (the subject of the next article, S3).
SMOTE assumes feature-space linearity between neighbors. For heavily non-linear manifolds, interpolation can still leave the manifold even with perfect k_neighbors.
Euclidean-distance neighbor selection is scale-sensitive; conclusions assume sensible scaling.

Practical Takeaways

Never leave k_neighbors blind. Default 5 is a starting point, not a recommendation.
Tune it inside CV with a scoring metric that respects imbalance (F1, PR-AUC / average_precision, ROC-AUC) — as shown in the illustrative snippet.
Smaller k when the minority is dense and low-dimensional (stays on-manifold, avoids distant pulls).
Larger k only when the minority is sparse and you have verified synthetic points are not landing in ambiguous space — and cap it (usually ≤ 10–15).
Standardize features first. Euclidean neighbors are meaningless on mixed scales.
Watch local dimensionality. High-dimensional options feature sets push the safe k down, not up. Reduce dimensions or use border-aware variants (Borderline-SMOTE, ADASYN, SVMSMOTE, KMeansSMOTE) when SMOTE misbehaves.
In our stack: apply SMOTE only in Layer 2 (EOD-audited XGBoost/LightGBM training core) inside walk-forward CV — never on Dhan WebSocket live data.

FAQ

Q1. What is the default k_neighbors in SMOTE?
5, per imbalanced-learn (SOURCE: library default). Chawla et al. 2002 described "k nearest neighbors" generally without mandating a number.

Q2. Can k_neighbors be larger than the number of minority samples?
No — it must be strictly less than the minority class count. imbalanced-learn will raise an error otherwise. (DERIVED from the constraint that you need k distinct neighbors within the minority class.)

Q3. Does a bigger k_neighbors mean more synthetic samples?
No. It changes which neighbors each point interpolates with, not how many points are generated. Count is set by sampling_strategy.

Q4. Is k_neighbors = 1 the same as duplication?
Not literally — k=1 still interpolates along the segment to the single neighbor, so you get a spread of points, not exact copies. But the spread is confined to a thin 1-D line, so it behaves like duplication in the sense of adding no new manifold coverage. (DERIVED.)

Q5. Should I use SMOTE on the test set?
Absolutely not. Fit SMOTE on training folds only, inside CV. Leaking synthetic points into evaluation inflates scores.

Q6. How is k_neighbors related to the kNN classifier?
Only structurally. SMOTE uses a kNN index to find minority neighbors for interpolation; it is not a classifier. But the same distance/scale sensitivities apply.

TL;DR

k_neighbors (default 5) sets the size of the neighbor pool each minority point interpolates with. (SOURCE: imbalanced-learn.)
Too small → thin, duplication-like synthetic shells that add volume without information.
Too large → reaches distant neighbors, dropping synthetic points into sparse, ambiguous regions that corrupt the boundary.
The sweet spot tracks local dimensionality: small k for dense/low-D, modestly larger k for sparse (but capped).
Always tune inside CV on imbalance-aware scoring, standardize first, and apply only in Layer 2 of our NSE stack.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research 16:321–357. [PRIMARY — SOURCE]
scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k★). SMOTE.k_neighbors implementation and documented API. [PRIMARY — SOURCE]
Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. [SOURCE — variant context]
He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN: Adaptive Synthetic Sampling Approach. [SOURCE — variant context]
Blagus, R., Lusa, L. (2013). On the effectiveness of SMOTE for imbalanced data classification with small and high-dimensional datasets. [SOURCE — counter-evidence]
All geometric examples and the dimensionality argument: [DERIVED] (analytical, not executed).

Author / Canonical

Shakti Tiwari — Nifty Option Trader, XGBoost Expert. NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only. This article is part of the SMOTE deep-dive series on optiontradingwithai.in. Engine context: TWO-LAYER NSE stack — Layer 1 Dhan WebSocket live capture (shadow/predict-only); Layer 2 EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage); imbalance handling applied in Layer 2 inside CV only.

Resources & Links

Previous article: SMOTE_S1_smote_algorithm (the SMOTE algorithm)
Next article: SMOTE_S3_sampling_strategy
Brand site: https://optiontradingwithai.in
About: https://about.me/shaktitiwari
WhatsApp: https://wa.me/919169650895
Books: https://www.amazon.in/dp/B0H9ZNTBPK • https://www.amazon.in/dp/B0HBBFKDQF

SMOTE Explained: The kNN Interpolation Algorithm Behind Synthetic Oversampling

shakti tiwari — Sun, 06 Sep 2026 04:31:04 +0000

SMOTE Explained: The kNN Interpolation Algorithm Behind Synthetic Oversampling

Quick Answer (40–100 words): SMOTE (Synthetic Minority Over-sampling Technique) fixes class imbalance by synthesizing new minority examples instead of copying them. For each minority point it finds k nearest minority neighbors and creates a synthetic point on the line segment between the point and a randomly chosen neighbor, with a random interpolation factor ∈ [0,1]. This lives in the _make_samples function of imbalanced-learn's BaseSMOTE. It beats naive random oversampling because it expands the decision boundary along real minority directions rather than memorizing duplicates.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Why This Matters

Imbalanced datasets are the silent killer of vanilla classifiers. If 99% of your NIFTY 5-minute bars are "no big move" and 1% are "big move," a model that predicts "no move" every time scores 99% accuracy and is useless for trading. Accuracy lies. You need the minority class (the rare, high-information event) to be learnable.

Two naive fixes exist: (1) undersample the majority (throw away data — bad, you lose signal), or (2) random oversample the minority (copy rows — bad, the model memorizes and overfits exact duplicates). SMOTE, introduced by Chawla et al., 2002 (Journal of Artificial Intelligence Research, "SMOTE: Synthetic Minority Over-sampling Technique"), broke this trade-off by generating new minority points in feature space. That single paper has 30,000+ citations — it is the foundation of modern imbalance handling.

The canonical, battle-tested implementation is imbalanced-learn (scikit-learn-contrib/imbalanced-learn, ~7.1k GitHub stars, MIT license) — the scikit-learn-contrib package purpose-built for exactly this. Everything below is sourced from that repo's source code and docstrings plus the original papers.

Research Question / Hypothesis

Hypothesis: SMOTE's value comes from interpolating between real minority neighbors (creating plausible in-between points) rather than duplicating them. Understanding the exact interpolation math (_make_samples) is what lets you tune k_neighbors and avoid generating points inside majority territory.

Every claim below is SOURCE (Chawla 2002; Han 2005; imbalanced-learn source imblearn/over_sampling/_smote/base.py) or DERIVED from the algorithm. No invented numbers.

Data & Methodology (how SMOTE is defined)

The algorithm, per Chawla 2002 and the imbalanced-learn BaseSMOTE._make_samples implementation:

Identify minority samples T and their count N.
For each minority sample x_i, compute its k_neighbors nearest neighbors within the minority class (using a kNN index, default k=5).
For each x_i, generate N synthetic samples (where N is set by sampling_strategy):
- Pick a random neighbor x_zi from the k nearest minority neighbors.
- Draw a random number r ∈ [0,1].
- Synthesize: x_new = x_i + r · (x_zi − x_i).
Repeat until the desired minority count is reached.

This is pure linear interpolation on the minority manifold. The r is drawn uniformly in [0,1] each time, which is why random_state controls reproducibility.

The math, precisely

Given minority point x_i and a chosen neighbor x_zi (both d-dimensional vectors), the synthetic sample is:

x_new = x_i + λ · (x_zi − x_i),   λ ~ U(0,1)

When λ=0 you get x_i itself (the original point); λ=1 gives the neighbor; values in between fill the segment. Because λ is uniform, points are spread evenly along each connecting segment — SMOTE does not bias toward either endpoint.

Results (what the literature and implementation show)

Primary finding — SMOTE vs Random Oversampling

SOURCE (Chawla 2002; subsequent benchmarks in imbalanced-learn docs): SMOTE consistently produces better-separated decision boundaries than random oversampling because it introduces variety rather than exact copies. Random oversampling makes the learner overfit the duplicated points (the tree/model can branch perfectly on an identical row), inflating train score while test generalizes worse. SMOTE's interpolated points cannot be memorized, forcing the model to learn the minority region's shape.

Secondary finding — the kNN choice matters

SOURCE (imbalanced-learn k_neighbors parameter, default 5): too small k (e.g. 1) collapses SMOTE toward random-duplicate behavior (only one neighbor to interpolate with). Too large k pulls in minority points that are far away, generating synthetic samples in sparse/ambiguous regions. The default 5 is a reasonable middle; the right value is dataset-specific (see S2 article).

Limitations found in the original

SOURCE (Chawla 2002; later critiques by Batista et al. 2004 "A Study of the Behavior of Several Methods for Balancing Machine Learning Training Data"):

SMOTE generates points along straight lines between neighbors — it assumes local linearity. If the true minority class is non-convex or lies on a curved manifold, SMOTE can place synthetic points in majority territory (between a minority point and a neighbor that straddles a majority cluster).
SMOTE does not consider the majority class at all when generating — it is "blind" to where majority points sit. This is exactly the weakness Borderline-SMOTE and SVMSMOTE later fix (see V2, V3).

Reproducibility (the canonical code shape)

The following mirrors the imbalanced-learn API exactly (SOURCE: imbalanced-learn docs). This is the standard usage pattern — no fabricated output, run it on your own data:

from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import f1_score

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y)
sm = SMOTE(sampling_strategy=0.5, k_neighbors=5, random_state=42)
X_res, y_res = sm.fit_resample(X_train, y_train)   # apply ONLY on train, never test

clf = RandomForestClassifier().fit(X_res, y_res)
print(f1_score(y_test, clf.predict(X_test)))

Critical: SMOTE must be fit inside a cross-validation loop / on the training split only. Fitting it before the split leaks test information into training (see P1 article — the #1 SMOTE mistake).

What Failed / Counter-Evidence

Myth: "SMOTE always improves imbalanced models." SOURCE (multiple benchmark studies, e.g. Blagus & Lusa 2013 on medical data): on high-dimensional or small datasets, SMOTE can hurt because it invents correlated noise that the model fits. SMOTE is not free lunch.
Myth: "More synthetic data = better." SOURCE (imbalanced-learn guidance): oversampling to a 1:1 ratio is often worse than a moderate ratio (e.g. 1:2 or 1:3) because you drown the majority signal. sampling_strategy should be tuned, not maxed.

Limitations (explicit non-claims)

SMOTE assumes features are numeric and on comparable scales. Always standardize/normalize before SMOTE — otherwise the Euclidean kNN distance is dominated by the largest-magnitude feature (SOURCE: imbalanced-learn user guide).
SMOTE on raw price levels (e.g. NIFTY close=24000) without scaling is meaningless; use returns or z-scored features.
SMOTE generates in input feature space; it does not know causal structure. Synthetic samples are statistically plausible, not economically real.
This article explains the algorithm. Per-variant tuning (Borderline, SVM, KMeans, categorical) is covered in the V-series.

Worked Numerical Example (2-D, so you can see it)

Suppose one minority point is x_i = [2.0, 1.0] and its 5 nearest minority neighbors include x_zi = [4.0, 3.0]. SMOTE picks x_zi and draws λ = 0.4. The synthetic point is:

x_new = [2.0, 1.0] + 0.4 · ([4.0, 3.0] − [2.0, 1.0])
      = [2.0, 1.0] + 0.4 · [2.0, 2.0]
      = [2.0, 1.0] + [0.8, 0.8]
      = [2.8, 1.8]

That point [2.8, 1.8] lies between the two real minority points — it is on the minority manifold, not a copy. Repeat this for every minority point with different random neighbors and different λ values and you populate the minority region with plausible new cases. The decision boundary the classifier learns now wraps the filled region instead of hugging a few isolated points.

This is exactly why SMOTE needs scaled features: if feature 1 ranged 0–5 and feature 2 ranged 0–50000, the Euclidean distance would be dominated by feature 2, so "nearest neighbors" would be chosen almost entirely on feature 2 alone — the interpolation would run along the wrong axis.

How SMOTE Connects to Its Variants (preview)

The vanilla SMOTE above picks neighbors randomly among all minority points. Every later variant changes which minority points get oversampled or how neighbors are chosen:

Borderline-SMOTE (Han 2005, V2): only synthesizes near the border between classes, where misclassification actually happens — not in the safe interior of the minority region.
SVMSMOTE (V3): uses an SVM to find the margin boundary, then generates near the support vectors.
KMeansSMOTE (V4): clusters the minority class first, then oversamples within clusters so dense and sparse sub-clusters are both represented.
SMOTENC / SMOTEN (V5/V6): extend the interpolation to categorical dimensions (SMOTEN uses the Value Difference Metric instead of Euclidean distance).
ADASYN (V7): weights synthesis toward minority points that are harder (have more majority neighbors), adaptively.

The interpolation formula x_i + λ(x_zi − x_i) stays the same everywhere — only the selection of x_i and x_zi differs. That is the one insight that makes the whole family click.

Practical Takeaways

SMOTE = interpolate, don't duplicate. That is the entire insight — generate on the line between minority neighbors.
Scale features first (StandardScaler / MinMax), then SMOTE, then model.
Apply inside CV / train-only. Never fit_resample on the full dataset before splitting.
Tune sampling_strategy and k_neighbors — defaults are starting points, not recommendations.
Two-layer engine note: in our NSE stack, imbalance handling lives in the EOD-audited XGBoost/LightGBM training core (Layer 2); the live Dhan WebSocket layer (Layer 1) only consumes predictions in shadow mode. scale_pos_weight (XGBoost native) is often preferred over SMOTE there for speed — see P4.
Pair with the right metric: optimize F1 / PR-AUC / recall-at-precision, never raw accuracy, on imbalanced data.

FAQ

Q: Is SMOTE the same as duplicating rows? A: No. Duplicating = exact copies (memorization risk). SMOTE = new points interpolated between neighbors.

Q: Where is the math implemented? A: imbalanced-learn BaseSMOTE._make_samples — the x_i + λ(x_zi − x_i) line.

Q: What is a good k_neighbors? A: Default 5; tune 3–10. Smaller = more local, larger = more global (see S2).

Q: Should I SMOTE before train/test split? A: Never. Fit on train only (see P1).

Q: Does SMOTE work on categorical features? A: Not vanilla SMOTE (distance is meaningless on one-hot). Use SMOTENC / SMOTEN (see V5, V6).

Q: SMOTE vs class_weight? A: class_weight (e.g. XGBoost scale_pos_weight) reweights the loss instead of fabricating data — often cheaper and leakage-free. Covered in P4.

Q: In the two-layer engine, where does SMOTE run? A: Training core (Layer 2) only, inside walk-forward CV with cost-and-slippage applied.

Q: Why does SMOTE use Euclidean distance? A: Because it picks neighbors via a kNN index on the numeric feature vectors. That is why scaling is mandatory — unscaled features make the distance metric meaningless (see the worked example above).

Q: Can SMOTE create points in majority territory? A: Yes — because vanilla SMOTE ignores majority points when interpolating. If a minority point's neighbor straddles a majority cluster, the synthetic point can land inside it. This is the core weakness Borderline-SMOTE and SVMSMOTE address by being boundary-aware.

When NOT to Use SMOTE

SOURCE (Blagus & Lusa 2013; imbalanced-learn guidance): SMOTE is not universal. Avoid or tune carefully when:

Very small datasets (< a few hundred minority samples) — too few neighbors means synthetic points are low-quality and the model overfits the synthetic noise.
Highly overlapping classes — if minority and majority are not linearly separable even in principle, inventing more minority points just adds confusion.
Rare-event time series with autocorrelation (e.g. NIFTY direction) — SMOTE treats rows as i.i.d., breaking the time order. Prefer walk-forward CV + scale_pos_weight or SMOTE applied per-fold with care (see P2, P4).
You only need probability ranking, not balanced classes — if your downstream metric is PR-AUC or you use class_weight, raw imbalance with reweighting is often enough.

The honest framing: SMOTE is a tool for enriching the minority manifold, not a guaranteed accuracy boost. Measure with F1 / PR-AUC / recall-at-precision, and always compare against a no-SMOTE baseline.

TL;DR

SMOTE generates synthetic minority points by linear interpolation between a point and its k nearest minority neighbors (x_new = x_i + λ(x_zi−x_i), λ~U(0,1)). It beats random duplication because it creates variety instead of memorization, but it is blind to the majority class and assumes local linearity — so scale first, apply train-only, and tune k_neighbors/sampling_strategy. Source: Chawla et al. 2002 + imbalanced-learn.

📚 Related Articles

👤 About the Author

Name:: Shakti Tiwari
Role:: Nifty Option Trader, XGBoost Expert
Cert:: NISM-Series-XII (Securities Markets Foundation)
Knows:: XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

Chawla, Bowyer, Hall, Kegelmeyer (2002). "SMOTE: Synthetic Minority Over-sampling Technique." Journal of Artificial Intelligence Research 16:321–357. (SOURCE — original algorithm)
Han, Wang, Mao (2005). "Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets." (SOURCE — variant)
He, Bai, Garcia, Li (2008). "ADASYN: Adaptive Synthetic Sampling." (SOURCE — variant)
scikit-learn-contrib/imbalanced-learn GitHub repo (~7.1k stars) — imblearn/over_sampling/_smote/base.py _make_samples (SOURCE — implementation)
imbalanced-learn user guide — "It is recommended to scale the data" before SMOTE (SOURCE)

Author / Canonical Attribution

Shakti Tiwari — Nifty Option Trader, XGBoost Expert. NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

About: https://about.me/shaktitiwari
Site: https://optiontradingwithai.in
WhatsApp: https://wa.me/919169650895
Next: S2 k_neighbors deep-dive | Cluster V: SMOTE variants
Books: Option Trading with AI (B0H9ZNTBPK) | The AI Opportunity (B0HBBFKDQF)

DEV Community: shakti tiwari

RandomOverSampler: The Baseline Every SMOTE Variant Is Measured Against

RandomOverSampler: The Baseline Every SMOTE Variant Is Measured Against

Quick Answer

Why This Matters

Research Question / Hypothesis

Data & Methodology

The mechanism, exactly

The math (DERIVED)

Worked numerical example (DERIVED)

Contrast with SMOTE (the whole point)

Results / Findings

Reproducibility (illustrative snippet — NOT a result I ran)

The shrinkage Parameter — RandomOverSampler's Anti-Memorization Knob

What Failed / Counter-Evidence

Limitations

Practical Takeaways

FAQ

TL;DR

📚 Related Articles

Sources

Author / Canonical

Resources & Links

ADASYN in imbalanced-learn: Adaptive Synthetic Sampling That Chases the Hard Minority Points

ADASYN in imbalanced-learn: Adaptive Synthetic Sampling That Chases the Hard Minority Points

Quick Answer

Why This Matters

Research Question / Hypothesis

Data & Methodology

Results / Findings

Reproducibility

What Failed / Counter-Evidence

Limitations

Practical Takeaways

FAQ

TL;DR

📚 Related Articles

Sources

Author / Canonical

Resources & Links

SMOTEN in imbalanced-learn: SMOTE for Purely Categorical Data via the Value Difference Metric

SMOTEN in imbalanced-learn: SMOTE for Purely Categorical Data via the Value Difference Metric

Quick Answer

Why This Matters

Research Question / Hypothesis

Data & Methodology

Results / Findings

Reproducibility (code shape — illustrative)

Worked Categorical Example (DERIVED math)

What Failed / Counter-Evidence

Limitations

Practical Takeaways (Production Checklist)

FAQ

TL;DR

📚 Related Articles

Sources (PRIMARY)

Author / Canonical

Resources & Links

SMOTENC — Synthetic Minority Over-sampling for Mixed Categorical + Continuous Data (with a NIFTY Tabular Walkthrough)

SMOTENC — Synthetic Minority Over-sampling for Mixed Categorical + Continuous Data (with a NIFTY Tabular Walkthrough)

Quick Answer

Why This Matters

Research Question / Hypothesis

Data & Methodology

Step 1 — Split the feature matrix

Step 2 — The median-standard-deviation trick (the clever bit)

Step 3 — Standard SMOTE interpolation on the continuous side

Step 4 — Categorical mode selection (the part that saves you)

Results / Findings (mechanical, derived from the algorithm)

Worked numerical example (DERIVED — illustrative only)

Reproducibility (code shape — illustrative, not executed)

What Failed / Counter-Evidence

Limitations

Practical Takeaways

FAQ

TL;DR

📚 Related Articles

Sources

Author / Canonical

Resources & Links

The `shrinkage` Parameter — RandomOverSampler's Anti-Memorization Knob