DEV Community

shakti tiwari
shakti tiwari

Posted on

Vanilla SMOTE in imbalanced-learn: The kNN Interpolation Engine Behind Balanced XGBoost

#smote #machinelearning #imbalanceddata #ai

Vanilla SMOTE in imbalanced-learn: The kNN Interpolation Engine Behind Balanced XGBoost

Shakti Tiwari — Nifty Option Trader, XGBoost Expert

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

This is article V1 in the SMOTE cluster. PREV: S1 — The SMOTE Algorithm (intuition). NEXT: V2 — BorderlineSMOTE.

Quick Answer

Vanilla SMOTE from imbalanced-learn fixes class imbalance by synthesizing new minority-class samples as random convex combinations between each minority point and one of its k nearest minority neighbors — instead of blindly duplicating rows like random oversampling does (SOURCE: Chawla et al. 2002, JAIR 16:321–357). In imbalanced-learn the defaults are sampling_strategy='auto' (balance every minority class up to the majority count), k_neighbors=5, and an optional random_state for reproducibility (SOURCE: scikit-learn-contrib/imbalanced-learn, ~7.1k★). It helps most when the minority class forms a connected, dense region; it can hurt on tiny, high-dimensional, or noisy sets where invented points land in the wrong territory.

Why This Matters

Class imbalance is the silent killer of every Nifty option signal model. Your rare-event labels — a sharp expiry-day reversal, an OTM strangle that actually pays — might be 1–3% of rows. Train XGBoost on that raw and the tree will happily ignore the minority entirely because predicting "no move" already nets a 97% accuracy (DERIVED from the definition of accuracy on a 97/3 split). That 97% is a lie you can trade against at your own peril.

This is precisely why our resampling discipline lives in Layer 2 of the engine. Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. You never synthesize imaginary ticks on the live feed — that would be fraud dressed as feature engineering.

Understanding vanilla SMOTE — the plain SMOTE class, no borderline logic, no adaptive weighting — is the foundation. Every variant (BorderlineSMOTE, SVMSMOTE, ADASYN, KMeansSMOTE) is a mutation of this same interpolation core. If you don't understand the base, the variants are just incantations.

Research Question / Hypothesis

RQ: What exactly distinguishes vanilla SMOTE from random oversampling, how does its kNN interpolation behave under the default sampling_strategy='auto' / k_neighbors=5 configuration, and under what data-geometry conditions does it improve a classifier versus degrade it?

Hypothesis (DERIVED): Because SMOTE injects new points along minority–minority segments rather than replaying identical rows, it should (a) reduce the overfitting that duplication causes and (b) smooth the decision boundary — provided the minority manifold is locally coherent. When that manifold is sparse or the features are noisy/high-dimensional, the synthetic points will be unreliable and can poison the boundary instead.

Data & Methodology

We do not run code in this article. The snippet below is illustrative only, reproduced from the canonical imbalanced-learn usage shape — it is NOT an experiment result (SOURCE: scikit-learn-contrib/imbalanced-learn documentation). Treat it as the API contract, nothing more.

## Illustrative only — API shape from imbalanced-learn docs, not an experiment result.
from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y)

smote = SMOTE(
    sampling_strategy='auto',   # balance all minority classes to majority count
    k_neighbors=5,              # nearest minority neighbors to interpolate between
    random_state=42             # reproducibility
)

X_res, y_res = smote.fit_resample(X_train, y_train)
Enter fullscreen mode Exit fullscreen mode

The methodology we describe is the algorithmic specification as given by Chawla et al. (2002) and as implemented in the imbalanced-learn repository (SOURCE for both). We break the vanilla procedure into its atomic steps and then derive the geometry of one synthetic point by hand in the next section.

The four atomic steps (SOURCE: Chawla 2002; SOURCE: imbalanced-learn SMOTE)

  1. Isolate the minority set. For each minority class C (when sampling_strategy='auto', every class smaller than the majority), take its instances T_C.
  2. Set the target count. sampling_strategy decides how many synthetic samples to make. With 'auto', the target equals the majority class size, so the number of new points for class C is N_majority − N_C (DERIVED).
  3. Find k nearest minority neighbors. For each minority point x_i, compute Euclidean distances to all other points of the same class and keep the k_neighbors closest. Default k_neighbors=5 (SOURCE: imbalanced-learn default).
  4. Interpolate. For each x_i, draw a random neighbor x_zi from its k-NN set, draw δ ∼ Uniform(0,1), and emit: x_new = x_i + δ · (x_zi − x_i) (SOURCE: Chawla 2002, Equation for synthetic sample generation).

Repeat step 4 until the required count of synthetics is reached. Note what SMOTE does not do: it never duplicates an existing row. Every emitted point is strictly a new coordinate on a segment between two real minority points (DERIVED from the convex-combination formula with δ∈[0,1]; at δ=0 it equals x_i and at δ=1 it equals x_zi, but δ is drawn continuously so exact endpoints are measure-zero events).

Results / Findings

Below are the concrete behavioral findings, each anchored to a primary source or a derivation from the formula above.

F1 — SMOTE differs from random oversampling in mechanism, not just magnitude. Random oversampling copies existing minority rows until balance. SMOTE manufactures points in the convex hull of minority pairs (SOURCE: Chawla 2002 motivation section; DERIVED: copying ⇒ identical rows in training set ⇒ a tree can split on a memorized row, while interpolation ⇒ novel points).

F2 — The default sampling_strategy='auto' fully balances all minorities to the majority count. In imbalanced-learn, 'auto' is documented to resample all classes except the majority to the majority's cardinality (SOURCE: imbalanced-learn user guide). If you have a 1000/50 split, 'auto' creates 950 synthetic minors. You can also pass a float (e.g. 0.2 ⇒ minority becomes 20% of majority) or a dict (per-class overrides) — but the smart default is full balance (SOURCE: imbalanced-learn API).

F3 — k_neighbors=5 sets the locality of invention. Smaller k ⇒ synthetics hug tighter neighborhoods (risk: over-local, can create tiny clusters); larger k ⇒ synthetics spread toward farther cousins (risk: blend across potentially distinct sub-manifolds). The choice is a bias–variance dial on the minority geometry (DERIVED from k-NN smoothing theory).

F4 — random_state makes the synthetics reproducible, not the model. Same random_state ⇒ same synthetic coordinates across runs; the downstream classifier still has its own seed. This is essential for the walk-forward CV in Layer 2 so that two backtests of the same window are comparable (DERIVED from determinism of fit_resample under fixed seed; SOURCE: imbalanced-learn design).

F5 — SMOTE operates in raw feature space with Euclidean distance. It has no notion of feature scale, categorical vs numeric, or label noise. That is both its strength (simplicity, speed) and its trap (DERIVED from the algorithm using raw coordinates).

Reproducibility (code shape — illustrative)

As stated up top, the following is the canonical imbalanced-learn call shape and is shown for reproducibility of API usage, not as a claim that we executed a model here. The pipeline pattern below is what we actually wire into Layer 2 CV (SOURCE: imbalanced-learn Pipeline + SMOTE examples).

## Illustrative only — shows where SMOTE sits: INSIDE CV, never on the test set.
from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTE
from xgboost import XGBClassifier
from sklearn.model_selection import StratifiedKFold

pipe = Pipeline([
    ('smote', SMOTE(sampling_strategy='auto', k_neighbors=5, random_state=42)),
    ('clf',   XGBClassifier(n_estimators=300, max_depth=4,
                            eval_metric='logloss', random_state=42))
])

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
## pipe.fit(X_train_fold, y_train_fold)  ->  scored on the untouched validation fold
Enter fullscreen mode Exit fullscreen mode

The golden rule, repeated because people violate it constantly: fit_resample must be called only on training folds, never on validation or test data, or you leak synthetic minorities into your evaluation and your backtest lies (DERIVED from CV hygiene; echoed in every imbalanced-learn tutorial — here SOURCE: imbalanced-learn "avoid leakage" guidance).

Worked 2-D Interpolation Example (DERIVED math)

Let's make the formula concrete in two dimensions so you see what SMOTE emits. Suppose one minority point is A = (2, 3) and its randomly chosen 5-NN neighbor is B = (4, 7). The segment AB has direction B − A = (2, 4).

SMOTE draws δ ∼ Uniform(0,1) and computes A + δ·(B−A). Three independent draws:

δ x = 2 + δ·2 y = 3 + δ·4 Synthetic point
0.25 2.50 4.00 (2.50, 4.00)
0.50 3.00 5.00 (3.00, 5.00) ← exact midpoint
0.80 3.60 6.20 (3.60, 6.20)

Every synthetic lies on the line segment between A and B, never beyond it, because δ∈[0,1] (DERIVED). Now scale this: with k_neighbors=5, each minority point gets 5 candidate neighbors, so the set of possible segments is rich, and SMOTE samples different neighbors and different δ to fill the minority region with a cloud of plausible in-between points rather than 5 copies of A. This is the entire conceptual difference from random oversampling, which would simply append (2,3) five more times (DERIVED comparison).

A second minority point C = (5, 1) with neighbor D = (6, 4) (D−C = (1,3)) and δ=0.6 yields (5.6, 2.8). Notice the synthetic cloud now spans two distinct minority sub-regions, and a classifier trained on the augmented set learns a boundary that interpolates between real minors instead of over-memorizing them (DERIVED; consistent with Chawla 2002's stated intent — SOURCE).

What Failed / Counter-Evidence

SMOTE is not a free lunch, and the literature is blunt about it.

C1 — Blagus & Lusa (2013) show SMOTE can degrade performance on small or high-dimensional datasets. When the minority class has few samples or the feature space is wide, the k-NN neighborhoods become unreliable — synthetic points get placed in regions that don't reflect the true minority distribution, and classifiers trained on them perform worse than on the raw imbalanced data (SOURCE: Blagus & Lusa 2013). This is the single most important caveat for options data, where rare events are genuinely sparse.

C2 — Synthetic points can cross the decision boundary into majority territory. If a minority point sits near the majority region, its neighbor-based interpolation can emit a point that actually belongs to the majority class. SMOTE has no label-check; it just stamps "minority" on whatever it creates (DERIVED from the algorithm lacking any label-consistency test; SOURCE-adjacent: this limitation motivates BorderlineSMOTE and SVMSMOTE).

C3 — Random oversampling sometimes matches or beats SMOTE. On datasets where the minority class is already well-separated, simply duplicating can be sufficient, and the added complexity of SMOTE buys nothing (SOURCE: multiple empirical comparisons in the imbalanced-learning literature; DERIVED: more synthetic noise without benefit when the boundary is already clear).

Limitations

  1. Scale blindness. Euclidean distance on unstandardized features lets large-magnitude columns dominate neighbor selection. Always scale inside the CV pipeline (DERIVED from distance geometry).
  2. No categorical handling. Vanilla SMOTE interpolates numerically; mixing it on one-hot or label-encoded categoricals produces impossible hybrids (e.g. 0.4 in a binary column). Use SMOTENC/SMOTEN for mixed types (SOURCE: imbalanced-learn SMOTENC/SMOTEN).
  3. Curse of dimensionality. In high-D spaces, "nearest" neighbors are often far, so synthetic points are wild (DERIVED from concentration of distances; SOURCE-adjacent: Blagus & Lusa 2013).
  4. Leakage risk. Applied outside CV, it contaminates evaluation (see Reproducibility).
  5. Assumes connected manifold. Discrete, multi-modal minorities with gaps violate the interpolation assumption (DERIVED).

Practical Takeaways (Production Checklist)

  • Use SMOTE(sampling_strategy='auto', k_neighbors=5, random_state=42) as the default baseline; tune k_neighbors only after baseline CV (SOURCE: imbalanced-learn defaults; DERIVED tuning order).
  • Always wrap SMOTE in a Pipeline so it fits per-fold, never on the full set (SOURCE: imbalanced-learn Pipeline guidance).
  • Scale features before SMOTE inside the same pipeline (DERIVED).
  • Compare against scale_pos_weight in XGBoost — often cheaper and leakage-free; SMOTE earns its place when the minority geometry is genuinely learnable (DERIVED from practice; consistent with imbalanced-learn recommendations).
  • For Nifty option rare-event labels: start with scale_pos_weight, add vanilla SMOTE only inside Layer 2 CV, and watch for boundary-crossing degradation on sparse expiries (DERIVED from our two-layer discipline).
  • If vanilla SMOTE underperforms, the next article (V2 BorderlineSMOTE) shows the targeted variant that only synthesizes near the decision border.

FAQ

Q: Is vanilla SMOTE the same as random oversampling?
No. Random oversampling duplicates existing minority rows; SMOTE creates new points interpolated between minority neighbors (SOURCE: Chawla 2002; DERIVED comparison).

Q: What is the default sampling_strategy?
'auto', which balances every minority class up to the majority count (SOURCE: imbalanced-learn).

Q: What does k_neighbors do?
It sets how many nearest minority neighbors each point can interpolate with; default 5 (SOURCE: imbalanced-learn default).

Q: Does random_state make my model reproducible?
It makes the synthetic samples reproducible. The classifier needs its own seed (DERIVED).

Q: Can I use SMOTE on live Nifty ticks?
No — never on live data. Our stack applies it only in Layer 2 EOD CV (two-layer engine note above; DERIVED from our discipline).

Q: When does SMOTE hurt?
Small/high-dimensional/noisy minorities where neighbors are unreliable (SOURCE: Blagus & Lusa 2013).

Q: What comes after vanilla SMOTE?
BorderlineSMOTE (V2), then SVMSMOTE, ADASYN, KMeansSMOTE — the variant family covered in later articles.

TL;DR

Vanilla SMOTE synthesizes minority samples as convex combinations x_i + δ·(x_zi − x_i) between a point and one of its k_neighbors (default 5) minority neighbors, balancing classes via sampling_strategy='auto'. Unlike random oversampling (which duplicates), it invents plausible in-between points — great on dense, connected minorities, risky on sparse/high-dimensional/noisy ones. Always run it inside CV, never on live or test data.

📚 Related Articles

👤 About the Author



Name:
Shakti Tiwari

Role:
Nifty Option Trader, XGBoost Expert

Cert:
NISM-Series-XII (Securities Markets Foundation)

Knows:
XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation



Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources (PRIMARY)

  • Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research (JAIR), 16:321–357. — original algorithm, interpolation formula, motivation vs random oversampling.
  • scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k★). imblearn.over_sampling.SMOTE — default parameters (sampling_strategy='auto', k_neighbors=5), fit_resample API, Pipeline/leakage guidance, SMOTENC/SMOTEN for categoricals.
  • Blagus, R., Lusa, L. (2013). SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. — evidence SMOTE can degrade on small/high-dimensional data.
  • Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE — previewed variant (V2).
  • He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN — adaptive variant.
  • Nguyen, H.M., Cooper, E.W., Kamei, K. (2011). SVM-SMOTE — SVM-margin variant.
  • KMeansSMOTE (2018, last-resort paper) — clustering-guided variant.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE engineering cluster, sibling to the XGBoost cluster. This article (V1, vanilla SMOTE) is the canonical reference for the base interpolation mechanism; later articles extend it.

Disclaimer (repeat): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

Top comments (0)