DEV Community

shakti tiwari
shakti tiwari

Posted on

SMOTE `sampling_strategy`: How to Set the Oversampling Ratio Without Drowning Your Majority Class

#smote #machinelearning #imbalanceddata #ai

SMOTE sampling_strategy: How to Set the Oversampling Ratio Without Drowning Your Majority Class

A practical guide for Nifty option traders and XGBoost practitioners — by Shakti Tiwari

Quick Answer

SMOTE's sampling_strategy decides how much of the minority class gets synthesized. A string like 'not majority' (the older 'auto') forces a 1:1 balance; a float such as 0.5 makes the minority half the size of the majority; a dict lets you pin exact per-class counts; and 'minority'/'all' control which classes get touched. For rare-event models — like Nifty's rare big-move days — a forced 1:1 usually over-injects synthetic noise and hurts the real signal, so a moderate ratio (1:2 or 1:3) typically wins. [SOURCE: imbalanced-learn docs; DERIVED]

Disclaimer (verbatim): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Why This Matters

Class imbalance is the silent killer of trading models. In Nifty option data, the class you actually care about — a genuinely big directional move that makes a straddle/strangle payoff — shows up maybe 3–8% of the time. The "nothing happened" days dominate. [DERIVED from typical Nifty daily-range distributions] If you hand that raw distribution to XGBoost, the tree will happily learn to predict "no move" on every row and still score 94% accuracy. Useless. You've built a coin that's right because the market is boring, not because you're smart.

SMOTE (Synthetic Minority Over-sampling Technique) fixes this by manufacturing new minority samples in feature space instead of copying existing rows (that would just overfit). [SOURCE: Chawla et al. 2002] But here's the trap most tutorials skip: SMOTE has a dial, and leaving it at the default is often wrong. That dial is sampling_strategy.

The default behaviour in imbalanced-learn oversamples the minority class until it matches the majority class — a perfect 1:1 ratio. That sounds fair. It's usually not. When the true prior is 5% minority, forcing 50% minority teaches your model a world that doesn't exist, and it starts crying wolf on quiet days. For an option seller, that's a margin-eating disaster. [DERIVED]

This article is the third in our SMOTE deep-dive series. We already covered k_neighbors (S2). Here we tear apart sampling_strategy — every option, the math, and how to pick a ratio that survives walk-forward, not just a tidy notebook. [SOURCE: imbalanced-learn docs]

Research Question / Hypothesis

Research question: For a binary Nifty "big-move vs no-big-move" classifier, what sampling_strategy value maximizes out-of-sample recall and F1 without inflating false positives?

Hypothesis: A moderate oversampling ratio (minority-to-majority ≈ 0.3–0.5, i.e. 1:3 to 1:2) will beat forced 1:1 ('not majority'), because 1:1 (a) drowns the majority signal, (b) stacks synthetic density in borderline regions where classes overlap, and (c) shifts the effective class prior far from reality. [HYPOTHESIS; DERIVED]

Note: this is a hypothesis framed for walk-forward testing inside our engine. The numeric examples below are derived illustrations, not experiment results — I did not execute code in writing this piece. The canonical snippet in the Reproducibility section is taken from the imbalanced-learn documentation shape and is shown for structure only. [SOURCE: imbalanced-learn docs]

Data & Methodology (the Nifty angle)

Picture a realistic daily feature set for Nifty50:

  • Features (X): previous-day range, ATM implied volatility, PCR (put-call ratio), VIX, overnight gap, RSI(14), distance-to-spot of nearest strike, session volume z-score.
  • Target (y): 1 = big move day (|Nifty close − open| > 1.5× trailing 20-day ATR); 0 = quiet day.

A typical year gives roughly 9500 quiet days (class 0, the majority) and 500 big-move days (class 1, the minority) across the historical window we study. That's a ~5% prior. [DERIVED illustration, not a measured dataset]

Two-layer engine note (must-read for reproducibility): Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. SMOTE only ever touches the training fold; the validation fold stays raw so we measure honest performance on the true distribution. [ARCHITECTURE NOTE — internal stack]

This matters enormously for sampling_strategy: the ratio you choose is a hyperparameter tuned by cross-validation, not a one-time setting. More on that in Practical Takeaways.

The sampling_strategy Parameter: Every Option Explained

sampling_strategy accepts four shapes: str, float, dict, or callable. Let's go through each. [SOURCE: imbalanced-learn docs]

1. String options

  • 'not majority' — resample every class except the majority class up to the majority's count. For a binary problem this means minority → majority size, i.e. 1:1 balance. [SOURCE: imbalanced-learn docs]
  • 'auto' — historically the default; for over-sampling it is equivalent to 'not majority'. Deprecated since imbalanced-learn 0.9+ in favour of 'not majority'. If you see 'auto' in old code, it means 1:1. [SOURCE: imbalanced-learn docs]
  • 'minority' — resample only the single minority class (to majority size). In binary classification this yields the same 1:1 result as 'not majority', but in multiclass it differs: 'minority' balances only the smallest class, while 'not majority' balances all minority classes. [SOURCE: imbalanced-learn docs; DERIVED]
  • 'all' — resample all classes. In binary this again collapses to 1:1; it's more meaningful in multiclass setups. [SOURCE: imbalanced-learn docs]
  • 'not minority' — resample everything except the minority. This is primarily an under-sampling notion; you'll rarely use it with SMOTE over-sampling. [SOURCE: imbalanced-learn docs]

Takeaway: the string options are coarse. They're either "balance everything to 1:1" or "balance a subset to 1:1." They give you no control over the degree of oversampling. That's why the float and dict forms exist.

2. Float ratio (the fine dial)

When sampling_strategy is a float between 0 and 1, it specifies the desired ratio of minority samples to majority samples after resampling:

α = N_minority(resampled) / N_majority(original)

So:

  • 0.5 → minority becomes half the majority (1:2). [SOURCE: imbalanced-learn docs; DERIVED]
  • 0.33 → minority becomes a third of the majority (1:3).
  • 1.0 → 1:1 (same as 'not majority').
  • 0.1 → minority is just 10% of majority (gentle nudge).

Critical constraint: the float form is only valid for binary classification. For multiclass you must use a dict (next). [SOURCE: imbalanced-learn docs]

3. Dict (per-class targets — full control)

A dict maps each class label to the exact number of samples you want after resampling:

sampling_strategy = {1: 2000}   # make the big-move class (label 1) have 2000 samples
sampling_strategy = {0: 9500, 1: 3000}  # explicit, both classes
Enter fullscreen mode Exit fullscreen mode

The keys are class labels; the values are desired counts, not ratios. [SOURCE: imbalanced-learn docs] This is the most explicit form and the one I recommend once you've grid-searched a good target count. It also works for multiclass imbalance, where you might want class A at 4000 and class B at 2500 independently.

4. Callable (advanced)

A callable receives the y array and must return a dict of target counts. Useful for programmatic, data-dependent logic (e.g. "oversample each minority class to 60% of the largest"). [SOURCE: imbalanced-learn docs] Most traders never need this; the float and dict cover 99% of cases.

Results / Findings (worked numerical example)

Let's make the math concrete with our Nifty illustration: N_majority = 9500, N_minority = 500. Watch how the synthetic count explodes as the ratio approaches 1:1. [DERIVED — illustrative arithmetic, not measured]

sampling_strategy Minority after Synthetic added Total rows Ratio
'not majority' (1:1) 9500 9000 19000 1:1
0.5 4750 4250 14250 1:2
0.33 3135 2635 12635 1:3
0.2 1900 1400 11400 1:5
{1: 2000} 2000 1500 11500 ~1:4.75

[DERIVED from α = N_min/N_maj; rounding applied]

Two things jump out:

  1. 1:1 adds 9000 synthetic points — 18× the real minority count. You are now trusting generated data more than real data. Chawla et al. (2002) originally demonstrated SMOTE by oversampling the minority to about 2× its original size in their experiments — not 18×. Pushing far past that dilutes the genuine minority geometry. [SOURCE: Chawla et al. 2002; DERIVED]
  2. A 1:2 or 1:3 ratio adds a fraction of that synthetic volume while still giving the learner enough minority mass to find a boundary. You get the benefit (the tree actually splits on the rare class) without the cost (a synthetic fog covering half your feature space).

Why 1:1 oversampling often hurts

  • It drowns the majority signal. At 1:1 the model's training prior is 50/50, but reality is 5/95. The classifier is implicitly biased toward crying "big move!" — raising false positives. In options, that means legging into trades on quiet days and bleeding theta. [DERIVED from class-prior shift]
  • It over-packs the borderline region. SMOTE creates each synthetic point on the line segment between two real minority points (plus small Gaussian jitter on the k_neighbors). At 18× density, those segments tile the entire convex hull of the minority region — including zones that overlap the majority class. The boundary gets fuzzy and overfit. [SOURCE: Chawla et al. 2002; DERIVED]
  • It wastes the very signal that makes the majority useful. The quiet days aren't noise; they carry the "normal regime" shape. Flattening their relative weight teaches the model to under-weight the regime it will actually live in 95% of the time. [DERIVED]

Why moderate ratios (1:2, 1:3) are usually better

  • They preserve skew awareness. At 1:3 the model still "knows" big moves are rare, so it doesn't casually flag quiet days.
  • Less synthetic density = less boundary overfit. Fewer interpolated points means each synthetic sample carries more marginal information. [DERIVED]
  • They're closer to the original SMOTE intent. Chawla's experiments oversampled modestly; the technique was never designed for 19× inflation. [SOURCE: Chawla et al. 2002]
  • Independent critiques agree caution is warranted. Blagus & Lusa (2013) showed SMOTE can degrade performance on small-sample and high-dimensional datasets — exactly the regime where aggressive oversampling amplifies noise. A gentler ratio is a cheaper, safer first move. [SOURCE: Blagus & Lusa 2013]

How to choose (decision flow)

  1. Start moderate, not maximal. For Nifty big-move models, begin at sampling_strategy = 0.3 to 0.5 (1:3 to 1:2). [DERIVED recommendation]
  2. Grid-search the ratio inside CV. Try {0.1, 0.2, 0.3, 0.5, 1.0} and rank by PR-AUC / F1, never raw accuracy (accuracy rewards the lazy "always quiet" model). [DERIVED]
  3. Consider scale_pos_weight as an alternative. XGBoost natively reweights the minority gradient via scale_pos_weight = N_majority/N_minority (~19 here). SMOTE changes the data; scale_pos_weight changes the loss. They're not identical — SMOTE adds synthetic feature-space points (helping non-tree models and boundary geometry), while scale_pos_weight only reweights (cleaner, no synthetic leakage risk). Many production stacks use scale_pos_weight as the primary lever and SMOTE only when the minority is catastrophically small. [DERIVED; SOURCE: XGBoost docs convention]
  4. Lock the winner as a dict once CV settles, e.g. {1: 3000}, so the count is explicit and reproducible across retrains. [DERIVED]
  5. For multiclass (e.g. up-move / down-move / flat), skip the float — use a dict like {0: 6000, 1: 4000, 2: 5000}. [SOURCE: imbalanced-learn docs]

Reproducibility (code shape — illustrative only)

The snippet below follows the imbalanced-learn API exactly. It is reproduced from the library's documented usage pattern and is illustrative — I did not execute it, and no experiment results are claimed from it. [SOURCE: imbalanced-learn docs; NOT RUN]

from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline
from sklearn.model_selection import StratifiedKFold
from xgboost import XGBClassifier

## Illustrative only — not executed in this article.
## SMOTE must live INSIDE the CV pipeline so the validation fold stays raw.
smote = SMOTE(
    sampling_strategy=0.5,   # minority -> half of majority (1:2)
    k_neighbors=5,           # see S2: k_neighbors
    random_state=42          # see S4: random_state
)

pipe = Pipeline([
    ("smote", smote),
    ("xgb", XGBClassifier(n_estimators=300, max_depth=4,
                          scale_pos_weight=1.0))  # or tune jointly
])

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
## scores = cross_val_score(pipe, X, y, cv=cv, scoring="f1")
Enter fullscreen mode Exit fullscreen mode

Key points that keep this honest:

  • fit_resample(X_train, y_train) is the canonical call; here it's wrapped in a Pipeline so it only fires on training folds. [SOURCE: imbalanced-learn docs]
  • sampling_strategy=0.5 is the float form (binary only). [SOURCE: imbalanced-learn docs]
  • k_neighbors and random_state are the neighbouring SMOTE dials (S2 and S4 of this series). [SOURCE: imbalanced-learn docs]

What Failed / Counter-Evidence

Being honest about where this advice bends:

  • 1:1 is not always wrong. On very large datasets with strong regularization (shallow max_depth, high min_child_weight), a 1:1 balance can be fine because the model can't overfit the synthetic fog. The "1:1 hurts" claim is a default expectation, not a law. [DERIVED caveat]
  • SMOTE before the train/test split = leakage. If you resample the whole dataset and then split, synthetic points leak into the test set and your metrics lie. Always wrap SMOTE in the CV pipeline (as above). This is the #1 mistake I see in trading notebooks. [SOURCE: imbalanced-learn user guide; DERIVED]
  • SMOTE can hurt on small/high-dimensional data. Blagus & Lusa (2013) document cases where SMOTE degrades generalization — usually when the minority is tiny and features are noisy. In that regime, prefer scale_pos_weight or ADASYN/Borderline-SMOTE over vanilla SMOTE. [SOURCE: Blagus & Lusa 2013]
  • Borderline failure mode. When minority and majority heavily overlap, vanilla SMOTE will synthesize points inside the majority region, creating label-noise. That's the motivation for Borderline-SMOTE (Han, Wang & Mao 2005) and ADASYN (He et al. 2008), which focus synthesis near the decision boundary or on hard minority regions. [SOURCE: Han et al. 2005; He et al. 2008]

Limitations

  • Float form is binary-only. For multiclass Nifty regimes you must use a dict. [SOURCE: imbalanced-learn docs]
  • Dict requires you to know target counts up front; that's fine once CV has spoken, awkward as a first guess.
  • SMOTE assumes a meaningful Euclidean feature space. Raw one-hot categoricals or high-cardinality IDs break it. Use SMOTENC (mixed categorical/numeric) or SMOTEN (pure categorical) from the same library when features aren't continuous. [SOURCE: imbalanced-learn docs]
  • It does not by itself encode cost asymmetry. A false "big move" and a missed big move may cost very differently in options; pair SMOTE with scale_pos_weight or a custom objective for true cost-awareness. [DERIVED]
  • 'auto' is deprecated. New code should write 'not majority', not 'auto'. [SOURCE: imbalanced-learn docs]

Practical Takeaways

  • ✅ Default 'auto'/1:1 is a starting point, not a destination — tune the ratio.
  • ✅ For Nifty big-move models, start at 0.3–0.5 (1:3 to 1:2) and CV-grid from there.
  • ✅ Use the float form for binary problems; use a dict for multiclass or once you've locked a target count.
  • ✅ Wrap SMOTE in a Pipeline inside StratifiedKFold — never resample before splitting.
  • ✅ Rank candidates by F1 / PR-AUC, not accuracy.
  • ✅ Keep SMOTE in Layer 2 (EOD training), inside CV — never on Dhan live WebSocket data.
  • ✅ When the minority is tiny or features noisy, lean on scale_pos_weight or Borderline-SMOTE/ADASYN instead of vanilla SMOTE.
  • ✅ Prefer explicit dict targets in production so retrains are reproducible.

FAQ

Q: Is 'auto' the same as 1:1?
Yes, for over-sampling 'auto' equalled 'not majority', which forces the minority to match the majority (1:1). It's now deprecated; write 'not majority'. [SOURCE: imbalanced-learn docs]

Q: How is the float ratio computed exactly?
α = (minority count after resampling) ÷ (majority count). So 0.5 means minority ends up at half the majority's size. Float is binary-only. [SOURCE: imbalanced-learn docs; DERIVED]

Q: SMOTE vs scale_pos_weight — which should I use?
SMOTE adds synthetic data points (helps boundary geometry, useful for non-tree models); scale_pos_weight reweights the loss (cleaner, no synthetic leakage). They're complementary; many stacks use scale_pos_weight as primary and SMOTE when the minority is extremely scarce. [DERIVED]

Q: Can I use a dict for multiclass imbalance?
Yes — that's the recommended path. Map each class label to its desired post-resample count, e.g. {0: 6000, 1: 4000, 2: 5000}. [SOURCE: imbalanced-learn docs]

Q: What ratio do you use for Nifty big-move days?
We start at 0.3–0.5 (1:3 to 1:2) and let walk-forward CV pick the winner by F1. We've rarely seen 1:1 win on this target. [DERIVED recommendation]

Q: Does SMOTE run on live data?
No. In our two-layer NSE stack, SMOTE lives only in Layer 2 (EOD-audited training) inside cross-validation. Layer 1 (Dhan WebSocket) is shadow/predict-only and never resampled. [ARCHITECTURE NOTE]

TL;DR

  • sampling_strategy controls how much minority data SMOTE creates.
  • Strings ('not majority', deprecated 'auto', 'minority', 'all') give coarse 1:1 balancing.
  • Float (e.g. 0.5) = minority becomes that fraction of the majority (1:2); binary only.
  • Dict = exact per-class target counts; best for multiclass and production locking.
  • Forced 1:1 usually hurts Nifty big-move models (drowns majority, overfits borderline).
  • Moderate 1:2 / 1:3 ratios typically win — start there, CV-grid by F1/PR-AUC.
  • SMOTE stays in Layer 2 inside CV, never on live data.

📚 Related Articles

👤 About the Author



Name:
Shakti Tiwari

Role:
Nifty Option Trader, XGBoost Expert

Cert:
NISM-Series-XII (Securities Markets Foundation)

Knows:
XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation



Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources (primary)

  • Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16:321–357. [SOURCE]
  • imbalanced-learn documentation — imblearn.over_sampling.SMOTE, sampling_strategy parameter semantics (str/float/dict/callable, 'not majority' replacing deprecated 'auto', binary-only float). scikit-learn-contrib/imbalanced-learn. [SOURCE]
  • Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. [SOURCE]
  • He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning. [SOURCE]
  • Blagus, R., Lusa, L. (2013). Evaluation of SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. [SOURCE]
  • imbalanced-learn — SMOTENC / SMOTEN (categorical handling) and SVMSMOTE / KMeansSMOTE implementations. [SOURCE]

Author / Canonical

Written by Shakti Tiwari — Nifty Option Trader, XGBoost Expert.
Brand site: optiontradingwithai.in · Profile: about.me/shaktitiwari

Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

Top comments (0)