DEV Community

shakti tiwari
shakti tiwari

Posted on

SMOTE random_state & Reproducibility: Why Seeded Synthetic Samples Matter for Auditable XGBoost Backtests

#smote #machinelearning #imbalanceddata #ai

SMOTE random_state & Reproducibility: Why Seeded Synthetic Samples Matter for Auditable XGBoost Backtests

Shakti Tiwari — Nifty Option Trader, XGBoost Expert

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

This is article S4 in the SMOTE cluster. PREV: S3 — sampling_strategy. NEXT: V3 — SVMSMOTE.

Quick Answer

SMOTE is not deterministic by default. For every synthetic minority sample it draws a random neighbor from the k-NN set and a random interpolation factor λ ∼ Uniform(0,1) — both pulled from a pseudo-random number generator (SOURCE: Chawla et al. 2002, JAIR 16:321–357; SOURCE: scikit-learn-contrib/imbalanced-learn, ~7.1k★). If you do not pass random_state, the two draws come from the global NumPy RNG and change every run. Pass random_state=<int> and fit_resample emits the exact same synthetic coordinates each time. That reproducibility is what makes a backtest auditable — and our Layer-2 training core depends on it.

Why This Matters

Here is the uncomfortable truth about options trading research: a backtest you cannot reproduce is a backtest you cannot trust. Aur bhai, agar kal ki run mein 18% return dikh raha tha aur aaj 11% — and you changed nothing — then your pipeline has a hidden non-determinism, and your "edge" might just be noise wearing a tuxedo.

Class imbalance is everywhere in Nifty option signals. Your rare-event label — a sharp expiry-day reversal, an OTM strangle that actually pays, a VIX-spike continuation — might be 1–3% of rows. To teach XGBoost that minority, we often synthesize minority samples with SMOTE inside the training fold. That synthesis is random. If the random draw is not pinned, two backtests of the same historical window produce different synthetic training sets, different models, and different reported metrics. Now tell me: which number do you show the client? Which number do you defend in an audit?

This is precisely why our resampling discipline lives in Layer 2 of the engine. Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data. Within that audited core, every fit_resample call is seeded so that a reviewer can reconstruct the exact training set that produced a given model and a given walk-forward metric. Reproducibility is not a nice-to-have here; it is the whole point of "audited."

Understanding random_state — what it controls, what it does not control, and how it tangles with the global NumPy/sklearn RandomState — is the difference between a research notebook that lies silently and one that can be replayed verbatim. Let's pull it apart.

Research Question / Hypothesis

RQ: What exactly does SMOTE's random_state parameter govern? How does it interact with the global NumPy/sklearn RandomState? And why is a fixed seed necessary — but not sufficient — for an auditable, reproducible backtest?

Hypothesis (DERIVED): Because _make_samples consumes two RNG draws per synthetic point — (a) a random neighbor index and (b) a random λ ∈ [0,1] — the entire synthetic set is a deterministic function of (X_minority, k_neighbors, sampling_strategy, random_state). Seeding it makes the synthetic set reproducible. But the model needs its own seed, and the global RNG can still leak into SMOTE if random_state=None, so a fixed SMOTE seed alone does not guarantee a reproducible end-to-end pipeline.

Data & Methodology

We do not run code in this article. The snippets below are illustrative only, reproduced from the canonical imbalanced-learn usage shape — they are NOT experiment results (SOURCE: scikit-learn-contrib/imbalanced-learn documentation). Treat them as the API contract, nothing more.

## Illustrative only — API shape from imbalanced-learn docs, not an experiment result.
from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y)

smote = SMOTE(
    sampling_strategy='auto',   # set by the PREV article (S3)
    k_neighbors=5,
    random_state=42             # <-- the subject of THIS article
)

X_res, y_res = smote.fit_resample(X_train, y_train)
Enter fullscreen mode Exit fullscreen mode

The methodology we describe is the algorithmic specification as given by Chawla et al. (2002) and as implemented in the imbalanced-learn repository (SOURCE for both). We then derive, from the interpolation formula and from sklearn's documented check_random_state semantics, exactly which RNG draws SMOTE makes and how a seed pins them.

Where the randomness actually lives (SOURCE: Chawla 2002; SOURCE: imbalanced-learn BaseSMOTE._make_samples)

Recall the synthetic-sample formula from the algorithm article (S1):

x_new = x_i + λ · (x_zi − x_i),   λ ~ U(0,1)
Enter fullscreen mode Exit fullscreen mode

For each minority point x_i that must be "grown," SMOTE must choose:

  1. A random neighbor x_zi — drawn from the k_neighbors nearest minority neighbors of x_i. This is one RNG draw (an integer index in [0, k_neighbors)).
  2. A random interpolation factor λ — drawn uniformly from [0,1]. This is a second RNG draw.

Both draws are taken from a numpy.random.RandomState object that SMOTE obtains through sklearn.utils.check_random_state(random_state) (SOURCE: scikit-learn check_random_state contract; SOURCE: imbalanced-learn uses this). Therefore the sequence of neighbor picks and λ values — and hence every synthetic coordinate — is fully determined by the seed fed into that RandomState. With random_state=None, the object is the global np.random singleton, which is mutated by every random call in your program (DERIVED from check_random_state's documented behaviour for the None case).

Results / Findings

Each finding is anchored to a primary source or derived from the formula / RNG contract above.

F1 — Without random_state, the synthetic set differs every run. Because neighbor index and λ are RNG draws, two consecutive fit_resample calls with random_state=None (and no intervening global seed reset) produce different synthetic points (DERIVED from the two-draw mechanism). Concretely, the same minority point can be paired with a different neighbor and a different λ, landing at a different coordinate (SOURCE: imbalanced-learn design; DERIVED).

F2 — With random_state=<int>, the synthetic set is bit-for-bit reproducible. Passing an integer creates a fresh np.random.RandomState(seed) that is independent of the global stream, so repeated fit_resample calls emit identical synthetics (SOURCE: check_random_state int-branch; DERIVED). This is the property an audited backtest needs: same data + same seed ⇒ same training set ⇒ same model ⇒ same metric.

F3 — random_state=None silently couples SMOTE to the global NumPy RNG. When random_state=None, check_random_state returns the global np.random singleton (SOURCE: sklearn contract). That means np.random.seed(7) somewhere else in your script shifts SMOTE's draws, and other np.random calls (shuffles, dropouts, initializations) consume from the same stream and displace SMOTE's draws. This is the single most common "why did my synthetic set move?" bug (DERIVED from global-singleton sharing).

F4 — Passing an int random_state decouples SMOTE from the global RNG. Once you pass 42, SMOTE builds its own RandomState(42) and ignores the global np.random state entirely (DERIVED from check_random_state int-branch). So np.random.seed(999) no longer moves SMOTE's synthetics — but it does still affect anything else using the global stream. Mixing the two mental models is where teams get confused (DERIVED).

F5 — random_state reproduces the synthetics, not the model. The classifier (XGBoost/LightGBM/RF) has its own seed. A seeded SMOTE gives you a reproducible training set; the model trained on it still needs its own random_state/seed for a reproducible model (DERIVED; SOURCE-adjacent: every imbalanced-learn SMOTE+Pipeline example seeds both SMOTE and the estimator).

F6 — Reproducibility ≠ validity. A perfectly reproducible synthetic set can still be a bad synthetic set (e.g. boundary-crossing points on sparse expiries). Seeding guarantees you can replay the mistake, not that the mistake is small (DERIVED; consistent with Blagus & Lusa 2013's caution that SMOTE can hurt on small/high-dimensional data — SOURCE).

Reproducibility (code shape — illustrative)

As stated up top, the following is the canonical imbalanced-learn call shape and is shown for reproducibility of API usage, not as a claim that we executed a model here. The pattern below is what we actually wire into Layer-2 walk-forward CV (SOURCE: imbalanced-learn Pipeline + SMOTE examples).

## Illustrative only — shows where random_state sits: on BOTH SMOTE and the estimator, INSIDE CV.
from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTE
from xgboost import XGBClassifier
from sklearn.model_selection import StratifiedKFold

SEED = 42   # ONE canonical seed, stored in config, version-controlled.

pipe = Pipeline([
    ('smote', SMOTE(sampling_strategy='auto', k_neighbors=5, random_state=SEED)),
    ('clf',   XGBClassifier(n_estimators=300, max_depth=4,
                            eval_metric='logloss', random_state=SEED))
])

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=SEED)
## pipe.fit(X_train_fold, y_train_fold) -> scored on the untouched validation fold
Enter fullscreen mode Exit fullscreen mode

The golden rule, repeated because people violate it constantly: seed everything that touches the random stream — SMOTE, the splitter, and the estimator — and store that seed in version control next to the data snapshot. A backtest is only auditable if a reviewer can re-run it and land on the same number (DERIVED from CV hygiene; echoed in imbalanced-learn "avoid leakage" guidance — SOURCE).

Worked Example: Same Data, Two Seeds, Two Synthetic Sets (DERIVED math)

Let's make the seed-dependence concrete in two dimensions. Take one minority point A = (2, 3). Its k_neighbors=2 minority neighbors are B = (4, 7) and C = (1, 5), so the candidate segment set is AB and AC.

SMOTE makes two draws per synthetic: (neighbor index ∈ {0,1}, λ ∼ U(0,1)).

Run 1 — random_state=42: Suppose the RNG yields (neighbor=0 → B, λ=0.25).

x_new = A + 0.25·(B − A) = (2,3) + 0.25·(2,4) = (2.50, 4.00)
Enter fullscreen mode Exit fullscreen mode

Run 2 — random_state=7: Suppose the RNG yields (neighbor=1 → C, λ=0.80).

x_new = A + 0.80·(C − A) = (2,3) + 0.80·(−1,2) = (1.20, 4.60)
Enter fullscreen mode Exit fullscreen mode

Two different seeds ⇒ two different neighbors ⇒ two different λ ⇒ two synthetic points that are not even on the same segment. Multiply this by hundreds of minority points and you get entirely different synthetic clouds from the same real minority set (DERIVED). This is exactly why an unseeded SMOTE poisons backtest comparability: you are not comparing models, you are comparing random draws.

Key nuance (DERIVED): the real minority points A, B, C never change — only the invented ones do. So unseeded SMOTE doesn't alter your ground truth; it alters the training fiction you built on top of it, and that fiction is what your model memorizes.

What Failed / Counter-Evidence

Seeding SMOTE is necessary, but several things still break end-to-end reproducibility even when random_state is set. Be honest about these:

C1 — Global np.random leakage via random_state=None. If any colleague "cleans up" the code and removes random_state (thinking the global np.random.seed() call covers it), SMOTE rejoins the global stream and the synthetic set drifts again (DERIVED from F3/F4). We have seen this exact regression in shared notebooks (SOURCE-adjacent: a well-known failure mode in imbalanced-learn GitHub issues).

C2 — The estimator's own non-determinism dominates. Even with a seeded SMOTE, XGBoost on GPU or with certain histogram threading can return slightly different trees across runs; LightGBM's feature-parallel splits can, too. So "I seeded SMOTE, why isn't the metric identical?" — because the model seed or hardware threading moved (DERIVED from XGBoost/LightGBM known non-determinism on parallel/GPU paths). SMOTE seeding is necessary but not sufficient (DERIVED from F5).

C3 — Library version drift changes the RNG stream. The exact sequence of draws produced by a given integer seed depends on the implementation of the sampler and the RNG backend. A different imbalanced-learn / scikit-learn / numpy version can emit a different synthetic set for the same random_state (DERIVED from version-sensitive RNG implementations; SOURCE-adjacent: sklearn's documented RNG/Generator migration notes). Pin your versions — reproducibility includes the software, not just the seed (DERIVED engineering principle).

C4 — Different seed per fold hides instability. Some teams reseed SMOTE per CV fold "to get more data." That means each fold trains on a different synthetic cloud, so fold metrics are not comparable across reruns and the variance of the synthetic step is masked (DERIVED). For auditability you want the same seed every fold, every run.

C5 — Pickling SMOTE without recording the seed. If you joblib.dump a fitted SMOTE and later fit_resample a fresh instance without the seed, you get a new synthetic set (DERIVED from F2). Store the seed alongside the artifact.

Limitations

  1. Seeding controls randomness, not correctness. A reproducible bad synthetic set is still bad (see F6; SOURCE: Blagus & Lusa 2013 on SMOTE's limits on small/high-dimensional data).
  2. random_state=None is the default trap. If you instantiate SMOTE() with no seed, you opted into global-stream coupling (DERIVED from defaults; SOURCE: imbalanced-learn default random_state=None).
  3. Global-singleton sharing is invisible. Nothing in the API warns you that random_state=None shares np.random with every other random call (DERIVED from check_random_state semantics).
  4. Parallelism can reintroduce non-determinism in the downstream model, even when SMOTE is perfectly seeded (see C2).
  5. Version sensitivity. Same seed, different library version ⇒ different synthetics (C3).
  6. Assumes you actually re-run on the same X/y snapshot. Reproducibility of synthetics is moot if the input data itself changed between runs (DERIVED — data versioning is part of the audit).

Practical Takeaways (Production Checklist)

  • Always pass random_state=<int> to SMOTE. Never leave it None in a research or production pipeline (SOURCE: imbalanced-learn API; DERIVED from F1/F3).
  • Pick ONE canonical seed (we use 42 as a documented default) and store it in config, version-controlled next to the data snapshot (DERIVED from audit discipline).
  • Seed the estimator too. XGBoost random_state, the StratifiedKFold random_state, and SMOTE's — all three (DERIVED from F5/C2).
  • Wrap SMOTE in a Pipeline so it fits per-fold, never on the full set, and the same seed applies consistently inside CV (SOURCE: imbalanced-learn Pipeline guidance).
  • Don't rely on np.random.seed() to control SMOTE — pass the int. Global seeding is fragile once any other random call exists (DERIVED from F3/F4).
  • Pin library versions (imbalanced-learn, scikit-learn, numpy, xgboost) in your environment lockfile; reproducibility includes the software (DERIVED from C3).
  • For Nifty option rare-event labels: apply SMOTE only inside Layer-2 EOD walk-forward CV, seed it, and compare against a scale_pos_weight baseline that is inherently seed-free (DERIVED from our two-layer discipline). When the minority is genuinely sparse, scale_pos_weight is often the more auditable choice.
  • Log the seed with every backtest run. An auditable run is (data hash, code hash, library versions, seed) → metric. Miss any one and you can't replay it (DERIVED).

FAQ

Q: Does SMOTE need random_state to work?

It works without it — but produces a different synthetic set every run. For any reproducible research or backtest, set it (SOURCE: imbalanced-learn; DERIVED from F1).

Q: What does random_state actually control in SMOTE?

The random neighbor index and the random λ ∈ [0,1] used in x_i + λ·(x_zi − x_i) — i.e. which synthetic points get made (SOURCE: Chawla 2002; DERIVED from _make_samples).

Q: If I set np.random.seed(42), is SMOTE reproducible?

Only if random_state=None — and even then, any other np.random call shifts SMOTE's draws. Pass random_state=42 to SMOTE directly; that decouples it from the global stream (DERIVED from F3/F4).

Q: Does seeding SMOTE make my XGBoost model reproducible?

No. It makes the training set reproducible. The model needs its own seed (F5).

Q: Why does reproducibility matter for trading backtests?

Because an auditable backtest must be replayable: same data + same seed + same code + same versions ⇒ same number. Otherwise you can't defend or compare results (DERIVED from our Layer-2 audit discipline).

Q: Can two seeds give totally different synthetic clouds?

Yes — different neighbor picks and λ values put synthetic points on different segments (see Worked Example). Same real minority, different fiction (DERIVED).

Q: Is random_state the same across SMOTE variants (SVMSMOTE, Borderline, ADASYN)?

Yes, they all inherit the same check_random_state contract, so the seeding principle is identical; only the selection of points differs (SOURCE: imbalanced-learn; DERIVED). The NEXT article (V3 SVMSMOTE) covers that variant.

Q: Can I use SMOTE on live Nifty ticks?

No — never on live data. Our stack applies it only in Layer-2 EOD CV (two-layer engine note above).

TL;DR

SMOTE draws a random neighbor and a random λ∼U(0,1) per synthetic sample, so without random_state the synthetic set changes every run. Pass random_state=<int> to make it bit-for-bit reproducible — essential for auditable Layer-2 backtests. But note: random_state=None silently couples SMOTE to the global NumPy RNG, an int seed decouples it, and the model still needs its own seed. Seeding is necessary, not sufficient: pin library versions, seed the splitter and estimator too, and log the seed with every run. Source: Chawla et al. 2002 + imbalanced-learn.

📚 Related Articles

👤 About the Author



Name:
Shakti Tiwari

Role:
Nifty Option Trader, XGBoost Expert

Cert:
NISM-Series-XII (Securities Markets Foundation)

Knows:
XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation



Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources (PRIMARY)

  • Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research (JAIR), 16:321–357. — original algorithm; interpolation formula x_new = x_i + λ·(x_zi − x_i) with λ drawn randomly; motivation vs random oversampling.
  • scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k★). imblearn.over_sampling.SMOTE — random_state parameter (default None), fit_resample API, BaseSMOTE._make_samples neighbour/λ draws, Pipeline/leakage guidance, SVMSMOTE/BorderlineSMOTE/ADASYN/KMeansSMOTE variants sharing the same check_random_state contract.
  • scikit-learn sklearn.utils.check_random_state contract — None ⇒ global np.random singleton; int ⇒ np.random.RandomState(int); RandomState instance ⇒ used as-is. (SOURCE: scikit-learn API docs; DERIVED behaviour described above.)
  • Blagus, R., Lusa, L. (2013). SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. — evidence SMOTE can degrade on small/high-dimensional data (reproducibility does not imply validity; F6/C1 context).
  • Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE — boundary-aware variant (previewed; NEXT family).
  • He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN — adaptive variant sharing the same seeding contract.
  • Nguyen, H.M., Cooper, E.W., Kamei, K. (2011). SVM-SMOTE — SVM-margin variant (NEXT: V3).
  • KMeansSMOTE (2018, last-resort paper) — clustering-guided variant.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE engineering cluster, sibling to the XGBoost cluster. This article (S4, random_state & reproducibility) is the canonical reference for seeding SMOTE and for understanding its interaction with the global NumPy/sklearn RandomState inside an auditable Layer-2 backtest. The PREV article (S3, sampling_strategy) covers how many synthetics to make; the NEXT article (V3, SVMSMOTE) covers which points the variant synthesizes.

Disclaimer (repeat): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Resources & Links

Top comments (0)