DEV Community

shakti tiwari
shakti tiwari

Posted on

SMOTE `k_neighbors` Parameter: How 5 Little Neighbors Decide If Your Synthetic Samples Help or Hurt

#smote #machinelearning #imbalanceddata #ai

SMOTE k_neighbors Parameter: How 5 Little Neighbors Decide If Your Synthetic Samples Help or Hurt

Part of the SMOTE deep-dive cluster for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in).

Disclaimer (verbatim): Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Two-layer engine note: Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data.

Quick Answer

The k_neighbors argument of SMOTE controls how many minority-class neighbors each synthetic sample is allowed to interpolate between. The default is 5 (SOURCE: scikit-learn-contrib/imbalanced-learn, default value in SMOTE; Chawla et al. 2002 uses "k nearest neighbors" without fixing a canonical number, the library chose 5). Too small and SMOTE collapses toward duplication — the new points pile onto thin line segments and stop covering new territory. Too large and it reaches across the feature space, dragging in distant neighbors so the synthetic points land in sparse, ambiguous regions that blur the class boundary. The right value tracks the local dimensionality of your minority manifold, and you tune it inside cross-validation, never on held-out data.

Why This Matters

Imbalanced datasets are the daily reality for anyone trading Nifty options. A profitable breakout, a genuine reversal, or a directional signal is rare relative to the thousands of "nothing happens" candles. Train XGBoost naively on that and the model quietly learns to always predict "no trade" — it scores 95% accuracy and is completely useless. (DERIVED: this is the textbook minority-class neglect problem; the accuracy paradox is well documented in the imbalance literature.)

SMOTE is the most-used fix, and k_neighbors is the knob most people leave at its default without understanding. But here is the uncomfortable truth: SMOTE is not automatically safe. Blagus & Lusa (2013) showed that on small, high-dimensional datasets SMOTE can degrade classifier performance. The reason it can backfire usually traces back to how the synthetic points are built — and k_neighbors is the parameter that decides the geometry of every single one of them.

If you are running SMOTE in a pipeline (as you should be, inside CV), a bad k_neighbors means you are spending compute to manufacture noise. A good one means you are genuinely thickening the minority manifold in a way your gradient-boosted model can learn. So this is not a cosmetic setting. It is the difference between an augmented dataset and a corrupted one.

Research Question / Hypothesis

Research question: How does the choice of k_neighbors in SMOTE trade off between two failure modes — under-diversification (synthetic samples collapse toward duplication) and over-generalization (synthetic samples are placed in sparse, ambiguous regions far from the true minority manifold)?

Hypothesis (DERIVED): There exists a problem-dependent sweet spot. When the minority class is locally dense and low-dimensional, a smaller k_neighbors (e.g. 3–5) is sufficient and safer because the nearest neighbors already lie on the manifold. When the minority class is sparse or sits in a high-dimensional feature space where nearest-neighbor distances grow fast (the curse of dimensionality), a slightly larger k_neighbors is needed to give SMOTE enough candidates — but pushing it too far pulls in genuinely distant points and hurts. The optimal value is therefore a function of local dimensionality, not a universal constant.

Data & Methodology

We reason from two primary sources and the library's own implementation, not from a fresh experiment we ran (we are explicit: no code was executed for this article; every snippet below is illustrative, taken from the canonical imbalanced-learn API shape).

Primary sources:

  • Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). "SMOTE: Synthetic Minority Over-sampling Technique." Journal of Artificial Intelligence Research 16:321–357. (SOURCE: original algorithm definition.)
  • scikit-learn-contrib/imbalanced-learn GitHub repository (~7.1k★). (SOURCE: the reference implementation of SMOTE.k_neighbors.)

The mechanism (DERIVED from Chawla 2002 + imbalanced-learn source):

SMOTE operates on the minority class only. For each minority sample x_i:

  1. Find its k_neighbors nearest neighbors within the minority class (Euclidean distance in the original feature space, by default).
  2. Randomly pick one neighbor x_zi from that pool.
  3. Generate x_new = x_i + (x_zi − x_i) · δ, where δ ~ Uniform(0, 1).

So k_neighbors literally sets the size of the candidate pool each point interpolates against. The library builds a kNN index (a NearestNeighbors object, backed by a BallTree or KDTree depending on the metric) over the minority points, queried with n_neighbors = k_neighbors + 1 so the point itself is excluded from its own neighbor list. (DERIVED: this is how imbalanced-learn wires the parameter internally; the "+1" is to skip the self-match.)

That single number therefore controls both the diversity of the interpolation and the maximum reach of any synthetic point. Hold that thought — it is the whole story.

Results / Findings

Finding 1 — Small k_neighbors collapses toward duplication

Take k_neighbors = 1. Each minority point has exactly one neighbor to interpolate with. Every synthetic point for x_i lies on the single line segment between x_i and that lone neighbor. When minority points cluster tightly, the lone nearest neighbor is often the same point for many rows, so the synthesizer keeps drawing on the same few line segments. The result is a synthetic cloud that is geometrically thin — it sits on a 1-D skeleton rather than filling a 2-D (or higher-D) region. Functionally it resembles duplication: you are adding volume without adding information about the manifold's shape.

Even at k_neighbors = 2 or 3, if the minority is sparse, the nearest couple of neighbors may still be very close together. Interpolating between near-identical points yields near-duplicates. This is why people sometimes report SMOTE "doing nothing" — it is not that SMOTE failed; the k_neighbors was too small to escape the local clump. (DERIVED.)

Finding 2 — Large k_neighbors over-reaches into ambiguous space

Now push to k_neighbors = 15 or 20 on a minority set that is genuinely scattered. The 15th or 20th nearest neighbor can be far from x_i — possibly across a gap that separates the minority cluster from the majority region. When x_zi is that distant point, the synthetic sample x_i + (x_zi − x_i)·δ lands somewhere in the middle of that gap. That middle is precisely the ambiguous zone the classifier should not be confidently trained on, because it is likely near the true decision boundary or even inside majority territory. You have manufactured a misleading label.

This is the classic SMOTE pitfall: synthetic points that cross into the majority's space create false confidence and widen the boundary incorrectly. (DERIVED; consistent with the rationale behind Borderline-SMOTE (Han, Wang, Mao 2005), which was invented precisely because vanilla SMOTE generates noisy points near the border.)

Finding 3 — Local dimensionality governs the sweet spot

Here is the subtle part. The distance from x_i to its k-th neighbor does not grow linearly with k — it grows with the intrinsic dimensionality of the local minority manifold. In a genuinely 2-D, well-sampled blob, the 5th neighbor is still close; in a 50-dimensional options feature space, even the 5th neighbor can already be surprisingly far because volume expands explosively with dimension (the curse of dimensionality). (DERIVED: standard concentration-of-measure argument.)

So the "safe" k_neighbors shrinks as local dimensionality rises. This is exactly why Blagus & Lusa (2013) found SMOTE can hurt on small/high-dimensional data: at high dimension the neighbor pool fills with distant points almost immediately, so any k_neighbors large enough to avoid duplication is also large enough to inject noise. The remedy there is not cranking k_neighbors — it is reducing dimensionality first (PCA, feature selection) or switching to a border-aware variant.

Worked 2-D Geometric Intuition

Let the minority class be five points in 2-D:

  • P1 = (0, 0)
  • P2 = (1, 0)
  • P3 = (0, 1)
  • P4 = (1, 1)
  • P5 = (2, 2) ← a stray outlier, far from the tight unit square

Focus on P1 = (0,0).

Case A — k_neighbors = 1. The only neighbor is P2 = (1,0) (distance 1). Every synthetic point for P1 is (δ, 0) with δ ∈ (0,1). All of them lie on the x-axis segment between P1 and P2. They never explore toward P3, P4, or the interior of the unit square. The synthetic manifold is a 1-D line, not the 2-D region the real minority occupies. If many minority rows behave like this, the augmentation is a thin shell — essentially duplication along one axis. (DERIVED.)

Case B — k_neighbors = 5 (default). The neighbors of P1 are P2 (1.00), P3 (1.00), P4 (1.41), P5 (2.83). The pool now includes the stray outlier P5. If the random draw selects P5, the synthetic point is (2.83·δ, 2.83·δ). At δ = 0.5 that is (1.41, 1.41) — far outside the tight unit-square cluster, landing in open space near the P5 outlier. This is over-generalization: SMOTE reached across a 2.83-unit gap to a stray point and manufactured a sample in a sparse, ambiguous region that the original minority data did not actually populate. (DERIVED.)

The lesson (DERIVED): With this geometry, k_neighbors around 3–4 would have been better — P1's three closest neighbors (P2, P3, P4) all sit in the dense unit square, so interpolation stays on-manifold and still has variety. k=1 is too thin; k=5 already drags in the outlier. No single default is right; the data's shape decides.

Finding 4 — The connection to the kNN index

k_neighbors is not just a math parameter; it is a query size against an index. Internally SMOTE fits a NearestNeighbors structure (BallTree/KDTree) and issues a k-NN query per minority point. Two consequences follow:

  1. Cost scales with k. Larger k_neighbors means a larger neighbor list per point and more interpolation candidates — modestly more compute, usually negligible, but real on very large minority sets.
  2. Index quality bounds the result. The neighbors returned are only as meaningful as the distance metric and the feature scaling. If your features are on wildly different scales (e.g. an options "strike distance" feature in thousands vs. an RSI in 0–100), Euclidean distance is dominated by the large-scale feature, so the "nearest" neighbors are nearest in the wrong sense. Standardize features before SMOTE. (DERIVED; standard preprocessing guidance, reinforced by imbalanced-learn's own pipeline examples.)

Reproducibility (code shape — illustrative only)

We did not run this. The snippet below shows the canonical imbalanced-learn usage shape so you can reproduce the experiment yourself. Treat it as illustrative, sourced from the library's documented API — not as an experiment result from us.

## ILLUSTRATIVE ONLY — not executed. Standard imbalanced-learn API shape.
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
from xgboost import XGBClassifier
from sklearn.preprocessing import StandardScaler

## Always scale BEFORE SMOTE (SMOTE uses Euclidean distance).
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("smote", SMOTE(random_state=42, k_neighbors=5)),  # <-- the knob
    ("clf", XGBClassifier(random_state=42, eval_metric="logloss")),
])

## Tune k_neighbors INSIDE CV — never on the test set.
param_grid = {"smote__k_neighbors": [3, 5, 7, 9, 11]}
search = GridSearchCV(pipe, param_grid,
                      scoring="f1",   # or average_precision / roc_auc
                      cv=5, n_jobs=-1)
search.fit(X_train, y_train)
Enter fullscreen mode Exit fullscreen mode

Key points reproduced from the docs (SOURCE: imbalanced-learn): SMOTE is imported from imblearn.over_sampling; it is fit via fit_resample(X, y); k_neighbors accepts an int (number of neighbors) or, in newer versions, an estimator object inheriting from KNeighborsMixin used to find neighbors. The int form is what 99% of users pass.

What Failed / Counter-Evidence

The hypothesis that "bigger k is always better for sparse classes" fails once you cross into the ambiguous zone (Finding 2). Empirical literature supports caution: Blagus & Lusa (2013) report cases where SMOTE worsens results, and the border-noise mechanism is the leading explanation. Borderline-SMOTE (Han et al. 2005) and ADASYN (He et al. 2008) were both invented to fix exactly the failure mode that vanilla SMOTE with a careless k_neighbors produces — generating points in the wrong place. So if grid search keeps preferring very large k_neighbors, suspect that your minority is too sparse or too high-dimensional for vanilla SMOTE at all, and consider a border-aware variant or dimensionality reduction instead.

Limitations

  • We did not execute code. All geometry here is derived analytically; the numbers are hand-computed illustrations, not measured from a run. Validation on your own data is mandatory.
  • k_neighbors only affects where points are drawn, not how many. The count of synthetic samples is controlled by sampling_strategy (the subject of the next article, S3).
  • SMOTE assumes feature-space linearity between neighbors. For heavily non-linear manifolds, interpolation can still leave the manifold even with perfect k_neighbors.
  • Euclidean-distance neighbor selection is scale-sensitive; conclusions assume sensible scaling.

Practical Takeaways

  1. Never leave k_neighbors blind. Default 5 is a starting point, not a recommendation.
  2. Tune it inside CV with a scoring metric that respects imbalance (F1, PR-AUC / average_precision, ROC-AUC) — as shown in the illustrative snippet.
  3. Smaller k when the minority is dense and low-dimensional (stays on-manifold, avoids distant pulls).
  4. Larger k only when the minority is sparse and you have verified synthetic points are not landing in ambiguous space — and cap it (usually ≤ 10–15).
  5. Standardize features first. Euclidean neighbors are meaningless on mixed scales.
  6. Watch local dimensionality. High-dimensional options feature sets push the safe k down, not up. Reduce dimensions or use border-aware variants (Borderline-SMOTE, ADASYN, SVMSMOTE, KMeansSMOTE) when SMOTE misbehaves.
  7. In our stack: apply SMOTE only in Layer 2 (EOD-audited XGBoost/LightGBM training core) inside walk-forward CV — never on Dhan WebSocket live data.

FAQ

Q1. What is the default k_neighbors in SMOTE?
5, per imbalanced-learn (SOURCE: library default). Chawla et al. 2002 described "k nearest neighbors" generally without mandating a number.

Q2. Can k_neighbors be larger than the number of minority samples?
No — it must be strictly less than the minority class count. imbalanced-learn will raise an error otherwise. (DERIVED from the constraint that you need k distinct neighbors within the minority class.)

Q3. Does a bigger k_neighbors mean more synthetic samples?
No. It changes which neighbors each point interpolates with, not how many points are generated. Count is set by sampling_strategy.

Q4. Is k_neighbors = 1 the same as duplication?
Not literally — k=1 still interpolates along the segment to the single neighbor, so you get a spread of points, not exact copies. But the spread is confined to a thin 1-D line, so it behaves like duplication in the sense of adding no new manifold coverage. (DERIVED.)

Q5. Should I use SMOTE on the test set?
Absolutely not. Fit SMOTE on training folds only, inside CV. Leaking synthetic points into evaluation inflates scores.

Q6. How is k_neighbors related to the kNN classifier?
Only structurally. SMOTE uses a kNN index to find minority neighbors for interpolation; it is not a classifier. But the same distance/scale sensitivities apply.

TL;DR

  • k_neighbors (default 5) sets the size of the neighbor pool each minority point interpolates with. (SOURCE: imbalanced-learn.)
  • Too small → thin, duplication-like synthetic shells that add volume without information.
  • Too large → reaches distant neighbors, dropping synthetic points into sparse, ambiguous regions that corrupt the boundary.
  • The sweet spot tracks local dimensionality: small k for dense/low-D, modestly larger k for sparse (but capped).
  • Always tune inside CV on imbalance-aware scoring, standardize first, and apply only in Layer 2 of our NSE stack.

📚 Related Articles

👤 About the Author



Name:
Shakti Tiwari

Role:
Nifty Option Trader, XGBoost Expert

Cert:
NISM-Series-XII (Securities Markets Foundation)

Knows:
XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation



Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

  • Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research 16:321–357. [PRIMARY — SOURCE]
  • scikit-learn-contrib/imbalanced-learn (GitHub, ~7.1k★). SMOTE.k_neighbors implementation and documented API. [PRIMARY — SOURCE]
  • Han, H., Wang, W.-Y., Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. [SOURCE — variant context]
  • He, H., Bai, Y., Garcia, E.A., Li, S. (2008). ADASYN: Adaptive Synthetic Sampling Approach. [SOURCE — variant context]
  • Blagus, R., Lusa, L. (2013). On the effectiveness of SMOTE for imbalanced data classification with small and high-dimensional datasets. [SOURCE — counter-evidence]
  • All geometric examples and the dimensionality argument: [DERIVED] (analytical, not executed).

Author / Canonical

Shakti Tiwari — Nifty Option Trader, XGBoost Expert. NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only. This article is part of the SMOTE deep-dive series on optiontradingwithai.in. Engine context: TWO-LAYER NSE stack — Layer 1 Dhan WebSocket live capture (shadow/predict-only); Layer 2 EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage); imbalance handling applied in Layer 2 inside CV only.

Resources & Links

Top comments (0)