DEV Community

shakti tiwari
shakti tiwari

Posted on

BorderlineSMOTE in imbalanced-learn: Oversample the Decision Boundary, Not the Safe Interior

#smote #machinelearning #imbalanceddata #ai

BorderlineSMOTE in imbalanced-learn: Oversample the Decision Boundary, Not the Safe Interior

Part of the SMOTE Family series. Previous: V1 vanilla SMOTE. Next: V3 SVMSMOTE.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

BorderlineSMOTE (from imblearn.over_sampling import BorderlineSMOTE) is a variant of SMOTE that refuses to waste synthetic samples in the safe interior of the minority class. It first classifies every minority point as SAFE, DANGER, or NOISE based on how many of its nearest neighbours belong to the majority class, then synthesises new points only for the DANGER set — the minority instances hugging the decision boundary, which are the ones a classifier actually struggles with. When the informative signal lives at the class boundary (the usual case in noisy financial data), BorderlineSMOTE typically beats vanilla SMOTE while adding less redundant noise. [DERIVED summary of Han et al. 2005 + imbalanced-learn implementation]

Why This Matters

If you trade Nifty options, you already live in an imbalanced world. Big directional moves that you want to catch — a sharp expiry-day reversal, a volatility expansion — are rare compared to the dull chop that fills most sessions. Train an XGBoost model naively on that history and it will quietly learn the lazy rule "predict the common class," because that minimises raw error while delivering a model that is useless for the trades that pay the bills.

SMOTE fixed part of that by manufacturing synthetic minority rows. But vanilla SMOTE sprays those synthetic rows everywhere — including deep inside minority territory where the classifier was already confident. That is wasted effort, and worse, it can blur the very boundary you care about. BorderlineSMOTE is the smarter cousin: it asks where the hard cases are and spends its synthetic budget there. [DERIVED from the conceptual motivation in Han et al. 2005]

Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data.

Research Question / Hypothesis

Research question: For a binary classifier trained on an imbalanced dataset, does restricting synthetic over-sampling to the minority "danger" region (BorderlineSMOTE) improve minority-class recall and balanced metrics (F1, G-mean) relative to unrestricted vanilla SMOTE, without inflating training cost?

Hypothesis (DERIVED): Yes — when the classes are separated by a boundary rather than thoroughly intermixed. The boundary minority points carry the most discriminative information; the interior minority points are already easy; the noise minority points (surrounded by majority) are usually mislabels. Concentrating generation on the danger zone should raise genuine recall while adding less artificial overlap. The gain collapses when the classes are so overlapping that "danger" ≈ "everything," or when the dataset is tiny/high-dimensional (see Limitations).

Data & Methodology

We describe the canonical experimental shape used throughout the SMOTE literature and the imbalanced-learn documentation. No experiment is executed in this article — the snippet below is illustrative, taken from the imbalanced-learn API, and is shown only to anchor the method. [SOURCE: scikit-learn-contrib/imbalanced-learn repo, BorderlineSMOTE docstring/example]

The pipeline shape is always:

  1. Stratified train/test split (never let SMOTE see the test set).
  2. Inside cross-validation only, fit BorderlineSMOTE on the training fold and fit_resample.
  3. Train the estimator on the resampled fold; validate on the untouched test fold.
  4. Optionally wrap with Pipeline so resampling is never leaked.

Illustrative API usage (from imbalanced-learn docs — not run here):

from collections import Counter
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import BorderlineSMOTE
from imblearn.pipeline import Pipeline
from imblearn.metrics import classification_report_imbalanced

## Synthetic illustrative data — do not treat as a real market dataset
X, y = make_classification(
    n_classes=2, class_sep=2, weights=[0.1, 0.9],
    n_informative=3, n_redundant=1, flip_y=0,
    n_features=20, n_clusters_per_class=1,
    n_samples=1000, random_state=10,
)
print("Original dataset shape", Counter(y))
## Original dataset shape Counter({1: 900, 0: 100})

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

## borderline-1 is the default kind
sm = BorderlineSMOTE(kind="borderline-1", random_state=42)
X_res, y_res = sm.fit_resample(X_train, y_train)
print("Resampled shape", Counter(y_res))
## Resampled shape Counter({0: 900, 1: 900})  -- illustrative from repo example

pipe = Pipeline([
    ("smote", BorderlineSMOTE(random_state=42)),
    ("clf", RandomForestClassifier(random_state=42)),
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report_imbalanced(y_test, y_pred))
Enter fullscreen mode Exit fullscreen mode

Key constructor parameters [SOURCE: imbalanced-learn repo, filter.py]:

  • k_neighbors (default 5) — number of nearest neighbours used to generate each synthetic point.
  • m_neighbors (default 10) — number of nearest neighbours used to decide whether a minority point is "in danger."
  • kind — "borderline-1" (default) or "borderline-2" (see below).
  • sampling_strategy (default "auto") — how much to oversample.
  • random_state — for reproducibility of the random interpolation gap.

A useful diagnostic the library exposes: sm.in_danger_indices_ — a dict mapping each resampled class to the indices of the original minority rows that were flagged "in danger" and used to seed synthesis. Inspecting it tells you exactly which rows BorderlineSMOTE considered boundary-relevant. [SOURCE: imbalanced-learn repo, BorderlineSMOTE.in_danger_indices_ attribute]

Results / Findings

Because this article does not run an experiment, the "findings" below are a synthesis of the primary sources, not measured numbers from this author's machine.

Finding 1 — Vanilla SMOTE over-invests in the interior. Vanilla SMOTE selects every minority point with equal probability and interpolates between two minority neighbours [SOURCE: Chawla et al. 2002]. Many of those points sit far from any majority instance, where the classifier already separates cleanly. Generating there adds rows the model did not need and, in the worst case, the interpolated segment between two minority points can cross majority space, fabricating ambiguous examples. [DERIVED]

Finding 2 — The boundary minority points are the informative ones. Han, Wang and Mao (2005) argue that a minority instance surrounded mostly by its own class is "safe" and contributes little new separable signal, while one sitting next to majority instances is on the frontier where misclassification actually happens [SOURCE: Han, Wang, Mao 2005]. Concentrating synthesis there strengthens the decision region where it is weakest. [DERIVED interpretation]

Finding 3 — The DANGER mask is the load-bearing mechanism. In imbalanced-learn, danger detection lives in _in_danger_noise: for each minority point, count how many of its m_neighbors nearest neighbours (default 10) belong to the majority class. A point is flagged DANGER when that count is at least half but not all (m/2 ≤ n_majority < m); it is flagged NOISE when all neighbours are majority; otherwise it is left alone (SAFE). Only DANGER points seed new synthetic rows. [SOURCE: imbalanced-learn repo, base.py _in_danger_noise]

Finding 4 — borderline-1 vs borderline-2 change where synthesis reaches. This is the part most practitioners miss.

  • borderline-1 (default): synthetic samples are generated by interpolating each DANGER minority point with its minority-class neighbours only. The new points stay on the minority side of the boundary, thickening the edge of the minority region. [SOURCE: imbalanced-learn repo, filter.py — X_to_sample_from = X_class]
  • borderline-2: the whole dataset is used as the neighbour pool (X_to_sample_from = X), so the DANGER point can also be interpolated toward its majority neighbours. This pushes synthetic points across the boundary into majority space, directly contesting the majority region. [SOURCE: imbalanced-learn repo, filter.py — X_to_sample_from = X]

The interpolation itself is the familiar SMOTE rule, here written for a DANGER seed x_i and a chosen neighbour x_z [DERIVED]:
x_new = x_i + (x_z − x_i) · δ, with δ ~ Uniform(0, 1).

For borderline-1, x_z is drawn from the minority k-NN of x_i; for borderline-2, x_z may be a minority or majority k-NN of x_i.

Finding 5 — Empirical direction from the literature. The original paper reports Borderline-SMOTE outperforming vanilla SMOTE on a range of imbalanced UCI-style datasets, particularly on F-measure and related balanced metrics, because it stops fabricating easy interior points [SOURCE: Han, Wang, Mao 2005]. We are not quoting exact tables here (this author did not re-run them); the qualitative claim is well established in the over-sampling literature. [DERIVED note on what the paper establishes]

Reproducibility

To reproduce a real comparison you would, at minimum:

  1. Pick a fixed random_state everywhere (data split, SMOTE, estimator).
  2. Compare three resamplers inside the same CV loop: SMOTE() (V1), BorderlineSMOTE(kind="borderline-1"), BorderlineSMOTE(kind="borderline-2").
  3. Score with minority recall, precision, F1, and G-mean — not raw accuracy, which is misleading under imbalance.
  4. Always fit_resample on the training fold only.

The code sketch in Data & Methodology is the canonical shape from the imbalanced-learn documentation and is presented illustratively — it was not executed for this article, and its printed outputs are quoted from the library's own example for reference. [SOURCE: imbalanced-learn repo docstring; not an experiment result of this author]

What Failed / Counter-Evidence

Borderline-SMOTE is not a free lunch, and the honest literature says so.

  • Blagus & Lusa (2013) show that SMOTE-family over-sampling can hurt on small or high-dimensional datasets, where the nearest-neighbour geometry is unreliable and synthetic points amplify noise rather than signal [SOURCE: Blagus & Lusa 2013]. Borderline-SMOTE narrows the problem but inherits it.
  • When classes are thoroughly intermixed, almost every minority point becomes "in danger," so BorderlineSMOTE degenerates toward vanilla SMOTE — you pay the complexity for little gain. [DERIVED]
  • Boundary noise is still boundary noise. If the minority points near the boundary are actually mislabels (NOISE that escaped the m/2 filter), BorderlineSMOTE will enthusiastically manufacture more of them. Over-focusing on the border can amplify labelling errors. [DERIVED]
  • Better variants exist for hard cases. SVMSMOTE (V3) uses an SVM to pick support-vector-like border points; KMeansSMOTE clusters first. BorderlineSMOTE is a strong default but not the final word. [SOURCE: imbalanced-learn repo; Nguyen et al. 2011 for SVM-SMOTE]

Limitations

  1. It still ignores majority density. BorderlineSMOTE synthesises minority points based on neighbour labels, not on how densely the majority class is packed. It can drop a synthetic minority point into a dense majority cluster, making that region even more ambiguous. [DERIVED]
  2. It can over-focus on border noise. As noted, the DANGER set includes genuinely hard points and mislabeled ones; the algorithm cannot tell them apart. [DERIVED]
  3. Threshold sensitivity. Results shift with m_neighbors and k_neighbors. Too small an m_neighbors makes the danger estimate noisy; too large blurs the boundary concept. These are hyperparameters you must tune inside CV, not set once. [DERIVED]
  4. Feature-space only. Like all SMOTE variants, it operates in the raw feature space and assumes Euclidean proximity means semantic similarity — fragile with mixed-type, high-cardinality, or heavily engineered features. (For those, SMOTENC/SMOTEN exist.) [SOURCE: imbalanced-learn repo]
  5. No guarantee of separability. Adding boundary minority points does not create a cleaner boundary; if the true boundary is irreducible noise, more synthetic points just thicken the fog. [DERIVED]
  6. Class imbalance ratio still matters. At extreme ratios (e.g., 1:1000), even BorderlineSMOTE may need to be paired with under-sampling of the majority or with scale_pos_weight in the estimator itself. [DERIVED]

Practical Takeaways

  • Reach for BorderlineSMOTE when your minority class forms reasonably coherent clusters with a definable boundary and you care about recall at that boundary — the classic imbalanced-classification setup, including many Nifty option signal problems.
  • Prefer borderline-1 as the default; try borderline-2 only when you suspect the minority region needs to actively push into majority space. Always validate both inside CV.
  • Never resample the test set. Resample inside the training fold only, ideally via imblearn.pipeline.Pipeline so it composes cleanly with scaling and the classifier.
  • Pair with cleaning. Because BorderlineSMOTE can still leave noisy majority regions, many practitioners follow it with Edited Nearest Neighbours (ENN) to prune clearly misplaced points — a "SMOTE + ENN" or "BorderlineSMOTE + ENN" combo.
  • Tune m_neighbors/k_neighbors as hyperparameters; don't accept defaults blindly on real data.
  • Compare against the cheap baseline. Before reaching for any SMOTE variant, try scale_pos_weight (XGBoost/LightGBM) or class weights — sometimes that alone closes the gap without synthesising a single row.
  • In our stack, all of this lives in Layer 2 (EOD-audited training), never in Layer 1 (live Dhan capture).

FAQ

Q1. What exactly is the difference between borderline-1 and borderline-2?
A1. Both select the same DANGER minority points. borderline-1 interpolates each DANGER point only with its minority neighbours, keeping new samples on the minority side. borderline-2 uses the whole dataset as the neighbour pool, so a DANGER point can also be interpolated toward majority neighbours, pushing synthetic points across the boundary. [SOURCE: imbalanced-learn repo]

Q2. How does BorderlineSMOTE decide a point is "in danger"?
A2. For each minority point it counts majority-class neighbours among its m_neighbors (default 10) nearest neighbours. If at least half but not all are majority → DANGER (used for synthesis). If all are majority → NOISE (skipped). Otherwise → SAFE (skipped). [SOURCE: imbalanced-learn repo, _in_danger_noise]

Q3. Is BorderlineSMOTE always better than vanilla SMOTE?
A3. No. It wins when the informative minority signal sits at a boundary and the interior is already separable. When classes are heavily intermixed or the dataset is tiny/high-dimensional, it can degenerate toward vanilla SMOTE or even hurt. [DERIVED from Han 2005 + Blagus & Lusa 2013]

Q4. Can I use BorderlineSMOTE with categorical features?
A4. BorderlineSMOTE works in continuous Euclidean space. For mixed numeric/categorical data, use SMOTENC; for all-categorical, use SMOTEN — both from imblearn.over_sampling. [SOURCE: imbalanced-learn repo]

Q5. Should I apply it to my live trading data?
A5. No — apply any over-sampling inside the training/CV loop of your audited model core, never on live inference. Our NSE stack keeps resampling strictly in Layer 2. [DERIVED from the two-layer engine design]

Q6. What comes after BorderlineSMOTE?
A6. V3 SVMSMOTE uses an SVM to locate border-support points, and KMeansSMOTE clusters before synthesising. They refine the "where to synthesise" question further. [SOURCE: imbalanced-learn repo; Nguyen et al. 2011]

TL;DR

  • BorderlineSMOTE synthesises minority samples only near the decision boundary, not in the safe interior. [DERIVED]
  • It tags each minority point SAFE / DANGER / NOISE by counting majority neighbours; only DANGER seeds new rows. [SOURCE: imbalanced-learn repo]
  • borderline-1 (default) keeps new points on the minority side; borderline-2 can push them into majority space. [SOURCE: imbalanced-learn repo]
  • It usually beats vanilla SMOTE when the boundary carries the signal, but ignores majority density and can amplify border noise. [DERIVED]
  • Always resample inside CV only; tune m_neighbors/k_neighbors; consider pairing with ENN. [DERIVED]
  • Next in series: V3 SVMSMOTE. Previous: V1 vanilla SMOTE.

📚 Related Articles

👤 About the Author



Name:
Shakti Tiwari

Role:
Nifty Option Trader, XGBoost Expert

Cert:
NISM-Series-XII (Securities Markets Foundation)

Knows:
XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation



Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

  • [SOURCE] Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. Advances in Intelligent Computing (ICIC 2005), Lecture Notes in Computer Science, vol. 3644, pp. 878–887. Springer. — original Borderline-SMOTE algorithm, SAFE/BORDERLINE/NOISE classification, borderline-1 vs borderline-2.
  • [SOURCE] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — vanilla SMOTE baseline.
  • [SOURCE] scikit-learn-contrib/imbalanced-learn GitHub repository — imblearn/over_sampling/_smote/filter.py (BorderlineSMOTE, m_neighbors, kind, in_danger_indices_) and imblearn/over_sampling/_smote/base.py (_in_danger_noise danger/noise mask logic). Primary source for the implementation described.
  • [SOURCE] Blagus, R., & Lusa, L. (2013). A note on the estimation of the optimal SMOTE oversampling ratio. / SMOTE can degrade performance on small, high-dimensional datasets. — counter-evidence.
  • [SOURCE] Nguyen, H. M., Cooper, E. W., & Kamei, K. (2011). Borderline over-sampling for imbalanced data classification. — basis for SVM-SMOTE (V3).
  • [DERIVED] Interpolation formula x_new = x_i + (x_z − x_i)·δ and the "where to synthesise" interpretation, derived from the cited sources.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE Family series (V2 of V6). This is educational content; not investment advice and not SEBI-registered research.

Resources & Links

Top comments (0)