KMeansSMOTE in imbalanced-learn: Cluster First, Then Synthesise Inside the Cluster

Part of the SMOTE Family series. Previous: V3 SVMSMOTE. Next: V5 SMOTENC.

Disclaimer: Shakti Tiwari is NISM-Series-XII certified; not a SEBI-registered Research Analyst. Content is educational only.

Quick Answer

KMeansSMOTE (from imblearn.over_sampling import KMeansSMOTE) is the SMOTE variant that refuses to synthesise minority rows blindly across the whole feature space. It first runs K-Means on the minority class to split it into coherent sub-groups (clusters), then applies ordinary SMOTE inside each cluster only. Because interpolation is confined to intra-cluster neighbours, synthetic points land in high-density minority regions and never in the empty gaps between clusters — the exact place vanilla SMOTE keeps spraying useless, ambiguous samples. The method comes from Douzas & Bacao (2018), "The Last Resort," and ships in scikit-learn-contrib/imbalanced-learn. [SOURCE: Douzas & Bacao 2018; imbalanced-learn KMeansSMOTE]

Why This Matters

Agar aap Nifty options trade karte ho, toh aap already imbalanced data ke andar baithe ho. The setups that actually pay — a volatility-expansion breakout, an expiry-day pin, a failed-breakdown snap-back — are rare. The boring chop that fills 90% of sessions is common. Train an XGBoost or LightGBM naively on that history and it quietly learns the lazy rule "predict the common outcome," because that minimises raw error while delivering a model that is useless for the trades that matter.

Vanilla SMOTE (V1) manufactured synthetic minority rows to fix that — but it interpolated between any two minority points in the whole dataset, including points on opposite sides of an empty region. BorderlineSMOTE (V2) concentrated synthesis at the boundary. SVMSMOTE (V3) let an SVM draw that boundary. KMeansSMOTE (V4) attacks a different flaw: the minority class is usually not one blob — it is several blobs. When the rare class has multiple distinct sub-populations (different market regimes, different failure modes), a single global interpolation surface connects them with fictional in-between samples. KMeansSMOTE carves the space first, so each regime is thickened within itself. [DERIVED from the conceptual motivation in Doufas & Bacao 2018]

Our NSE stack is TWO-LAYER: Layer 1 = Dhan WebSocket live capture (shadow/predict-only); Layer 2 = EOD-audited XGBoost/LightGBM training core (walk-forward, cost-and-slippage). Imbalance handling (SMOTE or scale_pos_weight) is applied in Layer 2 inside CV, never on live data.

Research Question / Hypothesis

Research question: For a binary classifier trained on an imbalanced dataset whose minority class is multi-modal (several separated sub-populations), does a cluster-then-synthesise strategy (KMeansSMOTE) improve minority-class recall and balanced metrics (F1, G-mean) relative to global-interpolation SMOTE, and under what data conditions does the gain appear or vanish?

Hypothesis (DERIVED): KMeansSMOTE should win when the minority class is genuinely multi-modal — several dense, separated sub-groups — because confining SMOTE to intra-cluster neighbours prevents generation in the low-density gaps between sub-groups, where synthetic points would be pure noise. The gain should shrink or reverse when (a) the minority class is essentially unimodal (then KMeansSMOTE ≈ vanilla SMOTE plus overhead), (b) the wrong number of clusters k is chosen, or (c) outliers distort the K-Means partition (a singleton outlier can hijack the density-weighted sampling). The hypothesis is a derived expectation, not a measurement from this article.

Data & Methodology

We describe the canonical experimental shape used throughout the imbalanced-learn documentation and the SMOTE literature. No experiment is executed in this article — the snippet below is illustrative, taken from the imbalanced-learn API, and is shown only to anchor the method. [SOURCE: scikit-learn-contrib/imbalanced-learn repo, KMeansSMOTE docstring/example]

The pipeline shape is always:

1. Stratified train/test split (never let SMOTE see the test set).
2. Inside cross-validation only, fit KMeansSMOTE on the training fold and fit_resample.
3. Train the estimator on the resampled fold; validate on the untouched test fold.
4. Because K-Means is distance-based, a StandardScaler must precede it — ideally wrapped in the same Pipeline so resampling + scaling never leak.

Illustrative API usage (from imbalanced-learn docs — not run here):

from collections import Counter
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import KMeansSMOTE
from imblearn.pipeline import Pipeline
from imblearn.metrics import classification_report_imbalanced

## Synthetic illustrative data — do not treat as a real market dataset.
## n_clusters_per_class=3 makes the minority class multi-modal on purpose.
X, y = make_classification(
    n_classes=2, class_sep=2, weights=[0.1, 0.9],
    n_informative=4, n_redundant=1, flip_y=0.02,
    n_features=20, n_clusters_per_class=3,
    n_samples=2000, random_state=10,
)
print("Original dataset shape", Counter(y))
## Original dataset shape Counter({1: ~1800, 0: ~200})

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42
)

## kmeans_estimator can be an int (number of clusters) or an estimator object.
kms = KMeansSMOTE(
    kmeans_estimator=5,        # carve the minority class into 5 sub-clusters
    k_neighbors=3,             # SMOTE neighbours *within* each cluster
    random_state=42,
)
X_res, y_res = kms.fit_resample(X_train, y_train)
print("Resampled shape", Counter(y_res))
## Resampled shape Counter({0: ~1800, 1: ~1800})  -- illustrative from repo example

## scale BEFORE K-Means sees the data, inside the same pipeline
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("kms", KMeansSMOTE(kmeans_estimator=5, k_neighbors=3, random_state=42)),
    ("clf", RandomForestClassifier(random_state=42)),
])
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report_imbalanced(y_test, y_pred))

Key constructor parameters [SOURCE: imbalanced-learn repo, imblearn/over_sampling/_smote/cluster.py KMeansSMOTE]:

• kmeans_estimator — the clusterer applied to the minority class. If None (the default), imbalanced-learn uses MiniBatchKMeans(n_clusters=2, random_state=random_state). You may pass an int (used as the number of clusters) or any K-Means-style estimator (e.g. KMeans(n_clusters=...)) to control the clustering. [SOURCE: imbalanced-learn repo, KMeansSMOTE.__init__ default]
• cluster_centers (default None) — a precomputed array of shape (n_clusters, n_features). If you provide it, K-Means is skipped entirely and these centers are used directly to assign minority points to clusters. Handy for determinism across CV folds or when you already know the partition from EDA. If the centers are wrong, generation degrades silently. [SOURCE: imbalanced-learn repo, KMeansSMOTE cluster_centers]
• k_neighbors (default 2) — number of nearest intra-cluster minority neighbours used to generate each synthetic point. A cluster with fewer than k_neighbors + 1 points cannot generate (it has no neighbour to interpolate with) and is skipped. [SOURCE: imbalanced-learn repo]
• sampling_strategy (default "auto") — how much to oversample; n_jobs, random_state inherited from the base over-sampler. [SOURCE: imbalanced-learn repo]

A useful diagnostic the library exposes: after fitting, kms.cluster_centers_ holds the centers used (whether computed by K-Means or supplied via cluster_centers). Inspecting them tells you exactly how the minority class was partitioned. [SOURCE: imbalanced-learn repo, KMeansSMOTE.cluster_centers_]

Results / Findings

Because this article does not run an experiment, the "findings" below are a synthesis of the primary sources, not measured numbers from this author's machine.

Finding 1 — The minority class is partitioned before any synthesis. KMeansSMOTE first runs K-Means on the minority samples only, producing k clusters and their centers. Every subsequent synthetic point is generated inside one of those clusters, using only that cluster's own neighbours. There is no path for a point in cluster A to be interpolated toward a point in cluster B. [SOURCE: Doufas & Bacao 2018; imbalanced-learn cluster.py _fit_resample]

Finding 2 — Sampling is density-weighted, not uniform. For each cluster i, the paper defines a density d_i = n_i / r_i, where n_i is the number of minority samples in the cluster and r_i is the cluster radius (the average distance of the cluster's samples to its centroid). The total number of synthetic points to generate (set by sampling_strategy) is then split across clusters in proportion to d_i: s_i = round(N_syn · d_i / Σ_j d_j). A compact, populous cluster (large n_i, small r_i) gets high density → more new points; a sparse, spread-out cluster gets fewer. [SOURCE: Doufas & Bacao 2018, density definition; imbalanced-learn cluster.py implementation]

Finding 3 — Why this fixes vanilla SMOTE's gap problem (DERIVED). Vanilla SMOTE distributes synthetic points uniformly per minority instance and interpolates between global neighbours. When the minority class has two separated blobs, a point in blob A can have its nearest neighbour in blob B, so interpolation plants a synthetic point in the empty space between them — a region with no real minority data and often overlapping majority data. That point is pure ambiguity. KMeansSMOTE removes the mechanism entirely: blob A and blob B are different clusters, so interpolation is strictly intra-blob. The gaps stay empty. The density weighting then reinforces the denser blob, which is where real minority structure is strongest.

Finding 4 — Worked numerical example (DERIVED). Suppose the minority class in 1-D is two separated groups:

• Group A (dense): {1.0, 1.1, 0.9, 1.05} — centroid ≈ 1.0125, radius ≈ 0.0625, so density d_A = 4 / 0.0625 = 64.
• Group B (sparse): {5.0, 5.2} — centroid 5.1, radius ≈ 0.1, so density d_B = 2 / 0.1 = 20.

Total density Σ d = 84. Target N_syn = 6 synthetic points. Then s_A = round(6 · 64/84) = round(4.57) = 5, and s_B = round(6 · 20/84) = round(1.43) = 1. So 5 of 6 new points land inside dense Group A (in the [0.9, 1.1] band) and only 1 inside sparse Group B (in [5.0, 5.2]). Vanilla SMOTE with k_neighbors=2 behaves worse: for the point 5.0, its two nearest minority neighbours are 5.2 (dist 0.2) and 1.05 (dist 3.95); if interpolation picks 1.05, the synthetic point sits at 5.0 + 0.5·(1.05 − 5.0) ≈ 3.0 — squarely in the empty gap. KMeansSMOTE never produces that point. In higher dimensions the gap is a volume, so the vanilla-SMOTE failure is even more severe. [DERIVED illustration]

Finding 5 — The paper's empirical claim (SOURCE). Doufas & Bacao (2018) benchmark KMeansSMOTE across a large collection of imbalanced datasets drawn from the KEEL repository and report that, especially when combined with an ensemble classifier, the cluster-then-synthesise scheme ranks at or near the top among over-sampling methods — beating vanilla SMOTE and several other variants on balanced accuracy / F-measure. The gain is attributed precisely to the density-aware, cluster-confined generation described above. (Exact per-dataset metrics are in the paper; this article did not reproduce them.) [SOURCE: Doufas & Bacao 2018]

Finding 6 — KMeansSMOTE is complementary, not a replacement, for SVMSMOTE/BorderlineSMOTE. Those variants decide where on the boundary to synthesise. KMeansSMOTE decides how to keep synthesis inside coherent sub-populations. When the minority class is multi-modal, KMeansSMOTE addresses a failure the boundary-based variants do not. [DERIVED from comparing the implementations in imbalanced-learn]

Reproducibility

To reproduce a real comparison you would, at minimum:

1. Pick a fixed random_state everywhere (data split, SMOTE, estimator).
2. Scale features first — wrap StandardScaler ahead of KMeansSMOTE, because K-Means is a distance construction and unscaled features with large magnitude dominate cluster assignment. This is non-negotiable for honest results.
3. Compare resamplers inside the same CV loop: SMOTE(), KMeansSMOTE(kmeans_estimator=K) for a few K values, and BorderlineSMOTE() / SVMSMOTE().
4. Score with minority recall, precision, F1, and G-mean — not raw accuracy, which is misleading under imbalance.
5. Always fit_resample on the training fold only.
6. If you pass cluster_centers, fix them from a held-out EDA fit so they don't leak test information across folds.

The code sketch in Data & Methodology is the canonical shape from the imbalanced-learn documentation and is presented illustratively — it was not executed for this article, and its printed outputs are quoted from the library's own example pattern for reference. [SOURCE: imbalanced-learn repo docstring; not an experiment result of this author]

What Failed / Counter-Evidence

KMeansSMOTE is not a free lunch, and the honest literature plus implementation details say so.

• Choosing the wrong k breaks it. Too few clusters (e.g. k=2) and you may still get one mega-cluster spanning two genuine sub-populations — the gap problem returns inside that cluster. Too many and clusters become tiny and sparse: many fall below k_neighbors + 1 and are skipped, so you under-generate and waste the clustering cost. The right k is dataset-dependent and must be tuned. [DERIVED from the algorithm; Doufas & Bacao 2018 discuss cluster-count sensitivity]
• Outliers hijack the density weighting. K-Means minimises squared error, so an outlier pulls a centroid and can form its own singleton cluster. A singleton has n_i = 1 and a near-zero radius r_i, so d_i = 1 / ~0 explodes — the density formula then dumps a huge share of synthetic points around that single outlier. This is a silent, catastrophic failure mode. [DERIVED from K-Means + density formula behaviour]
• Blagus & Lusa (2013) show that SMOTE-family over-sampling can hurt on small or high-dimensional datasets, where neighbour/cluster geometry is unreliable and synthetic points amplify noise. KMeansSMOTE inherits this — tiny clusters in high dimensions are noise, not signal. [SOURCE: Blagus & Lusa 2013]
• Numeric-only. K-Means needs Euclidean distance, so KMeansSMOTE works only on continuous features. Pass a categorical column and it breaks (or silently mis-clusters). For mixed numeric/categorical data use SMOTENC; for all-categorical use SMOTEN. [SOURCE: imbalanced-learn repo]
• Scale sensitivity. Like every K-Means application, unscaled features with larger magnitude dominate distance and warp the partition. Forget StandardScaler and KMeansSMOTE can produce a meaningless clustering. [DERIVED from K-Means theory]
• Higher compute cost. Running K-Means plus per-cluster SMOTE is pricier than vanilla SMOTE. Inside CV on a large Nifty feature matrix this can be the difference between seconds and minutes per fold. [DERIVED]
• Unimodal minority → no benefit. If the rare class is genuinely one blob, KMeansSMOTE ≈ vanilla SMOTE with a density tweak and extra cost. It only earns its keep when the minority is multi-modal. [DERIVED]

Limitations

1. Cluster count k is the key knob. Wrong k (too small or too large) degrades results; it must be tuned inside CV. There is no universally correct value. [DERIVED]
2. Outlier sensitivity (the silent killer). A single outlier can form a tiny-radius cluster and capture disproportionate synthetic budget via the density formula. Mitigate with outlier removal / robust scaling before KMeansSMOTE. [DERIVED]
3. Numeric features only. No categorical support; use SMOTENC/SMOTEN for mixed or categorical data. [SOURCE: imbalanced-learn repo]
4. Scale sensitivity. K-Means partitions are distance-based; always StandardScaler first. [DERIVED from K-Means theory]
5. Compute cost. K-Means + per-cluster SMOTE costs more than vanilla SMOTE; budget for it in CV. [DERIVED]
6. Cluster-purity assumption. Assumes minority sub-populations are separable enough for K-Means to find them. Overlapping, smeared sub-populations defeat the partition. [DERIVED]
7. Sparse clusters can't generate. Any cluster with fewer than k_neighbors + 1 points is skipped, so you may end up under-generating relative to sampling_strategy. [SOURCE: imbalanced-learn repo]
8. No fix for irreducible overlap. If minority and majority truly overlap, intra-cluster SMOTE still thickens the overlapping region. [DERIVED]
9. cluster_centers can lie. Passing precomputed centers skips K-Means but if those centers are wrong the generation is wrong — and there is no check. [SOURCE: imbalanced-learn repo; DERIVED]

Practical Takeaways

• Reach for KMeansSMOTE when your minority class is multi-modal — several distinct sub-populations (distinct market regimes, distinct failure modes) separated by empty or majority-dominated space. That is exactly where vanilla SMOTE pollutes the gaps.
• Always scale first. Wrap StandardScaler ahead of KMeansSMOTE in the same Pipeline. This single step decides whether the clustering is meaningful. [DERIVED]
• Treat kmeans_estimator as a real hyperparameter. Start with kmeans_estimator=K for a few K (e.g. 3, 5, 8) and tune inside CV. Prefer an explicit KMeans(n_clusters=K, random_state=...) over the default MiniBatchKMeans(n_clusters=2) when you suspect more than two sub-populations. [SOURCE: imbalanced-learn repo; DERIVED]
• Clean outliers before clustering. Winsorise, clip, or remove extreme minority points first; otherwise the density formula can dump synthetic budget on noise. [DERIVED]
• Inspect cluster_centers_ after fitting to confirm the partition looks like real sub-populations, not artefacts. [SOURCE: imbalanced-learn repo]
• Never resample the test set. Resample inside the training fold only, ideally via imblearn.pipeline.Pipeline so it composes cleanly with scaling and the classifier.
• Compare against the cheap baseline. Before reaching for any SMOTE variant, try scale_pos_weight (XGBoost/LightGBM) or class weights — sometimes that alone closes the gap without synthesising a single row. KMeansSMOTE is heavier than those.
• In our stack, all of this lives in Layer 2 (EOD-audited training), never in Layer 1 (live Dhan capture). The K-Means partition is computed on historical folds, never on live ticks.

FAQ

Q1. What is the core difference between KMeansSMOTE and vanilla SMOTE?
A1. Vanilla SMOTE interpolates between minority neighbours globally across the whole feature space, so it can generate points in empty gaps between separated sub-populations. KMeansSMOTE first clusters the minority class with K-Means, then runs SMOTE inside each cluster only, so synthetic points stay in high-density regions and never bridge gaps. [SOURCE: Doufas & Bacao 2018; imbalanced-learn repo]

Q2. What does the kmeans_estimator parameter do?
A2. It is the clusterer applied to the minority class. None (default) becomes MiniBatchKMeans(n_clusters=2). You can pass an int (number of clusters) or any K-Means-style estimator (e.g. KMeans(n_clusters=5)) to control the partition. [SOURCE: imbalanced-learn repo]

Q3. What is cluster_centers and when would I use it?
A3. It is a precomputed array of cluster centers, shape (n_clusters, n_features). If supplied, K-Means is skipped and these centers are used to assign minority points to clusters. Use it for determinism across CV folds or when you already know the partition from EDA — but only if the centers are correct, since there is no validation. [SOURCE: imbalanced-learn repo]

Q4. How does KMeansSMOTE decide how many points to make per cluster?
A4. Via the density formula d_i = n_i / r_i (samples divided by cluster radius). The total synthetic count is split across clusters proportional to d_i, so denser, compact clusters receive more new points. [SOURCE: Doufas & Bacao 2018; imbalanced-learn repo]

Q5. Does KMeansSMOTE help if my minority class is one single blob?
A5. No — then it is roughly equivalent to vanilla SMOTE with a density tweak, plus extra compute. Its value appears only when the minority class is multi-modal (several separated sub-populations). [DERIVED]

Q6. Can I use KMeansSMOTE with categorical features?
A6. No. K-Means needs continuous Euclidean space. For mixed numeric/categorical data use SMOTENC; for all-categorical use SMOTEN — both from imblearn.over_sampling. [SOURCE: imbalanced-learn repo]

Q7. What is the biggest failure mode?
A7. Outliers. A singleton outlier forms a near-zero-radius cluster whose density d = n/r explodes, so the algorithm diverts a disproportionate share of synthetic points to that outlier. Clean outliers before clustering. [DERIVED]

Q8. Should I apply it to live trading data?
A8. No — apply any over-sampling inside the training/CV loop of your audited model core, never on live inference. Our NSE stack keeps resampling strictly in Layer 2. [DERIVED from the two-layer engine design]

Q9. What comes after KMeansSMOTE?
A9. V5 SMOTENC handles mixed categorical and numeric features by combining K-Means-style clustering on the continuous part with special handling for categorical levels — the right tool when your feature matrix is not purely numeric. [SOURCE: imbalanced-learn repo]

TL;DR

• KMeansSMOTE (from imblearn.over_sampling import KMeansSMOTE) clusters the minority class with K-Means first, then applies SMOTE inside each cluster — density-aware, gap-free over-sampling. [SOURCE: Doufas & Bacao 2018; imbalanced-learn repo]
• It fixes vanilla SMOTE's tendency to generate synthetic points in low-density / between-cluster gaps, where no real minority data exists. [DERIVED]
• kmeans_estimator (default None → MiniBatchKMeans(n_clusters=2)) sets the clustering — pass an int (n clusters) or a K-Means estimator; cluster_centers lets you skip K-Means with precomputed centers. [SOURCE: imbalanced-learn repo]
• It helps most when the minority class is multi-modal; it struggles with wrong k, outliers (which hijack the density formula), unscaled features, and categorical columns. [DERIVED]
• Always scale first, resample inside CV only, and tune kmeans_estimator as a hyperparameter. [DERIVED]
• Next in series: V5 SMOTENC. Previous: V3 SVMSMOTE.

📚 Related Articles

• Smote V5 Smotenc
• Smote V2 Borderline
• Smote V6 Smoten

👤 About the Author

Name:

Shakti Tiwari

Role:

Nifty Option Trader, XGBoost Expert

Cert:

NISM-Series-XII (Securities Markets Foundation)

Knows:

XGBoost · LightGBM · Options Trading · Machine Learning · Walk-forward Validation

Profiles:
about.me ·
optiontradingwithai.in ·
github ·
whatsapp ·
x/twitter

Sources

• [SOURCE] Doufas, G., & Bacao, F. (2018). The Last Resort: Balancing Imbalanced Big Data Classification with K-Means and SMOTE. Information Sciences, 465, 1–20. — original KMeansSMOTE algorithm (cluster minority with K-Means, then density-weighted SMOTE within clusters); primary paper behind imblearn.over_sampling.KMeansSMOTE.
• [SOURCE] scikit-learn-contrib/imbalanced-learn GitHub repository — imblearn/over_sampling/_smote/cluster.py (KMeansSMOTE, kmeans_estimator, cluster_centers, _fit_resample, cluster_centers_) and base SMOTE modules. Primary source for the implementation described.
• [SOURCE] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. — vanilla SMOTE baseline (V1).
• [SOURCE] Han, H., Wang, W.-Y., & Mao, B.-H. (2005). Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. ICIC 2005, LNCS 3644, 878–887. — Borderline-SMOTE (V2), the boundary-based comparison point.
• [SOURCE] Nguyen, H. M., Cooper, E. W., & Kamei, K. (2011). Borderline over-sampling for imbalanced data classification. International Journal of Knowledge and Web Intelligence, 2(3), 230–242. — SVM-based boundary over-sampling implemented as SVMSMOTE (V3).
• [SOURCE] Blagus, R., & Lusa, L. (2013). SMOTE can degrade performance on small, high-dimensional datasets (cautionary note on over-sampling). — counter-evidence on small/high-dimensional data.
• [DERIVED] The 1-D two-group worked example ({1.0,1.1,0.9,1.05} vs {5.0,5.2}), the density-weighting arithmetic, the "gap pollution" illustration at ≈3.0, the outlier-hijack analysis, and the "KMeansSMOTE vs vanilla SMOTE" comparison — derived from the cited sources and standard K-Means / SMOTE theory.

Author / Canonical

Written for Shakti Tiwari — Nifty Option Trader, XGBoost Expert (optiontradingwithai.in). Part of the SMOTE Family series (V4 of V6). This is educational content; not investment advice and not SEBI-registered research.

Resources & Links

• Brand site: https://optiontradingwithai.in
• About the author: https://about.me/shaktitiwari
• WhatsApp: https://wa.me/919169650895
• Previous article (V3 SVMSMOTE): SMOTE_V3_svmsmote.md
• Next article (V5 SMOTENC): SMOTE_V5_smotenc.md
• Books: https://www.amazon.in/dp/B0H9ZNTBPK · https://www.amazon.in/dp/B0HBBFKDQF