DEV Community

Bechir Jamoussi
Bechir Jamoussi

Posted on

Low F1-Score due to Imbalanced Dataset even after resampling

I am performing a Binary Classification over an imbalanced dataset:

0: 16,263

1: 214

I have used multiple oversampling, undersampling, and combination techniques, below are the results that I have obtained:
I obtained this plots thanks to this piece of code:

def plot_resampling(X, y, sampler, ax, title=None):
    X_res, y_res = sampler.fit_resample(X, y)
    ax.scatter(X_res[:, 0], X_res[:, 1], c=y_res, alpha=0.8, edgecolor="k")
    if title is None:
        title = f"Resampling with {sampler.__class__.__name__}"
    ax.set_title(title)
    sns.despine(ax=ax, offset=10)
Enter fullscreen mode Exit fullscreen mode

Clarification: The X and y are the X_train and y_train and I used it to show the distribution of my data points before and after the resampling.

Comparison Initial Dataset and multiple oversampling techniques

For the RandomUnderSampler, the first one is without replacement and the second one is with replacement=True

Comparison Initial Dataset and multiple oversampling techniques

Use of Combination techniques

You need to know also that I have multiple outliers in my dataset, and hence, multiple columns are skewed, so I chose to use models that are not sensitive to skewness like:

  • SVC
  • Naive Bayes Classifier
  • Ensemble XGboost
  • KNN

For now, the best result that I have obtained is with SVC(kernel = "rbf") and using the SMOTE technique(Of course the sampling is only performed on the training dataset since the test one should represent the real population):

  • Test Accuracy: 0.75
  • Training Accuracy: 0.88

But the classification report is not good, the f1-score is 0.51, there is a real issue with the 1 class even after the resampling!! as you can see below:

enter image description here

Here is also the Confusion Matrix:

enter image description here

Can you please help me improve the f1 score, what is your analysis of the situation, and what are your suggestions?

Top comments (0)