How To Evaluate Machine Learning Model Performance Using Confusion Table

This article is the continuation of our discussion on evaluating the performance of a machine learning model.
To get the most out of this article, it is compulsory you read the first part of the two-series article.

How To Evaluate Machine Learning Model Performance Using Accuracy Score Metric

Victor Isaac Oshimua ・ Jun 26 '23

#machinelearning #datascience #tutorial #beginners

There we discussed how to evaluate using accuracy score.
The accuracy score does not provide a comprehensive measure of the model's performance in making predictions.
In this article, you will learn another way of evaluating your machine learning model.

Prerequisites

• A basic understanding of machine learning models(you have built at least one classification model).
• A basic understanding of evaluation metrics(this article will teach you how to implement accuracy score metrics).
• A basic understanding of python and data science libraries(Numpy, Pandas, Matplotlib, Scikit learn, etc).
• You must have read the first article of the series, or at least understands accuracy score metric and how to implement it.

What is confusion table

Confusion table or confusion matrix summarizes different types of errors and correct decisions that binary classifiers can make. This is done by displaying the counts of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions.

To interpret this confusion table, let's use our churn prediction scenario.

Positive: Customer will churn

• True positive (TP) means customer churned and our model predicted churn
• False positive (FP) means the customer did not churn and our model predicted churn

Negative: Customer will not churn

• True negative (TN) means the customer will not churn and our model predicted no churn
• False negative (FN) means the customer churned and our model predicted no churn

Implementing confusion table

We have previously built a classification model for our churn prediction, let's use confusion table to evaluate the model performance.
This is done by using the Scikit learn library.

from sklearn.metrics import confusion_matrix
prediction= model.predict(X_test)
confusion_matrix(Y_test,prediction)

Code output

Confusion table interpretation

The result of the code we run is a two-by-two numpy array.
We noticed that the confusion table has 0 counts for both false positives (FP) and true positives (TP), In other words, the model did not make any positive predictions and classified every customer as not churning.

This tells us that our dataset has class imbalance, where the majority of instances belong to the negative(Not churn) class, the model has learnt to predict negative for all instances.

Source code

Final thoughts

Confusion table is a way of summarizing the performance of a classification model, it tells you the errors the model made in making predictions. It provides a tabular representation that displays the proportions of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions made by the model.

Confusion table can be used to evaluate model performance when you have problems of class imbalance.
In most cases, the confusion matrix alone may not fully capture the model's performance. It is important to consider additional evaluation metrics.

If you have any questions, or suggestions or spot any errors, reach out to me on Twitter, or LinkedIn. I appreciate your engagement and look forward to hearing from you.