Machine Learning Algorithms Comparison: KNN vs SVM vs Decision Tree

#python #knn #svm #decisiontree

🇹🇷 Türkçe versiyon

🔄 What We Will Compare

This post compares three popular supervised ML algorithms:

K-Nearest Neighbors (KNN)
Support Vector Machine (SVM)
Decision Tree (DT)

We'll evaluate them based on:

Accuracy
Macro Avg Metrics (Precision, Recall, F1)
Weighted Avg Metrics (Precision, Recall, F1)

🔹 The Dataset: Glass Identification

We use the classic Glass Identification dataset which contains:

214 Rows
10 Columns: Refractive Index + 8 chemical properties + Glass Type

import pandas as pd
csv = pd.read_csv("glass.csv")
print(csv.columns)

🔍 Features

RI (Refractive Index)
Na, Mg, Al, Si, K, Ca, Ba, Fe (Chemical elements)
Type (Target class)

🌐 Resources for Learning Algorithms

Instead of long theoretical definitions, here are great visual + article resources for each algorithm:

🔢 KNN:

🌳 Decision Tree:

⚖️ SVM:

📅 Project Structure

.
├📄 glass.csv           <- Dataset
├🗋 Dt.py              <- Decision Tree class
├🗋 KNN.py             <- KNN class
├🗋 SVM.py             <- SVM class
├🗋 tools.py           <- Visualization tools
├🗋 db.py              <- Data inspection helper
└🗋 main.py            <- Main runner

📈 Data Preparation

Each class implements a common interface:

x_train, x_test, y_train, y_test = model.data_preprocces()

Steps:

Read CSV
Drop last column for X
Use last column as y
Split into train and test (80/20)

Supported by: Dt.py, KNN.py, SVM.py

💡 Model Integration

Each class supports a predict_report(choice, x_train, x_test, y_train, y_test) method with the following choices:

0: Return predictions
1: Return accuracy
2: Return confusion matrix
3: Return full classification report

🌿 Visualization Tools

The tools.py module provides utility methods to plot each metric using matplotlib:

Tools.Acc_table(report_list)
Tools.macro_prec(report_list)
Tools.macro_recall(report_list)
Tools.macro_f1(report_list)
Tools.wei_prec(report_list)
Tools.wei_recall(report_list)
Tools.wei_f1(report_list)