LinearBoost: Faster Than XGBoost and LightGBM, Outperforming Them on F1 Score on Seven Famous Benchmark Datasets

Hamidreza Keshavarz — Sun, 12 Jan 2025 17:11:10 +0000

Hi All!

The latest version of LinearBoost classifier is released!

https://github.com/LinearBoost/linearboost-classifier

In benchmarks on 7 well-known datasets (Breast Cancer Wisconsin, Heart Disease, Pima Indians Diabetes Database, Banknote Authentication, Haberman's Survival, Loan Status Prediction, and PCMAC), LinearBoost achieved these results:

It outperformed XGBoost on F1 score on all of the seven datasets
It outperformed LightGBM on F1 score on five of seven datasets
It reduced the runtime by up to 98% compared to XGBoost and LightGBM
It achieved competitive F1 scores with CatBoost, while being much faster

LinearBoost is a customized boosted version of SEFR, a super-fast linear classifier. It considers all of the features simultaneously instead of picking them one by one (as in Decision Trees), and so makes a more robust decision making at each step.

This is a side project, and authors work on it in their spare time. However, it can be a starting point to utilize linear classifiers in boosting to get efficiency and accuracy. The authors are happy to get your feedback!

DEV Community: Hamidreza Keshavarz

LinearBoost: Faster Than XGBoost and LightGBM, Outperforming Them on F1 Score on Seven Famous Benchmark Datasets