Classification Metrics: Why and When to Use Them

Classification models predict categorical outcomes, and evaluating their performance requires different metrics depending on the problem. Here’s a breakdown of key classification metrics, their importance, and when to use them.

---

1️. Accuracy
📌 Formula:
Accuracy=Correct Predictions /Total Predictions
✅ Use When: Classes are balanced (equal distribution of labels).
🚨 Avoid When: There’s class imbalance (e.g., fraud detection, where most transactions are legitimate).
📌 Example: If a spam classifier predicts 95 emails correctly out of 100, accuracy = 95%.

---

2️. Precision (Positive Predictive Value)
📌 Formula:
Precision=True Positives / (True Positives+False Positives)
✅ Use When: False positives are costly (e.g., diagnosing a disease when the patient is healthy).
🚨 Avoid When: False negatives matter more (e.g., missing fraud cases).
📌 Example: In cancer detection, high precision ensures fewer healthy people are incorrectly diagnosed.

---

3️. Recall (Sensitivity or True Positive Rate)
📌 Formula:
Recall=True Positives / (True Positives+False Negatives)
✅ Use When: Missing positive cases is dangerous (e.g., detecting fraud, security threats, or diseases).
🚨 Avoid When: False positives matter more than false negatives.
📌 Example: In fraud detection, recall ensures most fraud cases are caught, even at the cost of false alarms.

---

4️. F1 Score (Harmonic Mean of Precision & Recall)
📌 Formula:
F1=2×(Precision×Recall) / (Precision+Recall)
✅ Use When: You need a balance between precision and recall.
🚨 Avoid When: One metric (precision or recall) is more important than the other.
📌 Example: In spam detection, F1 ensures spam emails are detected (recall) while minimizing false flags (precision).

---

5️. ROC-AUC (Receiver Operating Characteristic – Area Under Curve)
📌 What it Measures: The model’s ability to differentiate between classes at various thresholds.
✅ Use When: You need an overall measure of separability (e.g., credit scoring).
🚨 Avoid When: Precise probability calibration is required.
📌 Example: A higher AUC means better distinction between fraud and non-fraud transactions.

---

6️. Log Loss (Cross-Entropy Loss)
📌 What it Measures: Penalizes incorrect predictions based on confidence level.
✅ Use When: You need probability-based evaluation (e.g., medical diagnoses).
🚨 Avoid When: Only class labels, not probabilities, matter.
📌 Example: In weather forecasting, log loss ensures a model predicting 90% rain probability is rewarded more than one predicting 60% if it actually rains.

---

Choosing the Right Metric
Scenario -- Best Metric
Balanced Dataset- Accuracy
Imbalanced Dataset- Precision, Recall, F1-Score
False Positives are Costly- Precision
False Negatives are Costly- Recall
Need Overall Performance- ROC-AUC
Probability-Based Prediction- Log Loss