"95% accurate!" — on a dataset that's 95% one class, a model that always guesses that class scores 95% and is useless. That's why classifiers are judged by the ROC curve and AUC instead. Here's both, live and draggable.
📈 Drag the threshold, watch the ROC: https://dev48v.infy.uk/ml/day20-roc-auc.html
A classifier outputs a score, you pick a threshold
Above the threshold = "positive." Move it and the confusion matrix (TP/FP/TN/FN) changes. From that you get:
- TPR (true positive rate / recall) = TP / (TP+FN)
- FPR (false positive rate) = FP / (FP+TN)
The ROC curve
Sweep the threshold from high to low and plot (FPR, TPR) at each step. The curve bows toward the top-left for a good classifier and hugs the diagonal (random) for a useless one. The demo's "separation" slider pulls the classes apart and you watch the curve bow out.
AUC
Area under the ROC curve. 0.5 = random, 1.0 = perfect. Beautifully, AUC = the probability that a random positive scores higher than a random negative. It's threshold-independent and robust to class imbalance — which is exactly why it beats raw accuracy.
(For rare positives, also look at the precision-recall curve.)
🔨 Built from data (sweep thresholds → TPR/FPR → plot → trapezoidal AUC) on the page: https://dev48v.infy.uk/ml/day20-roc-auc.html
Part of MachineLearningFromZero. 🌐 https://dev48v.infy.uk
Top comments (0)