Classification evaluation metrics.

We need various metrics to evaluate our models based on what we wish to achieve from our classification problem. For some we may require accuracy, in others we may prefer recall or precision. Below are the evaluation metrics used for classification models.

Accuracy
This is a measure of how accurate the model correctly classifies a variable. This is obtained by

$$Accuracy = \dfrac{TP + TN}{TP + TN + FP + FN}$$

Precision
This is a measure of how accurate the positive predictions are.
This is calculated by

$$Precision = \dfrac{TP}{TP + TN}$$

This measure is preferred when false positives need to be minimized to avoid unnecessary interventions, wasted resources, and potential harm to individuals or systems. Such instances include fraud detection. For instance, if our model kept flagging legitimate transactions as fraudulent, this might frustrate customers.

Recall
This is a measure of how many false positives were correctly predicted. This is calculated by

$$Recall = \dfrac{TP}{TP + FN}$$

This metric would be preferred when false negatives need to be minimized. A good example would be in diagnostics where a false negative for a patient who has cancer might lead to a delay of treatment which decreases chances of survival.

F1 score
This is the harmonic mean of precision and recall, taking into consideration both metrics.

$$F1 score = 2 * \dfrac{Precision * Recall}{Precision + Recall}$$

A high F1 score symbolizes a high precision as well as high recall. It presents a good balance between precision and recall and gives good results on imbalanced classification problems.