# ============================================================
# FINAL PREDICTION ANALYSIS
# ============================================================
final_predictions_df = pd.DataFrame({
"Actual_PCF": y_test_final_raw,
"Predicted_PCF": y_test_pred_raw
})
# Residual = Actual - Predicted
final_predictions_df["Residual"] = (
final_predictions_df["Actual_PCF"]
- final_predictions_df["Predicted_PCF"]
)
# Absolute error
final_predictions_df["Absolute_Error"] = (
final_predictions_df["Residual"].abs()
)
# Absolute percentage error
final_predictions_df["Absolute_Percentage_Error"] = (
final_predictions_df["Absolute_Error"]
/ np.maximum(
final_predictions_df["Actual_PCF"],
1e-8
)
) * 100
display(
final_predictions_df.head(10)
)
Final Prediction Analysis — Purpose
This cell creates a row-by-row comparison between the actual PCF and the final model's predicted PCF on the independent test set.
This is different from the previous evaluation cell where you calculated overall:
- MAE
- RMSE
- R²
Here, you are going deeper and asking:
“For each individual test observation, how close was the prediction to the actual PCF?”
This is useful for error analysis and interpretation.
1. Create the prediction DataFrame
```python id="v6p2k1"
final_predictions_df = pd.DataFrame({
"Actual_PCF": y_test_final_raw,
"Predicted_PCF": y_test_pred_raw
})
This creates a DataFrame with two columns:
```text id="9n2h7x"
Actual_PCF
Predicted_PCF
Actual_PCF
Contains the real PCF values from the independent test set.
Predicted_PCF
Contains the PCF values predicted by your final tuned XGBoost model.
So each row represents:
```text id="j6s8q3"
One test observation
↓
Actual PCF vs Predicted PCF
For example, your first row is:
```text id="8c4v1z"
Actual = 2300.00
Predicted = 1740.69
2. Calculate residual
```python id="m3r7k9"
final_predictions_df["Residual"] = (
final_predictions_df["Actual_PCF"]
- final_predictions_df["Predicted_PCF"]
)
The residual is:
**Residual = Actual − Predicted**
This tells you the **direction of the prediction error**.
### If residual is positive
```text id="8w1h5d"
Actual > Predicted
The model underestimated the PCF.
If residual is negative
```text id="2p7k4m"
Actual < Predicted
The model **overestimated** the PCF.
---
### Example: Row 0
```text id="y7k2q1"
Actual = 2300.00
Predicted = 1740.69
Residual = 2300 - 1740.69
= +559.31
Positive residual → the model underpredicted by approximately 559 kg CO₂e.
Example: Row 1
```text id="q8d3m6"
Actual = 580.00
Predicted = 688.73
Residual = 580 - 688.73
= -108.73
Negative residual → the model **overpredicted** by approximately 109 kg CO₂e.
---
# 3. Calculate absolute error
```python id="r5n2v8"
final_predictions_df["Absolute_Error"] = (
final_predictions_df["Residual"].abs()
)
.abs() takes the absolute value.
So:
```text id="f1s9k4"
Residual = +559 → Absolute Error = 559
Residual = -109 → Absolute Error = 109
The sign is removed.
Why?
Because absolute error is concerned only with:
> **How large was the error?**
not whether the model over- or underpredicted.
This is closely related to MAE.
---
# 4. Calculate Absolute Percentage Error
```python id="c7m4x1"
final_predictions_df["Absolute_Percentage_Error"] = (
final_predictions_df["Absolute_Error"]
/ np.maximum(
final_predictions_df["Actual_PCF"],
1e-8
)
) * 100
This calculates the Absolute Percentage Error (APE) for each observation.
The formula is:
$$
APE =
\frac{|Actual-Predicted|}
{Actual}
\times100
$$
So it tells you:
“The prediction error represents what percentage of the actual PCF?”
Example: Row 0
Actual:
```text id="x2q7m4"
2300
Absolute error:
```text id="r3k8p1"
559.31
Therefore:
$$
\frac{559.31}{2300}\times100
\approx24.32\%
$$
Your table shows:
```text id="s5j1q8"
24.317823%
So the prediction is approximately **24.3% away from the actual value**.
---
# 5. Why `np.maximum(..., 1e-8)`?
This part:
```python id="h2v6c9"
np.maximum(
final_predictions_df["Actual_PCF"],
1e-8
)
protects against division by zero.
Suppose:
```text id="p4n7x2"
Actual PCF = 0
Then:
```text id="k9m3s6"
Absolute error / 0
would create a division-by-zero problem.
So the denominator is forced to be at least:
```text id="e1r8w5"
0.00000001
### Important viva point
This does **not** make percentage error well-defined for zero actual values.
It only prevents a numerical division-by-zero error.
If the dataset contains actual PCF values of zero, percentage-based metrics should be interpreted carefully.
---
# 6. Display first 10 observations
```python id="u4q8z2"
display(
final_predictions_df.head(10)
)
.head(10) displays the first 10 rows.
It lets you inspect individual predictions without printing the entire test dataset.
Understanding your actual results
Your first 10 observations show:
| Row | Actual | Predicted | Residual | Absolute Error | APE |
|---|---|---|---|---|---|
| 0 | 2300.00 | 1740.69 | +559.31 | 559.31 | 24.32% |
| 1 | 580.00 | 688.73 | −108.73 | 108.73 | 18.75% |
| 2 | 58.00 | 144.31 | −86.31 | 86.31 | 148.81% |
| 3 | 2219.00 | 1447.95 | +771.05 | 771.05 | 34.75% |
| 4 | 15.00 | 16.25 | −1.25 | 1.25 | 8.33% |
| 5 | 66.16 | 96.84 | −30.68 | 30.68 | 46.37% |
| 6 | 17.10 | 18.84 | −1.74 | 1.74 | 10.15% |
| 7 | 584.00 | 655.04 | −71.04 | 71.04 | 12.16% |
| 8 | 6240.00 | 3961.29 | +2278.71 | 2278.71 | 36.52% |
| 9 | 2000.00 | 1740.86 | +259.14 | 259.14 | 12.96% |
The most interesting example: Row 2
Look at:
```text id="k3m7v2"
Actual = 58
Predicted = 144.31
The absolute error is only:
```text id="n6x1q8"
86.31 kg CO₂e
But the percentage error is:
```text id="w9c4s7"
148.81%
Why?
Because the actual value is very small.
$$
\frac{86.31}{58}\times100
\approx148.8\%
$$
This demonstrates an important limitation of percentage-based errors:
> **APE can become very large when the actual target is small, even when the absolute error is relatively modest.**
### Viva question
**“Why is the percentage error 148% when the absolute error is only 86?”**
Strong answer:
> “Because percentage error is relative to the actual value. The actual PCF is only 58 kg CO₂e, so an 86 kg CO₂e error is larger than the actual value itself. This is why I would not use APE alone to judge the model, particularly for small PCF values.”
Excellent answer.
---
# Row 8 — Large absolute error
Row 8:
```text id="3y7p2n"
Actual = 6240
Predicted = 3961.29
Absolute error:
```text id="7c4m9x"
2278.71 kg CO₂e
APE:
```text id="q5n8r1"
36.52%
This shows the opposite situation.
The percentage error is not as extreme as row 2, but the absolute error is very large.
This is why your project reports multiple metrics rather than relying on one metric.
Why do you need both residual and absolute error?
Because they answer different questions.
Residual
Shows direction:
```text id="r8x2k6"
Positive → underprediction
Negative → overprediction
### Absolute Error
Shows **magnitude**:
```text id="p4m7q1"
How far away was the prediction?
For example:
```text id="c9v3z5"
Residual = -108.73
means the model overpredicted.
But:
```text id="a6k1w8"
Absolute Error = 108.73
tells us the size of that error without caring about direction.
How this connects to MAE
Your:
```text id="x7n2m5"
Absolute_Error
column contains the individual absolute errors.
MAE is essentially:
$$
MAE = mean(|Actual-Predicted|)
$$
So:
```text id="h4q9s2"
Absolute_Error
↓
average all rows
↓
MAE
Therefore, this table lets you understand what is behind the overall MAE.
How this connects to RMSE
RMSE is particularly affected by large residuals.
For example, row 8 has an error of approximately:
```text id="z2k6v9"
2278.71
Large errors like this receive much more influence under RMSE because the residual is squared.
That's why your earlier:
```text id="v8q3m1"
Mean MAE ≈ 10,398
Mean RMSE ≈ 86,982
can differ dramatically.
Important limitation of this analysis
You are displaying only:
```python id="s4m8q2"
.head(10)
So this cell **does not yet analyse the entire test set**.
It only creates the full table and displays the first 10 observations.
If your supervisor asks:
**“Can you identify which observations have the largest errors?”**
This cell has the information available, but `head(10)` does not show the largest errors.
You would need to sort the DataFrame by `Absolute_Error`, for example:
```python
final_predictions_df.sort_values(
"Absolute_Error",
ascending=False
).head(10)
That would show the 10 largest absolute errors.
Similarly, you could sort by:
"Absolute_Percentage_Error"
to investigate the observations with the largest relative errors.
Very important viva distinction
If asked:
“Is this calculating your final model performance?”
Answer:
“Not directly. The overall final model performance is calculated using aggregate metrics such as MAE, RMSE and R². This cell performs row-level error analysis by comparing actual and predicted PCF values and calculating residuals, absolute errors and absolute percentage errors.”
That's the correct distinction.
30-second viva answer
“This cell performs row-level error analysis on the independent test predictions. I create a DataFrame containing the actual and predicted PCF values, then calculate residuals as actual minus predicted to identify under- and overprediction. I calculate absolute error to measure the magnitude of each error and absolute percentage error to express the error relative to the actual PCF. I use a small epsilon in the denominator to avoid division by zero. The analysis helps explain individual prediction behaviour behind the aggregate MAE, RMSE and R² results. However, percentage error can become very large for small actual PCF values, so I would not use it as the sole evaluation metric.”
Examiner trap
If they point to:
58 actual → 144 predicted → 148.8% error
and ask “Is your model bad?”
Don't immediately say yes.
Say:
“This individual prediction has a large relative error, but its absolute error is about 86 kg CO₂e. Because the actual value is small, the percentage error is amplified. I would assess it together with MAE, RMSE, R² and the distribution of errors across the complete test set.”
That answer shows you understand both the model and the limitations of the metric.
For your final error analysis
- Check whether errors vary by PCF size
- Inspect bias across the test set
Top comments (0)