Alex Agafonov

Posted on Nov 1

NumPy Raised No Error. Why Is Your Loss Calculating the Wrong Thing?

#python #numpy #testing #machinelearning

We need to compare three model predictions with three targets and calculate the mean squared error. By hand, the example below gives about 0.33. NumPy returns 2.33, with no exception, and the program continues running. Each prediction has been compared with all three targets, although we intended to match them one by one.

Let us find where the extra comparisons appeared. We can then state the input shapes explicitly and protect that decision with three tests. No model training or large dataset is needed for this example.

Check three samples by hand

The targets are 1, 2, and 3. The model predicts 1, 2, and 4. The first two predictions are correct; the last is off by one. The mean squared error should therefore be (0 + 0 + 1) / 3, approximately 0.3333.

Now store the predictions as a column and the targets as a one-dimensional array:

import numpy as np

predictions = np.array([[1.0], [2.0], [4.0]])  # (3, 1)
targets = np.array([1.0, 2.0, 3.0])           # (3,)

residuals = predictions - targets
print(residuals.shape)
print(residuals)
print(np.mean(residuals ** 2))

The output is:

(3, 3)
[[ 0. -1. -2.]
 [ 1.  0. -1.]
 [ 3.  2.  1.]]
2.3333333333333335

Instead of three residuals, we have a (3, 3) matrix. The first row compares prediction 1 with every target, the second does the same for prediction 2, and the third for prediction 4.

The intended errors are on the diagonal: 0, 0, and 1. But mean averages all nine squared values. Their sum is 21, so the result is 21 / 9, approximately 2.3333. The input values are unchanged, but the calculation now answers a different question.

Why NumPy allows it

Broadcasting rules compare dimensions from the right. Here, (3, 1) and (3,) produce (3, 3): the last dimension of the first array expands, and the missing leading axis of the second is treated as size 1.

We intended both arrays to describe the same ordered list of samples. The library does not know that. Its rules permit these pairwise comparisons, so no exception is raised. The application has to express which prediction belongs to which target.

We can inspect the resulting shape with numpy.broadcast_shapes before creating the residual matrix:

print(np.broadcast_shapes(predictions.shape, targets.shape))
# (3, 3)

This exposes the shape change. A (3, 3) result is not inherently wrong: another task might require exactly such a pairwise matrix.

State the input contract

For this function, we will require two nonempty one-dimensional arrays of identical shape. Position i in both arrays must refer to the same sample. The mean is taken over samples.

import numpy as np


def mse_per_sample(predictions, targets):
    predictions = np.asarray(predictions, dtype=np.float64)
    targets = np.asarray(targets, dtype=np.float64)
    if predictions.ndim != 1 or targets.ndim != 1:
        raise ValueError("Expected two one-dimensional arrays")
    if predictions.shape != targets.shape or predictions.size == 0:
        raise ValueError("Expected matching, nonempty sample axes")
    return float(np.mean((predictions - targets) ** 2))

The conversion to float64 is explicit. This example assumes finite numerical values in an ordinary range; it does not separately handle NaN, infinities, or overflow.

We can adapt the original column at the point where the meaning of its second axis is known:

aligned_predictions = predictions.squeeze(axis=1)
print(aligned_predictions.shape)  # (3,)
print(mse_per_sample(aligned_predictions, targets))
# 0.3333333333333333

We have removed the axis containing the single output value. That matches a task with one prediction per sample. Multiple outputs would require a separate decision about target alignment and averaging.

Even identical shapes do not prove that rows refer to the same samples: one array might have been sorted differently. The shape check catches this particular broadcasting error. Row alignment remains part of data preparation.

Why squeeze names an axis

One detail of numpy.squeeze matters here. Without axis, it removes every dimension of size 1. With a single sample, (1, 1) becomes (), removing the sample axis as well.

Using squeeze(axis=1) retains (1,). It also raises an error if that second dimension stops being size 1. That is a useful signal that the model output has changed and the conversion needs another look.

For the same reason, I would avoid fixing an unknown shape with reshape(-1) or flatten(). They can merge samples, features, or time steps. First establish what each axis means, then choose the transformation.

Protect the expected behavior

The first test checks the hand calculation and reproduces the original error. The second rejects inputs outside our contract. The third keeps just one sample and checks that its axis survives.

Save the function in sample_loss.py and the tests in test_sample_loss.py:

import unittest

import numpy as np

from sample_loss import mse_per_sample


class LossShapeTests(unittest.TestCase):
    def test_tiny_example_matches_manual_calculation(self):
        predictions = np.array([[1.0], [2.0], [4.0]])
        targets = np.array([1.0, 2.0, 3.0])
        residuals = predictions - targets
        self.assertEqual(residuals.shape, (3, 3))
        self.assertAlmostEqual(float(np.mean(residuals ** 2)), 7 / 3)
        self.assertAlmostEqual(
            mse_per_sample(predictions.squeeze(axis=1), targets), 1 / 3
        )

    def test_loss_rejects_shapes_outside_the_contract(self):
        for predictions, targets in [
            ([[1.0], [2.0]], [1.0, 2.0]),
            ([1.0, 2.0], [1.0]),
            ([], []),
            ([[1.0, 2.0]], [[1.0, 2.0]]),
        ]:
            with self.subTest(predictions=predictions, targets=targets):
                with self.assertRaises(ValueError):
                    mse_per_sample(predictions, targets)

    def test_one_sample_retains_its_sample_axis(self):
        predictions = np.array([[4.0]])
        targets = np.array([3.0])
        aligned = predictions.squeeze(axis=1)
        self.assertEqual(aligned.shape, (1,))
        self.assertAlmostEqual(mse_per_sample(aligned, targets), 1.0)
        self.assertEqual(predictions.squeeze().shape, ())
        with self.assertRaises(ValueError):
            mse_per_sample(predictions.squeeze(), targets)


if __name__ == "__main__":
    unittest.main()

Run python -m unittest -v test_sample_loss.py. The example and all three tests were checked with Python 3.12.14 and NumPy 2.3.5.

Averaging left us with a single number and hid the matrix of extra comparisons that came before it. Before that reduction, it helps to inspect the intermediate shape and try a small case with a known answer. Here, that is enough to find the problem before training a model.

DEV Community