DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

Raising the Batch Size Makes Batch Norm's Inference Statistics 13x Worse

In training, batch norm normalises each row with the statistics of the batch it happens to be in. At inference there is no batch, so it uses a running average collected during training.

Those are two different functions, and the layer swaps one for the other silently.

See it: https://dev48.infy.uk/dl/day78-batch-norm-at-inference.html

Every score is a real forward pass over an enumerated 512-row population, so the errors are exact means rather than estimates.

Same input, same weights, two answers

train mode vs eval mode RMSE 0.3219
worst running-variance error 5.36 on a single feature

Bigger batches make inference worse

The intuition is that a larger batch gives a better estimate, so the running average should improve. The first half is true. The second does not follow — the running average is an exponential moving average over updates, and a bigger batch means fewer of them.

batch size updates running stats vs truth
8 64 0.1036
32 16 0.1754
64 8 0.4506
256 2 1.3771

13× worse at 256 than at 8. With momentum 0.9 and two updates, the initialisation still carries weight 0.81 — the running variance never leaves its starting value of 1, on features whose real variance is nowhere near 1.

Two things move in opposite directions as the batch grows, and the second wins. The page checks the first separately, so the mechanism is not assumed: a bigger batch really does estimate the mean better. The loss is entirely in the number of updates.

The knob that makes training more stable makes the thing training hands to production less accurate.

And the tests cannot see any of it

Here is everything a batch-norm implementation can assert about itself without a second implementation to compare against:

  • the training output has mean ≈ β per feature
  • and variance ≈ γ² per feature
  • eval mode gives the same answer twice
  • the output has the input's shape

Checked on shuffled data and on data sorted by its first feature — which is what you get whenever rows arrive grouped by class, by user, or by time.

All four pass in both. Meanwhile the train/eval divergence goes from 0.3219 to 0.6648 — it doubles — because sorted batches are not exchangeable, so each batch's statistics describe its own slice and not the data. Nothing in the four properties mentions the order of the rows.

60 verifier asserts, 14 in-page checks, 0 failures.

Top comments (0)