The question under the question
Every guide on straightening a photo before you enlarge it compares two pipelines against a reference image and reports a decibel or two. I did that test too, and my first version reported a 1.9 dB advantage for one order — which turned out to be an artefact of the reference, not the ordering. So before trusting any ordering number, there is a cleaner question with an exact answer:
Is a separable resampler commutative with a 90-degree transpose?
It should be. Transposing an image swaps its two axes. A separable filter applies the same one-dimensional kernel along each axis in turn. Swap the axes and you swap the order of the two passes — and two passes of the same operation should land in the same place.
If that fails, then every "rotate before or after upscale" comparison in this space inherits an error floor that has nothing to do with image content. That is worth measuring exactly, because it is cheap to measure and it is either zero or it is not.
The test, and the control that makes it trustworthy
Seven real photographs, centre-cropped to 384×384, resized with four filters at three scales. For each, two paths:
- Arm A: transpose the image, resize, transpose back.
- Arm B: resize, then transpose.
Both outputs have identical dimensions and identical orientation, so the only difference is the arithmetic. The metric is the raw code-value difference between them — not PSNR against some reference, which is what lets this test have an exact pass condition.
The control is NEAREST. Nearest-neighbour performs no arithmetic at all; it selects pixels. So it must commute exactly, in every image, at every scale. If NEAREST fails, my harness is broken and nothing below means anything.
| Filter | Scale | Max diff (codes) | Mean abs diff | % pixels differing | PSNR between arms |
|---|---|---|---|---|---|
| NEAREST | 2× | 0 | 0.0000 | 0.00% | exact |
| NEAREST | 3× | 0 | 0.0000 | 0.00% | exact |
| NEAREST | 4× | 0 | 0.0000 | 0.00% | exact |
| BILINEAR | 2× | 1 | 0.1051 | 10.51% | 57.96 dB |
| BICUBIC | 2× | 12 | 0.1422 | 14.12% | 56.54 dB |
| LANCZOS | 2× | 17 | 0.1766 | 17.46% | 55.49 dB |
| LANCZOS | 4× | 19 | 0.1495 | 14.78% | 56.25 dB |
The control passed 21 out of 21 cases. NEAREST is bit-exact in every image at every scale, which rules out the obvious failure modes — a misaligned crop, a transposed stride, a wrong shape. The harness sees the same pixels through both paths.
And then the interpolating filters do not commute. LANCZOS at 2× differs by up to 17 code values on 17.5% of pixels. That is not a rounding hair.
Why it fails: the two passes are not in the same arithmetic
The kernel itself is not the problem. Probing a single-pixel impulse through a 2× bicubic resize gives horizontal taps of 50, 192, 192, 50 and vertical taps of 50, 192, 192, 50 — identical, and summing to 484 in both directions. The filter is symmetric. In exact arithmetic this test cannot fail.
What fails is the intermediate. Resizing an 8-bit image in two passes means the first pass produces 8-bit values, and the second pass reads those rounded values. The two orderings therefore round at different points, and once the first pass has quantised, the second pass cannot recover what the first discarded.
I can show that this is the mechanism rather than an assumption. Composing the two axis passes by hand on the same image:
| Composition | vs a single PIL resize call |
|---|---|
| width-then-height | max 0 codes, 0.00% pixels differ |
| height-then-width | max 9–12 codes, 15.09–16.41% pixels differ |
The single call matches one of the two orderings exactly and differs from the other. So the library is not doing something exotic — it is doing width-then-height, and the transposed call makes it do height-then-width. Same kernel, same code, different rounding path, different output. That is the whole effect.
The number that makes it practical, and the one that makes it irrelevant
A 17-code difference sounds alarming until you ask whether it survives delivery. It does, and it gets slightly worse after encoding, because the encoder is now deciding how to spend bits on two slightly different images:
| JPEG quality | Max diff after encode | Mean abs diff | PSNR between arms |
|---|---|---|---|
| 95 | 19 | 0.5162 | 49.01 dB |
| 90 | 22 | 0.5448 | 48.47 dB |
| 85 | 22 | 0.5362 | 48.31 dB |
| 75 | 33 | 0.5119 | 47.84 dB |
| 60 | 35 | 0.4694 | 47.23 dB |
So the difference is not a lab curiosity that a codec washes out. It persists, at roughly the same magnitude, down to quality 60. If you were hoping the answer was "it vanishes, stop worrying", it is not.
Where the error actually lives — the part I did not expect
Here is the result that changes how I would act on this. I assumed a resampling difference would concentrate on edges, where the filter has the most work to do. Splitting the error by local gradient strength says the opposite:
| Local gradient band | Share of pixels | Mean abs error | Max error |
|---|---|---|---|
| quietest 25% | 24.8% | 0.0302 | 4.00 |
| 25–50% | 24.7% | 0.1168 | 3.00 |
| 50–75% | 25.5% | 0.1792 | 5.00 |
| 75–90% | 15.0% | 0.2166 | 6.33 |
| busiest 10% | 10.0% | 0.2511 | 16.00 |
The maximum error is on edges, as expected — 16 codes in the busiest decile. But the mean error in the quietest quarter of the image is 0.03 codes, and the busiest decile is only 0.25. The error is small and spread thinly everywhere rather than concentrated where the eye is drawn.
That is the reassuring reading, and it is the one I would have stopped at if I had only looked at the maximum. But it is also the reading that matters least. A sub-code difference on a smooth gradient is exactly the condition that produces banding once the image is quantised again, and smooth areas are where banding is visible. A 16-code difference on a hard edge is invisible; a 0.25-code systematic bias across a sky is not necessarily. This measurement cannot settle that — it tells you the error is spread, not whether spread error is harmless.
Does it compound? Yes, slowly
Real pipelines are not one pass. If you upscale, straighten, then upscale again, the two orderings drift further apart at each step:
| Passes | Max diff | Mean abs diff | PSNR between arms |
|---|---|---|---|
| 1 | 17 | 0.1766 | 55.49 dB |
| 2 | 19 | 0.2624 | 53.48 dB |
| 3 | 20 | 0.3124 | 52.29 dB |
| 4 | 21 | 0.3419 | 51.59 dB |
It compounds, but sub-linearly: mean error roughly doubles from one pass to four while PSNR falls about 4 dB. It does not blow up, and it does not cancel. Four passes of 2× upscaling is already an implausible workflow, so the practical ceiling here is modest.
What I would actually do with this
Nothing dramatic, and I want to be honest about that rather than sell a 17-code scare.
- Do not reorder your pipeline for this. The ordering premium measured against a proper size-matched reference is around 0.7–0.9 dB, and this axis artefact is well below that. Reordering buys you less than the thing you were already arguing about. If the geometry itself is what you are worried about, the EXIF orientation sweep is the measurement that actually decides whether a normalise-first step buys anything, and it finds that for the eight standard tag values the premium is exactly zero.
- Do fix the multi-pass habit. If your pipeline resizes, then rotates, then resizes again, the second resize is working on already-quantised data. Doing all geometry in one operation, at the end, removes both the ordering question and one rounding stage. The same reasoning is why chained resize steps lose against a single step at a fractional factor — that comparison is measured in the fractional-factor study, where step-chaining loses consistently.
- Do not use NEAREST as evidence. It commutes perfectly and that is not a virtue — it commutes because it does no arithmetic, which is the same reason it looks bad. A filter that passes this test by not filtering has told you nothing about quality.
The general lesson is the one I keep relearning: an exact test with a control is worth more than a plausible number. I had a 1.9 dB result that was an artefact of my reference, and a 17-code result that is real but small. Only the second one survived being checked against a case where the answer was known in advance.
Reproducing it
Both scripts are plain numpy and Pillow, no SciPy, and run in about two minutes on the seven sample photographs:
python3 tracking/commute_axis.py # commutation table + NEAREST control
python3 tracking/commute_practical.py # encode survival, error location, compounding
The control asserts that NEAREST is bit-exact in all 21 image/scale cases and exits non-zero if it is not. If you change the image set, the control is the part to keep — everything else in the table is only meaningful while it passes.
If you want to see what the two orderings look like on your own file rather than trusting a table, you can try it in the browser — no install, and the comparison is side by side.
Top comments (0)