Chroma siting is the one setting nobody checks, and it decides where every colour edge in your upscaled frame lands. The short version: in 4:2:0, a single colour sample represents a 2×2 block of luma pixels — but where inside that block the sample sits is a convention, not a fact of the image. Two conventions dominate, they differ by half a chroma sample, and if the decoder assumes one while the encoder used the other, the colour plane lands one full luma pixel off while the luma plane stays exactly where it should be.
That asymmetry is the whole story. Detail is crisp, geometry is silently wrong, and nothing in the file tells you.
The two conventions differ by exactly one pixel
| Convention | Where the sample sits | Used by |
|---|---|---|
| co-sited | luma position (2k, 2l) — top-left of the 2×2 block | MPEG-2, H.264 default |
| centered | luma position (2k+1, 2l+1) — middle of the block | JPEG, MPEG-1 |
Half a chroma sample is one luma pixel. At 4:2:0 a chroma sample covers 2 luma pixels in each axis, so the offset between the two conventions, expressed in luma, is exactly 1. Not "about one" — structurally exactly one.
I measured this on seven real photographs, cropping 512×512 from the centre, converting to YCbCr and working on the Cb and Cr planes directly rather than on RGB. Working in the colour planes matters: RGB hides the problem because the conversion smears the displacement across all three channels, and a mean-error number computed on RGB makes a one-pixel colour shift look like ordinary resampling noise.
The cost is a 1.66–2.55× increase in chroma error
For each plane I did the honest round trip — subsample with centered siting, reconstruct with centered siting — and then the mistaken one, subsample centered and reconstruct as if co-sited:
| Photo | Cb correct | Cb mismatched | ratio | Cr correct | Cr mismatched | ratio |
|---|---|---|---|---|---|---|
| 048d3778 | 0.255 | 0.546 | 2.14 | 0.234 | 0.461 | 1.97 |
| 1344197c | 0.257 | 0.654 | 2.55 | 0.242 | 0.540 | 2.23 |
| 6766a320 | 0.271 | 0.666 | 2.46 | 0.197 | 0.339 | 1.72 |
| 792e8b64 | 0.286 | 0.692 | 2.42 | 0.259 | 0.531 | 2.05 |
| ac7b6755 | 0.311 | 0.723 | 2.32 | 0.315 | 0.748 | 2.37 |
| b4fd2338 | 0.258 | 0.609 | 2.36 | 0.231 | 0.511 | 2.21 |
| c079d084 | 0.197 | 0.441 | 2.24 | 0.186 | 0.310 | 1.66 |
Median ratio 2.24×, range 1.66–2.55×. The absolute numbers are small — a quarter of one level to two-thirds of a level on a 0–255 scale — and that is the point worth being honest about. This is not a visible-in-a-thumbnail error. It is a systematic one that sits underneath everything else, and it compounds with the scale factor.
The shift stays 1 pixel; the relative damage multiplies
This is the part I got wrong first, and it's worth walking through because the wrong version sounds more dramatic.
My initial scaling loop divided output coordinates by a hardcoded 2.0 regardless of the upscale factor. That reported a 1-pixel shift at k=1, k=2 and k=4 — and it was an artefact. Correcting it, I discovered the opposite: in a genuine 4:2:0 output the chroma plane is always half the luma raster, so chroma-sample spacing is a constant 2 output pixels at every scale. The displacement really is a constant 1 output pixel. What grows is not the shift but its weight relative to everything else:
| Upscale | error, matched siting | error, mismatched | ratio | shift |
|---|---|---|---|---|
| 1× | 0.250 | 0.555 | 2.22× | 1 px |
| 2× | 0.125 | 0.331 | 2.66× | 1 px |
| 4× | 0.033 | 0.170 | 5.14× | 1 px |
Read the two error columns together. Correct siting gets better with upscaling — 0.250 down to 0.033 — because interpolating a finer grid is a better-conditioned problem. Mismatched siting also improves in absolute terms, but four times more slowly. The ratio climbs from 2.22× to 5.14×. So upscaling doesn't create the error; it just removes everything else that was masking it.
Along a hard colour edge, the fringe widens
A single error number understates what a viewer notices. What a viewer notices is a colour fringe hugging an edge, and that width does grow:
| Upscale | fringe, matched siting | fringe, mismatched | |
|---|---|---|---|
| 1× | 1 px | 1 px | same |
| 2× | 2 px | 3 px | +1 px |
| 4× | 2 px | 5 px | +3 px |
At 4× the mismatch paints a fringe more than twice as wide as the correct pipeline leaves. On red-to-blue transitions — the classic worst case for chroma — this is the visible form of the problem, and it is why chroma handling deserves the same attention as sharpening. If you are already working through edge behaviour, this is the same family of question as what happens to the grid when the factor is not a whole number — geometry that survives the resize numerically while going wrong relationally.
Why ten controls, and what they caught
Two of the numbers above were wrong in an earlier run, and the controls are the reason they didn't ship.
The first control failure was mine, not the code's: I used a linear ramp to test whether a mismatch produces a shift. A mean-removed linear ramp is invariant to integer translation — shifting it just adds a constant — so the correlator returned lag 0 on a case that genuinely was displaced. The estimator was blind, and a blind estimator reports "no effect" perfectly consistently. Switching to a Gaussian bump, which has a unique peak, made the same test report −1 px immediately.
The second was the hardcoded 2.0 described above, caught by asserting the geometry rather than trusting the printout.
The remaining controls pin down what is not being measured:
- A flat plane under matched siting is bit-exact (max error 0.0).
- Matched siting keeps a Gaussian bump at lag 0, and mismatched siting moves it by exactly ±1.
- The two mismatch directions are symmetric, so this isn't a bias in the correlator.
- A constant colour plane is immune — no gradient, nothing to displace.
- A checkerboard at the chroma Nyquist is destroyed by 4:2:0 subsampling alone, at a mean error of 127.50, whoever is right about siting. That separates the two failure modes: subsampling loses high-frequency colour outright; siting only moves what survives.
- Self-correlation of an unshifted copy reports lag 0, confirming the estimator is not inventing displacement.
All ten pass, and the script re-runs to the same values.
What actually goes wrong in a pipeline
The mechanism is unglamorous. An upscaler decodes a JPEG to RGB, resizes, re-encodes. Each of those steps may hold a different assumption about siting, and the assumption is usually implicit — it lives in whatever library function was called, not in a setting you can read. The decode-up-sample step in most JPEG libraries assumes centered siting because JPEG specifies it. A video pipeline that later hands that frame to something expecting co-sited chroma has already introduced the offset, and the subsequent resize faithfully preserves the mistake while making it relatively more prominent, exactly as the table above shows.
The practical consequence: if you compare two upscalers on a colour-critical subject and one looks subtly "off" in a way you can't localise, siting is a plausible cause and it will not show up in a PSNR computed on RGB. It shows up as a consistent half-sample colour lead or lag on edges.
There is no single setting to flip, because the fix belongs at the boundary: know which convention each stage assumes, and don't chain a co-sited stage into a centered one without accounting for the half-sample. If the half-sample is the kind of detail you want to see rather than read about, run the same photo through an upscaler in a browser tab and compare edges at 400% — the displacement is small enough to hide in a full frame and large enough to see once you know which edge to look at.
The takeaway
Chroma siting is a half-sample convention that becomes a full-pixel colour displacement, and it survives upscaling intact while everything around it improves. Measured on seven photographs, assuming the wrong convention costs 1.66–2.55× in chroma error and widens edge fringes from 2 px to 5 px at 4×. It is not a resolution problem and no amount of additional pixels fixes it, because the error is in the position of the colour, not in the amount of it.
If you want the sampling-density side of this — what 4:2:0 itself throws away before siting ever enters the picture — the chroma subsampling notes cover that separately. The two errors are independent and should be counted separately.
Top comments (0)