I spent an afternoon trying to answer a question that sounded simple: when a
video has black bars, should you crop them before upscaling or after?
Everyone crops first. I did too. But I wanted the number that proves it, and the
number turned out to be a bad choice of instrument.
This post is about that instrument. The crop-order result is real, but the more
useful finding is that PSNR — the metric you'd naturally reach for — is
structurally blind to this class of error, and there is one cheap check that
catches it.
The setup
A 960×720 picture inside a 1280×720 frame: 160 px of black bar on each side.
That's a 16:9 picture in a 4:3 frame, which is what happens the moment you
letterbox anything for an old display, a vertical platform, or a fixed-aspect
player.
Two paths:
-
Crop first:
crop=960:720:160:0on the source, then upscale the picture. - Crop after: upscale the whole 1280×720 frame, then cut the bars off the scaled output.
The picture has high-contrast detail deliberately placed at x=0, right where the
bar meets it — the worst case, not the average case.
import numpy as np
from PIL import Image
BAR, PW, PH = 160, 960, 720 # 160px bar each side, 960x720 picture
x = np.arange(PW, dtype=np.float32)
pic = np.clip((200.0 - 120.0 * (x / (PW - 1)))[None, :]
* np.linspace(0.85, 1.15, PH, dtype=np.float32)[:, None], 0, 255)
pic = np.clip(pic + 18.0 * np.sin(2 * np.pi * x / 8.0)[None, :], 0, 255)
yy, xx = np.mgrid[0:PH, 0:PW]
pic[((xx - 480) ** 2 + (yy - 360) ** 2) < 200 ** 2] += 35.0
pic = np.clip(pic, 0, 255).astype(np.uint8)
frame = np.zeros((PH, PW + 2 * BAR), dtype=np.uint8)
frame[:, BAR:BAR + PW] = pic
def up(a, s, k):
return np.asarray(Image.fromarray(a).resize((a.shape[1] * s, a.shape[0] * s), k),
dtype=np.float32)
def db(mse):
return 10 * np.log10(255.0 ** 2 / mse) if mse > 0 else 99.0
for S in (2, 4, 8):
truth = up(pic, S, Image.LANCZOS) # crop first
wrong = up(frame, S, Image.LANCZOS)[:, BAR * S:BAR * S + PW * S] # crop after
err = (wrong - truth) ** 2
col = err.mean(axis=0)
band = 0
while band < len(col) and db(col[band]) < 50:
band += 1
print(f"S={S}x band={band}px ({band/S:.2f} src) "
f"1px={db(col[0]):.2f}dB 8px={db(err[:, :8].mean()):.2f}dB "
f"whole={db(err.mean()):.2f}dB")
Actual output on this machine:
S=2x band=5px (2.50 src) 1px=15.73dB 8px=23.39dB whole=46.72dB
S=4x band=9px (2.25 src) 1px=11.31dB 8px=19.16dB whole=45.49dB
S=8x band=19px (2.38 src) 1px=9.62dB 8px=16.19dB whole=45.22dB
Read the second number in each line. At 4×, the first column of the picture is
11.31 dB against ground truth. That is not "slightly soft". At 11 dB the
pixel is essentially unrelated to what it should be. Eight columns in, still
19.16 dB. And the whole-picture figure — the number you'd put in a test report
— is 45.49 dB, which sounds excellent.
The band is a property of the kernel, not the scale
I expected the damage to grow with the scale factor. It doesn't, in source
pixels:
| Scale | Kernel | Band (output px) | Band (source px) | 1px PSNR | 8px PSNR | Whole picture |
|---|---|---|---|---|---|---|
| 2× | lanczos | 5 | 2.50 | 15.73 | 23.39 | 46.72 |
| 2× | bicubic | 3 | 1.50 | 15.97 | 24.42 | 47.79 |
| 2× | bilinear | 1 | 0.50 | 14.16 | 23.19 | 46.54 |
| 4× | lanczos | 9 | 2.25 | 11.31 | 19.16 | 45.49 |
| 4× | bicubic | 6 | 1.50 | 11.33 | 19.80 | 46.13 |
| 4× | bilinear | 2 | 0.50 | 10.63 | 19.20 | 45.55 |
| 8× | lanczos | 19 | 2.38 | 9.62 | 16.19 | 45.22 |
| 8× | bicubic | 12 | 1.50 | 9.66 | 16.58 | 45.85 |
| 8× | bilinear | 4 | 0.50 | 9.28 | 15.97 | 45.33 |
Lanczos sits at 2.25–2.50 source px at 2×, 4×, and 8×. Bicubic is pinned at
exactly 1.50. Bilinear at 0.50. That's the filter support, which is fixed in
source coordinates — the number of output pixels moves, the number of source
pixels doesn't.
This is where the intuition "it only affects a pixel or two at the edge" goes
wrong. It affects two source pixels, which at 8× is nineteen output columns.
Scaling up doesn't spread the damage; it just renders the same damage in more
pixels.
Bilinear looking "cleanest" here (0.50 source px) is also a trap — it wins that
column by having the narrowest support, which is the same property that makes it
the softest kernel overall. It's not doing a better job; it's touching less.
Whole-frame PSNR prefers the broken frame
Here is the part that made me rewrite this post. Four candidates at 4×, all
scored against the same ground truth:
| Candidate | Whole PSNR | Worst column | Columns < 30 dB | 8px left edge |
|---|---|---|---|---|
| Crop-first, lanczos (correct) | — | — | 0 | — |
| Upscale frame, then crop (bleed) | 45.49 | 11.31 | 6 | 19.16 |
| Crop-first, bilinear (uniformly soft) | 49.25 | 42.11 | 0 | 45.14 |
The bleed path scores 45.49 dB. The uniformly soft path scores 49.25 dB.
If you rank by whole-frame PSNR, the soft frame wins by 3.76 dB — and the soft
frame is the one where every edge in the picture is mush.
The bleed frame has six columns below 30 dB. The soft frame has none. PSNR's
mean-squared aggregate is dominated by area, and the damaged area is 18 columns
out of 3840 — 0.47% of the picture. It cannot outvote 99.5% of correct pixels, no
matter how wrong those 18 columns are.
I also threw in a "pre-blur 1.5 px then lanczos" candidate expecting it to score
between the two. It scored 99.00 dB — bit-identical to ground truth. That was a
bug in my candidate list (the round-trip through a same-size resize was a no-op,
so I compared ground truth to itself). I'm leaving it in the post because it's
the exact failure mode this article is about: an instrument that reports a
perfect score and means nothing. If a number looks too good, check that you
actually changed the thing you think you changed.
How wide would the damage have to be to register?
Taking the measured first-column error at 4× lanczos (11.31 dB) and asking how
much of the picture would have to be that bad before the whole-frame number
moved:
| Target whole-frame PSNR | Band width needed | % of picture width |
|---|---|---|
| 45 dB | 2 px | 0.05% |
| 40 dB | 5 px | 0.13% |
| 35 dB | 16 px | 0.42% |
So the whole-frame metric can see a 16-pixel band — but it needs the entire
band to be at first-column severity. The real band decays as you move inward
(11 dB, 23, 26, 21, 23, then clear), so the average over the band is far milder
than its first column, and the aggregate never fires. The math isn't wrong; it's
answering a different question than the one you asked.
The video path agrees
I didn't want to publish a stills-only result, so I ran the same comparison
through ffmpeg with lossless intermediates (-crf 0 -pix_fmt yuv444p) to avoid
letting the encoder hide or create the artifact:
# picture master, no bars
ffmpeg -loop 1 -i pic.png -t 2 -r 25 -c:v libx264 -crf 0 -pix_fmt yuv444p pic.mp4
# add bars, keep it lossless
ffmpeg -i pic.mp4 -vf "pad=1600:900:200:0:black" -c:v libx264 -crf 0 -pix_fmt yuv444p frame.mp4
# crop first, then scale
ffmpeg -i frame.mp4 -vf "crop=1200:900:200:0,scale=4800:3600:flags=lanczos" \
-c:v libx264 -crf 0 -pix_fmt yuv444p A_cropfirst.mp4
# scale first, then crop
ffmpeg -i frame.mp4 -vf "scale=6400:3600:flags=lanczos,crop=4800:3600:800:0" \
-c:v libx264 -crf 0 -pix_fmt yuv444p B_upscalefirst.mp4
Per-frame PSNR against the lossless picture master, 5 frames:
| Path | Whole frame | Left band | Right band | 1px | 8px left |
|---|---|---|---|---|---|
| Crop first | 99.00 | 0 px | 0 px | 99.00 | 99.00 |
| Upscale first, then crop | 45.97 | 9 px (2.25 src) | 9 px | 10.57 | 18.88 |
Same 9-pixel band, same 2.25 source pixels, arrived at through a completely
different pipeline. Two independent harnesses agreeing on the geometry is worth
more than either one alone.
Note the asymmetry in the stills test that doesn't appear here: the left edge
is always worse than the right (1px 11.31 dB vs right-edge 8px 28.59 dB). That's
because the test picture's brightness ramps left to right, so the left bar is a
much bigger step than the right one. Bar bleed scales with the contrast across
the seam, not with distance from it.
Cost
Upscaling the frame instead of the picture is 33.3% more pixels (921,600 vs
691,200 at this size). Measured, 4×:
| Kernel | Upscale-then-crop | Crop-first | Ratio |
|---|---|---|---|
| lanczos | 123 ms | 95 ms | 1.30× |
| bicubic | 95 ms | 73 ms | 1.31× |
| bilinear | 72 ms | 54 ms | 1.34× |
About 1.3× the compute for a worse result — you pay 30% more to break the edge.
On a 10-second 30 fps clip that's 300 frames, so on the order of 8 seconds per
clip at this size, and the ratio holds.
What I'd actually check
- Never compare whole frames when the defect is a thin band. Compute per-column MSE and look at the first and last 8 columns against ground truth. That's the metric that sees this.
-
Crop, pad, or shift before the resample — not after. In ffmpeg,
cropgoes first in the filter chain. This is not a style preference; the filter order is the fix. - If a metric reports a suspiciously perfect score, verify you changed the input. My pre-blur candidate scored 99.00 dB because it was a no-op. That's a more common failure than a genuinely perfect result.
-
Sanity-check bar detection, don't trust the default.
cropdetect's default limit of 24 usesmax >= limit, so a bar with a luma of exactly 24 gets treated as picture. A noisy bar at luma 20 ± 4 also hits 24 and gets kept. Raising the limit past that starts eating genuinely dark picture instead — there is a range where no threshold is both sensitive and safe.
The clip-order question has a boring answer, which is why it's easy to get wrong
in a hurry: crop first. The interesting part is that the standard metric would
have let you ship the wrong one and reported 45 dB while doing it.
If you want to see these comparisons run on real footage rather than a synthetic
test pattern, upgolab.com has the same pipeline available
in the browser — it's the same crop-order logic, and it's free to try on your own
files.
Further reading, all measured the same way: crop the black bars before upscaling: the measured band that whole-frame PSNR cannot see; upgolab-ai-upscaler-guide; anamorphic SD and non-square pixels: what the 360 cycles/width ceiling costs, measured.
Top comments (0)