Why upscaling old low-res video makes it look worse, and the ffmpeg filters that fix it

If you've ever run an old 240p YouTube rip or a 2012 phone video through an AI upscaler, you've probably seen this: the output is bigger and sharper, and somehow looks worse. Faces turn into mosaics. Flat walls grow a grid of tiles. Edges get a crunchy halo.

I build a video restoration app, and this was one of the first quality bugs I had to understand properly. The short version: the upscaler is doing its job. It's sharpening exactly what you gave it, and what you gave it was mostly compression damage.

This post covers what that damage is, how to tell which kind your clip has, and the ffmpeg filters I run before any upscaler touches the video.

Two kinds of old video, two different problems

Most "old video" falls into one of two buckets, and they need opposite treatment.

Tape and DVD (480i / 576i). VHS, Hi8, MiniDV captures and DVDs are usually interlaced: each frame is two half-pictures (fields) captured a fraction of a second apart. On motion you get the comb pattern, thin horizontal stripes on anything moving. There's also analog noise: chroma smear, grain, tracking wobble.

Low-res internet and phone video (≤ 288p, sometimes 360p and 480p). Old uploads, early phone cameras, clips that have been re-shared five times. These are usually progressive, so there's nothing to deinterlace, but they're crushed by compression. The encoder split each frame into 8×8 (or larger) blocks and threw away most of the detail inside each one. What's left is a picture made of tiles.

The mistake I made at first was treating both the same way: a light deinterlace and a gentle denoise, then upscale. On tape that's fine. On a 240p download it's a disaster.

Why the blocks get worse

A neural upscaler is trained to find edges and texture and make them crisp. A block boundary is an edge as far as the model is concerned: a hard, straight, high-contrast line. So it sharpens it.

Upscale a 320×240 clip 4× and every 8×8 block becomes a 32×32 tile with a crisp outline. A gentle tape denoise doesn't help, because it's tuned for random grain and leaves the block grid alone.

You can't fix this afterwards. Once the upscaler has decided the grid is real detail, it's baked into every frame. The fix has to happen before upscaling, while the blocks are still small.

Step 1: find out what you have

ffprobe tells you most of what you need:

ffprobe -v error -select_streams v:0 \
  -show_entries stream=width,height,field_order,codec_name,bit_rate \
  -of default=nw=1 input.mp4

Roughly:

• field_order is tt, bb, tb or bt → interlaced. Treat it as tape.
• Height is 480, 486 or 576 with no field order reported → probably a tape or DVD capture that lost its metadata. Check a frame with motion for combing.
• Height is 288 or less → assume compression blocks, even if it looks "fine" at its native size.

That last rule is the one that matters. At 240p on a phone screen the blocks are barely visible. At 1080p they're the whole picture.

Step 2a: the tape path (deinterlace + light denoise)

For interlaced sources:

ffmpeg -i tape.mp4 \
  -vf "yadif=0:-1:0,hqdn3d=2.0:1.5:3.0:2.5" \
  -c:v libx264 -crf 12 -preset fast -c:a copy \
  tape_clean.mp4

• yadif=0:-1:0 deinterlaces to one frame per frame, auto-detects field order, and processes every frame. If you want smoother motion and can afford double the frames, yadif=1 (or bwdif=1) outputs one frame per field instead.
• hqdn3d=2.0:1.5:3.0:2.5 is a mild spatial + temporal denoise. The temporal part (the last two numbers) does most of the work on tape grain, because grain changes every frame and the picture doesn't.

Keep the denoise mild. Tape footage that has been denoised hard looks waxy, and an upscaler will happily sharpen wax.

Step 2b: the low-res path (deblock first)

For ≤ 288p progressive sources:

ffmpeg -i lowres.mp4 \
  -vf "spp=5,deblock=filter=strong:block=8,hqdn3d=3.2:2.4:5.5:4.0" \
  -c:v libx264 -crf 12 -preset fast -c:a copy \
  lowres_clean.mp4

Three filters, in this order:

1. spp=5, simple post-processing. It re-runs the DCT at shifted positions and averages the results, which is very good at removing block artifacts and ringing without blurring real edges much. Quality 5 is slow but worth it at these resolutions; the frames are tiny anyway.
2. deblock=filter=strong:block=8 smooths what's left along the 8×8 grid specifically.
3. hqdn3d with stronger settings than the tape path, to clean up the mosquito noise around edges that both of the above leave behind.

Notice there's no yadif. These clips are progressive, and deinterlacing a progressive source just throws away vertical resolution you can't spare.

The result will look softer than the original at native size. That's expected and it's the point: you've removed fake detail (the grid) so the upscaler can work on real detail.

Step 3: keep intermediates lossless

This one cost me more than I'd like to admit. If your pipeline extracts frames, processes them and re-encodes, possibly several times, don't use JPEG for the intermediate frames.

I had frames passing between stages as JPEG to save disk space. Every stage re-compressed them, adding a fresh layer of 8×8 blocks, and the sharpening at the end then brought all of those blocks out. I'd carefully deblocked the source and then put the blocks back myself.

Use PNG. You don't need maximum compression; PNG at level 1 is fast and still lossless:

ffmpeg -i lowres_clean.mp4 -compression_level 1 frames/%06d.png

Same idea for the intermediate video files: -crf 12 (or lower) on the pre-clean output, as above, so you're not adding a new generation of compression before the upscaler sees it.

Step 4: don't throw away pixels to save memory

If you use a heavy model (a diffusion upscaler, for example) on a small GPU, it's tempting to downscale the input a bit more so it fits. Don't do this with a source that's already tiny.

I had a memory fallback that dropped a 320-wide clip to 256 wide before inference. On paper that's a 20% cut. In practice, a 320×240 clip inferred at 256×192 and then stretched to 1080p is about a 5.6× stretch, and it looked pixelated no matter what came after. Shorter processing chunks at the native width were the better trade: slower, but the detail you start with is all the detail you'll ever have.

What to expect

With the right pre-clean, a 240p clip upscaled to 1080p looks like a soft but clean video: real faces instead of mosaics, flat walls that are actually flat. It won't look like it was shot in HD, and nothing honest will make it look that way. The goal is "the best version of what was recorded," not new detail invented by a model.

Tape footage, deinterlaced and lightly denoised, usually scales much better, because a 480i capture has more real information in it than its resolution suggests.

Summary

1. Check field_order and height with ffprobe before choosing filters.
2. Interlaced tape/DVD: yadif + mild hqdn3d.
3. ≤ 288p internet/phone video: spp + deblock + stronger hqdn3d, and no deinterlacing.
4. Deblock before upscaling. You can't remove a grid the upscaler has already sharpened.
5. Lossless intermediates (PNG, low CRF). Every JPEG generation adds blocks back.
6. Don't downscale an already-tiny source to save memory.

I use all of this in Phoenix Upscaler, a desktop app that picks the right path for each clip automatically. I wrote more about the two common cases on the site: restoring old phone video and VHS restoration. But the ffmpeg commands above work on their own, in front of any upscaler you like.

If you've found filter settings that work better on a particular kind of source (Hi8, early-2000s webcam, 3GP phone video), I'd like to hear about them in the comments.