Try it yourself: UpgoLAB runs in the browser and takes a 720p clip to 1080p or 4K without installing anything. Free, no watermark.
The numbers in this post are from my own test clips, not marketing material.
Pause an upscaled clip on any single frame and it looks great. Play it and something is wrong — the image seems to breathe. Edges crawl left and right, skin texture shimmers, a patch of wall that never moves in the source starts to boil.
This is the most common failure mode in AI video upscaling, and it is not a resolution problem. The frames are individually excellent; the sequence is not, because a frame-by-frame upscaler maximises image quality when a video is a signal in time.
The model has no idea frame 41 and frame 42 are the same wall
A still-image upscaler is a function f(x) -> y, where x is H×W×3 and y is 2H×2W×3. Nothing in that signature refers to time. Feed it frame 41 and frame 42 as separate calls and it has no representation of the fact that they depict the same scene a fortieth of a second apart. Each call is an independent inference.
If f were linear this would be harmless — nearly identical inputs would give nearly identical outputs. The problem is that the interesting part of f is not linear: a stack of convolutions with nonlinear activations, and in generative variants, a discriminator trained to reward plausible high-frequency texture.
That last part is the poison. Asked to reconstruct detail that is not in the source — hair strands, fabric weave, brick grain — a GAN-based upscaler does not interpolate. It samples from a conditional distribution of what the texture could plausibly be, given the low-resolution patch. Two inputs differing by a fraction of a pixel of noise land in different regions of that distribution and produce visibly different textures, because "visibly different" is exactly what the discriminator was trained to make possible.
So the high-frequency band of the output is not deterministic. It is a sample. Independent samples viewed at 24 or 30 per second are the definition of noise.
Why the eye is so unforgiving: flicker beats acuity
Human vision does not evaluate sharpness frame by frame. It integrates over roughly a tenth of a second, and its sensitivity to temporal modulation is not flat. Contrast sensitivity for flicker peaks in the low single digits to low-teens Hertz and falls off above that — which is why 24 Hz display is acceptable but a 10 Hz flicker in a light bulb is unbearable. Random per-frame texture resynthesis lands squarely in that band, so it reads not as noise but as the object itself wobbling.
Visibility also depends on the stability around it. A crawling edge on a moving camera is masked by motion; the same edge on a locked-off shot is the only thing moving in the frame, and motion detectors lock onto it instantly. That is why architectural footage, tripod interviews and product turntables — the content people most want to upscale — are the worst offenders.
Four mechanisms, not one
"Temporal inconsistency" is usually treated as one defect. It is at least four.
1. Stochastic texture resynthesis
Fine-grain boiling in textured regions — foliage, hair, gravel, knitwear, concrete. The giveaway is that it survives at the highest detail level: zoom in and individual specks are redrawn, not shifted.
2. Edge quantisation on sub-pixel motion
A one-pixel line moving half a pixel between frames is a substantial change in its sampled profile. The upscaler resolves that into a sharp edge at a slightly different sub-pixel position with a slightly different gradient falloff. Across a sequence the edge vibrates by a pixel or two and the gradient pulses. Thin structures are worst because their energy sits above the source's Nyquist limit — the band the model is guessing in. Power lines and small text are the reliable detectors.
3. GOP-boundary pulsing
I-frames are coded from scratch; P/B frames are motion-compensated residuals, with a measurably different noise floor and blocking character. The upscaler is sensitive to exactly that difference, because compression artefacts are in the distribution it learned to repair. So its output statistics shift at every keyframe — typically every one to two seconds in consumer encodes. Sharpness rises at the keyframe, softens through the GOP, snaps back.
4. Chroma/temporal interaction
Most delivery video is 4:2:0, so chroma is stored at half resolution in both axes and reconstructed by interpolation. Colour edges are already soft and imprecisely positioned before the upscaler sees them. When the model sharpens luma while chroma drifts, fringing moves frame to frame — a rainbow crawl on high-contrast boundaries that reads as a focus problem but is a sampling problem.
That comparison is a still, and stills are exactly the wrong way to judge this defect. It shows what a conservative preset does spatially, not stability — that only exists between frames.
A detection protocol that needs no lab
- Export as an image sequence. PNG, lossless — sampling from a re-encoded MP4 reintroduces the noise you are isolating.
- Pick a static crop. A region with no intended motion: a wall, a background post, a patch of sky. Tight, maybe 128×128.
- Compute frame-to-frame difference inside the crop. In an NLE, stack consecutive frames with a difference blend and boost gain. In ffmpeg or Python, take mean absolute difference.
- Watch the trend, not the value. A stable upscale gives a low flat line; a shimmering one gives a spiky line, spikes aligned to GOP boundaries confirming mechanism 3.
- Repeat on the source. If the source crop is also noisy you are measuring grain, not hallucination.
The working rule: if the static-region difference in the output is not in the same ballpark as the source's noise floor, the upscaler is adding motion that was not there. Calibrate once on a clean clip rather than trusting anyone's figure, mine included. And never evaluate by scrubbing — that reads as stills, the view that makes every upscale look fine.
The counter-intuitive fix: accept less sharpness
Every mitigation trades spatial acuity for temporal stability, and most resist because they optimise the wrong axis.
- Use deterministic or low-stochasticity modes where offered. A fixed seed does not fix sub-pixel motion, but removes mechanism 1 almost entirely.
- Denoise temporally, not just spatially. A spatial denoiser injects fresh residual noise every frame, which the upscaler then interprets differently each time — you manufactured the problem. Motion-compensated temporal denoising makes the input stable, and fixes more shimmer than any model swap.
- Treat grain as an enemy, then add it back. Grain is random per frame by construction. Strip it, upscale clean, overlay synthetic grain at output resolution.
- Ask for 2× instead of 4×. At 4× the model invents 16 output pixels per input pixel; at 2× it invents 4.
- Deliver below the upscale target. A 1080p deliverable from a 4K upscale, downsampled at the end, averages away much of the per-frame variance.
Decision table
| Source situation | Shimmer risk | Do this first | Avoid |
|---|---|---|---|
| Locked-off tripod, static background | Very high — nothing masks the crawl | Temporal denoise, conservative preset | Aggressive sharpen presets |
| Handheld or gimbal, constant motion | Moderate — motion masks a lot | Denoise, upscale, downsample | Chasing invisible sharpness |
| Heavy grain or high-ISO night | Very high — grain is the cause | Strip grain, upscale, re-add | Upscaling grain and hoping |
| Thin structures: wire, hair, text | High and localised | Accept softness; test 2× first | More sharpening |
| Low-bitrate, blocky source | High, plus GOP pulsing | Light denoise, then upscale | Strong deblocking |
| Archival film, grain is aesthetic | High, but removal is creative | Stabilise, then re-grain | Judging on a still |
| Short social clip, heavy cuts | Low — cuts reset attention | Standard per-frame pipeline | Paying for temporal work |
When you should not do this
Stills pulled from video. If the deliverable is one frame — a thumbnail, key art, a print grab — temporal consistency is meaningless and the pass only softens the frame.
Grain-as-content. Restoring Super 8 or a film print, the grain is part of the object. Removing and re-synthesising it is a regrade, not a restoration.
Very fast motion or heavy shake. Temporal methods estimate inter-frame motion. When motion is large relative to the frame, or shake is bad enough that flow estimation fails, warping adds smearing and tearing worse than the shimmer being fixed.
The defect is elsewhere. Soft because it was out of focus, captured at the wrong resolution, or re-encoded six times? Upscaling a defocused clip yields a consistently defocused upscale.
You cannot measure. Without a way to compare output against source in motion you cannot tell improvement from degradation. Take the conservative path — denoise, 2×, downsample.
What this changes about how you evaluate
Stop judging upscalers on stills. A still tells you about spatial reconstruction, the easy half every tool has largely solved. It says nothing about whether the tool has a temporal model, and the absence of one is what makes footage look cheap. Play the clip, watch a static region. If it boils, the frames were never the problem.
The pipeline I use — pick a preset for the source's damage, then verify in motion rather than on a paused frame — is documented at upgolab.com, including which presets trade spatial sharpness for sequence stability.
Also published as a lightweight static page: English
All languages: EN · 日本語 · DE · ES · PT
📖 Practical companion guide (EN): Why original video quality decides 80% of the result
The measurements behind this post, with the browser tests reproduced: making a 4K thumbnail that survives compression.
Everything else in the cluster: AI upscaling notes.
Measured separately: chroma subsampling and what 4:2:0 costs before an upscale — luma is bit-identical, chroma loses 9.7 dB at 2x, and a 1px colour line keeps exactly 50% of its contrast.

Top comments (0)