Extending a clip is one of those tasks that sounds like a filter and turns out to be a small rendering pipeline. Here is what I learned building it to run in a browser tab, with the parts that matter and the ones that are a waste of time.
Slow it down, then synthesise
Playing the same frames over a longer duration gives you a stutter, not a longer clip. The useful approach is to synthesise the missing intermediate frames: take the original pair of frames, estimate the motion between them, and generate one or more frames that sit between. A 10-second clip at 30 fps becomes 10 seconds at 60 fps of source material, which then reads as smooth slow motion.
Motion estimation is the expensive part
Block matching on the CPU is fine for a proof of concept and too slow for anything above 720p. Two things made it practical:
- Do the search in a downscaled luminance image. Estimate at quarter resolution, then refine the final vector only around the best candidate.
- Cap the search radius. Hair and foliage produce noise that a large search window chases forever for no visible gain.
Blend, do not snap
Taking the nearest frame produces visible judder at edges. A weighted blend between the two source frames along the motion vector - closer to the source, more weight - removes most of it. Around occlusion boundaries the vectors are wrong, so a small bilateral smoothing pass on the vector field is worth the few milliseconds.
Interpolation does not create detail
This is the honest limitation. If the source is soft, the extended clip is soft. If there is real camera motion, the vector field is easier to estimate; if the subject moves and the camera does not, expect artefacts around limbs. Quality scales with how much the source already gives you.
Where it fits
The practical uses are making a short clip hold a punchline, fitting a fixed slot in a social format, or giving an editor headroom to trim. All of that needs the output to look native, which is why the interpolation step is not optional.
I have the pipeline running at AI Video Extender if you want to compare its output with your own implementation.
Top comments (0)