DEV Community: Voor AI

How to Extract Audio From a Video With FFmpeg (Copy vs Re-encode, With pytest)

Voor AI — Sat, 07 Nov 2026 00:38:22 +0000

Want the soundtrack out of an MP4 without guessing containers? Use FFmpeg’s -vn -map 0:a:0 path, pick stream copy when the codec already matches your target, and re-encode only when you need MP3/WAV. Below is a small Python wrapper plus four pytest checks that passed on 2026-11-07.

Prerequisites

ffmpeg and ffprobe on PATH
Python 3.10+
pytest (uvx --with pytest pytest works if system Python lacks it)

Two honest extract commands

Lossless-ish keep (AAC inside MP4 → .m4a):

ffmpeg -i clip.mp4 -vn -map 0:a:0 -c:a copy out.m4a

Need MP3:

ffmpeg -i clip.mp4 -vn -map 0:a:0 -c:a libmp3lame -b:a 192k out.mp3

-vn drops video. -map 0:a:0 takes the first audio stream. copy refuses to transmux into a mismatched container — that failure is a feature.

Python helper

# extract_audio.py (abridged)
def extract_audio(src, dest, *, codec="copy", audio_bitrate=None):
    if not has_audio(src):
        raise ExtractError(f"no audio stream in {src}")
    cmd = ["ffmpeg", "-y", "-i", str(src), "-vn", "-map", "0:a:0"]
    if codec == "copy":
        cmd += ["-c:a", "copy"]
    else:
        cmd += ["-c:a", codec]
        if audio_bitrate:
            cmd += ["-b:a", audio_bitrate]
    cmd.append(str(dest))
    subprocess.run(cmd, check=True, capture_output=True)

Gate silent files with ffprobe before you spend a render slot.

pytest — four checks

test_has_audio_true
test_has_audio_false
test_stream_copy_m4a          # codec_name == aac, duration close
test_reencode_mp3_and_reject_silent

Run:

uvx --with pytest pytest -q test_extract_audio.py
# ....  4 passed in 0.45s

Fixtures build a 1.2 s blue sine clip and a silent twin via lavfi so CI stays offline-friendly after ffmpeg is installed.

Review checklist

ffprobe shows an audio stream before extract
Stream-copy outputs keep the original codec_name
Re-encode path is used only when the container/codec must change
Silent inputs raise a clear error instead of writing an empty file

Troubleshooting

Symptom	Likely cause	Fix
`Could not find tag for codec`	`copy` into wrong container	Use `.m4a` for AAC, or re-encode
Output ~0 bytes	No audio / wrong map	`has_audio()` gate + `-map 0:a:0`
Duration drift on MP3	Encoder padding	Allow ~0.25 s tolerance in tests

Limitations

Stream copy cannot change codec. Very long files still need disk headroom. This article does not cover multi-track selection beyond 0:a:0.

Optional next step on Voor

After you have a clean voice bed, you can build a new forward clip on the Grok AI Video Generator — upload a plate, attach audio-driven motion elsewhere on Voor, or start a fresh prompt. The engineering work above stands alone; the CTA is optional.

Final link: open Grok AI Video Generator.

How to Reverse a Video With FFmpeg (and Prove the Last Frame Became the First)

Voor AI — Fri, 06 Nov 2026 00:31:27 +0000

Reversing a clip in FFmpeg looks like a one-liner until you ask a sharper question: did the last decoded frame actually become the first frame of the output? Stream copy cannot answer that. A lossy H.264 round-trip can look right while failing a byte-level check. This article shows both a PNG-sequence proof that is exact at the ends, and the everyday reverse filter path with a PSNR sanity check.

If you need a brand-new forward clip instead of reversing someone else's footage, generate one on Grok AI Video Generator on Voor.

What we are proving

Source clip used in the tests: forward.mp4, 1280×720, 24 fps, 72 frames, 3.000 s.

Pass criteria:

Geometry preserved after reverse.
sha256(source[0]) == sha256(reversed[-1]) on PNG frames.
sha256(source[-1]) == sha256(reversed[0]) on PNG frames.
Filter-based H.264 reverse stays visually close to the PNG-path reverse (PSNR average ≥ 30; we measured ~47).

Method A — PNG sequence (exact ends)

# 1) explode
ffmpeg -i forward.mp4 -fps_mode passthrough f-%04d.png

# 2) rebuild in reverse order (copy/rename into rev/f-0001.png …)
ffmpeg -framerate 24 -i rev/f-%04d.png \
  -c:v libx264 -pix_fmt yuv420p -r 24 reversed.mp4

In Python the same idea is: extract → list(reversed(pngs)) → rebuild. That is what reverse_via_pngs() in the companion module does.

Method B — filter reverse (delivery one-liner)

ffmpeg -i forward.mp4 -vf reverse -an \
  -c:v libx264 -pix_fmt yuv420p reversed_filter.mp4

Notes:

reverse is a filter, so -c copy is not an option.
Audio is dropped here on purpose (-an). To reverse audio too, add areverse in a filter graph.
Re-encoding changes pixels; do not expect SHA equality against the source.

Minimal helper

def reverse_filter_h264(src, dest, ffmpeg="ffmpeg"):
    subprocess.run(
        [ffmpeg, "-y", "-i", str(src), "-vf", "reverse", "-an",
         "-c:v", "libx264", "-pix_fmt", "yuv420p", str(dest)],
        check=True,
    )

Pytest results (2026-11-06)

....                                                                     [100%]
4 passed in 1.36s

The suite checks geometry, exact PNG end-swap, PSNR between PNG-path and filter-path outputs, and a non-empty delivery file.

When not to reverse

Copyrighted music videos you do not have rights to alter.
Clips where reversed motion implies a false real-world event.
Any case where you actually need a new forward shot — generate one instead of reversing.

Voor alternative

Open Grok AI Video Generator, write a forward brief (or attach a still), and generate a new clip rather than reversing a source you should not republish.

Disclosure

This article was drafted with an autonomous assistant. All FFmpeg commands and pytest checks were run locally on the sample clip before publishing.

How to Crop a Video With FFmpeg (16:9 to 9:16, One Window per Shot)

Voor AI — Thu, 05 Nov 2026 00:35:14 +0000

To crop a video with FFmpeg, use the crop filter with an output width, height and top-left offset, re-encode the video and copy the audio:

ffmpeg -i in.mp4 -vf "crop=606:1080:(iw-ow)/2:0,setsar=1" \
  -c:v libx264 -crf 18 -pix_fmt yuv420p -c:a copy out.mp4

That turns a 1920×1080 clip into a centered 606×1080 vertical (9:16) clip. It is also where most vertical crops go wrong: a single centered window is fine for one static shot, but an edited clip has several shots, and the subject is rarely in the middle of all of them. Below I crop a real four-shot clip, show where the center crop fails, and fix it with one crop window per shot driven by FFmpeg's own scene detection. Everything is checked with 11 pytest tests.

The test clip

A 15.16 s, 1920×1080, 24 fps clip with AAC audio: a four-shot classroom scene with two students. It is one of the text-to-video examples on Voor's HappyHorse 1.1 page, so you can download the same file and reproduce every number here. ffprobe with -count_frames reports 363 frames.

Crop syntax in 30 seconds

crop=w:h:x:y keeps a w×h rectangle whose top-left corner is at (x, y). Inside the expressions, iw/ih are the input size and ow/oh the output size, so (iw-ow)/2 centers the window horizontally. Add setsar=1 so players do not stretch the result.

Cropping changes pixels, so the video has to be re-encoded. If you try -c:v copy, FFmpeg refuses with "Filtering and streamcopy cannot be used together". The audio is untouched, so -c:a copy is safe and keeps it bit-exact.

The width you ask for is not always the width you get

9:16 of 1080 is 607.5 px, and yuv420p needs even dimensions. Here is what FFmpeg 9.0.1 actually did:

Filter	Result
`crop=ih*9/16:ih`	608×1080, no warning
`crop=607:1080`	606×1080, no warning
`scale=607:1080` + libx264	fails: `width not divisible by 2 (607x1080)`

crop quietly rounds to the chroma grid, so your script and your file can disagree by two pixels without any error. That matters when you later compute offsets or stack crops side by side. Compute an even size yourself:

def even(n: float) -> int:
    return int(n) // 2 * 2

def crop_size(src_w, src_h, ratio_w, ratio_h):
    w, h = src_h * ratio_w / ratio_h, src_h
    if w > src_w:                       # source narrower than target: crop height instead
        w, h = src_w, src_w * ratio_h / ratio_w
    return even(w), even(h)

crop_size(1920, 1080, 9, 16)   # (606, 1080)
crop_size(1920, 1080, 1, 1)    # (1080, 1080)

Why the center crop fails on this clip

The top row of the image above is the plain centered crop. Shot 2 cuts straight between the two faces, and in shot 4 the window lands on the blurry background between them. Nothing is technically wrong with the file; the framing is just useless.

Step 1: find the shot boundaries with `scdet`

FFmpeg's scdet filter scores every frame for a scene change and can print the cut times:

import re, subprocess

def detect_shots(path, threshold=10.0):
    out = subprocess.run(
        ["ffmpeg", "-hide_banner", "-nostats", "-i", path, "-vf",
         f"scdet=threshold={threshold},metadata=print:file=-", "-an", "-f", "null", "-"],
        capture_output=True, text=True, check=True).stdout
    return [float(t) for t in re.findall(r"lavfi\.scd\.time=([0-9.]+)", out)]

detect_shots("happyhorse-t2v.mp4")   # [1.083333, 4.25, 9.0]

Three cuts, four shots. Pull one frame from the middle of each shot (-ss 0.5, 2.6, 6.6, 12) into a contact sheet and pick one x per shot by eye. Here that took four numbers: 980, 1040, 633 and 1150.

Step 2: one crop expression that changes `x` at each cut

crop evaluates x per frame, and t is the frame time, so a nested if(lt(t,…)) switches windows exactly at the cuts. Commas inside a filter argument must be escaped:

def piecewise_x(cuts, xs):
    assert len(xs) == len(cuts) + 1, "need one x per shot"
    expr = str(xs[-1])
    for cut, x in reversed(list(zip(cuts, xs[:-1]))):
        expr = f"if(lt(t\\,{cut:.4f})\\,{x}\\,{expr})"
    return expr

def crop(src, dst, w, h, x, y=0, crf=18):
    vf = f"crop={w}:{h}:{x}:{y},setsar=1"
    return subprocess.run(
        ["ffmpeg", "-v", "error", "-y", "-i", src, "-vf", vf,
         "-c:v", "libx264", "-crf", str(crf), "-preset", "medium", "-pix_fmt", "yuv420p",
         "-c:a", "copy", "-movflags", "+faststart", dst],
        capture_output=True, text=True)

w, h = crop_size(1920, 1080, 9, 16)
crop("happyhorse-t2v.mp4", "vertical.mp4", w, h,
     piecewise_x(detect_shots("happyhorse-t2v.mp4"), [980, 1040, 633, 1150]))

The bottom row of the first image is the result: every shot keeps one readable face. Shot 4 still loses one of the two speakers. At 9:16 there is no single window that keeps both, so either pick the speaker that matters or reframe the shot.

Step 3: prove the output with tests

These are the checks that run against the real clip (abridged):

def test_crop_silently_rounds_odd_width_but_scale_fails(): ...
    # crop=607:1080 -> 606 wide, rc 0; scale=607:1080 -> "not divisible by 2"

def test_shot_cuts_match_the_example():
    assert [round(c, 2) for c in detect_shots(SRC)] == [1.08, 4.25, 9.0]

def test_frames_duration_and_audio_survive(per_shot):
    a, b = probe(SRC), probe(per_shot)
    assert (b["w"], b["h"], b["sar"]) == (606, 1080, "1:1")
    assert b["frames"] == a["frames"] == 363
    assert abs(b["duration"] - a["duration"]) < 1 / 24

@pytest.mark.parametrize("t,shot", [(0.5, 0), (2.6, 1), (6.6, 2), (12.0, 3)])
def test_each_shot_uses_its_own_window(per_shot, t, shot):
    # output frame vs the same region cut from the source frame with Pillow
    assert psnr(out, right) > 30
    assert psnr(out, right) > psnr(out, centre) + 3

def test_cropdetect_finds_letterbox(): ...   # 1920x800 padded to 1080 -> crop=1920:800:0:140
def test_stream_copy_cannot_crop(): ...

11 passed in 5.36s

The PSNR test is the useful one: it pulls a frame from each shot, cuts the window you meant from the source frame with Pillow, and requires the output to match it better than the centered window. If the cut detection drifts by a few frames or someone edits the offsets, it fails.

Bonus: remove black bars with `cropdetect`

If the input is letterboxed, let FFmpeg measure the picture area instead of guessing:

ffmpeg -t 2 -i boxed.mp4 -vf cropdetect=limit=24:round=2:reset=0 -f null - 2>&1 | grep -o "crop=[0-9:]*" | sort | uniq -c

On a 1920×800 picture padded to 1920×1080 it reports crop=1920:800:0:140. Feed that straight into -vf crop=….

When not to crop at all

Cropping 16:9 to 9:16 throws away about 68% of the width (606 of 1920 px). If the vertical version is the one you actually need, it is often better to generate or shoot it vertical. The same HappyHorse 1.1 model has a 9:16 aspect ratio option; on Voor, 9:16 at 720p was quoted at 244 credits for 5 s and 732 credits for 15 s when I checked (Dialogue scene preset, prompt required). You can generate the vertical version directly and skip the crop, but you will still want the test above if you crop anything else.

Limitations

The per-shot x values were picked by eye from a contact sheet. There is no subject tracking here; a subject that moves within a shot needs keyframed offsets or a tracker.
scdet thresholds are content-dependent. Fades and dissolves may not register as cuts; check the list against the contact sheet.
Measurements are from FFmpeg 9.0.1 on macOS (Apple Silicon) with libx264; older builds may round crop sizes differently.

How to Speed Up a Video With FFmpeg (Without Losing Frames or Chipmunking the Audio)

Voor AI — Wed, 04 Nov 2026 00:31:29 +0000

To speed up a video with FFmpeg, change the video timestamps with setpts and the audio tempo with atempo in the same command, then tell FFmpeg what frame rate you want out. For 2x:

ffmpeg -i in.mp4 \
  -filter_complex "[0:v]setpts=PTS/2,fps=48[v];[0:a]atempo=2[a]" \
  -map "[v]" -map "[a]" -c:v libx264 -crf 18 -c:a aac out.mp4

That last part, the frame rate, is the one most snippets skip, and it decides whether you keep your frames or silently throw half of them away. Below I measure every variant on a real clip, then wrap it in a small Python helper with pytest checks for frame count, A/V sync and pitch.

The test clip

A 5.04 s, 1280×720, 24 fps clip with AAC audio (a ramen cook lifting noodles, steam, street noise). It is the public text-to-video example on Voor's Seedance 2.0 Mini page, so you can grab the same file and reproduce the numbers. ffprobe with -count_frames reports 121 video frames.

What the naive one-liner actually does

The common answer is -filter:v "setpts=PTS/2". It works, in the sense that the video is half as long. But the MP4 muxer keeps the input's 24 fps, so FFmpeg drops every second frame to fit:

Command (2x)	Frames out	fps	Video / audio
source	121	24	5.04 s / 5.09 s
`setpts=PTS/2` only	62	24	2.58 s / 2.54 s
`setpts=PTS/2,fps=48`	121	48	2.52 s / 2.54 s
`setpts=PTS/2,fps=24`	61	24	2.54 s / 2.54 s

So there are really two different jobs:

Keep every frame (smooth, larger file, 48 fps): set fps to source fps × factor.
Keep 24 fps (smaller, platform-friendly): drop frames on purpose with fps=24, and know that motion gets choppier.

Both are fine. Doing the second one by accident is the bug.

Audio: use atempo, never asetrate

atempo changes tempo without changing pitch. asetrate changes the sample rate, which speeds the audio up and raises the pitch, so voices go chipmunk. I checked it on a 440 Hz test tone:

On older FFmpeg builds one atempo stage only accepted 0.5–2.0, so the usual trick is to chain stages (atempo=2,atempo=2 for 4x). FFmpeg 9.0.1 accepted atempo=4 in one stage, but the chain works everywhere, so the helper below always builds one.

Slowing down has the opposite problem

setpts=PTS*2 (0.5x) with no rate keeps all 121 frames but stretches them over 10 s, so the file ends up at about 12 fps and stutters. Your options:

Command (0.5x)	Frames out	fps	Encode time
`setpts=PTS/0.5` only	121	12	0.71 s
`...,fps=24` (duplicate frames)	242	24	1.04 s
`...,minterpolate=fps=24:mi_mode=mci`	239	24	7.71 s

Duplicated frames look like a step-motion slow-mo; minterpolate synthesizes in-between frames and is about 7x slower here. Check it for warping around steam, hair and edges before you ship it.

Stream copy cannot do this

You might hope to skip re-encoding with -c copy. FFmpeg refuses:

Filtergraph 'setpts=PTS/2' was specified, but codec copy was selected.
Filtering and streamcopy cannot be used together.

Changing speed means new timestamps for every frame, so it is always a re-encode.

A small helper

import json, subprocess

def atempo_chain(factor: float) -> str:
    """Split a tempo factor into atempo stages within 0.5..2.0."""
    if factor <= 0:
        raise ValueError("factor must be > 0")
    stages, f = [], factor
    while f > 2.0:
        stages.append(2.0); f /= 2.0
    while f < 0.5:
        stages.append(0.5); f /= 0.5
    stages.append(f)
    return ",".join(f"atempo={s:.6g}" for s in stages)

def source_fps(path):
    out = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "v:0",
                          "-show_entries", "stream=r_frame_rate", "-of", "csv=p=0", path],
                         capture_output=True, text=True, check=True).stdout.strip()
    num, den = out.split("/")
    return float(num) / float(den)

def speed_cmd(src, dst, factor, fps="keep", interpolate=False):
    """fps="keep": source fps * factor; fps=N: resample; fps=None: naive."""
    v = f"setpts=PTS/{factor}"
    if fps == "keep":
        v += f",fps={source_fps(src) * factor:g}"
    elif fps and interpolate:
        v += f",minterpolate=fps={fps}:mi_mode=mci"
    elif fps:
        v += f",fps={fps}"
    return ["ffmpeg", "-v", "error", "-y", "-i", src,
            "-filter_complex", f"[0:v]{v}[v];[0:a]{atempo_chain(factor)}[a]",
            "-map", "[v]", "-map", "[a]",
            "-c:v", "libx264", "-crf", "18", "-pix_fmt", "yuv420p",
            "-c:a", "aac", "-b:a", "160k", "-movflags", "+faststart", dst]

def speed_up(src, dst, factor, fps="keep", interpolate=False):
    subprocess.run(speed_cmd(src, dst, factor, fps, interpolate), check=True)
    return dst

def probe(path):
    out = subprocess.run(["ffprobe", "-v", "error", "-count_frames",
                          "-show_entries", "stream=codec_type,duration,nb_read_frames,avg_frame_rate",
                          "-of", "json", path], capture_output=True, text=True, check=True).stdout
    s = {x["codec_type"]: x for x in json.loads(out)["streams"]}
    num, den = s["video"]["avg_frame_rate"].split("/")
    return {"v_dur": float(s["video"]["duration"]),
            "a_dur": float(s["audio"]["duration"]) if "audio" in s else None,
            "frames": int(s["video"]["nb_read_frames"]),
            "fps": float(num) / float(den)}

The clip must have an audio stream; for silent clips drop the [0:a] branch and the second -map.

Tests that catch the silent failures

import subprocess
import numpy as np
import pytest
from speed import atempo_chain, speed_up, probe

SRC = "inputs/mini-t2v-720p.mp4"
FRAME = 1 / 24

@pytest.fixture(scope="module")
def out(tmp_path_factory):
    return tmp_path_factory.mktemp("speed")

def test_atempo_chain_splits_large_and_small_factors():
    assert atempo_chain(4) == "atempo=2,atempo=2"
    assert atempo_chain(0.25) == "atempo=0.5,atempo=0.5"
    assert atempo_chain(3) == "atempo=2,atempo=1.5"

def test_2x_keep_every_frame(out):
    src = probe(SRC)
    r = probe(speed_up(SRC, str(out / "keep.mp4"), 2))
    assert r["frames"] == src["frames"]
    assert r["fps"] == pytest.approx(48)
    assert abs(r["v_dur"] - r["a_dur"]) < 2 * FRAME

def test_naive_2x_silently_drops_half_the_frames(out):
    src = probe(SRC)
    r = probe(speed_up(SRC, str(out / "naive.mp4"), 2, fps=None))
    assert r["frames"] <= src["frames"] // 2 + 2

def dominant_hz(path):
    pcm = subprocess.run(["ffmpeg", "-v", "error", "-i", str(path), "-ac", "1",
                          "-ar", "44100", "-f", "s16le", "-"],
                         capture_output=True, check=True).stdout
    x = np.frombuffer(pcm, dtype=np.int16).astype(float)
    spec = np.abs(np.fft.rfft(x * np.hanning(len(x))))
    return np.fft.rfftfreq(len(x), 1 / 44100)[spec.argmax()]

def test_atempo_keeps_pitch(out):
    tone, fast = out / "tone.wav", out / "fast.wav"
    subprocess.run(["ffmpeg", "-v", "error", "-y", "-f", "lavfi",
                    "-i", "sine=frequency=440:duration=2", str(tone)], check=True)
    subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", str(tone),
                    "-af", atempo_chain(2), str(fast)], check=True)
    assert dominant_hz(fast) == pytest.approx(440, abs=5)

The full suite (8 tests, also covering 4x sync, slow-motion frame rate, the asetrate pitch shift and the stream-copy refusal) passes in about 4 s on FFmpeg 9.0.1 with Python 3.13:

uvx --python 3.13 --with numpy pytest -q test_speed.py
........                                                    [100%]
8 passed

Checklist

Decide first: keep every frame (fps = source × factor) or a fixed delivery rate (fps=24/30).
Always pair setpts with atempo in one -filter_complex, never asetrate.
Chain atempo stages for portability beyond 2x.
Verify with ffprobe -count_frames: frame count, fps, and video vs audio duration within a frame or two.
For slow motion, choose between duplicated frames (fast, steppy) and minterpolate (slow, check for warping).

When not to speed up at all

Speeding up shortens the clip: the 5 s example becomes 2.5 s. If what you actually need is a longer or differently paced shot, it is often cleaner to generate it at that length. On Voor, Seedance 2.0 Mini takes 4–15 s durations at 480p or 720p with generated audio; the form quotes 126 credits for 480p at 5 s and 270 credits for 720p at 5 s before you press Generate. You can try a brief in the Seedance 2.0 Mini generator and then run the clip through the helper above if you still want a time-lapse feel.

How to Trim a Video With FFmpeg (and Prove the Cut Starts on the Right Frame)

Voor AI — Tue, 03 Nov 2026 00:36:52 +0000

There are two ways to trim a video with FFmpeg, and they don't do the same thing:

Fast, no re-encode: ffmpeg -ss 1 -i in.mp4 -t 2 -c copy out.mp4. This is fast, but the video can only start on a keyframe.
Exact, re-encode: ffmpeg -ss 1 -i in.mp4 -t 2 -c:v libx264 -crf 18 -c:a aac out.mp4. This starts on the frame you asked for, but the video is encoded again.

Most "my trim is off by a second" bugs come from using the first command and expecting the second. Below, I trim a real clip both ways, measure which source frame each output really starts on, and turn the results into pytest checks you can keep in a media pipeline. Everything was run with FFmpeg 9.0.1 on macOS.

The test clip

The fixtures are three short AI-generated clips published as examples on Voor's Vidu Q3 page: 1952×1062, 30 fps, H.264 + AAC. These clips have very few keyframes, which makes the difference between the two methods easy to see. List the keyframes first:

ffprobe -v error -select_streams v:0 -skip_frame nokey \
  -show_entries frame=pts_time -of csv=p=0 vidu-q3-2.mp4
# 0.000000
# 2.200000

vidu-q3-2.mp4 (5.108 s, 153 frames) has only two keyframes, at 0.0 s and 2.2 s. vidu-q3-1.mp4 (4.11 s) has just one, at 0.0 s. That tells you in advance which cuts -c copy can make cleanly: almost none.

The helpers

All measurements come from ffprobe and from decoding frames, not from the command line we meant to run:

def run(cmd):
    return subprocess.run(cmd, capture_output=True, text=True)

def probe(path):
    r = run([FFPROBE, "-v", "error", "-show_entries",
             "format=duration:stream=index,codec_type,codec_name,duration,nb_frames,start_time",
             "-of", "json", str(path)])
    return json.loads(r.stdout)

def keyframes(path):
    r = run([FFPROBE, "-v", "error", "-select_streams", "v:0", "-skip_frame", "nokey",
             "-show_entries", "frame=pts_time", "-of", "csv=p=0", str(path)])
    return [float(x.strip(",")) for x in r.stdout.split() if x.strip(",")]

def frame_count(path):
    """Frames a player will actually show (decoded, after edit lists)."""
    r = run([FFPROBE, "-v", "error", "-select_streams", "v:0", "-count_frames",
             "-show_entries", "stream=nb_read_frames", "-of", "csv=p=0", str(path)])
    return int(r.stdout.strip())

To find out which source frame an output really starts on, decode both as tiny grayscale images and pick the closest match:

def gray_frame(path, index=0, size=(64, 36)):
    """Decode one displayed frame as small grayscale bytes for comparison."""
    w, h = size
    r = subprocess.run([FFMPEG, "-v", "error", "-i", str(path), "-vf",
                        f"select=eq(n\\,{index}),scale={w}:{h},format=gray", "-fps_mode", "passthrough",
                        "-frames:v", "1", "-f", "rawvideo", "-"], capture_output=True)
    return r.stdout

def diff(a, b):
    """Mean absolute pixel difference between two gray frames (0 = identical)."""
    return sum(abs(x - y) for x, y in zip(a, b)) / max(1, len(a))

def nearest_source_frame(src, clip, clip_index=0, search=range(0, 160)):
    """Index of the source frame that looks most like frame `clip_index` of `clip`."""
    g = gray_frame(clip, clip_index)
    best, score = None, 1e9
    for i in search:
        s = gray_frame(src, i)
        if not s:
            break
        d = diff(g, s)
        if d < score:
            best, score = i, d
    return best, score

At 64×36 the comparison is cheap, and in this clip neighbouring frames differ enough that the best match is unambiguous. Re-encoded frames score about 0.1, unrelated frames score in the tens.

Trim 1: stream copy

def trim_copy(src, dst, start, length):
    """Fast: no re-encode. Video can only begin on a keyframe."""
    return run([FFMPEG, "-y", "-v", "error", "-ss", str(start), "-i", str(src), "-t", str(length),
                "-map", "0", "-c", "copy", "-avoid_negative_ts", "make_zero", str(dst)])

I asked for 1.000 s → 3.000 s. What came back:

	Asked	Got
Duration	2.00 s	3.09 s
Frames	60	92
First frame	source frame 30 (1.000 s)	source frame 0 (0.000 s)

With -c copy FFmpeg can't create a new keyframe at 1.0 s, so the output starts at the previous keyframe, 0.0 s, and keeps everything up to 3.0 s. The cover image shows the difference: the frame you wanted is the mushrooms, the frame you got is the insect close-up from the start of the clip.

Trim 2: re-encode

def trim_exact(src, dst, start, length, crf=18):
    """Frame-accurate: decode from the seek point and re-encode."""
    return run([FFMPEG, "-y", "-v", "error", "-ss", str(start), "-i", str(src), "-t", str(length),
                "-map", "0", "-c:v", "libx264", "-crf", str(crf), "-preset", "medium",
                "-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "160k",
                "-movflags", "+faststart", str(dst)])

Now the first frame matches source frame 30 (score 0.1), there are exactly 60 frames, and both audio and video streams last 2.000 s. The cost is encode time and one generation of quality loss. -crf 18 keeps that loss below what you'll notice on a clip like this, but it's still not a bit-exact copy.

Trim 3: snap to a keyframe and copy

If you don't need a specific frame, for example when cutting a preview or removing a slow start, move the cut to a keyframe and keep the speed of -c copy:

def snap_to_keyframe(path, start):
    """Latest keyframe at or before `start` - the only place stream copy can begin cleanly."""
    ks = [k for k in keyframes(path) if k <= start + 1e-6]
    return ks[-1] if ks else 0.0

snap_to_keyframe(clip, 2.5) returns 2.2. Copying from 2.2 s gives 62 frames that start exactly on source frame 66 (2.2 × 30), with no re-encoding. The rule is simple: stream copy is exact only when the start time is a keyframe.

The `-to` trap

-to is an output timestamp, so where you put -ss changes what it means:

def trim_to(src, dst, start, end, input_seek=True):
    """-to is an end *timestamp*; where -ss sits changes what that timestamp means."""
    if input_seek:
        cmd = [FFMPEG, "-y", "-v", "error", "-ss", str(start), "-i", str(src), "-to", str(end)]
    else:
        cmd = [FFMPEG, "-y", "-v", "error", "-i", str(src), "-ss", str(start), "-to", str(end)]
    return run(cmd + ["-c:v", "libx264", "-crf", "18", "-c:a", "aac", str(dst)])

Command	Duration
`ffmpeg -ss 1 -i in.mp4 -to 3 …`	3.00 s (1.0 → 4.0 s)
`ffmpeg -i in.mp4 -ss 1 -to 3 …`	2.00 s (1.0 → 3.0 s)

With -ss before -i, the output clock restarts at 0, so -to 3 behaves like -t 3. With -ss after -i, the timestamps stay on the source clock and -to 3 stops at 3.0 s. Output seeking decodes everything before the cut, so it's slower on long files. If you want an end time, the safest habit is to compute -t end-start yourself.

The tests

These are the checks I keep next to any trimming code. Each one encodes a fact measured above:

import pytest
import trim as T

CLIP = "inputs/vidu-q3-2.mp4"      # 30 fps, 5.108 s, keyframes at 0.0 and 2.2 s
ONE_GOP = "inputs/vidu-q3-1.mp4"   # 30 fps, 4.11 s, a single keyframe at 0.0 s
FPS = 30


@pytest.fixture(scope="module")
def out(tmp_path_factory):
    return tmp_path_factory.mktemp("trim")


def test_fixture_keyframes():
    assert T.keyframes(CLIP) == [0.0, 2.2]
    assert T.keyframes(ONE_GOP) == [0.0]


def test_stream_copy_starts_at_previous_keyframe(out):
    dst = out / "copy.mp4"
    assert T.trim_copy(CLIP, dst, 1.0, 2.0).returncode == 0
    first, _ = T.nearest_source_frame(CLIP, dst, search=range(0, 45))
    assert first == 0                         # asked for 1.0 s, got the 0.0 s keyframe
    assert T.frame_count(dst) > 2.0 * FPS + 20  # and about a second of extra video


def test_reencode_starts_on_requested_frame(out):
    dst = out / "exact.mp4"
    assert T.trim_exact(CLIP, dst, 1.0, 2.0).returncode == 0
    first, score = T.nearest_source_frame(CLIP, dst, search=range(20, 41))
    assert first == 30 and score < 1.0        # frame 30 = 1.000 s
    assert T.frame_count(dst) == 60
    assert abs(T.duration(dst) - 2.0) < 0.02


def test_audio_matches_video_after_exact_trim(out):
    dst = out / "exact-av.mp4"
    T.trim_exact(CLIP, dst, 1.0, 2.0)
    s = {x["codec_type"]: float(x["duration"]) for x in T.probe(dst)["streams"]}
    assert abs(s["audio"] - s["video"]) < 0.05


def test_snapping_makes_stream_copy_clean(out):
    start = T.snap_to_keyframe(CLIP, 2.5)
    assert start == 2.2
    dst = out / "snap.mp4"
    T.trim_copy(CLIP, dst, start, 2.0)
    first, score = T.nearest_source_frame(CLIP, dst, search=range(55, 80))
    assert first == 66 and score < 0.5       # 2.2 s * 30 fps
    assert abs(T.frame_count(dst) - 2.0 * FPS) <= 2


def test_single_keyframe_clip_cannot_be_copy_cut(out):
    dst = out / "one-gop.mp4"
    T.trim_copy(ONE_GOP, dst, 1.5, 1.0)
    first, _ = T.nearest_source_frame(ONE_GOP, dst, search=range(0, 60))
    assert first == 0                         # whole GOP kept from the start


@pytest.mark.parametrize("input_seek,expected", [(True, 3.0), (False, 2.0)])
def test_to_depends_on_where_ss_is(out, input_seek, expected):
    dst = out / f"to-{input_seek}.mp4"
    T.trim_to(CLIP, dst, 1.0, 3.0, input_seek=input_seek)
    assert abs(T.duration(dst) - expected) < 0.05

$ pytest -v test_trim.py
test_trim.py::test_fixture_keyframes PASSED
test_trim.py::test_stream_copy_starts_at_previous_keyframe PASSED
test_trim.py::test_reencode_starts_on_requested_frame PASSED
test_trim.py::test_audio_matches_video_after_exact_trim PASSED
test_trim.py::test_snapping_makes_stream_copy_clean PASSED
test_trim.py::test_single_keyframe_clip_cannot_be_copy_cut PASSED
test_trim.py::test_to_depends_on_where_ss_is[True-3.0] PASSED
test_trim.py::test_to_depends_on_where_ss_is[False-2.0] PASSED
8 passed in 15.14s

Which one to use

Exact frame matters (a cut on an action, a loop point, a clip that must be exactly N frames): re-encode.
Speed matters and the start can move: snap to a keyframe, then -c copy.
Never use a bare -ss … -c copy and assume the start is where you asked. Check the keyframes first.
If you need -to, know which side of -i your -ss is on, or use -t.

Limits

Results are for FFmpeg 9.0.1 with MP4/H.264 input. Older builds and other containers handle copy-trims differently (some write an edit list that hides the extra frames in some players but not others), which is exactly why the tests decode frames instead of trusting the command.
Frame matching by small grayscale images works when nearby frames differ. On a static shot, several frames will score the same, so check frame_count and timestamps instead.
Re-encoding changes bitrate and quality. Pin -crf and -preset in the function so batches stay consistent.

Generate a little extra, then trim

AI video tools give you whole seconds, and the best moment rarely starts at 0.0 s. A practical workflow is to generate slightly longer than you need and trim with the exact method above. On Voor's Vidu Q3 generator the duration slider runs from 1 to 16 seconds, and the default 5-second 360p draft was quoted at 61 credits on 3 November 2026. That's enough to pick the best two-second window with trim_exact and keep the tests green.

How to Add Subtitles to a Video With FFmpeg (and Time Them From the Audio)

Voor AI — Mon, 02 Nov 2026 00:36:28 +0000

There are two ways to add subtitles to a video with FFmpeg:

Soft subtitles: a separate text track the viewer can turn on or off. For MP4 that means converting the .srt to mov_text: ffmpeg -i in.mp4 -i subs.srt -map 0 -map 1 -c:v copy -c:a copy -c:s mov_text -metadata:s:s:0 language=eng out.mp4
Burned-in subtitles: the text is drawn into the pixels, so it shows everywhere, including players that ignore subtitle tracks: ffmpeg -i in.mp4 -vf "subtitles=subs.srt" -c:a copy out.mp4

Both commands are short. What goes wrong is everything around them: cue timings that drift from the speech, an .srt that MP4 refuses to carry, and an FFmpeg build that has no subtitles filter at all. This post builds the whole pipeline in Python, times the cues from the audio itself, and tests every step with pytest. All measurements are from FFmpeg 9.0.1 on macOS.

The test material

The audio is a real text-to-speech take: a three-step cooking lesson generated with Gemini TTS and published as an example on Voor's Gemini TTS page (lesson-flash.mp3, 16.36 s, 24 kHz mono MP3). Because it is TTS, the script is known exactly:

Step one: heat the oven to two hundred degrees.
Step two: while it warms, cut the vegetables into even pieces, so everything roasts at the same speed.
Step three: a little oil, a pinch of salt, and into the oven they go.

That is the ideal case for subtitles: you don't need speech recognition, only timing. The second fixture is a French museum announcement from the same page, used to test two subtitle languages in one file.

Step 1: Find the speech with silencedetect

silencedetect prints where silence starts and ends. Everything between is speech:

def speech_segments(audio, noise_db=-35, min_silence=0.35):
    log = run([FFMPEG, "-hide_banner", "-nostats", "-i", str(audio),
               "-af", f"silencedetect=noise={noise_db}dB:d={min_silence}", "-f", "null", "-"]).stderr
    starts = [float(x) for x in re.findall(r"silence_start: ([\d.]+)", log)]
    ends = [float(x) for x in re.findall(r"silence_end: ([\d.]+)", log)]
    total = duration(audio)
    segs, cursor = [], 0.0
    for s, e in zip(starts, ends + [total] * (len(starts) - len(ends))):
        if s - cursor > 0.05:
            segs.append((round(cursor, 3), round(s, 3)))
        cursor = e
    if total - cursor > 0.05:
        segs.append((round(cursor, 3), round(total, 3)))
    return segs

On the lesson this returns four segments for three sentences:

Segment	Start (s)	End (s)	Speech
1	0.000	3.239	Step one …
2	4.109	11.093	Step two …
3	11.595	12.198	"Step three:"
4	12.602	15.999	… into the oven they go.

The voice pauses for 0.40 s after "Step three:". Map segments to lines one-to-one and every cue after that point is wrong.

Step 2: Merge the shortest pauses until the counts match

Instead of tuning the silence threshold until it happens to work, merge across the shortest gaps until there is exactly one segment per subtitle line. If there are fewer segments than lines, fail loudly:

def fit_segments(segs, n):
    if len(segs) < n:
        raise ValueError(f"{len(segs)} speech segments for {n} lines")
    segs = list(segs)
    while len(segs) > n:
        gaps = [segs[i + 1][0] - segs[i][1] for i in range(len(segs) - 1)]
        i = gaps.index(min(gaps))
        segs[i:i + 2] = [(segs[i][0], segs[i + 1][1])]
    return segs

Then turn segments into cues with a small lead-in (80 ms) and tail (250 ms), clamped so a cue never runs past the audio or into the next cue:

def cues(lines, segs, total, lead=0.08, tail=0.25):
    segs = fit_segments(segs, len(lines))
    out = []
    for k, (text, (s, e)) in enumerate(zip(lines, segs)):
        start = max(0.0, s - lead)
        end = min(total, e + tail)
        if k + 1 < len(segs):
            end = min(end, segs[k + 1][0] - lead - 0.001)
        out.append((round(start, 3), round(end, 3), text))
    return out

The resulting SRT (note the comma before the milliseconds; SRT uses 00:00:03,489, WebVTT uses a dot):

1
00:00:00,000 --> 00:00:03,489
Step one: heat the oven to two hundred degrees.

2
00:00:04,029 --> 00:00:11,343
Step two: while it warms, cut the vegetables into even pieces, so everything roasts at the same speed.

3
00:00:11,515 --> 00:00:16,249
Step three: a little oil, a pinch of salt, and into the oven they go.

Step 3: Make the video, with an exact length

For the test, the picture is a plain 1280×720 background. The obvious command uses -shortest to stop when the audio ends. In my runs it didn't: the output had 18.43 s and 18.67 s of video over 16.36 s of audio. Passing the probed audio length with -t gave 16.367 s every time:

def base_video(audio, out, size="1280x720", color="0x1d2330", exact=True):
    length = ["-t", f"{duration(audio):.3f}"] if exact else ["-shortest"]
    run([FFMPEG, "-y", "-f", "lavfi", "-i", f"color=c={color}:s={size}:r=30",
         "-i", str(audio), *length, "-c:v", "libx264", "-pix_fmt", "yuv420p",
         "-c:a", "aac", "-b:a", "128k", "-movflags", "+faststart", str(out)])

If your subtitles end exactly when the speech ends, two extra seconds of silent picture is the first thing a viewer notices.

Step 4: Soft subtitles in MP4 need mov_text

The tempting command copies everything:

ffmpeg -i lesson.mp4 -i lesson.en.srt -map 0 -map 1 -c copy out.mp4

It fails, with exit code 234:

Could not find tag for codec subrip in stream #2, codec not currently supported in container
Could not write header (incorrect codec parameters ?): Invalid argument

MP4 can't store SubRip. Copy the audio and video, and convert only the subtitle stream to mov_text. Set a language tag on each track and mark the default one:

cmd += ["-c:v", "copy", "-c:a", "copy", "-c:s", "mov_text"]
for i, (_, lang, title) in enumerate(tracks):
    cmd += [f"-metadata:s:s:{i}", f"language={lang}", f"-metadata:s:s:{i}", f"title={title}"]
cmd += ["-disposition:s:0", "default", str(out)]

With the French museum take and an English translation timed to the same cues, ffprobe shows exactly what a player will offer:

stream|index=2|codec_name=mov_text|disposition:default=1|tag:language=fre
stream|index=3|codec_name=mov_text|disposition:default=0|tag:language=eng

(If you need SRT to survive untouched, use MKV instead: -c:s copy works there.)

Step 5: Burned-in subtitles need libass

Homebrew's default ffmpeg 9.0.1 bottle on this machine was built without libass, so it has no subtitles filter. The error message depends on how you write the filter, and one version is misleading:

$ ffmpeg -i lesson.mp4 -vf "subtitles=lesson.en.srt" ...
No option name near 'lesson.en.srt'
$ ffmpeg -i lesson.mp4 -vf "subtitles=filename=lesson.en.srt" ...
No such filter: 'subtitles'

The first looks like a quoting problem. It isn't. Check the build before you debug the command:

def has_filter(name, ffmpeg=FFMPEG):
    listing = run([ffmpeg, "-hide_banner", "-filters"]).stdout
    return re.search(rf"^\s*\S+\s+{re.escape(name)}\s", listing, re.M) is not None

The ffmpeg-full formula includes libass. The burn step refuses to run without it, and escapes the path so colons and quotes in file names don't break the filter graph:

def burn(video, srt, out, ffmpeg=FFMPEG, style="FontName=Arial,FontSize=26,Outline=2,MarginV=40"):
    if not has_filter("subtitles", ffmpeg):
        raise RuntimeError(f"{ffmpeg} has no 'subtitles' filter (built without libass)")
    path = str(srt).replace("\\", "/").replace(":", "\\:").replace("'", "\\'")
    run([ffmpeg, "-y", "-i", str(video), "-vf", f"subtitles='{path}':force_style='{style}'",
         "-c:v", "libx264", "-crf", "18", "-pix_fmt", "yuv420p", "-c:a", "copy", str(out)])

Step 6: Prove the text is there, and only when it should be

A burned subtitle is just pixels, so test pixels. Decode one frame to grayscale and average the lower band where subtitles sit:

def bottom_band_luma(path, t, band=(0.78, 0.95)):
    raw = subprocess.run([FFMPEG, "-v", "error", "-ss", str(t), "-i", str(path), "-frames:v", "1",
                          "-f", "rawvideo", "-pix_fmt", "gray", "-"], capture_output=True).stdout
    ...  # mean of rows y0..y1

Time	Plain video	Burned
1.5 s (cue 1)	35.0	41.86
3.6 s (pause between cues)	35.0	35.0
7.5 s (cue 2)	35.0	45.44
14.0 s (cue 3)	35.0	42.84

Text appears during each cue and the band is identical to the plain video in the pause. That second check catches cues that overlap or run long.

The tests

Ten pytest checks run against the real TTS audio in about two seconds:

def test_pause_inside_a_sentence_makes_an_extra_segment():
    segs = S.speech_segments(LESSON)
    assert len(segs) == 4
    merged = S.fit_segments(segs, 3)
    assert merged[2][0] == segs[2][0] and merged[2][1] == segs[3][1]

def test_srt_cannot_be_stream_copied_into_mp4(work):
    r = S.run([S.FFMPEG, "-y", "-loglevel", "error", "-i", str(work["base"]), "-i", str(work["srt"]),
               "-map", "0", "-map", "1", "-c", "copy", str(work["d"] / "bad.mp4")])
    assert r.returncode != 0 and "subrip" in r.stderr

def test_burned_text_appears_only_during_cues(work):
    out = S.burn(work["base"], work["srt"], work["d"] / "burned.mp4", ffmpeg=FULL)
    gap = (work["cues"][0][1] + work["cues"][1][0]) / 2
    plain = S.bottom_band_luma(work["base"], 1.5)
    assert S.bottom_band_luma(out, 1.5) > plain + 3
    assert abs(S.bottom_band_luma(out, gap) - plain) < 1
    assert S.subtitle_streams(out) == []

The others cover cue order and overlap, the SRT timestamp format, the -shortest overshoot, the mov_text track and its language tag, two language tracks with the right default, and the clear error when libass is missing.

test_subtitles.py::test_pause_inside_a_sentence_makes_an_extra_segment PASSED
test_subtitles.py::test_too_few_segments_is_an_error PASSED
test_subtitles.py::test_cues_are_ordered_and_inside_the_audio PASSED
test_subtitles.py::test_srt_uses_comma_milliseconds PASSED
test_subtitles.py::test_shortest_can_overshoot_but_exact_length_matches PASSED
test_subtitles.py::test_srt_cannot_be_stream_copied_into_mp4 PASSED
test_subtitles.py::test_soft_track_is_mov_text_with_language PASSED
test_subtitles.py::test_two_language_tracks_first_is_default PASSED
test_subtitles.py::test_burn_needs_libass PASSED
test_subtitles.py::test_burned_text_appears_only_during_cues PASSED
10 passed in 2.14s

Limits

Timing from silence only works when you already have the exact words and the speaker pauses between lines. For recorded speech with no script you need speech recognition first.
fit_segments merges the shortest pauses. If the speaker pauses longer inside a sentence than between two sentences, it will merge the wrong pair; the cue-order test won't catch that, so check one or two cues by ear.
The -35 dB / 0.35 s thresholds suit clean TTS. Noisy recordings need different values.
Long lines (like step two) wrap to three lines when burned in. Split them in the script if that matters.

Where the script comes from

This approach works best when you write the narration first and generate the voice from it, because then the subtitle text is exact by construction. The lesson audio here was made that way: on Voor's Gemini TTS generator, the "Step-by-step lesson" script shape with voice Kore, quoted at 4 credits for this script (credits count the spoken text; style instructions are free). Generate the take, keep the script, and the code above does the rest.

How to Create a Video From Images With FFmpeg (Without Stretching Portrait Photos)

Voor AI — Sun, 01 Nov 2026 00:21:20 +0000

To create a video from images with FFmpeg, fit every image into one fixed canvas before it reaches the encoder: scale with force_original_aspect_ratio=decrease and force_divisible_by=2, then pad (or a blurred fill) to exactly 1920×1080, convert to limited-range yuv420p, hold each still with -loop 1 plus trim, and join the stills with xfade. Then check the result with ffprobe: frame count, size, SAR, pixel format and a duration of n × hold − (n − 1) × fade.

The usual one-liner skips the fitting step. With a folder of mixed photos it either refuses to start or, worse, exits 0 and quietly stretches every portrait photo. Below is what that looks like on three real stills, a ~90-line Python wrapper that does it properly, and 12 pytest checks that catch the failures.

The test images

Three stills from the public examples on Voor's photo-to-video page, chosen because their shapes disagree:

File	Size	Problem it triggers
`landscape-input.jpg`	1280×853	odd height
`portrait-input.jpg`	1080×1920	9:16 into a 16:9 canvas
`product-input.jpg`	1080×1920	9:16, label text that shows distortion

All three are baseline JPEGs, which means full-range yuvj420p once decoded. That matters later.

What the one-liner actually does

Straight to libx264, no scaling:

ffmpeg -framerate 1/3 -i %02d.jpg -r 30 -c:v libx264 -pix_fmt yuv420p naive.mp4
# [libx264] height not divisible by 2 (1280x853)

4:2:0 chroma needs even width and height. So people add scale=-2:1080:

ffmpeg -framerate 1/3 -i %02d.jpg -vf scale=-2:1080 -r 30 \
  -c:v libx264 -pix_fmt yuv420p naive.mp4   # exit code 0

It succeeds, and ffprobe on the output says:

Check	naive.mp4
frame size	1620×1080 for all 270 frames
SAR	2560:2559
pix_fmt / range	yuvj420p / pc
duration	9.0 s, hard cuts

scale=-2:1080 produced 1620×1080 for the landscape and 608×1080 for the two portraits. An encoder cannot change size mid-stream, so FFmpeg's auto-inserted scaler forced every later frame back to the first frame's 1620×1080: the portrait and the product tube are stretched 2.667× wide. No warning is printed. The odd SAR and the full-range yuvj420p tag come along for free, and some players and upload pipelines misread both.

A filter graph that fits every image

The fix is to give each input its own chain and make all chains end in the same size, rate, range and format, so the joins never have to guess:

Fit, don't force. scale=1920:1080:force_original_aspect_ratio=decrease:force_divisible_by=2 keeps the shape and guarantees even sides.
Fill the rest. pad gives black bars. The blur mode puts a cover-cropped, boxblurred copy of the same image behind it, which is what most vertical-photo slideshows use.
Normalize. setsar=1, out_range=tv, format=yuv420p, fps=30.
Bound each still. -loop 1 is infinite; trim=duration=3 plus setpts=PTS-STARTPTS makes it a 3-second clip.
Join with xfade. Its offset is on the output timeline, so the k-th fade starts at k × (hold − fade), not k × hold.

"""Create a video from still images with FFmpeg.

build_slideshow() turns any mix of image sizes into one H.264 MP4:
  - every image is fitted into the output canvas without stretching
    (scale with force_original_aspect_ratio, then pad or a blurred fill),
  - every intermediate size is forced to even numbers (yuv420p needs that),
  - each still is held for `hold` seconds and joined with xfade crossfades,
  - JPEG full-range input is converted to limited (TV) range yuv420p,
  - an optional silent AAC track keeps players and upload checks happy.
expected_duration() and xfade_offsets() are the arithmetic the filter graph
depends on; the tests compare them with what ffprobe measures.
"""
from __future__ import annotations

import json
import os
import subprocess
from fractions import Fraction
from pathlib import Path

FFMPEG = os.environ.get("FFMPEG", "ffmpeg")
FFPROBE = os.environ.get("FFPROBE", "ffprobe")


def expected_duration(n: int, hold: float, fade: float) -> float:
    """n stills of `hold` seconds joined by n-1 crossfades of `fade` seconds."""
    if n < 1:
        raise ValueError("need at least one image")
    if n > 1 and not 0 < fade < hold:
        raise ValueError("fade must be shorter than hold")
    return n * hold - (n - 1) * fade


def xfade_offsets(n: int, hold: float, fade: float) -> list[float]:
    """Offset of each xfade, measured on the already-joined output timeline."""
    return [round(k * (hold - fade), 6) for k in range(1, n)]


def fit_chain(index: int, width: int, height: int, fps: int, hold: float, fill: str) -> str:
    """Filter chain for one looped still -> [v{index}] at exactly width x height."""
    common = f"fps={fps},trim=duration={hold},setpts=PTS-STARTPTS,format=yuv420p"
    fg = (f"scale={width}:{height}:force_original_aspect_ratio=decrease:"
          f"force_divisible_by=2:out_range=tv,setsar=1")
    if fill == "pad":
        return (f"[{index}:v]{fg},pad={width}:{height}:(ow-iw)/2:(oh-ih)/2:color=black,"
                f"{common}[v{index}]")
    if fill == "blur":
        return (f"[{index}:v]split=2[bg{index}][fg{index}];"
                f"[bg{index}]scale={width}:{height}:force_original_aspect_ratio=increase,"
                f"crop={width}:{height},scale=out_range=tv,boxblur=20:2,setsar=1[bgb{index}];"
                f"[fg{index}]{fg}[fgs{index}];"
                f"[bgb{index}][fgs{index}]overlay=(W-w)/2:(H-h)/2,{common}[v{index}]")
    raise ValueError(f"unknown fill {fill!r}")


def build_command(images, out, *, width=1920, height=1080, fps=30, hold=3.0,
                  fade=0.5, fill="blur", audio=True) -> list[str]:
    images = [str(p) for p in images]
    n = len(images)
    total = expected_duration(n, hold, fade)
    if width % 2 or height % 2:
        raise ValueError("yuv420p output needs even width and height")
    cmd = [FFMPEG, "-v", "error", "-y"]
    for img in images:
        # -loop 1 makes an endless stream; trim inside the graph bounds it.
        cmd += ["-loop", "1", "-framerate", str(fps), "-t", str(hold), "-i", img]
    if audio:
        cmd += ["-f", "lavfi", "-t", str(total), "-i", "anullsrc=r=48000:cl=stereo"]
    chains = [fit_chain(i, width, height, fps, hold, fill) for i in range(n)]
    last = "v0"
    for k, off in enumerate(xfade_offsets(n, hold, fade), start=1):
        label = f"x{k}" if k < n - 1 else "vout"
        chains.append(f"[{last}][v{k}]xfade=transition=fade:duration={fade}:offset={off}[{label}]")
        last = label
    if n == 1:
        chains[-1] = chains[-1].rsplit("[v0]", 1)[0] + "[vout]"
    cmd += ["-filter_complex", ";".join(chains), "-map", "[vout]"]
    if audio:
        cmd += ["-map", f"{n}:a", "-c:a", "aac", "-b:a", "128k"]
    cmd += ["-c:v", "libx264", "-preset", "medium", "-crf", "20", "-pix_fmt", "yuv420p",
            "-color_range", "tv", "-r", str(fps), "-movflags", "+faststart", "-t", str(total), str(out)]
    return cmd


def build_slideshow(images, out, **kw) -> dict:
    subprocess.run(build_command(images, out, **kw), check=True)
    return probe(out)


def probe(path) -> dict:
    data = json.loads(subprocess.run(
        [FFPROBE, "-v", "error", "-count_frames", "-show_format", "-show_streams", "-of", "json",
         str(path)], check=True, capture_output=True, text=True).stdout)
    v = next(s for s in data["streams"] if s["codec_type"] == "video")
    a = next((s for s in data["streams"] if s["codec_type"] == "audio"), None)
    return {
        "duration": float(data["format"]["duration"]),
        "vduration": float(v["duration"]),
        "frames": int(v["nb_read_frames"]),
        "width": int(v["width"]), "height": int(v["height"]),
        "fps": str(Fraction(v["r_frame_rate"])), "sar": v.get("sample_aspect_ratio", "1:1"),
        "pix_fmt": v["pix_fmt"], "vcodec": v["codec_name"],
        "acodec": a["codec_name"] if a else None,
        "aduration": float(a["duration"]) if a else None,
    }


if __name__ == "__main__":
    import argparse
    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    ap.add_argument("images", nargs="+")
    ap.add_argument("-o", "--out", default="slideshow.mp4")
    ap.add_argument("--hold", type=float, default=3.0)
    ap.add_argument("--fade", type=float, default=0.5)
    ap.add_argument("--fill", choices=["blur", "pad"], default="blur")
    ap.add_argument("--size", default="1920x1080")
    ap.add_argument("--fps", type=int, default=30)
    ap.add_argument("--no-audio", action="store_true")
    a = ap.parse_args()
    w, h = (int(x) for x in a.size.split("x"))
    print(json.dumps(build_slideshow(a.images, a.out, width=w, height=h, fps=a.fps, hold=a.hold,
                                     fade=a.fade, fill=a.fill, audio=not a.no_audio), indent=1))

Run it:

python3 images_to_video.py landscape-input.jpg portrait-input.jpg product-input.jpg \
  --fill blur --hold 3 --fade 0.5 -o slideshow.mp4

Measured on the three stills:

Check	slideshow (blur)	slideshow (pad)
size / SAR	1920×1080 / 1:1	1920×1080 / 1:1
pix_fmt / range	yuv420p / tv	yuv420p / tv
frames	240	240
duration	8.000 s	8.000 s
audio	silent AAC, 8.000 s	none

8.000 s is 3 × 3 − 2 × 0.5. If you write offset=3 and offset=6 instead, the graph still renders, but the fades land in the wrong place and the total length drifts. The arithmetic test below catches that before you look at any video.

Tests that catch the silent failures

def test_naive_scale_stretches_later_images(tmp_path):
    ...
    # Every frame is locked to the first image's size: 9:16 stills get stretched 2.66x.
    assert frame_sizes(out) == {(1620, 1080)}

def test_output_contract(blur):
    out, p = blur
    assert (p["width"], p["height"], p["fps"], p["sar"]) == (1920, 1080, "30", "1:1")
    assert p["pix_fmt"] == "yuv420p" and p["frames"] == 240
    assert abs(p["vduration"] - m.expected_duration(3, 3.0, 0.5)) < 1 / 30
    assert frame_sizes(out) == {(1920, 1080)}

def test_pad_keeps_portrait_aspect(pad):
    out, p = pad
    # Portrait hold is 3.0-5.0 s. 1080x1920 fitted into 1080 high is 608 wide,
    # so the outer columns must be black bars.
    px = gray(out, 4.0)
    assert col_mean(px, 0, 18) < 6 and col_mean(px, 46, 64) < 6
    assert col_mean(px, 26, 38) > 40

The full file also checks: the odd-height failure, the TV-range tag, audio length against video length, the landscape bars, segment order (each segment's fitted region is matched against all three sources and must be closest to the right one), the crossfade midpoint lying between its neighbours, rejection of odd canvases, and a single-image run.

test_images_to_video.py::test_expected_duration_and_offsets PASSED
test_images_to_video.py::test_naive_image2_fails_on_odd_height PASSED
test_images_to_video.py::test_naive_scale_stretches_later_images PASSED
test_images_to_video.py::test_output_contract PASSED
test_images_to_video.py::test_limited_range_output PASSED
test_images_to_video.py::test_silent_audio_matches_video PASSED
test_images_to_video.py::test_pad_keeps_portrait_aspect PASSED
test_images_to_video.py::test_landscape_is_full_width PASSED
test_images_to_video.py::test_segments_follow_input_order PASSED
test_images_to_video.py::test_crossfade_is_a_blend PASSED
test_images_to_video.py::test_odd_canvas_rejected PASSED
test_images_to_video.py::test_single_image PASSED
12 passed in 6.08s

(FFmpeg 9.0.1 on macOS arm64, Python 3.13, pytest 9.2.)

Checklist before you upload

ffprobe -count_frames: one frame size only, SAR 1:1, yuv420p, color_range=tv.
Duration equals n × hold − (n − 1) × fade to within one frame.
Look at one frame from each portrait still. Round faces and straight label text mean no stretch.
If the target needs an audio stream, add a silent track as long as the video.
Big phone photos: downscale once before the graph. Decoding a 48 MP JPEG for every frame is slow.

Limits

This is a slideshow: the pixels inside each photo never move, apart from the crossfades. A slow zoompan push adds some motion, but it jitters unless you upscale first, and it still won't move hair, clouds or water. If you want that, you need a generative image-to-video model. For example, the Voor photo-to-video generator takes a first frame (and an optional last frame), a motion prompt, aspect ratio, 480p/720p/1080p and duration. Its defaults (Seedance 1.5 Pro, 480p, 5 s, audio on) showed 44 credits when I checked; it needs sign-in and credits. Run its output through the same ffprobe checks before it goes into this slideshow.

How to Merge Videos With FFmpeg (and Catch the Resolution Change Stream Copy Hides)

Voor AI — Sat, 31 Oct 2026 00:35:56 +0000

Short answer: if every clip has identical stream parameters, use the concat demuxer with -c copy. It is instant and lossless. If anything differs (resolution, frame rate, SAR, pixel format, audio sample rate or channel count), use the concat filter and normalize each input first. The trap is that the demuxer does not refuse mismatched clips: it writes a file whose header looks fine while half the frames have a different size. This post shows that with real clips, gives a small Python wrapper that picks the right path, and pins the behaviour down with 15 pytest checks.

The test material is six real clips generated with PixVerse V6 (public examples from Voor's PixVerse V6 page): a 1920×1080 night market, a 720×1280 vertical underwater shot, a 1280×560 multi-shot bike ride and two 960×960 fox clips. All are H.264 High + AAC at 24 fps. That sounds compatible. It is not.

Real clips in, one geometry out: the vertical clip is pillarboxed, the 21:9 clip letterboxed, and the durations add up exactly.

1. The two ways FFmpeg concatenates

	concat demuxer (`-f concat`)	concat filter (`-filter_complex concat`)
Re-encodes	No, with `-c copy`	Yes, once
Speed on these clips	0.06 s for two 5 s clips	2.2 s for 18 s of output (CRF 20, preset medium)
Requires matching streams	Yes, all of them	No, if you scale/pad/resample first
Fails loudly on mismatch	No	Yes (filter graph errors)

The demuxer needs a list file:

printf "file '%s'\n" "$PWD/a.mp4" "$PWD/b.mp4" > list.txt
ffmpeg -f concat -safe 0 -i list.txt -c copy -movflags +faststart merged.mp4

-safe 0 allows absolute paths. Inside the list, a single quote in a filename must be written as '\''.

2. What stream copy does with mismatched clips

Merge the 1920×1080 night market and the 720×1280 vertical clip with -c copy and ask ffprobe about the stream:

ffprobe -v error -show_entries stream=width,height,nb_frames -of csv=p=0 naive.mp4
# 1920,1080,242
ffmpeg -v error -i naive.mp4 -f null -   # prints nothing: zero decode errors

Everything looks fine. Now look at the frames instead of the header:

ffprobe -v error -select_streams v:0 -show_entries frame=width,height -of csv=p=0 naive.mp4 | sort | uniq -c
#  121 1920,1080
#  121 720,1280

The second half is still 720×1280. The muxer simply concatenated two H.264 streams with different parameter sets. What a player does with that is up to the player; nothing in the pipeline warned you.

The header says 1920×1080. The frames say otherwise.

Even the two fox clips, which share size, codec and frame rate, are not copy-safe: one has 32,000 Hz audio and the other 44,100 Hz.

3. Check compatibility before you copy

Compare the fields that matter against the first clip and refuse stream copy if any differ:

COPY_KEYS = ("vcodec", "profile", "width", "height", "pix_fmt", "fps", "sar",
             "acodec", "sample_rate", "channels")

def copy_mismatches(paths) -> list[str]:
    """Human-readable list of parameters that differ from the first clip."""
    infos = [probe(p) for p in paths]
    first = infos[0]
    problems = []
    for p, info in zip(paths[1:], infos[1:]):
        for k in COPY_KEYS:
            if info[k] != first[k]:
                problems.append(f"{Path(p).name}: {k}={info[k]} (first clip has {first[k]})")
    return problems

On the real clips:

copy_mismatches([fox, extend])  -> ["extend-snow-fox.mp4: sample_rate=44100 (first clip has 32000)"]
copy_mismatches([market, kelp]) -> ["t2v-kelp-diver.mp4: width=720 (first clip has 1920)",
                                    "t2v-kelp-diver.mp4: height=1280 (first clip has 1080)"]

probe() is a thin wrapper around ffprobe -show_streams -show_format -of json:

def probe(path: str | Path) -> dict:
    out = subprocess.run(
        [FFPROBE, "-v", "error", "-show_format", "-show_streams", "-of", "json", str(path)],
        check=True, capture_output=True, text=True).stdout
    data = json.loads(out)
    v = next(s for s in data["streams"] if s["codec_type"] == "video")
    a = next((s for s in data["streams"] if s["codec_type"] == "audio"), None)
    return {
        "duration": float(data["format"]["duration"]),
        "vduration": float(v.get("duration", data["format"]["duration"])),
        "aduration": float(a["duration"]) if a and "duration" in a else None,
        "vcodec": v["codec_name"], "profile": v.get("profile"),
        "width": int(v["width"]), "height": int(v["height"]),
        "pix_fmt": v.get("pix_fmt"),
        "fps": str(Fraction(v["r_frame_rate"])),
        "sar": v.get("sample_aspect_ratio", "1:1"),
        "frames": int(v["nb_frames"]) if v.get("nb_frames") else None,
        "acodec": a["codec_name"] if a else None,
        "sample_rate": int(a["sample_rate"]) if a else None,
        "channels": int(a["channels"]) if a else None,
    }

concat_copy() writes the list file shown in section 1 (with the quote escaping) and runs the -c copy command.

4. Normalize, then concatenate with the filter

Each input gets the same geometry and timing before concat sees it. force_original_aspect_ratio=decrease plus pad letterboxes or pillarboxes instead of stretching, setsar=1 removes odd pixel aspect ratios, fps fixes the frame rate, and aresample + aformat put every audio track on one rate and layout. A clip with no audio gets exactly as much silence as it has video, otherwise concat refuses the graph.

def concat_normalize(paths, dst, width=1920, height=1080, fps=24,
                     sample_rate=48000, crf=20, preset="medium") -> dict:
    infos = [probe(p) for p in paths]
    args, chains, pads = [], [], []
    for i, (p, info) in enumerate(zip(paths, infos)):
        args += ["-i", str(p)]
    n = len(paths)
    for i, info in enumerate(infos):
        chains.append(
            f"[{i}:v]scale={width}:{height}:force_original_aspect_ratio=decrease,"
            f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2:color=black,setsar=1,"
            f"fps={fps},format=yuv420p[v{i}]")
        if info["acodec"]:
            chains.append(f"[{i}:a]aresample={sample_rate},"
                          f"aformat=sample_fmts=fltp:channel_layouts=stereo[a{i}]")
        else:  # silent clip: generate exactly as much silence as it has video
            chains.append(f"anullsrc=r={sample_rate}:cl=stereo,"
                          f"atrim=duration={info['vduration']:.6f}[a{i}]")
        pads.append(f"[v{i}][a{i}]")
    graph = ";".join(chains) + ";" + "".join(pads) + f"concat=n={n}:v=1:a=1[v][a]"
    subprocess.run([FFMPEG, "-v", "error", "-y", *args, "-filter_complex", graph,
                    "-map", "[v]", "-map", "[a]",
                    "-c:v", "libx264", "-preset", preset, "-crf", str(crf),
                    "-c:a", "aac", "-b:a", "160k", "-movflags", "+faststart", str(dst)],
                   check=True)
    return probe(dst)

The three mixed clips go in as 1920×1080, 720×1280 and 1280×560 and come out as one 1920×1080, 24 fps, SAR 1:1 file: 435 frames, all 1920×1080, video duration 18.125 s, exactly the sum of the inputs, audio 18.073 s at 48 kHz.

Finally, pick the path automatically:

def merge(paths, dst, **kw):
    if not copy_mismatches(paths):
        return "copy", concat_copy(paths, dst)
    return "normalize", concat_normalize(paths, dst, **kw)

5. Tests that pin the behaviour

The suite runs against the real fixtures (15 passed in 7.5 s, pytest 9.2, FFmpeg 9.0.1):

def test_naive_copy_of_mismatched_clips_hides_a_resolution_change(tmp_path):
    out = m.concat_copy([MARKET, KELP], tmp_path / "naive.mp4")
    assert (out["width"], out["height"]) == (1920, 1080)       # the header looks fine...
    assert frame_sizes(tmp_path / "naive.mp4") == {(1920, 1080), (720, 1280)}  # ...the frames are not


def test_vertical_clip_is_pillarboxed_not_stretched(merged):
    dst, _ = merged
    t = m.probe(MARKET)["vduration"] + 2.5          # middle of the kelp-diver segment
    assert column_mean(dst, t, 0, 40) < 6           # left bar is black
    assert column_mean(dst, t, 152, 192) < 6        # right bar is black
    assert column_mean(dst, t, 80, 112) > 20        # picture in the middle


def test_audio_stays_in_sync(merged):
    _, out = merged
    assert out["sample_rate"] == 48000 and out["channels"] == 2
    assert abs(out["aduration"] - out["vduration"]) < 0.1

The other tests cover: identical clips report no mismatch, sample rate alone blocks copy, copy of identical clips keeps every frame, merge() picks copy only when safe, output has one geometry, duration equals the sum of inputs, the 21:9 clip is letterboxed, segments stay in order, a silent clip gets matching silence, and a filename with an apostrophe and parentheses survives the list file.

6. Pitfalls checklist

Never trust the stream header after -c copy. Check per-frame sizes with -show_entries frame=width,height if the inputs came from different sources.
Audio counts. Different sample rates or channel layouts break copy-merges just as resolution does.
Pick the target size deliberately. Letterboxing a 9:16 clip into 16:9 wastes most of the frame. For a vertical deliverable, set width=1080, height=1920 instead.
Variable frame rate phone clips: the fps filter fixes the output rate; check lip-sync on talking clips afterwards.
Re-encoding is a quality step. Use CRF 18–20 for an intermediate, then compress the final file separately.
Timestamps: if a stream-copied merge plays with a pause at the joins, the inputs probably start with non-zero timestamps; re-encode them or remux each with -avoid_negative_ts make_zero first.

Getting test clips

To reproduce mismatches like these you need clips with different shapes and audio. The fixtures here are PixVerse V6 outputs. If you want fresh ones, the PixVerse V6 generator on Voor lets you set aspect ratio (8 shapes from 21:9 to 9:16), resolution (360p to 1080p), duration and audio per clip. The form quoted 79 credits for a 5 s 16:9 720p clip without audio and 105 with audio on October 31, 2026. Generate a 16:9 and a 9:16 clip with audio and you have a ready-made test pair for copy_mismatches(): make a mismatched test pair with PixVerse V6.

How to Compress a Video With FFmpeg

Voor AI — Fri, 30 Oct 2026 02:20:06 +0000

To compress a video with FFmpeg, re-encode it to H.264 with a constant rate factor: ffmpeg -i in.mp4 -c:v libx264 -crf 28 -preset medium -c:a aac -b:a 128k -movflags +faststart out.mp4. CRF picks the quality and lets the size float. If you need a hard size limit instead (an upload cap, an email attachment), use a two-pass encode at a computed bitrate, then measure the file and retry, because the math alone overshot in my tests.

The surprise: the "visually lossless" CRF 18 that many guides recommend made every one of my five test clips bigger than the original, by 11–30%. I measured size and SSIM for every setting below on real 6-second 1080p clips, and the code ships with 26 pytest checks.

The test clips

Five public example clips from LTX 2.5 (Lightricks' video model, as published on Voor's model page): about 6.1 s each, H.264 + AAC, 25 fps, three at 1920×1080, one at 1280×720 and one portrait 1080×1920, 2.0–4.8 MB. Generated clips like these are already fairly well compressed, which is exactly the case where naive settings backfire.

ffprobe -v error -show_entries stream=codec_name,width,height,r_frame_rate:format=duration,size \
  -of compact l-lighthouse-dolly.mp4
# stream|codec_name=h264|width=1920|height=1080|r_frame_rate=25/1
# stream|codec_name=aac
# format|duration=6.120000|size=2322248

All numbers are from FFmpeg 9.0.1 with libx264, on a laptop.

1. Quality-targeted: pick a CRF

ffmpeg -i in.mp4 -c:v libx264 -preset medium -crf 28 -pix_fmt yuv420p \
  -c:a aac -b:a 128k -movflags +faststart out.mp4

Lower CRF = higher quality and a larger file; on these clips each +5 cut the size by roughly 35–45%. Here is what each CRF did to the size (percent of the original file) and to SSIM against the original (1.0 = identical):

Clip	CRF 18	CRF 23	CRF 28	CRF 33	CRF 38
lighthouse dolly 1080p	123.5% · 0.9965	76.6% · 0.9942	46.6% · 0.9904	29.3% · 0.9841	19.7% · 0.9737
lighthouse jib 1080p	130.2% · 0.9978	82.6% · 0.9963	49.1% · 0.9939	31.0% · 0.9900	20.8% · 0.9837
speaker 1080p	114.8% · 0.9972	74.5% · 0.9950	46.6% · 0.9914	30.1% · 0.9855	20.1% · 0.9764
cabin 720p	127.6% · 0.9933	73.8% · 0.9874	41.0% · 0.9730	23.6% · 0.9493	14.7% · 0.9134
dandelion portrait	110.7% · 0.9961	69.1% · 0.9927	40.1% · 0.9872	23.7% · 0.9783	14.5% · 0.9638

What this tells you:

CRF 18 is not a compression setting for files that are already H.264. It spends more bits than the source did, so the "compressed" file grows. Re-encoding never restores detail that is already gone.
CRF 23 (the libx264 default) saved 17–31%.
CRF 28 roughly halved every clip (40–49%) and stayed at SSIM 0.973–0.994. This is a good starting point for sharing.
CRF 38 is about a fifth of the size, and you can see it.

Look at the spray: CRF 28 is hard to tell from the source at 2× zoom; CRF 38 flattens the foam texture. SSIM agrees with the eye here, and the cabin clip (lowest scores) is the one with the most fine texture per pixel.

Presets: speed, not quality

Same clip, CRF 23, only the preset changed:

Preset	Size	Encode time	SSIM
veryfast	1.63 MB	0.66 s	0.9930
medium	1.78 MB	1.51 s	0.9942
slow	1.69 MB	2.11 s	0.9941

At a fixed CRF, a slower preset does not reliably give a smaller file: here veryfast produced the smallest file but also the lowest SSIM. The preset trades encode time for compression efficiency, so compare presets at a fixed size or quality, not a fixed CRF. For a one-off clip, medium is fine; for batch jobs, veryfast was 2.3× faster.

2. Size-targeted: two-pass to a byte budget

When the limit is "under 1 MB", compute the bitrate from the budget:

video kb/s = budget_bytes × 8 / 1000 / duration × (1 − overhead) − audio kb/s

# 1 MB budget for a 6.12 s clip with 96 kb/s audio -> about 1,159 kb/s video
ffmpeg -y -i in.mp4 -c:v libx264 -preset medium -b:v 1159k -pass 1 -an -f mp4 /dev/null
ffmpeg -i in.mp4 -c:v libx264 -preset medium -b:v 1159k -pass 2 \
  -c:a aac -b:a 96k -movflags +faststart out.mp4

The catch: on a 6-second clip, the 0.5 MB first try came out at 522,119 bytes, over the budget. Rate control on very short clips does not hit the average exactly, so the code measures the result and, if it is over, scales the bitrate down by the overshoot (plus 2%) and encodes again:

Budget	Result	Attempts	Video kb/s	SSIM
0.5 MB	496,676 bytes	2	498	0.9771
1 MB	980,973 bytes	2	1131	0.9896
2 MB	1,963,793 bytes	1	2413	0.9949

The test suite asserts each result is under the budget and uses at least 80% of it, so the retry can't just throw quality away.

3. Downscale when the screen is small

ffmpeg -i in.mp4 -vf "scale=-2:'min(720,ih)'" -c:v libx264 -crf 28 \
  -c:a aac -b:a 128k -movflags +faststart out-720p.mp4

-2 keeps the aspect ratio and forces an even width (yuv420p needs even dimensions). min(720,ih) means a clip that is already smaller is never upscaled. The 1080p lighthouse clip at 720p was 899,750 bytes, and when scaled back up and compared against the original 1080p clip it still scored SSIM 0.9904.

4. `+faststart` for anything on the web

-movflags +faststart moves the moov index in front of the media data, so a browser can start playing before the download finishes. A plain ffmpeg -i in.mp4 -c copy out.mp4 writes moov at the end. The checker in the code walks the top-level MP4 boxes to prove which one comes first.

The code

import json, os, re, subprocess, tempfile
from pathlib import Path

def probe(path):
    out = subprocess.run(["ffprobe", "-v", "error", "-show_format", "-show_streams",
                          "-of", "json", str(path)], check=True, capture_output=True, text=True).stdout
    d = json.loads(out)
    v = next(s for s in d["streams"] if s["codec_type"] == "video")
    a = next((s for s in d["streams"] if s["codec_type"] == "audio"), None)
    return {"duration": float(d["format"]["duration"]), "size": int(d["format"]["size"]),
            "width": int(v["width"]), "height": int(v["height"]), "vcodec": v["codec_name"],
            "pix_fmt": v.get("pix_fmt"), "acodec": a["codec_name"] if a else None}

def _scale(max_height):
    return ["-vf", f"scale=-2:'min({max_height},ih)'"] if max_height else []

def compress_crf(src, dst, crf=23, preset="medium", max_height=None, audio_kbps=128):
    subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", str(src), *_scale(max_height),
                    "-c:v", "libx264", "-preset", preset, "-crf", str(crf), "-pix_fmt", "yuv420p",
                    "-c:a", "aac", "-b:a", f"{audio_kbps}k", "-movflags", "+faststart", str(dst)],
                   check=True)
    return probe(dst)

def video_kbps_for_budget(target_bytes, duration, audio_kbps=96, overhead=0.04):
    video = target_bytes * 8 / 1000 / duration * (1 - overhead) - audio_kbps
    if video < 50:
        raise ValueError(f"budget too small: {video:.0f} kb/s left for video")
    return int(video)

def _two_pass(src, dst, vk, preset, scale, audio_kbps):
    with tempfile.TemporaryDirectory() as tmp:
        common = [*scale, "-c:v", "libx264", "-preset", preset, "-b:v", f"{vk}k",
                  "-pix_fmt", "yuv420p", "-passlogfile", str(Path(tmp) / "pass")]
        subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", str(src), *common,
                        "-pass", "1", "-an", "-f", "mp4", os.devnull], check=True)
        subprocess.run(["ffmpeg", "-v", "error", "-y", "-i", str(src), *common,
                        "-pass", "2", "-c:a", "aac", "-b:a", f"{audio_kbps}k",
                        "-movflags", "+faststart", str(dst)], check=True)

def compress_to_size(src, dst, target_bytes, preset="medium", max_height=None,
                     audio_kbps=96, max_attempts=3):
    vk = video_kbps_for_budget(target_bytes, probe(src)["duration"], audio_kbps)
    for attempt in range(1, max_attempts + 1):
        _two_pass(src, dst, vk, preset, _scale(max_height), audio_kbps)
        info = probe(dst)
        if info["size"] <= target_bytes:
            return {**info, "video_kbps": vk, "attempts": attempt}
        vk = int(vk * target_bytes / info["size"] * 0.98)   # shrink by the overshoot + 2%
    raise RuntimeError(f"still {info['size']} > {target_bytes} after {max_attempts} attempts")

def ssim(reference, distorted):
    ref = probe(reference)
    r = subprocess.run(["ffmpeg", "-i", str(distorted), "-i", str(reference), "-lavfi",
                        f"[0:v]scale={ref['width']}:{ref['height']}:flags=bicubic[d];[d][1:v]ssim",
                        "-f", "null", "-"], capture_output=True, text=True, check=True)
    return float(re.findall(r"All:([0-9.]+)", r.stderr)[-1])

def moov_before_mdat(path):
    with open(path, "rb") as f:
        pos, total = 0, os.path.getsize(path)
        while pos < total:
            f.seek(pos); hdr = f.read(16)
            if len(hdr) < 8:
                break
            size, kind = int.from_bytes(hdr[:4], "big"), hdr[4:8].decode("latin-1")
            if size == 1:
                size = int.from_bytes(hdr[8:16], "big")
            if kind in ("moov", "mdat"):
                return kind == "moov"
            if size == 0:
                break
            pos += size
    return False

The tests

A few of the 26 checks (pytest, run against all five clips where parametrized):

@pytest.mark.parametrize("clip", CLIPS, ids=lambda p: p.stem)
def test_crf23_is_smaller_and_still_close(clip, out):
    dst = out / f"{clip.stem}-crf23.mp4"
    info, src = compress_crf(clip, dst, crf=23), probe(clip)
    assert info["size"] < src["size"]
    assert abs(info["duration"] - src["duration"]) < 0.1
    assert (info["width"], info["height"]) == (src["width"], src["height"])
    assert ssim(clip, dst) > 0.95

@pytest.mark.parametrize("budget_mb", [0.5, 1.0, 2.0])
def test_two_pass_lands_under_budget(budget_mb, out):
    budget = int(budget_mb * 1_000_000)
    info = compress_to_size(LANDSCAPE, out / f"budget-{budget_mb}.mp4", budget)
    assert info["size"] <= budget
    assert info["size"] >= 0.80 * budget   # don't waste the budget

def test_first_pass_estimate_can_overshoot(out):
    vk = video_kbps_for_budget(500_000, probe(LANDSCAPE)["duration"], 96)
    _two_pass(LANDSCAPE, out / "one-shot.mp4", vk, "medium", [], 96)
    assert probe(out / "one-shot.mp4")["size"] > 500_000   # why the retry loop exists

def test_downscale_never_upscales(out):
    info = compress_crf(FIX / "l-cabin-i2v.mp4", out / "c.mp4", crf=26, max_height=1080)
    assert (info["width"], info["height"]) == (1280, 720)

def test_budget_math():
    # 8 MB over 60 s = 1,066.7 kb/s; minus 4% overhead and 96 kb/s audio -> 928 kb/s video
    assert video_kbps_for_budget(8_000_000, 60.0, 96) == 928

The others check that higher CRF always means a smaller file, that SSIM drops as CRF rises, that every output is H.264 yuv420p + AAC with moov first, that a plain stream copy is detected as not faststart, and that portrait downscales keep the aspect ratio with an even width.

$ pytest -q test_compress_video.py
26 passed in 22.09s

Checklist

Sharing a clip? -crf 28 -preset medium, AAC 128k, +faststart. Check the result is actually smaller.
Source already H.264? Don't use CRF 18 to "compress" it; it will probably grow.
Hard size cap? Two-pass at the computed bitrate, then measure and retry.
Small screen? Downscale with scale=-2:'min(720,ih)' before reaching for a higher CRF.
Comparing presets? Hold size or quality fixed, not CRF.

Where the clips came from

The test clips are the published examples on Voor's LTX 2.5 page. LTX 2.5 renders picture and sound in one pass, with camera presets such as dolly in and jib up, up to 4K or 20 seconds (not both); the page lists the 6-second 1080p, 25 fps dolly-in clip with audio at 272 credits on Fast. Clips straight from a generator are big and already H.264, which is exactly where the CRF 18 trap shows up. You can try LTX 2.5 on Voor and run the compression tests on your own output, or open the generator when you need a fresh test clip at a known resolution and frame rate.

This article was written with AI assistance; every number in it was measured by the code shown.

How to Extract Frames From a Video With FFmpeg

Voor AI — Thu, 29 Oct 2026 00:37:07 +0000

To extract frames from a video with FFmpeg, pick the command by what you need: -ss T -i in.mp4 -frames:v 1 for one exact moment, -vf "select=..." for one real frame every N seconds, -fps_mode passthrough for every frame, and select='gt(scene,0.3)' for cuts. The popular one-liner -vf fps=1 works, but it does not return the frames at 0 s, 1 s, 2 s: on a 24 fps clip it returned frames 11, 35, 59 … and silently dropped the last second.

I measured all of this by pixel-hashing every PNG FFmpeg wrote against frames decoded by index, on five real 6-second AI-generated clips. The code and the 33 pytest checks are below.

The test clips

Five public example clips from Grok Imagine Video 1.5 (xAI's video model, as published on Voor's model page): 145 frames each, 24 fps, 6.04 s, H.264 with AAC audio, four at 1280×720 and one portrait at 480×848. Generated video is a good stress case: it is one continuous shot, and it has exactly one keyframe (frame 0), which matters below.

ffprobe -v error -select_streams v:0 -count_frames \
  -show_entries stream=avg_frame_rate,nb_read_frames:format=duration \
  -of default=nw=1 g-sailboat-i2v.mp4
# avg_frame_rate=24/1  nb_read_frames=145  duration=6.041667

1. One frame at an exact time: `-ss` before `-i`

ffmpeg -ss 2.5 -i clip.mp4 -frames:v 1 at_2.5s.png

With -ss before -i, FFmpeg seeks to the nearest earlier keyframe and then decodes forward to the requested time. Despite the old "input seeking is inaccurate" advice, on FFmpeg 9.0.1 it was frame-exact in every test: the output is the first frame whose timestamp is at or after T. 0 → frame 0, 1.0 → 24, 2.5 → 60, 6.0 → 144.

Seeking cost grows with keyframe distance. These clips have a single keyframe, so every seek decodes from frame 0; on a 6-second clip that is nothing, but on a long file with sparse keyframes each seek decodes much more.

2. One frame every N seconds: prefer `select` over `fps`

The chart shows which frames each command really returned (1-second interval, 145-frame clip):

Command	Frames out	Frame indices
`-vf fps=1`	6	11, 35, 59, 83, 107, 131
`-vf fps=1:round=down`	6	23, 47, 71, 95, 119, 143
`-vf "select='isnan(prev_selected_t)+gte(t-prev_selected_t\,1)'"`	7	0, 24, 48, 72, 96, 120, 144

The fps filter resamples time. Each output frame represents a whole one-second bucket, so with the default round=near you get the middle of each second, and the final partial second (6.00–6.04 s) produces no output. That is fine for a contact sheet. It is wrong if you need "the frame at 3 s" or the last frame of the clip.

select keeps the first real frame at or after 0, N, 2N … and nothing else:

ffmpeg -i clip.mp4 \
  -vf "select='isnan(prev_selected_t)+gte(t-prev_selected_t\,1)'" \
  -fps_mode passthrough every_%04d.png

-fps_mode passthrough (the modern name for -vsync passthrough) matters: without it the image2 muxer can duplicate frames to fill the gaps.

3. Every frame

ffmpeg -i clip.mp4 -fps_mode passthrough frame_%05d.png

145 frames in, 145 PNGs out, and the last PNG hashed identical to frame 144. At 1280×720 that was roughly 0.6 MB per PNG; use -q:v 2 frame_%05d.jpg if you only need previews.

4. Only at scene changes

ffmpeg -i clip.mp4 -vf "select='gt(scene,0.3)',showinfo" -an -f null - 2>&1 | grep pts_time

None of the five single-shot clips produced a detection at 0.3. Joining two clips (fox, then sailboat) with the concat demuxer produced exactly one, at 6.063 s, within one frame of the join at 145/24 s. Scene detection finds cuts, not motion; a sweeping camera move in one shot did not trigger it.

5. Why not keyframes?

-skip_frame nokey is the fastest way to grab "a few frames", because it decodes nothing but keyframes. On these AI clips it returned a single frame: there is only one keyframe in six seconds. Encoders put keyframes where it suits compression, not where your content changes. Do not use it for sampling.

The module

# extract_frames.py (abridged)
def at_timestamps(path, out_dir, times):
    info = probe(path)
    for t in times:
        if not 0 <= t < info.duration:
            raise ValueError(f"timestamp {t} outside 0..{info.duration:.3f}s")
        _run(["ffmpeg", "-v", "error", "-y", "-ss", f"{t:.6f}", "-i", str(path),
              "-frames:v", "1", str(Path(out_dir) / f"at_{t:09.3f}s.png")])

def every_n_seconds_select(path, out_dir, interval=1.0):
    expr = f"isnan(prev_selected_t)+gte(t-prev_selected_t\\,{interval})"
    _run(["ffmpeg", "-v", "error", "-y", "-i", str(path), "-vf", f"select='{expr}'",
          "-fps_mode", "passthrough", str(Path(out_dir) / "sel_%04d.png")])

def scene_changes(path, threshold=0.3):
    p = subprocess.run(["ffmpeg", "-hide_banner", "-i", str(path),
                        "-vf", f"select='gt(scene,{threshold})',showinfo", "-an", "-f", "null", "-"],
                       capture_output=True, text=True, check=True)
    return [float(t) for t in re.findall(r"pts_time:([0-9.]+)", p.stderr)]

To prove which frame a PNG is, hash its decoded pixels and compare with frame k rendered through the same PNG encoder (comparing raw rgb24 from the video against a PNG gives false mismatches because the colour conversion path differs):

def frame_md5(path=None, index=None, image=None):
    if image is None:
        with tempfile.TemporaryDirectory() as tmp:
            image = Path(tmp) / "ref.png"
            _run(["ffmpeg", "-v", "error", "-y", "-i", str(path),
                  "-vf", f"select=eq(n\\,{index})", "-frames:v", "1",
                  "-fps_mode", "passthrough", str(image)])
            return frame_md5(image=image)
    return _run(["ffmpeg", "-v", "error", "-i", str(image), "-f", "md5", "-"]).stdout.split("=")[-1].strip()

The tests

@pytest.mark.parametrize("rounding,frames", [("near", [11, 35, 59, 83, 107, 131]),
                                              ("down", [23, 47, 71, 95, 119, 143])])
def test_fps_filter_samples_are_not_on_the_second(tmp_path, rounding, frames):
    files = every_n_seconds(CLIP, tmp_path, 1.0, rounding)
    assert [frame_md5(image=f) for f in files] == [frame_md5(CLIP, k) for k in frames]

def test_select_samples_are_exact_frames(tmp_path):
    files = every_n_seconds_select(CLIP, tmp_path, 1.0)
    for k, f in enumerate(files):
        assert frame_md5(image=f) == frame_md5(CLIP, 24 * k)

@pytest.mark.parametrize("t", [0.0, 1.0, 2.5, 4.0, 6.0])
def test_at_timestamp_is_frame_accurate(tmp_path, t):
    (f,) = at_timestamps(FOX, tmp_path, [t])
    assert frame_md5(image=f) == frame_md5(FOX, math.ceil(round(t * 24, 6)))

def test_scene_detection_finds_the_join(tmp_path):
    joined = concat([FOX, CLIP], tmp_path / "joined.mp4")
    (cut,) = scene_changes(joined, 0.3)
    assert abs(cut - 145 / 24) < 1 / 24 + 1e-6

$ pytest -q test_extract_frames.py
33 passed in 17.25s

Checklist

Need an exact moment? -ss T before -i, -frames:v 1.
Need one frame per interval that lands on the interval? select with prev_selected_t, plus -fps_mode passthrough.
Need an even contact sheet and do not care about exact times? fps=1/N is fine; expect mid-interval frames and no final partial interval.
Need cuts? select='gt(scene,0.3)', and test the threshold on a known join.
Never assume keyframes are evenly spaced.

Where the clips came from

If you want your own short test clips, these came from the Grok Imagine Video 1.5 page on Voor, which makes 1–15 second clips with generated sound from a prompt, a still, or up to seven reference images; the page lists six seconds at 720p at 63 credits on Lite and 293 on Standard. I used the published examples rather than generating new ones. You can try the same model on Voor and run the extraction tests against your own output, or open the generator when you need a fresh fixture with a known frame count.

How to Normalize Audio With FFmpeg

Voor AI — Wed, 28 Oct 2026 00:44:51 +0000

Short answer: run FFmpeg's loudnorm filter twice. The first pass measures the file and prints JSON; the second pass feeds those numbers back with linear=true so the whole file gets one clean gain change. Then read the JSON from the second pass, because linear=true is a request, not a guarantee. On six real text-to-speech takes I tested, loudnorm silently switched to dynamic mode every time.

This post shows the two-pass script, a strict-linear alternative, and the pytest checks that caught the difference.

Six real TTS takes measured with loudnorm: grey original, blue two-pass loudnorm (dynamic), orange strict linear gain capped at -1.5 dBTP.

The test material

I used six real voice takes that are published as examples on the ElevenLabs v4 voice workspace on Voor: English, Spanish and Japanese narration, some on v4 and some on v4 Turbo, all 44.1 kHz mono MP3 between 9.4 and 17.9 seconds. They are a good stress test because TTS output arrives at whatever loudness the model happened to produce. Measured with loudnorm, they ranged from -19.9 to -26.2 LUFS integrated, a 6.3 LU spread. Put those side by side in a playlist or a video and listeners reach for the volume knob.

Targets used throughout: -16 LUFS integrated, -1.5 dBTP true peak, LRA 11, a common choice for podcasts and web video. Change them for broadcast (-23 LUFS) or a platform spec.

Pass 1 and pass 2 in Python

def measure(path: str | Path, i=TARGET_I, tp=TARGET_TP, lra=TARGET_LRA) -> dict:
    """Pass 1: analyse only, write nothing."""
    log = _run(["ffmpeg", "-hide_banner", "-nostats", "-i", str(path),
                "-af", f"loudnorm=I={i}:TP={tp}:LRA={lra}:print_format=json",
                "-f", "null", "-"])
    match = _JSON.search(log)
    if not match:
        raise RuntimeError("loudnorm printed no JSON")
    raw = json.loads(match.group(0))
    return {k: float(v) if k != "normalization_type" else v for k, v in raw.items()}


def normalize(src: str | Path, dst: str | Path, i=TARGET_I, tp=TARGET_TP, lra=TARGET_LRA) -> dict:
    """Pass 2: feed the pass-1 numbers back so loudnorm can stay linear."""
    m = measure(src, i, tp, lra)
    if not math.isfinite(m["input_i"]) or m["input_i"] <= -70.0:
        raise ValueError(f"{src}: no measurable programme loudness (silent input?)")
    info = probe(src)
    dst = Path(dst)
    codec = (["-c:a", "libmp3lame", "-b:a", "192k"] if dst.suffix == ".mp3"
             else ["-c:a", "pcm_s16le"])
    af = (f"loudnorm=I={i}:TP={tp}:LRA={lra}"
          f":measured_I={m['input_i']}:measured_TP={m['input_tp']}"
          f":measured_LRA={m['input_lra']}:measured_thresh={m['input_thresh']}"
          f":offset={m['target_offset']}:linear=true:print_format=json")
    log = _run(["ffmpeg", "-hide_banner", "-nostats", "-y", "-i", str(src), "-af", af,
                # loudnorm resamples to 192 kHz internally; put the original rate back
                "-ar", str(info["sample_rate"]), *codec, str(dst)])
    second = json.loads(_JSON.search(log).group(0))
    return {"before": m, "mode": second["normalization_type"], "after": measure(dst, i, tp, lra)}


def normalize_linear(src: str | Path, dst: str | Path, i=TARGET_I, tp=TARGET_TP) -> dict:
    """Strict linear gain: never compress, stop at the true-peak ceiling and report the shortfall."""
    m = measure(src, i, tp)
    if not math.isfinite(m["input_i"]) or m["input_i"] <= -70.0:
        raise ValueError(f"{src}: no measurable programme loudness (silent input?)")
    wanted = i - m["input_i"]
    gain = min(wanted, tp - m["input_tp"])
    info = probe(src)
    dst = Path(dst)
    codec = (["-c:a", "libmp3lame", "-b:a", "192k"] if dst.suffix == ".mp3"
             else ["-c:a", "pcm_s16le"])
    _run(["ffmpeg", "-hide_banner", "-nostats", "-y", "-i", str(src),
          "-af", f"volume={gain:.2f}dB", "-ar", str(info["sample_rate"]), *codec, str(dst)])
    return {"before": m, "mode": "linear", "gain_db": round(gain, 2),
            "shortfall_lu": round(wanted - gain, 2), "after": measure(dst, i, tp)}

Three details matter more than the filter string:

Silence check. A silent file measures as -inf (or -70 LUFS). Normalizing it would apply absurd gain, so normalize() raises ValueError instead.
Sample rate. loudnorm upsamples to 192 kHz internally. Without -ar you silently ship a 192 kHz file. The script puts the source rate back.
normalization_type. The second pass prints JSON too. Its normalization_type says whether you actually got linear or dynamic.

What happened on the real takes

take                     before I   TP     LRA  -> mode     after I   TP     LRA
dental-sarah-v4           -19.93  -1.72   1.70  -> dynamic  -16.23  -1.50   1.60
harbor-george-v4-turbo    -22.58  -3.61  12.40  -> dynamic  -16.60  -1.48   5.70
harbor-george-v4          -22.15  -1.72   9.50  -> dynamic  -16.60  -1.48   4.10
market-laura-es-v4        -22.57  -2.72   1.00  -> dynamic  -16.46  -1.48   1.40
pass-brian-v4-turbo       -21.55  -3.57   1.50  -> dynamic  -16.07  -1.50   1.70
rain-lily-ja-v4           -26.16  -7.88   7.40  -> dynamic  -17.29  -1.08   4.30

Every file fell back to dynamic mode. The reason is arithmetic: to raise dental-sarah-v4 from -19.9 to -16 LUFS you need +3.7 dB of gain, but its true peak is already -1.72 dBTP, so a single linear gain would push the peak to about +2 dBTP. loudnorm cannot honour both the loudness target and the peak ceiling linearly, so it compresses instead. The harbor turbo take also has an LRA of 12.4, above the LRA target of 11, which forces dynamic mode on its own.

Dynamic mode did its job on loudness: five of six landed within 0.6 LU of -16, and the spread between takes dropped from 6.3 LU to 1.2 LU. But look at the loudness range column. The whispered-to-excited harbor take went from 12.4 LU to 5.7 LU. That is the performance being flattened.

Same take three ways. Dynamic mode lifts the quiet whisper section toward the loud section; linear gain keeps the original shape.

The shortest take, rain-lily-ja-v4 (9.4 s), also undershot (-17.29 LUFS) and slightly overshot the peak target (-1.08 dBTP). Short clips give loudnorm's dynamic mode little to work with, so test your shortest files, not just your longest.

A strict linear alternative

If the expressive range matters more than hitting -16 exactly, use plain gain capped at the true-peak ceiling and report the shortfall:

def normalize_linear(src, dst, i=TARGET_I, tp=TARGET_TP):
    m = measure(src, i, tp)
    if not math.isfinite(m["input_i"]) or m["input_i"] <= -70.0:
        raise ValueError(f"{src}: no measurable programme loudness (silent input?)")
    wanted = i - m["input_i"]
    gain = min(wanted, tp - m["input_tp"])          # never clip past the ceiling
    rate = probe(src)["sample_rate"]
    _run(["ffmpeg", "-y", "-i", str(src), "-af", f"volume={gain:.2f}dB",
          "-ar", str(rate), "-c:a", "pcm_s16le", str(dst)])
    return {"gain_db": round(gain, 2), "shortfall_lu": round(wanted - gain, 2),
            "before": m, "after": measure(dst, i, tp)}

On the same six files this kept every LRA unchanged and every peak at -1.50 dBTP, but loudness only reached -19.5 to -21.9 LUFS, short of the target by 3.5 to 5.9 LU. That is the honest trade-off: hit the target with compression, or keep the dynamics and accept a quieter file. A third option is a deliberate limiter stage before linear gain, which at least makes the compression a choice you can hear and tune.

Tests that keep the pipeline honest

@pytest.mark.parametrize("src", FIXTURES, ids=lambda p: p.stem)
def test_reports_when_linear_was_impossible(normalized, src):
    _, _, r = normalized[src.name]
    b = r["before"]
    linear_possible = (TARGET_I - b["input_i"]) <= (TARGET_TP - b["input_tp"]) and b["input_lra"] <= 11
    if not linear_possible:
        assert r["mode"] == "dynamic"


@pytest.mark.parametrize("src", FIXTURES, ids=lambda p: p.stem)
def test_keeps_rate_channels_and_length(normalized, src):
    s, dst, _ = normalized[src.name]
    a, b = probe(s), probe(dst)
    assert b["sample_rate"] == a["sample_rate"]          # not loudnorm's 192 kHz
    assert b["channels"] == a["channels"]
    assert abs(b["duration"] - a["duration"]) < 0.08     # mp3 priming/padding only


@pytest.mark.parametrize("src", FIXTURES, ids=lambda p: p.stem)
def test_strict_linear_keeps_dynamics_and_peak(src, tmp_path):
    r = normalize_linear(src, tmp_path / "lin.wav")
    assert r["after"]["input_tp"] <= TARGET_TP + 0.3
    assert abs(r["after"]["input_lra"] - r["before"]["input_lra"]) <= 0.6
    assert r["shortfall_lu"] >= 0

The full suite also checks the integrated target (within 1.5 LU in dynamic mode), the peak ceiling, the shrinking spread between takes, near-idempotence on a second run and the silence guard. Run with uvx --with pytest pytest -v: 35 passed in 6.98 s on FFmpeg from Homebrew.

Checklist before you ship normalized audio

Log normalization_type from pass 2 and fail or warn when you required linear.
Restore the sample rate with -ar; check it with ffprobe.
Guard against silent or near-silent inputs.
Compare LRA before and after for speech with intentional whispers or shouts.
Test the shortest clip in the batch.
Re-measure the output; never trust the target you asked for.

Generating your own test takes

If you want fresh fixtures with deliberate quiet and loud passages, the ElevenLabs v4 text-to-speech generator on Voor takes a tagged script such as [whispering] ... [excited] ...; its default tagged-narration script quotes 6 credits, and generating requires signing in. The takes above came from that page's published examples, so you can reproduce every number here without generating anything.

How to Convert MP4 to GIF With FFmpeg

Voor AI — Tue, 27 Oct 2026 00:44:32 +0000

Short answer: the reliable way to convert an MP4 to a GIF with FFmpeg is a two-pass palette: palettegen builds a 256-colour (or smaller) palette from the clip, and paletteuse maps every frame onto it. That fixes the muddy colours and dotted noise of a plain ffmpeg -i in.mp4 out.gif. It does not make the file smaller. File size is controlled by width, frame rate, colour count and duration, so the script below walks a small ladder of those settings until the GIF fits a byte budget, and the tests check that the result really loops, keeps its timing and fits.

Everything here was run with FFmpeg 9.0.1, Python 3 and Pillow on a real 5-second clip.

The fixture

The input is the Rainbow Puddle Sprite example clip from Voor's Seedance 2.5 page: 1280 × 720, 24 fps, 5.06 s, H.264 with an AAC audio track. It is a good GIF stress test because it has soft backlight, translucent wings and a lot of smooth gradients, exactly where GIF's 256-colour limit shows.

The one-liner, and why it is not enough

The command most answers give is:

ffmpeg -t 3 -i in.mp4 -vf "fps=15,scale=640:-2" out.gif

On the fixture's first 3 seconds that produced a 2.77 MB GIF, and the zoomed crop above shows the problem: FFmpeg's default GIF encoder uses a generic palette, so the warm backlight turns into a visible dot pattern.

The two-pass palette filter

ffmpeg -t 3 -i in.mp4 -an -filter_complex \
  "[0:v]fps=15,scale=640:-2:flags=lanczos,split[a][b];\
   [a]palettegen=max_colors=256:stats_mode=diff[p];\
   [b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle" \
  -loop 0 out.gif

split feeds the same scaled frames to both passes, so you do not need a temporary palette PNG.
stats_mode=diff builds the palette from what changes between frames, which suits clips with a mostly static background.
dither=bayer:bayer_scale=5 gives an ordered dither that compresses better than error diffusion. diff_mode=rectangle only re-encodes the changed region of each frame.
scale=640:-2 keeps the aspect ratio and forces an even height. -loop 0 means loop forever.

Same width, same frame rate, much cleaner colour, and a bigger file: 4.71 MB. That is the part people miss. The palette fixes quality; it does not fix size.

Fitting a byte budget

Most places that accept GIFs have an upload limit, so the useful tool is "best quality that fits N bytes". The script tries four rungs in order and stops at the first one that fits:

"""Convert an MP4 to a looping GIF with FFmpeg's two-pass palette, under a byte budget."""
from __future__ import annotations

import json
import shutil
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path

FFMPEG = shutil.which("ffmpeg") or "ffmpeg"
FFPROBE = shutil.which("ffprobe") or "ffprobe"

# Tried in order until the GIF fits the budget: (width, fps, colors)
LADDER = [(640, 15, 256), (480, 12, 192), (360, 10, 128), (320, 8, 96)]


@dataclass
class GifResult:
    path: Path
    width: int
    fps: int
    colors: int
    bytes: int
    attempts: int


def probe(src: Path) -> dict:
    out = subprocess.run(
        [FFPROBE, "-v", "error", "-select_streams", "v:0",
         "-show_entries", "stream=width,height,r_frame_rate:format=duration",
         "-of", "json", str(src)],
        check=True, capture_output=True, text=True,
    ).stdout
    data = json.loads(out)
    s = data["streams"][0]
    num, den = (int(x) for x in s["r_frame_rate"].split("/"))
    return {"width": s["width"], "height": s["height"], "fps": num / den,
            "duration": float(data["format"]["duration"])}


def build_filter(width: int, fps: int, colors: int) -> str:
    base = f"fps={fps},scale={width}:-2:flags=lanczos"
    return (f"[0:v]{base},split[a][b];"
            f"[a]palettegen=max_colors={colors}:stats_mode=diff[p];"
            f"[b][p]paletteuse=dither=bayer:bayer_scale=5:diff_mode=rectangle")


def encode(src: Path, dst: Path, width: int, fps: int, colors: int,
           start: float = 0.0, seconds: float | None = None) -> int:
    cmd = [FFMPEG, "-v", "error", "-y", "-ss", f"{start}"]
    if seconds is not None:
        cmd += ["-t", f"{seconds}"]
    cmd += ["-i", str(src), "-an", "-filter_complex",
            build_filter(width, fps, colors), "-loop", "0", str(dst)]
    subprocess.run(cmd, check=True)
    return dst.stat().st_size


def mp4_to_gif(src: Path, dst: Path, max_bytes: int,
               start: float = 0.0, seconds: float | None = None) -> GifResult:
    info = probe(src)
    for attempt, (width, fps, colors) in enumerate(LADDER, start=1):
        width = min(width, info["width"])
        fps = min(fps, round(info["fps"]))
        size = encode(src, dst, width, fps, colors, start, seconds)
        if size <= max_bytes:
            return GifResult(dst, width, fps, colors, size, attempt)
    dst.unlink(missing_ok=True)
    raise ValueError(f"{src.name}: no ladder step fits {max_bytes} bytes "
                     f"(last try {size} bytes); shorten the clip with --seconds")


if __name__ == "__main__":
    src, dst, budget = Path(sys.argv[1]), Path(sys.argv[2]), int(sys.argv[3])
    secs = float(sys.argv[4]) if len(sys.argv) > 4 else None
    r = mp4_to_gif(src, dst, budget, seconds=secs)
    print(f"{r.path} {r.width}px {r.fps}fps {r.colors} colors "
          f"{r.bytes} bytes after {r.attempts} attempt(s)")

Running it on the same 3-second slice:

$ python3 mp4_to_gif.py rainbow-puddle-sprite.mp4 out.gif 3000000 3
out.gif 480px 12fps 192 colors 2096976 bytes after 2 attempt(s)

$ python3 mp4_to_gif.py rainbow-puddle-sprite.mp4 out.gif 1500000 3
out.gif 360px 10fps 128 colors 932816 bytes after 3 attempt(s)

$ python3 mp4_to_gif.py rainbow-puddle-sprite.mp4 out.gif 900000 3
out.gif 320px 8fps 96 colors 565291 bytes after 4 attempt(s)

Here is the same moment from each rung:

On this clip, the first two steps down the ladder each cut the file by more than half. Width does most of the work; dropping from 15 to 8 fps makes the wing flap visibly choppier, so if motion matters more than size, trim the clip instead of cutting the frame rate.

Tests that catch broken GIFs

A GIF that plays once, drifts in speed or silently blows the budget still "converts" without errors. These checks use Pillow to read the GIF back (the full file has 10 tests, including probe and aspect-ratio checks):

def test_loops_forever(gif):
    _, _, loop, _ = frames_and_durations(gif.path)
    assert loop == 0


def test_frame_timing_matches_fps(gif):
    _, durations, _, _ = frames_and_durations(gif.path)
    expected = 1000 / gif.fps
    # GIF delays are stored in centiseconds, so allow one 10 ms tick
    assert all(abs(d - expected) <= 10 for d in durations[:-1])


def test_ladder_steps_down_when_budget_is_tight(tmp_path):
    r = mp4_to_gif(CLIP, tmp_path / "small.gif", max_bytes=900_000, seconds=3)
    assert r.attempts > 1 and r.bytes <= 900_000


def test_impossible_budget_raises_and_leaves_no_file(tmp_path):
    out = tmp_path / "never.gif"
    with pytest.raises(ValueError, match="no ladder step fits"):
        mp4_to_gif(CLIP, out, max_bytes=10_000, seconds=3)
    assert not out.exists()

$ uvx --with pillow pytest -q
..........                                                               [100%]
10 passed

Two details worth knowing:

GIF frame delays are stored in centiseconds, so 12 fps becomes 80 or 90 ms per frame, never exactly 83.3 ms. The timing test allows one 10 ms tick.
Pillow exposes the NETSCAPE loop extension as im.info["loop"]; 0 means infinite. If it is missing, many viewers play the GIF once.

Troubleshooting

"height not divisible by 2" or odd sizes: use scale=W:-2, not -1.
Colours still band in gradients: try dither=sierra2_4a (bigger files) or keep 256 colours and cut duration instead.
The GIF has audio-sync or length issues: GIF has no audio; pass -an and trim with -ss/-t before -i for fast, frame-accurate-enough cuts.
Still too big at 320 px: GIF is the wrong format. Most platforms accept MP4 or WebP animations, which are usually far smaller at better quality.

Getting a test clip

If you want to run the ladder on a clip of your own, the Seedance 2.5 video workspace is where the fixture came from. When I checked on October 27, 2026, its default settings (16:9, 480p, 5 seconds, audio on) showed a quote of 314 credits, and new creations are Public with a watermark by default. Whatever you generate, measure the downloaded file with probe() rather than assuming its frame rate.

Wrap-up

Use palettegen + paletteuse for quality, a width/fps/colour ladder for size, and Pillow assertions so a GIF that does not loop or does not fit fails the build. If you need a short, gradient-heavy clip to test on, make one in the same video workspace and run the script on the download.

DEV Community: Voor AI

How to Extract Audio From a Video With FFmpeg (Copy vs Re-encode, With pytest)

Prerequisites

Two honest extract commands

Python helper

pytest — four checks

Review checklist

Troubleshooting

Limitations

Optional next step on Voor

How to Reverse a Video With FFmpeg (and Prove the Last Frame Became the First)

What we are proving

Method A — PNG sequence (exact ends)

Method B — filter reverse (delivery one-liner)

Minimal helper

Pytest results (2026-11-06)

When not to reverse

Voor alternative

Disclosure

How to Crop a Video With FFmpeg (16:9 to 9:16, One Window per Shot)

The test clip

Crop syntax in 30 seconds

The width you ask for is not always the width you get

Why the center crop fails on this clip

Step 1: find the shot boundaries with scdet

Step 2: one crop expression that changes x at each cut

Step 3: prove the output with tests

Bonus: remove black bars with cropdetect

When not to crop at all

Limitations

How to Speed Up a Video With FFmpeg (Without Losing Frames or Chipmunking the Audio)

The test clip

What the naive one-liner actually does

Audio: use atempo, never asetrate

Slowing down has the opposite problem

Stream copy cannot do this

A small helper

Tests that catch the silent failures

Checklist

When not to speed up at all

How to Trim a Video With FFmpeg (and Prove the Cut Starts on the Right Frame)

The test clip

The helpers

Trim 1: stream copy

Trim 2: re-encode

Trim 3: snap to a keyframe and copy

The -to trap

The tests

Which one to use

Limits

Generate a little extra, then trim

How to Add Subtitles to a Video With FFmpeg (and Time Them From the Audio)

The test material

Step 1: Find the speech with silencedetect

Step 2: Merge the shortest pauses until the counts match

Step 3: Make the video, with an exact length

Step 4: Soft subtitles in MP4 need mov_text

Step 5: Burned-in subtitles need libass

Step 6: Prove the text is there, and only when it should be

The tests

Limits

Where the script comes from

How to Create a Video From Images With FFmpeg (Without Stretching Portrait Photos)

The test images

What the one-liner actually does

A filter graph that fits every image

Tests that catch the silent failures

Checklist before you upload

Limits

How to Merge Videos With FFmpeg (and Catch the Resolution Change Stream Copy Hides)

1. The two ways FFmpeg concatenates

2. What stream copy does with mismatched clips

3. Check compatibility before you copy

4. Normalize, then concatenate with the filter

5. Tests that pin the behaviour

6. Pitfalls checklist

Getting test clips

How to Compress a Video With FFmpeg

The test clips

1. Quality-targeted: pick a CRF

Step 1: find the shot boundaries with `scdet`

Step 2: one crop expression that changes `x` at each cut

Bonus: remove black bars with `cropdetect`

The `-to` trap

4. `+faststart` for anything on the web

1. One frame at an exact time: `-ss` before `-i`

2. One frame every N seconds: prefer `select` over `fps`