Detect variable frame rate uploads with ffprobe and normalize them before they break your pipeline

TL;DR

Screen recordings from OBS, QuickTime, Loom and phone capture are variable frame rate. Feed them to a transcoder that assumes constant frame rate and you get audio drift that grows over the clip. We'll build a two-tier ffprobe gate (cheap header check, then a real per-frame timestamp check) and normalize with -fps_mode cfr on FFmpeg 8.

The bug

Audio and video start in sync and drift apart as the clip plays. Four minutes in, it's obviously wrong. Thirty seconds in, nobody notices. That growing gap is the signature of a variable frame rate (VFR) source hitting a pipeline that assumed constant frame rate (CFR).

Audio runs on a fixed sample clock. VFR video does not. When your transcoder stamps out frames at a target rate against a source that never agreed to one, the error accumulates instead of resetting.

Let's detect it properly and fix it at ingest.

1. Why you keep receiving VFR 🎥

Screen capture tools generate a frame only when the screen content changes. That covers OBS, QuickTime screen recording, Loom, phone screen recorders, and most game capture. If your product accepts user uploads of demos, tutorials, bug reports or gameplay, a real chunk of your ingest is VFR.

Check a file you already have:

ffprobe -v error \
  -select_streams v:0 \
  -show_entries stream=r_frame_rate,avg_frame_rate,nb_frames,duration \
  -of default=noprint_wrappers=1 \
  input.mp4

r_frame_rate=30000/1001
avg_frame_rate=23856/1001
nb_frames=7152
duration=300.033333

Those two rates disagree, so this file is worth a closer look.

2. Why the popular check is only a heuristic ⚠️

Nearly every tutorial stops at "if r_frame_rate != avg_frame_rate, it's VFR." That is not true, and believing it will cost you a day.

• avg_frame_rate is total frames divided by duration. Real.
• r_frame_rate is FFmpeg's guess at a base rate, computed like a lowest common multiple of observed timings so every timestamp lands on a whole tick. It is not the actual frame rate.

So a mismatch suggests VFR. It does not prove it, and identical values do not prove CFR either.

⚠️ Treat the header comparison as a cheap first-pass filter, not a verdict. Confirm with per-frame timestamps.

3. The real check: per-frame deltas

Dump packet timestamps and look at the gaps between them. Uniform gaps mean the file is effectively constant no matter what the headers claim.

ffprobe -v error \
  -select_streams v:0 \
  -show_entries packet=pts_time \
  -of csv=p=0 \
  input.mp4 | head -20

0.000000
0.033367
0.066733
0.100100
0.133467
0.166833
0.250250
0.283617
0.316983
0.550550

Look at that jump from 0.166833 to 0.250250, and the bigger one to 0.550550. Uneven. That's genuine VFR.

Here's the check as a script. It only pays for the expensive packet dump when the cheap check trips.

# vfr_gate.py (python 3.11+, ffmpeg/ffprobe 8.x)
import json
import subprocess
from fractions import Fraction

def _probe(args: list[str]) -> dict:
    out = subprocess.run(
        ["ffprobe", "-v", "error", "-of", "json", *args],
        capture_output=True, text=True, check=True,
    ).stdout
    return json.loads(out)

def header_mismatch(path: str) -> tuple[bool, Fraction, Fraction]:
    """Tier 1: cheap. Compares r_frame_rate against avg_frame_rate."""
    data = _probe([
        "-select_streams", "v:0",
        "-show_entries", "stream=r_frame_rate,avg_frame_rate",
        path,
    ])
    s = data["streams"][0]
    r = Fraction(s["r_frame_rate"])
    a = Fraction(s["avg_frame_rate"])
    return (r != a), r, a

Now tier two, which only runs on files that tripped tier one:

# vfr_gate.py (continued)
def timestamps_are_irregular(path: str, tolerance: float = 0.002,
                             max_packets: int = 3000) -> bool:
    """Tier 2: expensive. Are the gaps between presentation timestamps uniform?"""
    data = _probe([
        "-select_streams", "v:0",
        "-show_entries", "packet=pts_time",
        "-read_intervals", "%+#" + str(max_packets),
        path,
    ])
    pts = sorted(
        float(p["pts_time"]) for p in data.get("packets", [])
        if p.get("pts_time") not in (None, "N/A")
    )
    if len(pts) < 3:
        return False

    deltas = [b - a for a, b in zip(pts, pts[1:])]
    baseline = sorted(deltas)[len(deltas) // 2]   # median, robust to one bad gap
    return any(abs(d - baseline) > tolerance for d in deltas)

def is_vfr(path: str) -> bool:
    mismatch, _, _ = header_mismatch(path)
    return mismatch and timestamps_are_irregular(path)

💡 Tip: -read_intervals "%+#3000" caps how many packets ffprobe reads. On a 90-minute file, dumping every packet is slow and you don't need all of them to see irregularity.

Run it:

$ python vfr_gate.py screen-recording.mp4
VFR: True   (r=30000/1001  avg=23856/1001)

$ python vfr_gate.py camera-export.mp4
VFR: False  (r=30000/1001  avg=30000/1001)

4. Normalize with -fps_mode cfr

Once you know, the fix is one flag applied as the first transcoding step, before segmenting, thumbnailing, or anything else forms an opinion about the file.

ffmpeg -i input.mp4 \
  -fps_mode cfr -r 30 \
  -c:v libx264 -preset medium -crf 21 \
  -c:a aac -b:a 128k -ar 48000 \
  -movflags +faststart \
  normalized.mp4

⚠️ Old tutorials use -vsync cfr or -vsync 1. That family is deprecated in FFmpeg 8.x. Use -fps_mode cfr in anything new.

Pin -r explicitly. Letting FFmpeg infer the output rate is how you got here.

Picking the rate:

Content	Rate	Why
Screen capture, talking head, general UGC	30	Safe default, cheapest
Gameplay, sports, high motion	60	Motion detail is the point
Anything above the source's peak rate	don't	Buys nothing, costs bitrate

5. Wire it into ingest

# ingest.py
import subprocess
from vfr_gate import is_vfr

def normalize_if_needed(src: str, dst: str, fps: int = 30) -> str:
    if not is_vfr(src):
        return src  # already CFR, skip the re-encode

    subprocess.run([
        "ffmpeg", "-v", "error", "-y", "-i", src,
        "-fps_mode", "cfr", "-r", str(fps),
        "-c:v", "libx264", "-preset", "medium", "-crf", "21",
        "-c:a", "aac", "-b:a", "128k", "-ar", "48000",
        "-movflags", "+faststart",
        dst,
    ], check=True)
    return dst

Emit a metric every time this fires. You want to know what fraction of your ingest is VFR, because it directly sizes your transcode fleet.

The cost, honestly

Normalizing a screen recording is not free. Twelve seconds of a static slide that occupied a handful of VFR frames becomes 360 frames at 30fps CFR. Your encoder does more work.

The output usually isn't much bigger, because near-identical consecutive frames compress extremely well and modern rate control handles them fine. But your encode time goes up on exactly the content that was cheapest before. Size your workers for screen capture as the worst case, not the best.

Errors you'll hit

avg_frame_rate=0/0: the container has no duration, common with fragmented MP4 and some live recordings. Skip tier one and go straight to timestamps.

Audio still drifts after normalization: check whether the source audio sample rate is what you think. Pin -ar 48000 and re-test. A resample happening implicitly is its own drift source.

Application provided invalid, non monotonically increasing dtx: the source has out-of-order timestamps on top of being VFR. Add -fflags +genpts before -i.

What's next

Frame rate is one assumption your pipeline makes about incoming files without checking. There are others, and they fail in unrelated-looking ways:

• Rotation metadata: the file is 1080x1920 with a 90-degree rotate flag, and half your tools honor it.
• Color space / transfer characteristics: the reason HDR sources come out washed on SDR screens.
• Audio channel layout: 5.1 sources silently downmixed, or not.

Pick one, write the ffprobe check, run it over a week of real uploads, and see what comes back. That number is usually more interesting than the fix.