DEV Community: Mason K

Build a capped-CRF ABR ladder with FFmpeg and prove it with VMAF (x264, NVENC, SVT-AV1)

Mason K — Tue, 06 Oct 2026 11:57:27 +0000

TL;DR

We'll convert a 4-rung CBR ladder to capped CRF, write a small Python sweep that encodes a set of clips at several CRF and cap values, score each with VMAF, and report average VMAF, lowest-frame VMAF, bitrate, and the share of seconds that hit the cap. Then we'll port the winning settings to NVENC and SVT-AV1.

A fixed-bitrate ladder spends the same on every second of every title. Capped CRF lets the encoder spend less on easy content and holds a ceiling on hard content. The catch is that it has two knobs (the CRF value and the cap) and most people only tune one.

We'll measure instead of guessing. You need ffmpeg 7.x or 8.x built with libx264, libvmaf, and optionally libsvtav1 and NVENC; ffprobe; and python 3.11+. Check your build:

ffmpeg -hide_banner -version | head -1
ffmpeg -hide_banner -filters | grep -E "libvmaf"
ffmpeg -hide_banner -encoders | grep -E "libx264|libsvtav1|h264_nvenc"

ffmpeg version 8.1.2 Copyright (c) 2000-2026 the FFmpeg developers
 ..C libvmaf           VV->V      Calculate the VMAF between two video streams.
 V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
 V..... libsvtav1            SVT-AV1(Scalable Video Technology for AV1) encoder
 V....D h264_nvenc           NVIDIA NVENC H.264 encoder

If libvmaf is missing, the static builds from BtbN (Linux/Windows) and Homebrew's ffmpeg (macOS) include it.

1. 📐 The baseline: a CBR-ish ladder

Here's the kind of ladder most teams inherit. Four rungs, each a fixed target:

# scripts/ladder_cbr.sh
IN=$1
common="-c:v libx264 -preset slow -profile:v high -g 48 -keyint_min 48 -sc_threshold 0 -c:a aac -b:a 128k"
ffmpeg -y -i "$IN" -vf scale=-2:1080 $common -b:v 6000k -maxrate 6000k -bufsize 12000k out_1080_cbr.mp4
ffmpeg -y -i "$IN" -vf scale=-2:720  $common -b:v 3000k -maxrate 3000k -bufsize 6000k  out_720_cbr.mp4
ffmpeg -y -i "$IN" -vf scale=-2:480  $common -b:v 1500k -maxrate 1500k -bufsize 3000k  out_480_cbr.mp4
ffmpeg -y -i "$IN" -vf scale=-2:360  $common -b:v 800k  -maxrate 800k  -bufsize 1600k  out_360_cbr.mp4

Keep the GOP flags (-g 48 -keyint_min 48 -sc_threshold 0 for 24 fps, 2-second segments). Capped CRF changes rate control, not segmenting, and your HLS packager still needs aligned keyframes across rungs.

2. 🎚️ Convert it to capped CRF

Capped CRF is three flags: a quality target, a ceiling, and a VBV buffer that enforces the ceiling.

# scripts/ladder_capped.sh
IN=$1; CRF=${2:-23}
common="-c:v libx264 -preset slow -profile:v high -g 48 -keyint_min 48 -sc_threshold 0 -c:a aac -b:a 128k"
ffmpeg -y -i "$IN" -vf scale=-2:1080 $common -crf $CRF -maxrate 6000k -bufsize 6000k out_1080_crf.mp4
ffmpeg -y -i "$IN" -vf scale=-2:720  $common -crf $CRF -maxrate 3000k -bufsize 3000k out_720_crf.mp4
ffmpeg -y -i "$IN" -vf scale=-2:480  $common -crf $CRF -maxrate 1500k -bufsize 1500k out_480_crf.mp4
ffmpeg -y -i "$IN" -vf scale=-2:360  $common -crf $CRF -maxrate 800k  -bufsize 800k  out_360_crf.mp4

What each flag does:

-crf 23: hold this quality; spend whatever bitrate that takes.
-maxrate 6000k: but never sustain more than this.
-bufsize 6000k: the VBV buffer the encoder uses to enforce maxrate. -maxrate does nothing without it. A buffer equal to the maxrate (1 second of video at the cap) is a common starting point; a larger buffer allows more short-term variability.

💡 Tip: -crf without -maxrate/-bufsize is "unbounded CRF". It's great for archives and useless for streaming, because one explosion scene can spike past anything your ladder planned for.

Run both scripts on a sample and compare file sizes:

./scripts/ladder_cbr.sh samples/lecture.mp4 && ./scripts/ladder_capped.sh samples/lecture.mp4 23
ls -l out_1080_*.mp4

On easy content (a lecture, a screen recording) the capped-CRF file will be noticeably smaller. On a sports clip it'll be about the same size, because the cap is doing the work. That difference is the whole point, and the next step turns it into numbers.

3. 📊 The sweep: encode, score, count cap hits

We'll sweep CRF values and cap values across a folder of representative clips, score each encode with VMAF against the source, and record four things per encode: mean VMAF, lowest-frame VMAF, average bitrate, and the fraction of one-second windows that sit at or near the cap.

# sweep.py
import csv, json, subprocess, sys
from pathlib import Path

CLIPS = sorted(Path("samples").glob("*.mp4"))
CRFS = [21, 23, 25, 27]
CAPS_K = [6000, 9000, 12000]
HEIGHT = 1080
FPS = 24  # match your source; used for the per-second bitrate windows

def run(cmd):
    return subprocess.run(cmd, check=True, capture_output=True, text=True)

def encode(src, crf, cap_k):
    out = Path("out") / f"{src.stem}_crf{crf}_cap{cap_k}.mp4"
    out.parent.mkdir(exist_ok=True)
    run(["ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-i", str(src),
         "-vf", f"scale=-2:{HEIGHT}", "-c:v", "libx264", "-preset", "slow",
         "-g", "48", "-keyint_min", "48", "-sc_threshold", "0",
         "-crf", str(crf), "-maxrate", f"{cap_k}k", "-bufsize", f"{cap_k}k",
         "-an", str(out)])
    return out

def vmaf(src, enc):
    log = enc.with_suffix(".vmaf.json")
    # distorted first, reference second; scale the reference to match
    run(["ffmpeg", "-hide_banner", "-loglevel", "error",
         "-i", str(enc), "-i", str(src),
         "-lavfi", f"[1:v]scale=-2:{HEIGHT}[ref];[0:v][ref]libvmaf=log_fmt=json:log_path={log}:n_threads=8",
         "-f", "null", "-"])
    d = json.loads(log.read_text())
    frames = [f["metrics"]["vmaf"] for f in d["frames"]]
    return d["pooled_metrics"]["vmaf"]["mean"], min(frames)

def bitrate_profile(enc, cap_k):
    # per-second bitrate from packet sizes; count seconds at >= 95% of cap
    out = run(["ffprobe", "-v", "error", "-select_streams", "v:0",
               "-show_entries", "packet=pts_time,size", "-of", "csv=p=0", str(enc)]).stdout
    buckets = {}
    for line in out.strip().splitlines():
        pts, size = line.split(",")
        buckets[int(float(pts))] = buckets.get(int(float(pts)), 0) + int(size)
    kbps = [b * 8 / 1000 for b in buckets.values()]
    avg = sum(kbps) / len(kbps)
    at_cap = sum(1 for k in kbps if k >= 0.95 * cap_k) / len(kbps)
    return avg, at_cap

with open("results.csv", "w", newline="") as f:
    w = csv.writer(f)
    w.writerow(["clip", "crf", "cap_k", "vmaf_mean", "vmaf_min_frame", "avg_kbps", "pct_seconds_at_cap"])
    for src in CLIPS:
        for crf in CRFS:
            for cap in CAPS_K:
                enc = encode(src, crf, cap)
                mean, low = vmaf(src, enc)
                avg, at_cap = bitrate_profile(enc, cap)
                w.writerow([src.name, crf, cap, f"{mean:.2f}", f"{low:.2f}", f"{avg:.0f}", f"{at_cap:.0%}"])
                print(f"{src.name:24} crf={crf} cap={cap}k  vmaf={mean:.2f} low={low:.2f}  {avg:.0f} kbps  at_cap={at_cap:.0%}", file=sys.stderr)

python sweep.py

lecture.mp4              crf=21 cap=6000k  vmaf=97.1 low=88.4  2410 kbps  at_cap=0%
lecture.mp4              crf=23 cap=6000k  vmaf=96.3 low=86.9  1680 kbps  at_cap=0%
lecture.mp4              crf=25 cap=6000k  vmaf=95.0 low=84.7  1190 kbps  at_cap=0%
football.mp4             crf=23 cap=6000k  vmaf=92.8 low=61.2  5840 kbps  at_cap=91%
football.mp4             crf=26 cap=12000k vmaf=92.9 low=74.0  4720 kbps  at_cap=12%
...

Those lines show the shape of the output, not numbers you should expect; your clips will differ. But the two patterns are the ones to look for:

Easy clip, at_cap=0%. CRF is in control the whole time. Raising CRF from 21 to 25 drops bitrate a lot and VMAF a little. This is the content you've been overpaying for.
Hard clip, at_cap high. The cap is in control; the CRF value barely matters. Lowest-frame VMAF is poor because a hard ceiling starves hard frames. A higher cap with a higher CRF can lower the average bitrate while raising the lowest-frame score, because the encoder stops smearing bits evenly across frames that need them unevenly. This counterintuitive result is documented in Jan Ozer's February 2026 case study, where a sports clip went from 5542 to 4593 kbps average by moving from CRF 24 at a 6 Mbps cap to CRF 26 at 12 Mbps, with lowest-frame VMAF rising from 63.81 to 75.21.

⚠️ Note: pooled_metrics.vmaf.mean is the headline, but vmaf_min_frame is where viewer complaints live. A 93 average with a 60 floor means one scene looks broken. Track both.

4. 🎯 Pick the target, then pick per genre

A VMAF of 93 is the widely quoted "good enough" line (most viewers judge the encode indistinguishable from the source or noticeably but not annoyingly different). Scores well above 95 are bandwidth nobody notices.

Open results.csv and, per clip, find the cheapest (CRF, cap) row with vmaf_mean >= 93 and an acceptable floor. You'll likely find they cluster by genre, not by file. Ozer's advice is 10 to 20 test files per genre before you give a genre its own profile. Then the production config is a tiny table:

# app/encode_profiles.py
PROFILES = {
    #        1080p             720p             480p             360p
    "talking_head": [(25, 4000), (25, 2000), (26, 1000), (27, 600)],
    "sports":       [(26, 12000), (26, 6000), (26, 3000), (27, 1500)],
    "animation":    [(24, 5000), (24, 2500), (25, 1200), (26, 700)],
    "default":      [(23, 6000), (23, 3000), (24, 1500), (25, 800)],
}
# (crf, maxrate_kbps); bufsize = maxrate unless you've measured a reason otherwise

Those numbers are placeholders shaped like real ones. Fill them from your own results.csv.

5. 🔁 Same idea, other encoders

The decision transfers across codecs; the flags don't.

Encoder	Capped-CRF form	Notes
`libx264` / `libx265`	`-crf N -maxrate Xk -bufsize Yk`	Single pass.
`h264_nvenc` / `hevc_nvenc` / `av1_nvenc`	`-cq N -rc:v vbr -maxrate Xk -bufsize Yk`	CQ scale differs from CRF; re-sweep.
`libsvtav1`	`-crf N -maxrate Xk -bufsize Yk`	CRF + maxrate/bufsize is supported by FFmpeg's SVT-AV1 wrapper.
`libvpx-vp9`	`-crf N -b:v 0 -maxrate Xk -bufsize Yk` (or low `-b:v` + `-qmax` as a floor)	Ozer's client used the `-qmax` approach for VP9 and found it cheaper than capped CRF at equal quality.

# NVENC H.264, same shape
ffmpeg -y -i in.mp4 -c:v h264_nvenc -preset p5 -cq 32 -rc:v vbr -maxrate 6000k -bufsize 6000k -an out_nvenc.mp4

# SVT-AV1
ffmpeg -y -i in.mp4 -c:v libsvtav1 -preset 6 -crf 35 -maxrate 4000k -bufsize 4000k -g 48 -an out_av1.mp4

Change encode() in sweep.py to take an encoder string and you have the same report for every codec you ship.

6. ✅ Sanity checks before you ship the ladder

# keyframes still aligned across rungs (first 10)
for f in out_1080_crf.mp4 out_720_crf.mp4; do
  ffprobe -v error -select_streams v:0 -skip_frame nokey -show_entries frame=pts_time -of csv=p=0 "$f" | head -10 | tr '\n' ' '; echo
done

0.000000 2.000000 4.000000 6.000000 8.000000 10.000000 12.000000 14.000000 16.000000 18.000000
0.000000 2.000000 4.000000 6.000000 8.000000 10.000000 12.000000 14.000000 16.000000 18.000000

And a bandwidth check: the HLS BANDWIDTH attribute per rung should be the measured peak, not the old nominal target. With capped CRF the peak is bounded by maxrate (plus VBV slack), so set BANDWIDTH to your cap and AVERAGE-BANDWIDTH to the measured average from results.csv.

What's next

Wire PROFILES into your upload pipeline with a genre tag at ingest, and keep the default profile for untagged content.
Add a weekly job that re-runs sweep.py on a random sample of new uploads, so your CRF values track the catalogue as it changes.
If you're already on a managed encoder, check whether it exposes a quality target plus cap (some call it "constrained quality" or "content-adaptive"); the measuring approach here still applies to what it produces.

Prove SRT survives packet loss and RTMP doesn't: MediaMTX, ffmpeg, and tc netem on one laptop

Mason K — Tue, 06 Oct 2026 11:56:55 +0000

TL;DR

We'll run MediaMTX locally, publish a test pattern to it over RTMP and over SRT, pull both back out, then add packet loss to the network path with tc netem. RTMP stalls and reconnects; SRT retransmits inside its latency window and keeps going. Then we'll size that window properly from a real RTT and push it to a relay.

RTMP rides on TCP, so one lost packet blocks everything behind it until it's retransmitted. SRT rides on UDP with its own selective retransmission and a latency budget you choose. Everyone says this; almost nobody has watched it happen. Let's watch it happen.

You need Linux (netem is a Linux qdisc; on macOS run this inside a Linux VM or container with NET_ADMIN), ffmpeg 7.x or 8.x built with libsrt (check with ffmpeg -protocols | grep srt), and MediaMTX. We'll do everything on loopback so there's nothing to deploy.

1. 🛠️ Start MediaMTX with RTMP and SRT ingest

MediaMTX is a single static binary that speaks SRT, RTMP, RTSP, WebRTC, HLS and more, with no dependencies. Grab the latest release for your platform from the GitHub releases page (1.19.x at the time of writing) and run it:

mkdir srt-lab && cd srt-lab
# download + extract mediamtx for your platform, then:
./mediamtx

2026/10/05 10:02:11 INF MediaMTX v1.19.0
2026/10/05 10:02:11 INF configuration loaded from /home/me/srt-lab/mediamtx.yml
2026/10/05 10:02:11 INF [RTSP] listener opened on :8554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
2026/10/05 10:02:11 INF [RTMP] listener opened on :1935
2026/10/05 10:02:11 INF [HLS] listener opened on :8888
2026/10/05 10:02:11 INF [WebRTC] listener opened on :8889 (HTTP), :8189 (ICE/UDP)
2026/10/05 10:02:11 INF [SRT] listener opened on :8890 (UDP)

The shipped mediamtx.yml enables both RTMP (:1935) and SRT (:8890) by default. Leave it alone for now.

💡 Tip: with Docker, docker run --rm --network host bluenviron/mediamtx does the same thing. Host networking matters later, because we'll shape the host's loopback interface.

2. 📡 Publish the same source over both protocols

We'll use a synthetic source so the test is repeatable and nobody has to find a sample file: testsrc2 with a burned-in timestamp, plus a sine tone. Two publishers, two paths.

# publish_rtmp.sh
ffmpeg -re -f lavfi -i "testsrc2=size=1280x720:rate=30" -f lavfi -i "sine=frequency=440" \
  -vf "drawtext=text='%{pts\:hms}':fontsize=48:fontcolor=white:x=20:y=20:box=1:boxcolor=black@0.6" \
  -c:v libx264 -preset veryfast -tune zerolatency -b:v 3000k -maxrate 3000k -bufsize 6000k -g 60 \
  -c:a aac -b:a 128k \
  -f flv rtmp://127.0.0.1:1935/rtmp_test

# publish_srt.sh
ffmpeg -re -f lavfi -i "testsrc2=size=1280x720:rate=30" -f lavfi -i "sine=frequency=440" \
  -vf "drawtext=text='%{pts\:hms}':fontsize=48:fontcolor=white:x=20:y=20:box=1:boxcolor=black@0.6" \
  -c:v libx264 -preset veryfast -tune zerolatency -b:v 3000k -maxrate 3000k -bufsize 6000k -g 60 \
  -c:a aac -b:a 128k \
  -f mpegts "srt://127.0.0.1:8890?streamid=publish:srt_test&pkt_size=1316&latency=120000"

Three things to notice in the SRT URL:

streamid=publish:<path> is how MediaMTX knows this connection is a publisher and which path it feeds. The same server reads with streamid=read:<path>.
pkt_size=1316 keeps MPEG-TS packets inside a single UDP datagram (7 × 188 bytes). Leave it.
latency=120000 is in microseconds. That's 120 ms, the ffmpeg/OBS default, and it's deliberately too small for this experiment. We'll fix it in step 5.

Run both in separate terminals. MediaMTX logs should show both paths ready:

INF [RTMP] [conn 127.0.0.1:51822] is publishing to path 'rtmp_test', 2 tracks (H264, MPEG-4 Audio)
INF [SRT] [conn 127.0.0.1:51823] is publishing to path 'srt_test', 2 tracks (H264, MPEG-4 Audio)

3. 👀 Read both back and log frame arrivals

To see what the viewer sees, pull each path back out and log received frames per second. Plain ffmpeg -f null with -progress is enough; no player needed.

# read_rtmp.sh  (reads over RTMP too, so the impaired leg is encoder -> server -> reader)
ffmpeg -hide_banner -i rtmp://127.0.0.1:1935/rtmp_test -f null - -progress pipe:1 2>/dev/null \
  | grep --line-buffered -E "^(frame|drop_frames)=" | paste - - | ts '%H:%M:%S'

# read_srt.sh
ffmpeg -hide_banner -i "srt://127.0.0.1:8890?streamid=read:srt_test&latency=120000" -f null - -progress pipe:1 2>/dev/null \
  | grep --line-buffered -E "^(frame|drop_frames)=" | paste - - | ts '%H:%M:%S'

(ts is from moreutils; apt install moreutils. If you'd rather not, drop the last pipe.)

With no impairment, both tick along at 30 frames per second:

10:05:01 frame=30   drop_frames=0
10:05:02 frame=60   drop_frames=0
10:05:03 frame=90   drop_frames=0

4. 💥 Inject loss with tc netem

netem is the Linux kernel's network emulator. It attaches to an interface's egress and can delay, jitter, reorder and drop packets. We'll put it on loopback, which affects every leg (publisher → server and server → reader) in both directions. That's fine for a comparison, because both protocols get the identical impairment; just know that the numbers you see are "loss applied twice".

Start gentle:

sudo tc qdisc add dev lo root netem delay 40ms 10ms loss 2%

That's 40 ms of delay with 10 ms jitter and 2 percent random loss, roughly a bad hotel Wi-Fi. Watch the two reader terminals.

The SRT reader keeps counting, maybe with the odd dropped frame. The RTMP reader starts to hitch: frame counts stall for a second or two, then jump. Now make it realistic. Real loss is bursty, and netem has a state model for that:

sudo tc qdisc change dev lo root netem delay 40ms 10ms loss state 8% 25%

Within a few seconds the RTMP side looks like this:

10:07:14 frame=4110  drop_frames=0
10:07:18 frame=4110  drop_frames=0      <- stalled, TCP waiting on a retransmit
10:07:22 frame=4111  drop_frames=0

Then the counter stops updating entirely. Remove the 2>/dev/null and you'll see why: ffmpeg reports an Input/output error on the RTMP input and exits. The RTMP publisher terminal usually dies around the same time with a broken pipe or a timeout. In OBS this is the moment the status bar goes red and the encoder reconnects.

The SRT reader at 120 ms latency is not happy either, but it's a different kind of unhappy:

10:07:14 frame=4108  drop_frames=14
10:07:15 frame=4131  drop_frames=21
10:07:16 frame=4159  drop_frames=23

It drops frames it couldn't recover in time and keeps going. No stall, no reconnect. Those line values are illustrative of the shape; your counts will differ run to run.

⚠️ Note: netem on lo shapes your own machine's loopback. If you're doing this on a remote box over SSH, do NOT attach to the interface carrying your SSH session; put MediaMTX in a network namespace or use a second interface. And always clean up: sudo tc qdisc del dev lo root.

5. 🎛️ Size the latency window properly

SRT's latency is the budget it's allowed to spend recovering a packet before giving up on it. The OBS project's rule, which matches my experience: at least 2.5 × round-trip time, and more if loss comes in bursts. The URL value is in microseconds.

With our 40 ms one-way delay applied in both directions, the RTT is about 80 ms plus jitter, so 2.5× is 200 ms. Give it headroom for the bursty model and try 600 ms on both ends (publisher and reader each set their own; the connection negotiates up to the larger):

# publisher
... -f mpegts "srt://127.0.0.1:8890?streamid=publish:srt_test&pkt_size=1316&latency=600000"
# reader
ffmpeg -i "srt://127.0.0.1:8890?streamid=read:srt_test&latency=600000" ...

Re-run the bursty loss. Dropped frames on the SRT reader should fall to near zero while RTMP still dies on schedule. If you still see drops, double the latency; you're trading delay for resilience, and that trade is the whole feature.

For a real deployment, measure instead of guessing:

ping -c 20 ingest.example.com | tail -1
# rtt min/avg/max/mdev = 96.4/101.2/118.9/5.1 ms

2.5 × 101 ms ≈ 250 ms minimum; on a link with bursty loss I'd start at 1 s (latency=1000000) and lower it only if the glass-to-glass delay actually matters for the use case.

6. 🔁 The relay pattern (because Twitch still wants RTMP)

The big consumer platforms ingest RTMP, not SRT. So in production, SRT covers the bad first mile (venue → relay) and RTMP covers the good last mile (relay → platform, from a machine with a solid connection). The OBS wiki documents the ffmpeg version of this:

# on the relay VM, near the platform's ingest
ffmpeg -i "srt://:8890?mode=listener&transtype=live&latency=1000000" \
  -c copy -f flv rtmp://live-xxx.twitch.tv/app/$STREAM_KEY

Or let MediaMTX do it: publish SRT into a path and forward the path to an RTMP destination from mediamtx.yml. In OBS, point Settings > Stream > Custom at srt://relay-host:8890?streamid=publish:live&latency=1000000, key blank. SRT output has been in OBS since 25.0.

Add passphrase=...&pbkeylen=16 to both ends if the relay is on the public internet; SRT encrypts the payload with AES and the passphrase is the shared secret.

7. 🧹 Clean up

sudo tc qdisc del dev lo root
pkill -f publish_ && pkill mediamtx

Wrapping up

	RTMP (TCP)	SRT (UDP + ARQ)
Lost packet	Blocks everything behind it until retransmitted	Only that packet is re-requested
Sustained loss	Stall, then disconnect	Degrades: late packets are dropped, stream continues
Tuning knob	None	`latency` (≥ 2.5 × RTT, microseconds in URLs)
Platform ingest	Twitch, YouTube, etc.	Your relay, media servers, managed live APIs that accept SRT
Encryption	RTMPS (TLS)	Built-in AES via `passphrase`

What's next

Repeat step 4 with loss state 15% 40% and see where SRT's recovery stops keeping up; the OBS wiki quotes roughly 15 percent as the range SRT can recover from. RIST, a sibling protocol, claims higher.
Scrape MediaMTX's Prometheus metrics endpoint during the test to graph the two paths side by side instead of eyeballing terminals.
If you serve viewers from the same box, add LL-HLS on MediaMTX's :8888 and measure glass-to-glass latency at each SRT latency setting, so the number you pick is a measured trade-off rather than a guess.

Build a members-only video library in Python: private media, RS256 playback tokens, and tokenized thumbnails

Mason K — Tue, 06 Oct 2026 11:56:30 +0000

TL;DR

We'll build the backend for a members-only video library: upload assets as private, create an RSA signing key, mint a short-lived JWT per viewer in a FastAPI endpoint, and hand the player a stream URL plus a thumbnail URL that both carry the token. Then we'll prove the thumbnail is locked too, because that's the one everybody forgets.

The naive version of a paywalled video library is: check the session, render a player, done. The .m3u8 URL in that player is public. Anyone who copies it from DevTools has your content forever.

The fixed version has three parts: the asset has no public address, every playback URL is a token minted for one viewer and expiring soon, and the stills (thumbnail, scrub-bar spritesheet) require the same token. We'll use FastPix as the video API because its signing-key flow is a single endpoint and the thumbnails honour the playback token. The pattern is the same shape on Mux or Cloudflare Stream; only the claim names change.

You need python 3.11+ (the FastPix SDK itself supports 3.9.2 or later), a FastPix account with an Access Token ID and Secret Key, and a public URL for a webhook (ngrok is fine for dev).

1. 🧱 Project setup

mkdir members-video && cd members-video
python -m venv .venv && source .venv/bin/activate
pip install fastapi "uvicorn[standard]" httpx "pyjwt[crypto]" fastpix-python python-dotenv

# .env
FASTPIX_TOKEN_ID=...
FASTPIX_SECRET=...
FASTPIX_SIGNING_KEY_ID=      # filled in step 2
FASTPIX_PRIVATE_KEY_B64=     # filled in step 2
DATABASE_URL=sqlite:///./library.db

The [crypto] extra on PyJWT matters: RS256 needs the cryptography package, and without it you get NotImplementedError: Algorithm 'RS256' could not be found at the worst possible time.

2. 🔑 Create a signing key (once)

A signing key on FastPix is an RSA key pair. You call the endpoint, the API generates a 2048-bit pair, returns the private half to you once (base64-encoded) and keeps only the public half for verification. It does not save your private key, so if you lose it, you make a new key.

curl -s -X POST https://api.fastpix.com/v1/iam/signing-keys \
  -u "$FASTPIX_TOKEN_ID:$FASTPIX_SECRET" | python -m json.tool

{
  "success": true,
  "data": {
    "id": "fc9d9368-6ee5-4b16-ae50-880a2374bdc4",
    "privateKey": "LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2...",
    "createdAt": "2026-10-05T09:14:06.618993Z"
  }
}

Put data.id into FASTPIX_SIGNING_KEY_ID and data.privateKey into FASTPIX_PRIVATE_KEY_B64. Decode it once at startup:

# app/signing.py
import base64, os, time
import jwt

SIGNING_KEY_ID = os.environ["FASTPIX_SIGNING_KEY_ID"]
PRIVATE_KEY_PEM = base64.b64decode(os.environ["FASTPIX_PRIVATE_KEY_B64"]).decode()

def playback_token(playback_id: str, *, ttl_seconds: int = 3600) -> str:
    now = int(time.time())
    payload = {
        "kid": SIGNING_KEY_ID,
        "aud": f"media:{playback_id}",
        "iat": now,
        "exp": now + ttl_seconds,
    }
    return jwt.encode(payload, PRIVATE_KEY_PEM, algorithm="RS256",
                      headers={"kid": SIGNING_KEY_ID})

Three claims do the work. kid names the key so you can rotate later. aud is media:<playbackId>, which scopes the token to one asset. exp is the whole revocation story (more on that in step 6).

⚠️ Note: the FastPix JWT guide shows a generic HS256 example with a shared secret. The key the API issues you is an RSA private key and FastPix verifies with the stored public key, so the matching algorithm is RS256. If a token is rejected, test the same inputs in their hosted signer at jwt.fastpix.co and compare the decoded header.

3. 📤 Upload assets as private

We'll ingest from a URL with the official Python SDK. The one field that matters for this tutorial is access_policy="private". A private asset gets no public playback ID at all.

# app/ingest.py
import os
from fastpix_python import Fastpix, models

def create_private_media(source_url: str, title: str, course_id: str) -> dict:
    with Fastpix(security=models.Security(
        username=os.environ["FASTPIX_TOKEN_ID"],
        password=os.environ["FASTPIX_SECRET"],
    )) as fastpix:
        res = fastpix.input_video.create_media(
            inputs=[{"type": "video", "url": source_url}],
            access_policy="private",
            metadata={"courseId": course_id, "title": title},
        )
    return res.model_dump(mode="json", by_alias=True)["data"]

The create response's data is an object (not an array) carrying id, status and playbackIds[]. Store id against your lesson row now; the playback ID is in that list, but don't serve it until the asset is ready.

Readiness arrives on a webhook. Point your FastPix workspace's webhook URL at /webhooks/fastpix and handle the video.media.ready event:

# app/webhooks.py
from fastapi import APIRouter, Request
from .db import mark_ready

router = APIRouter()

@router.post("/webhooks/fastpix")
async def fastpix_webhook(request: Request):
    event = await request.json()
    if event.get("type") == "video.media.ready":
        media = event["data"]
        mark_ready(media_id=media["id"],
                   playback_id=media["playbackIds"][0]["id"])
    return {"ok": True}

💡 Tip: treat the webhook as idempotent. Processing-pipeline events can arrive more than once; mark_ready should be an UPDATE that's safe to repeat.

4. 🎟️ The playback endpoint: membership check, then mint

This is the only place tokens get created. The membership check happens first; the token is proof that it passed.

# app/main.py
from fastapi import FastAPI, Depends, HTTPException
from .auth import current_user          # your session logic
from .db import get_lesson, user_can_watch
from .signing import playback_token
from .webhooks import router as webhooks

app = FastAPI()
app.include_router(webhooks)

STREAM = "https://stream.fastpix.com/{pid}.m3u8?token={tok}"
THUMB  = "https://images.fastpix.com/{pid}/thumbnail.jpg?token={tok}"
SPRITE = "https://images.fastpix.com/{pid}/spritesheet.jpg?token={tok}"

@app.get("/api/lessons/{lesson_id}/playback")
def playback(lesson_id: int, user=Depends(current_user)):
    lesson = get_lesson(lesson_id)
    if not lesson or not lesson.playback_id:
        raise HTTPException(404)
    if not user_can_watch(user.id, lesson.course_id):
        raise HTTPException(403)

    tok = playback_token(lesson.playback_id, ttl_seconds=2 * 3600)
    return {
        "playbackId": lesson.playback_id,
        "token": tok,
        "streamUrl": STREAM.format(pid=lesson.playback_id, tok=tok),
        "thumbnailUrl": THUMB.format(pid=lesson.playback_id, tok=tok),
        "spritesheetUrl": SPRITE.format(pid=lesson.playback_id, tok=tok),
        "expiresInSeconds": 2 * 3600,
    }

Run it:

uvicorn app.main:app --reload
# INFO:     Uvicorn running on http://127.0.0.1:8000

curl -s -b session=... http://127.0.0.1:8000/api/lessons/42/playback | python -m json.tool

{
  "playbackId": "b331e0d8-bef4-4ad2-8760-757fdb2818b7",
  "token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImZjOWQ5MzY4...",
  "streamUrl": "https://stream.fastpix.com/b331e0d8-...m3u8?token=eyJ...",
  "thumbnailUrl": "https://images.fastpix.com/b331e0d8-.../thumbnail.jpg?token=eyJ...",
  "spritesheetUrl": "https://images.fastpix.com/b331e0d8-.../spritesheet.jpg?token=eyJ...",
  "expiresInSeconds": 7200
}

Notice there is no user ID inside the token. There doesn't need to be. The token is scoped to one playback ID and one time window; who it was issued to is your database's business, not the CDN's.

5. ▶️ Player side

The FastPix web player takes the token as an attribute, so the front-end is a fetch and a render:

<!-- templates/lesson.html -->
<script type="module">
  const r = await fetch(`/api/lessons/${LESSON_ID}/playback`);
  const { playbackId, token } = await r.json();
  const p = document.createElement("fastpix-player");
  p.setAttribute("playback-id", playbackId);
  p.setAttribute("token", token);
  p.setAttribute("stream-type", "on-demand");
  document.querySelector("#stage").replaceChildren(p);
</script>

If you'd rather use hls.js or a native <video>, hand it streamUrl directly. The token is just a query parameter on the manifest URL.

For the poster frame, use thumbnailUrl from the same response. Do not build a thumbnail URL without the token "because it's only an image". That image is a frame of the paid content.

6. Expiry, rotation, and the retrofit trap

Expiry is the revocation mechanism. When a membership lapses, stop minting; the current token dies on its own. Pick a TTL that covers one sitting of your longest content and add a refresh call for the long tail. Before you commit to a TTL, run this test: mint a 120-second token, start playback, and watch what happens at 180 seconds. That tells you whether your provider validates on the manifest only or on every segment, and therefore whether your refresh has to be proactive.

Rotation is why kid is in every token. Suspect a leak? Create a second signing key, switch FASTPIX_SIGNING_KEY_ID and the PEM, then delete the old key through the signing-keys API. Tokens signed by the deleted key stop verifying. No flag day, no downtime.

The retrofit trap. If your library was public before, setting the media to private does not revoke public playback IDs that already exist; the FastPix docs call this out explicitly. List the playback IDs for each asset and delete the public ones:

curl -s -u "$FASTPIX_TOKEN_ID:$FASTPIX_SECRET" \
  https://api.fastpix.com/v1/on-demand/$MEDIA_ID/playback-ids | python -m json.tool
# look for "accessPolicy": "public" entries, then DELETE each by id

Create a fresh private one if the asset has none:

curl -s -X POST https://api.fastpix.com/v1/on-demand/$MEDIA_ID/playback-ids \
  -u "$FASTPIX_TOKEN_ID:$FASTPIX_SECRET" -H 'Content-Type: application/json' \
  -d '{"accessPolicy": "private"}'

Optional belt-and-braces for embed abuse: per-playback-ID domain restrictions (up to 25 entries, wildcards like *.yourdomain.com) via PATCH /v1/on-demand/{mediaId}/playback-ids/{playbackId}/domains. It's not a substitute for tokens; it's a cheap extra fence.

7. ✅ The four-request launch test

PID=b331e0d8-bef4-4ad2-8760-757fdb2818b7
curl -s -o /dev/null -w "%{http_code}\n" "https://stream.fastpix.com/$PID.m3u8"
curl -s -o /dev/null -w "%{http_code}\n" "https://images.fastpix.com/$PID/thumbnail.jpg"
curl -s -o /dev/null -w "%{http_code}\n" "https://images.fastpix.com/$PID/spritesheet.jpg"
curl -s -o /dev/null -w "%{http_code}\n" "https://stream.fastpix.com/$PID.m3u8?token=$EXPIRED_TOKEN"

All four should come back as client errors (not 200). If the thumbnail line prints 200, you've shipped a public trailer for private content. Then one more: the token from step 4 on the stream URL should return 200 and a manifest.

Wrapping up

Layer	What it stops	Where it lives
`accessPolicy: private`	Any URL without a token	On the asset
Per-viewer RS256 JWT, short `exp`	Shared links outliving a session	Your backend
Token on thumbnail + spritesheet	Leaking frames via "just images"	Same token, image host
`kid` + key rotation	A leaked private key	Signing-keys API
Domain allow-list	Hot-linked embeds	Per playback ID

What's next

The FastPix token guide covers the claim set, the hosted signer for debugging, and custom-domain signed URLs if you serve from media.yourdomain.com: https://fastpix.com/docs/video-security/generate-jwts-for-secure-media
Add a /playback/refresh endpoint and have the player call it a minute before expiresInSeconds runs out.
The same design works on Mux (signed playback IDs) and Cloudflare Stream (signed URLs); the claim names differ, the three layers don't.

Test your video pipeline in CI for free: synthetic fixtures, ffprobe goldens, and webhook replay with pytest

Mason K — Tue, 29 Sep 2026 09:13:06 +0000

TL;DR

We'll give a video pipeline a real test suite that runs on every pull request, costs nothing, and stores no video files in git. Fixtures are generated with FFmpeg's lavfi source, outputs are asserted with ffprobe JSON snapshots (not byte comparisons), and webhook handlers are tested by replaying recorded payloads. pytest, FFmpeg 8.1 pinned in GitHub Actions, Python 3.12.

What we're building

A tests/ directory for a pipeline that looks like most of them: a probe() step, a build_ffmpeg_args() step that picks a ladder, a transcode() step that shells out, and a webhook handler that advances state when the encoder (local or managed) reports back.

The four tiers, in order of how often they run:

Tier	Needs FFmpeg?	Needs network?	Runs on
1. Pure logic (arg builders, ladder policy)	no	no	every commit
2. FFmpeg in the loop on synthetic fixtures	yes	no	every commit
3. Webhook replay from recorded payloads	no	no	every commit
4. One real provider call	yes	yes	nightly, budgeted

This post builds tiers 2 and 3 in full and shows the skeleton for 1 and 4.

1. Project layout

pipeline/
  probe.py          # ffprobe wrapper
  ladder.py         # pure: probe result -> list of renditions
  transcode.py      # shells out to ffmpeg
  webhooks.py       # FastAPI handler
tests/
  conftest.py       # fixture factory
  test_ladder.py    # tier 1
  test_transcode.py # tier 2
  test_webhooks.py  # tier 3
  fixtures/webhooks/*.json
  goldens/*.json
.github/workflows/ci.yml

python -m venv .venv && source .venv/bin/activate
pip install pytest pytest-snapshot fastapi httpx

2. The fixture factory (no binaries in git) 🎬

FFmpeg can synthesise a test pattern and a tone from nothing. A two-second 320x240 clip is a few hundred KB and takes well under a second to make.

# tests/conftest.py
import subprocess
from pathlib import Path
import pytest

def make_fixture(
    out: Path,
    seconds: float = 2,
    size: str = "320x240",
    fps: int = 24,
    audio: bool = True,
    vcodec: str = "libx264",
) -> Path:
    cmd = ["ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
           "-f", "lavfi", "-i", f"testsrc2=duration={seconds}:size={size}:rate={fps},format=yuv420p"]
    if audio:
        cmd += ["-f", "lavfi", "-i", f"sine=frequency=440:duration={seconds}"]
    cmd += ["-c:v", vcodec, "-preset", "ultrafast", "-crf", "28"]
    if audio:
        cmd += ["-c:a", "aac", "-shortest"]
    cmd += [str(out)]
    subprocess.run(cmd, check=True)
    return out

@pytest.fixture
def fixture_factory(tmp_path):
    def _make(name="in.mp4", **kw):
        return make_fixture(tmp_path / name, **kw)
    return _make

Now every test can ask for exactly the input it needs:

fixture_factory()                                   # 2s, 320x240, 24fps, with audio
fixture_factory("vertical.mp4", size="240x426")     # portrait source
fixture_factory("silent.mp4", audio=False)          # no audio stream
fixture_factory("hfr.mp4", fps=60)                  # high frame rate

💡 Tip: testsrc2 produces a pattern with motion and text, which is closer to real content than plain colour bars and makes encoder behaviour (keyframes, bitrate) more realistic.

3. Tier 1: pure logic, no FFmpeg

The refactor that makes everything else possible: keep "decide what to run" separate from "run it".

# pipeline/ladder.py
from dataclasses import dataclass

@dataclass(frozen=True)
class Rendition:
    height: int
    video_kbps: int

LADDER = [Rendition(1080, 5000), Rendition(720, 2800), Rendition(480, 1400), Rendition(360, 800)]

def pick_ladder(width: int, height: int) -> list[Rendition]:
    """Never upscale. Portrait sources are judged by their short edge."""
    short_edge = min(width, height)
    return [r for r in LADDER if r.height <= short_edge]

# tests/test_ladder.py
from pipeline.ladder import pick_ladder

def test_720p_source_gets_three_rungs():
    assert [r.height for r in pick_ladder(1280, 720)] == [720, 480, 360]

def test_portrait_source_uses_short_edge():
    assert [r.height for r in pick_ladder(1080, 1920)] == [1080, 720, 480, 360]

def test_tiny_source_gets_nothing_above_itself():
    assert pick_ladder(320, 240) == []

These run in milliseconds and catch a surprising share of real bugs, because ladder logic is where the edge cases live.

4. Tier 2: FFmpeg in the loop, asserted with ffprobe

Encoders are not bit-exact across versions, so never compare output bytes. Probe the output and assert on the properties your pipeline is responsible for.

# pipeline/probe.py
import json, subprocess
from pathlib import Path

def probe(path: Path) -> dict:
    out = subprocess.run(
        ["ffprobe", "-v", "error", "-print_format", "json", "-show_streams", "-show_format", str(path)],
        capture_output=True, text=True, check=True,
    ).stdout
    return json.loads(out)

STABLE_STREAM_KEYS = ("codec_type", "codec_name", "width", "height", "pix_fmt", "r_frame_rate", "channels", "sample_rate")

def stable_view(p: dict) -> dict:
    """Keep only fields that survive FFmpeg upgrades. Drop bit_rate, size, tags."""
    return {
        "streams": [{k: s.get(k) for k in STABLE_STREAM_KEYS if k in s} for s in p["streams"]],
        "duration": round(float(p["format"]["duration"]), 1),
        "nb_streams": p["format"]["nb_streams"],
    }

# pipeline/transcode.py
import subprocess
from pathlib import Path
from .ladder import Rendition

def transcode(src: Path, dst: Path, r: Rendition) -> Path:
    cmd = ["ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-i", str(src),
           "-vf", f"scale=-2:{r.height}", "-c:v", "libx264", "-preset", "veryfast",
           "-b:v", f"{r.video_kbps}k", "-g", "48", "-keyint_min", "48", "-sc_threshold", "0",
           "-c:a", "aac", "-b:a", "128k", "-movflags", "+faststart", str(dst)]
    subprocess.run(cmd, check=True)
    return dst

Now the tests. One direct assertion, one golden snapshot.

# tests/test_transcode.py
import json
from pipeline.ladder import Rendition
from pipeline.probe import probe, stable_view
from pipeline.transcode import transcode

def test_360p_rendition_has_expected_shape(fixture_factory, tmp_path):
    src = fixture_factory(size="640x360", seconds=2)
    out = transcode(src, tmp_path / "out.mp4", Rendition(360, 800))
    view = stable_view(probe(out))

    video = next(s for s in view["streams"] if s["codec_type"] == "video")
    audio = next(s for s in view["streams"] if s["codec_type"] == "audio")
    assert video["codec_name"] == "h264"
    assert (video["width"], video["height"]) == (640, 360)
    assert video["pix_fmt"] == "yuv420p"
    assert audio["codec_name"] == "aac"
    assert view["duration"] == 2.0

def test_360p_rendition_matches_golden(fixture_factory, tmp_path, snapshot):
    src = fixture_factory(size="640x360", seconds=2)
    out = transcode(src, tmp_path / "out.mp4", Rendition(360, 800))
    snapshot.snapshot_dir = "tests/goldens"
    snapshot.assert_match(json.dumps(stable_view(probe(out)), indent=2, sort_keys=True), "360p.json")

First run creates the golden; later runs diff against it.

pytest tests/test_transcode.py --snapshot-update   # once, to create tests/goldens/360p.json
pytest tests/test_transcode.py
# tests/test_transcode.py ..                                              [100%]
# 2 passed in 1.84s

⚠️ Note: if a golden ever includes bit_rate or size, it will break on the next FFmpeg upgrade and someone will delete the test. stable_view() exists to stop that.

The keyframe cadence you set with -g 48 is stable across builds and worth asserting too:

def keyframe_indices(path):
    out = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "v:0",
                          "-show_entries", "frame=key_frame", "-of", "csv=p=0", str(path)],
                         capture_output=True, text=True, check=True).stdout.split()
    return [i for i, k in enumerate(out) if k == "1"]

def test_keyframes_every_48_frames(fixture_factory, tmp_path):
    src = fixture_factory(size="640x360", seconds=4, fps=24)
    out = transcode(src, tmp_path / "out.mp4", Rendition(360, 800))
    assert keyframe_indices(out) == [0, 48]

5. Tier 3: webhook replay from recorded payloads

The provider is not part of your test suite. Its payloads are. Capture one real payload per event type once (from a dev environment, with a request logger), strip anything sensitive, and commit the JSON.

// tests/fixtures/webhooks/media_ready.json
{
  "type": "media.ready",
  "id": "evt_01",
  "data": { "id": "media_abc123", "status": "ready", "playback_url": "https://cdn.example.com/media_abc123/index.m3u8" }
}

// tests/fixtures/webhooks/media_failed.json
{
  "type": "media.failed",
  "id": "evt_02",
  "data": { "id": "media_abc123", "status": "failed", "error": { "message": "unsupported codec" } }
}

Field names will differ per provider; the shape of the tests does not.

# pipeline/webhooks.py
from fastapi import FastAPI, Request

app = FastAPI()
STATE: dict[str, str] = {}          # media_id -> status; swap for your DB
SEEN: set[str] = set()              # event ids, for idempotency

@app.post("/webhooks/video")
async def video_webhook(req: Request):
    evt = await req.json()
    if evt["id"] in SEEN:
        return {"ok": True, "duplicate": True}
    SEEN.add(evt["id"])
    media_id = evt["data"]["id"]
    if evt["type"] == "media.ready":
        STATE[media_id] = "ready"
    elif evt["type"] == "media.failed":
        STATE[media_id] = "failed"
    return {"ok": True}

# tests/test_webhooks.py
import json
from pathlib import Path
import pytest
from httpx import ASGITransport, AsyncClient
from pipeline import webhooks

FIX = Path("tests/fixtures/webhooks")
load = lambda name: json.loads((FIX / f"{name}.json").read_text())

@pytest.fixture(autouse=True)
def reset_state():
    webhooks.STATE.clear(); webhooks.SEEN.clear()

@pytest.fixture
async def client():
    async with AsyncClient(transport=ASGITransport(app=webhooks.app), base_url="http://t") as c:
        yield c

async def test_ready_marks_media_ready(client):
    r = await client.post("/webhooks/video", json=load("media_ready"))
    assert r.status_code == 200
    assert webhooks.STATE["media_abc123"] == "ready"

async def test_duplicate_ready_is_idempotent(client):
    await client.post("/webhooks/video", json=load("media_ready"))
    r = await client.post("/webhooks/video", json=load("media_ready"))
    assert r.json()["duplicate"] is True
    assert webhooks.STATE == {"media_abc123": "ready"}

async def test_failed_after_ready_is_a_decision_you_made(client):
    await client.post("/webhooks/video", json=load("media_ready"))
    await client.post("/webhooks/video", json=load("media_failed"))
    # Pick your policy and pin it. Here: last event wins.
    assert webhooks.STATE["media_abc123"] == "failed"

async def test_missing_optional_field_does_not_crash(client):
    evt = load("media_ready"); del evt["data"]["playback_url"]
    r = await client.post("/webhooks/video", json=evt)
    assert r.status_code == 200

You just tested three things a manual staging upload can never produce on demand: a duplicate delivery, an out-of-order failure, and a payload with a field missing.

6. CI: pin FFmpeg, run everything

# .github/workflows/ci.yml
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - uses: FedericoCarboni/setup-ffmpeg@v3
        with:
          ffmpeg-version: "8.1"      # pin it; unpinned runners can drift between OSes
      - run: pip install -r requirements.txt
      - run: pytest -q

setup-ffmpeg@v3 installs and caches the requested build and exposes both ffmpeg and ffprobe. Its README is explicit that without a pinned version, different operating systems may get different FFmpeg versions, which is exactly the drift your goldens would catch and you'd rather not debug.

# expected CI output
tests/test_ladder.py ...                                                  [ 33%]
tests/test_transcode.py ...                                               [ 66%]
tests/test_webhooks.py ....                                               [100%]
10 passed in 3.41s

7. Tier 4 skeleton: the one paid call

# .github/workflows/nightly-contract.yml
on:
  schedule: [{ cron: "0 3 * * *" }]
jobs:
  contract:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: FedericoCarboni/setup-ffmpeg@v3
        with: { ffmpeg-version: "8.1" }
      - run: pytest -q -m contract
        env: { PROVIDER_TOKEN: "${{ secrets.PROVIDER_TOKEN }}" }

Mark those tests @pytest.mark.contract, exclude the marker from the default run, upload one two-second generated fixture, poll until ready, fetch the playback manifest, assert a 200. One asset, one run a night, a hard timeout. That's the whole budget.

Things to know

Generated fixtures are deterministic in shape, not in bytes. Assert on probe output, never on checksums of media.
-preset ultrafast for fixtures, your real preset for the code under test. The fixture just needs to be a valid input; the transcode is what you're testing.
Record webhooks from a dev environment, not production. Sanitise IDs anyway.
Keep goldens small. stable_view() should produce a few dozen lines, not the full ffprobe dump.
Re-baseline on purpose. When you bump the pinned FFmpeg, run --snapshot-update in the same PR and review the diff. That diff is the changelog for your outputs.

What's next

Add a tier-2 test per rendition in your ladder and one per "weird input" (VFR, no audio, portrait, HDR) as you meet them in production.
If your webhook handler is the idempotent Postgres-backed one, point the replay tests at a test database instead of the dict and keep the same fixtures.
Wire stable_view() into your production pipeline as a post-encode QC check; it's the same function.

Post your fixture factory variants under #python or #devops; the "weird input" list is the part every team builds from scratch.

Build a three-stage video dedup worker: SHA-256, perceptual hash, then FFmpeg's MPEG-7 signature

Mason K — Tue, 29 Sep 2026 09:12:35 +0000

TL;DR

We'll build a dedup worker that catches re-encoded, resized, watermarked and trimmed copies of videos you already have. Stage 1 is a byte hash, stage 2 is a 64-bit perceptual hash checked by Hamming distance, stage 3 is FFmpeg's built-in MPEG-7 signature filter for "does A contain B, and where". Python 3.12, FFmpeg 7 or newer, SQLite.

What we're building

A dedup.py module with one entry point, check(path) -> Verdict, that returns exact, near, contains, or new, plus a SQLite index so it gets faster as the library grows. No ML, no GPUs, nothing that isn't pip install or apt install.

The reason for three stages instead of one: the cheap check should run on every upload before you spend money transcoding, and the expensive check should only run on candidates the cheap check surfaced.

1. Set up

sudo apt-get install -y ffmpeg     # 7.x or 8.x; the signature filter has shipped since 2017
python3 -m venv .venv && source .venv/bin/activate
pip install videohash2            # maintained fork of videohash; pin the current release in requirements.txt
ffmpeg -hide_banner -filters | grep signature

You should see:

 ... signature          N->V       Calculate the MPEG-7 video signature

If that line is missing, your FFmpeg build was configured without it; grab a static build.

2. Make some test inputs 🎬

We'll synthesise a source and three "duplicates" so the tests are reproducible and the repo stays binary-free.

# scripts/make_fixtures.sh
set -e
mkdir -p fixtures && cd fixtures

# 20-second source
ffmpeg -y -f lavfi -i "testsrc2=duration=20:size=640x360:rate=25,format=yuv420p" \
       -f lavfi -i "sine=frequency=440:duration=20" \
       -c:v libx264 -crf 20 -c:a aac -shortest source.mp4

# A: re-encode at lower quality and resolution
ffmpeg -y -i source.mp4 -vf scale=320:180 -c:v libx264 -crf 32 -c:a aac dup_reencode.mp4

# B: watermark in the corner
ffmpeg -y -i source.mp4 -vf "drawbox=x=10:y=10:w=80:h=30:color=white@0.8:t=fill" \
       -c:v libx264 -crf 23 -c:a copy dup_watermark.mp4

# C: 6-second excerpt starting at 0:07 (containment case)
ffmpeg -y -ss 7 -i source.mp4 -t 6 -c:v libx264 -crf 23 -c:a aac excerpt.mp4

# D: unrelated video
ffmpeg -y -f lavfi -i "mandelbrot=size=640x360:rate=25" -t 20 \
       -c:v libx264 -crf 23 unrelated.mp4

sha256sum fixtures/*.mp4 gives you four different hashes for what a human would call two videos. That's the problem.

3. Stage 1: the byte hash (keep it, it's free)

# dedup/stage1.py
import hashlib
from pathlib import Path

def sha256(path: Path, chunk: int = 1 << 20) -> str:
    h = hashlib.sha256()
    with path.open("rb") as f:
        while blk := f.read(chunk):
            h.update(blk)
    return h.hexdigest()

This catches the accidental double upload and nothing else. It stays because it costs nothing and it's the only stage with zero false positives.

4. Stage 2: perceptual hash + Hamming gate 🔍

videohash2 samples one frame per second, shrinks each to 144x144, tiles them into a collage, and wavelet-hashes the collage into 64 bits. Two re-encodes of the same clip land a few bits apart.

# dedup/stage2.py
from pathlib import Path
from videohash2 import VideoHash

def phash(path: Path) -> int:
    vh = VideoHash(path=str(path))
    return int(vh.hash_hex, 16)          # 64-bit int

def hamming(a: int, b: int) -> int:
    return (a ^ b).bit_count()

Try it on the fixtures:

>>> from pathlib import Path
>>> from dedup.stage2 import phash, hamming
>>> src = phash(Path("fixtures/source.mp4"))
>>> for name in ["dup_reencode", "dup_watermark", "excerpt", "unrelated"]:
...     print(name, hamming(src, phash(Path(f"fixtures/{name}.mp4"))))

You'll see the re-encode and the watermark land close to zero and unrelated land far away. The interesting one is excerpt: it will not be close. That's by design and the library says so in its docs: a whole-video collage hash cannot tell you that one video is a part of another. That's what stage 3 is for.

⚠️ Note: pick the Hamming threshold conservatively and log near-misses. A false positive here means one user's upload gets served as another user's asset. Start tight (single digits), review what you're rejecting, and loosen only with evidence.

5. Stage 3: MPEG-7 signature for containment

FFmpeg's signature filter fingerprints every frame and, given two or more inputs, searches for matching sequences. First, generate and store a signature per asset. Generation is the slow part; the maintainer of one tool built on this filter reports 19 minutes to fingerprint 96 files versus 45 seconds to compare them all. Never regenerate.

# one-time per asset: write a binary signature next to it
ffmpeg -hide_banner -i fixtures/source.mp4 \
  -vf "signature=format=binary:filename=fixtures/source.sig" -f null -

Then compare two inputs in one run:

ffmpeg -hide_banner -i fixtures/source.mp4 -i fixtures/excerpt.mp4 \
  -filter_complex "signature=nb_inputs=2:detectmode=full" -f null - 2>&1 | grep -i match

Expected shape of the output (offsets will differ):

[Parsed_signature_0 @ 0x...] matching of video 0 at 7.000000 and 1 at 0.000000, 150 frames matching

It found the excerpt inside the source and told you where. Run the same against unrelated.mp4 and you get no matching line at all. Run it against dup_reencode.mp4 and you get whole video matching.

Wrap it:

# dedup/stage3.py
import re, subprocess
from pathlib import Path

MATCH = re.compile(r"matching of video 0 at ([\d.]+) and 1 at ([\d.]+), (\d+) frames matching")
WHOLE = re.compile(r"whole video matching")

def signature_compare(a: Path, b: Path) -> dict | None:
    cmd = ["ffmpeg", "-hide_banner", "-nostats", "-i", str(a), "-i", str(b),
           "-filter_complex", "signature=nb_inputs=2:detectmode=full:th_di=50",
           "-f", "null", "-"]
    out = subprocess.run(cmd, capture_output=True, text=True).stderr
    m = MATCH.search(out)
    if not m:
        return None
    return {
        "offset_a": float(m.group(1)),
        "offset_b": float(m.group(2)),
        "frames": int(m.group(3)),
        "whole": bool(WHOLE.search(out)),
    }

th_di=50 sets the minimum matching sequence to 50 frames (two seconds at 25 fps) so a single similar frame doesn't count. The filter's other thresholds (th_d, th_dc, th_xh, th_it) have sane defaults; leave them until you have a labelled set to tune against.

6. The denominator decision

frames alone isn't a verdict. You have to divide by something, and the choice is a policy, not a constant.

Denominator	6s excerpt in 20s source	Answers
source frames (500)	150 / 500 = 30%	"Is most of my video in theirs?"
shorter video's frames (150)	150 / 150 = 100%	"Is one of these inside the other?"

For dedup you want the second. Most hand-rolled implementations use the first because it's the natural thing to write, and then wonder why excerpts never get flagged.

# dedup/verdict.py
def coverage(frames: int, frames_a: int, frames_b: int) -> float:
    return frames / min(frames_a, frames_b)

Get frame counts from ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of csv=p=0 file.mp4, or cheaper, duration * r_frame_rate from a plain probe.

7. Put it together with a SQLite index

# dedup/worker.py
import sqlite3
from dataclasses import dataclass
from pathlib import Path
from .stage1 import sha256
from .stage2 import phash, hamming
from .stage3 import signature_compare
from .verdict import coverage

PHASH_MAX_DISTANCE = 6
MIN_COVERAGE = 0.4

@dataclass
class Verdict:
    kind: str            # exact | near | contains | new
    match_id: int | None = None
    detail: dict | None = None

def check(path: Path, db: sqlite3.Connection, frames_of) -> Verdict:
    digest = sha256(path)
    row = db.execute("SELECT id FROM assets WHERE sha256=?", (digest,)).fetchone()
    if row:
        return Verdict("exact", row[0])

    h = phash(path)
    candidates = []
    for aid, other_hash, other_path in db.execute("SELECT id, phash, path FROM assets"):
        d = hamming(h, int(other_hash))
        if d <= PHASH_MAX_DISTANCE:
            return Verdict("near", aid, {"hamming": d})
        if d <= 20:                          # loose band: worth a stage-3 look
            candidates.append((aid, Path(other_path)))

    for aid, other in candidates:
        m = signature_compare(other, path)
        if m and coverage(m["frames"], frames_of(other), frames_of(path)) >= MIN_COVERAGE:
            return Verdict("contains", aid, m)

    db.execute("INSERT INTO assets(path, sha256, phash) VALUES (?,?,?)",
               (str(path), digest, str(h)))
    db.commit()
    return Verdict("new")

Schema:

-- dedup/schema.sql
CREATE TABLE IF NOT EXISTS assets (
  id     INTEGER PRIMARY KEY,
  path   TEXT NOT NULL,
  sha256 TEXT NOT NULL UNIQUE,
  phash  TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_phash ON assets(phash);

💡 Tip: the linear scan over phash is fine for tens of thousands of assets. Past that, bucket by the top 16 bits of the hash, or move to a BK-tree, before you reach for a vector database.

Run it over the fixtures in order:

python -m dedup.cli fixtures/source.mp4 fixtures/dup_reencode.mp4 \
                    fixtures/dup_watermark.mp4 fixtures/excerpt.mp4 fixtures/unrelated.mp4
# source.mp4        new
# dup_reencode.mp4  near      (hamming=<small>)
# dup_watermark.mp4 near      (hamming=<small>)
# excerpt.mp4       contains  (offset_a≈7.0, frames≈150, coverage≈1.00)
# unrelated.mp4     new
# exact distances and frame counts depend on your FFmpeg build; the verdicts should not

8. Things to know

Store signatures as derived assets, like thumbnails. Write the .sig once and keep it next to the media; stage 3 becomes comparison-only.
Stage 2 fails on rotation and reversal. Rotation beyond roughly ten degrees or a reversed clip produces a different hash. If that matters for you, generate hashes for the four rotations at index time.
Stage 3 is block-based and the FFmpeg docs say it is "not very robust to some geometric modification". Heavy crops and picture-in-picture will slip through. It's still far better than nothing.
Sampling rate is a cost knob. One tool's experiment moving from 5 fps to 15 fps sampling saw 1.53x the signature-generation time, 5.61x the comparison time, and about 3x the storage. Start low.
Run in log-only mode first. Record verdicts for a week without acting on them, then look at the near and contains rows by hand before you let the worker skip an encode.

What's next

Hook check() into your upload webhook or queue consumer so it runs before the transcode job is enqueued; the whole point is skipping the encode.
Add a reference table for takedown assets and run stage 3 against it on every upload, not just nightly.
If you want to see the frames instead of the numbers, ffmpeg -ss <offset_a> -i source.mp4 -frames:v 1 a.png next to the same from the candidate is a five-second visual sanity check.

Tag it #python and #ffmpeg if you write up your own thresholds; the tuning data is the part nobody shares.

Build a React video player hook that actually releases the player (and a Playwright test that proves it)

Mason K — Tue, 29 Sep 2026 09:12:14 +0000

TL;DR

Single-page apps leak video players on every route change unless three things are torn down in the right order: the hls.js instance, the <video> element's decoder, and your own references. We'll build a useHlsPlayer hook that does it correctly, then a Playwright test that mounts the player 100 times and fails CI if Chrome's "too many WebMediaPlayers" error ever appears.

What we're building

A React hook, useHlsPlayer, that owns the whole player lifecycle, and an end-to-end test that catches the regression when someone deletes the cleanup. Stack: React 19, hls.js 1.7, Vite, Playwright. Node 22.

Why this matters: Chrome counts your players

Since around Chrome 92, Chromium enforces a cap on media players per frame: 75 on desktop, 40 on mobile. Cross it and the next <video> you try to play logs this and refuses:

Blocked attempt to create a WebMediaPlayer as there are too many WebMediaPlayers already in existence.

The Chromium team picked those numbers as the 99.9th percentile of what real pages create, specifically to stop infinite-scroll video sites from holding thousands of players in memory. If your app can reach the cap, it is leaking players. Let's make sure it can't.

1. Set up the project

npm create vite@latest react-hls-teardown -- --template react-ts
cd react-hls-teardown
npm install hls.js@^1.7
npm install -D @playwright/test
npx playwright install chromium

Add a test stream. Any public HLS URL works; the hls.js demo streams are fine for a leak test since we only care about player creation, not content.

2. The naive hook (this is what leaks)

Here is the version that shows up in most codebases. It works on first mount and leaks on every remount.

// src/useHlsPlayerNaive.ts  (DO NOT SHIP)
import { useEffect, useRef } from "react";
import Hls from "hls.js";

export function useHlsPlayerNaive(src: string) {
  const videoRef = useRef<HTMLVideoElement>(null);

  useEffect(() => {
    const video = videoRef.current!;
    const hls = new Hls();
    hls.loadSource(src);
    hls.attachMedia(video);
    // no cleanup returned
  }, [src]);

  return videoRef;
}

Every time src changes or the component remounts, a new Hls instance is created and the old one is still alive, still attached, still holding a MediaSource. Remove the <video> from the DOM and the browser-side player is still there too, because nothing told it to release.

3. The hook with correct teardown

Three layers, torn down in this order.

// src/useHlsPlayer.ts
import { useEffect, useRef } from "react";
import Hls from "hls.js";

export function useHlsPlayer(src: string) {
  const videoRef = useRef<HTMLVideoElement>(null);

  useEffect(() => {
    const video = videoRef.current;
    if (!video) return;

    let hls: Hls | null = null;

    if (Hls.isSupported()) {
      hls = new Hls({ enableWorker: true });
      hls.loadSource(src);
      hls.attachMedia(video);
    } else if (video.canPlayType("application/vnd.apple.mpegurl")) {
      video.src = src; // Safari native HLS
    }

    return () => {
      // 1. Library first. destroy() detaches media and removes listeners
      //    internally, in that order. Do not call removeAllListeners() before it.
      hls?.destroy();
      hls = null;

      // 2. Browser-side player. Removing src alone does nothing per spec;
      //    load() is what resets the element and releases the decoder.
      video.pause();
      video.removeAttribute("src");
      video.load();
    };
  }, [src]);

  return videoRef;
}

⚠️ Note: there is a maintainer-confirmed footgun here. If you call hls.removeAllListeners() before hls.destroy(), destroy cannot complete, because hls.js's internal components listen for the DESTROYING event to clean themselves up. You end up with a half-dead instance that still ticks on media events and throws Cannot read properties of null (reading 'getAppendedFrag') on the next route. Let destroy() do the listener removal itself.

💡 Tip: if you ever set video.src to a blob or object URL, add URL.revokeObjectURL(url) to the cleanup. Clearing src does not free the Blob.

4. The player component

// src/Player.tsx
import { useHlsPlayer } from "./useHlsPlayer";

export function Player({ src }: { src: string }) {
  const videoRef = useHlsPlayer(src);
  return <video ref={videoRef} controls muted playsInline style={{ width: 480 }} />;
}

And a tiny app that mounts and unmounts it on demand, so the test can drive it. In a real app this is your router; here we keep it explicit.

// src/App.tsx
import { useState } from "react";
import { Player } from "./Player";

const SRC = "https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8";

export default function App() {
  const [mounted, setMounted] = useState(true);
  const [naive, setNaive] = useState(false);
  return (
    <main>
      <button id="toggle" onClick={() => setMounted((m) => !m)}>toggle player</button>
      <label><input type="checkbox" onChange={(e) => setNaive(e.target.checked)} /> use naive hook</label>
      {mounted && <Player src={SRC} key={naive ? "naive" : "safe"} />}
    </main>
  );
}

(Wire naive through to useHlsPlayerNaive in your own copy if you want to watch the failing case; I left it out of the snippet to keep Player honest.)

5. Verify by hand first

Run the dev server, open chrome://media-internals in a second tab, and click toggle player twenty times.

npm run dev
# Local: http://localhost:5173/

With the safe hook, the players list in media-internals grows by one and shrinks by one on each toggle. With the naive hook, it only grows. On the 76th mount you get the blocked-player error in the console.

You can also open DevTools, Memory panel, and record a Detached elements profile. Detached <video> nodes retained by JavaScript are exactly what the naive hook produces.

6. The Playwright leak test

Now let's make CI catch it. Two assertions: Chrome never logged the blocked-player error, and after 100 mount/unmount cycles a fresh player can still load metadata (which it cannot once the cap is hit).

// tests/player-leak.spec.ts
import { test, expect } from "@playwright/test";

const BLOCKED = /Blocked attempt to create a WebMediaPlayer/;

test("mounting the player 100 times does not leak players", async ({ page }) => {
  const consoleErrors: string[] = [];
  page.on("console", (msg) => {
    if (msg.type() === "error" || msg.type() === "warning") consoleErrors.push(msg.text());
  });

  await page.goto("/");
  const toggle = page.locator("#toggle");

  for (let i = 0; i < 100; i++) {
    await toggle.click(); // unmount
    await toggle.click(); // mount
    await page.locator("video").waitFor({ state: "attached" });
  }

  // 1. Chrome never refused to create a player.
  expect(consoleErrors.filter((t) => BLOCKED.test(t))).toHaveLength(0);

  // 2. The 101st player is a real player: it reaches HAVE_METADATA.
  await expect
    .poll(() => page.evaluate(() => document.querySelector("video")!.readyState), { timeout: 15_000 })
    .toBeGreaterThanOrEqual(1);
});

The console assertion is the teeth, because the Chromium cap turns a slow leak into a deterministic error string. The readyState poll is the belt to go with the braces: a blocked player never gets metadata. With the naive hook this test fails around the 76th mount. With the safe hook it passes in under a minute.

💡 Tip: if you want to see what is leaking rather than that it leaks, open the same page in Chrome, run twenty toggles by hand, and record a Detached elements profile in the Memory panel. It lists the <video> nodes that left the DOM but are still retained by JavaScript, with the retaining path.

npx playwright test
# Running 1 test using 1 worker
#   ✓ tests/player-leak.spec.ts:6:1 › mounting the player 100 times does not leak players (41.2s)
# 1 passed (42s)

Playwright config only needs webServer pointed at Vite and use.baseURL set to the dev server.

// playwright.config.ts
import { defineConfig } from "@playwright/test";
export default defineConfig({
  webServer: { command: "npm run dev", url: "http://localhost:5173", reuseExistingServer: true },
  use: { baseURL: "http://localhost:5173", headless: true },
});

⚠️ Note: run this in Chromium only. The WebMediaPlayer cap is a Chromium behaviour; Firefox and WebKit will not emit the error, so the test would pass there even with a leak.

7. Things to know

Route transitions are the common trigger, not tab switches. Any place React unmounts and remounts the component (route change, modal, list re-key, Suspense boundary) creates a new player. If the old one isn't released, the count only goes up.
Feeds are the worst case. A virtualised list that recycles rows mounts and unmounts players by the dozen. The safe hook is mandatory there, and you should also cap how many players exist at once by only mounting <video> for rows near the viewport.
Safari native HLS has the same teardown. No hls.js instance, but you still need pause(), removeAttribute("src"), load().
Android WebView enforces the mobile cap (40). If your app ships inside a WebView, you hit it faster.
Don't reuse one <video> across hls.js instances without destroy() first. The library's own issue tracker has the exact error you'll get if you skip it.

What's next

Add a second test that mounts five players simultaneously and asserts they are all released, to cover carousels and grids.
If you're on Shaka or dash.js, the shape is the same: player.destroy() returns a promise, so await it in an async cleanup before you reset the element.
Read the hls.js API docs on destroy() and the Hls.Events.DESTROYING event if you want to hook your own cleanup into the library's, rather than running it alongside.

If you're posting about this on #webdev or #react, the one-line version is: a route change is not a page load, and nothing frees your player for you.

Build a conformance gate that skips the encode when the upload is already fine

Mason K — Thu, 24 Sep 2026 12:21:46 +0000

TL;DR

We're building a conformance gate in Python: probe an upload once with ffprobe, run it through ordered predicates, and get back one of three verdicts (passthrough, remux, transcode). The gate includes the keyframe-cadence check that most passthrough implementations forget, which is why they break at segmentation time.

Most upload pipelines transcode every file by reflex. But a lot of what lands in a UGC pipeline is already H.264 8-bit 4:2:0 straight off a phone, and re-encoding it costs compute and adds a second generation of loss for very little gain.

The fix isn't "stop encoding." It's classifying each file so you do the cheapest correct operation. Let's build the classifier.

Requirements: ffmpeg 8.x (7.x is fine too) and python 3.11+.

ffmpeg -version | head -1
python3 --version

1. One probe call, everything we need

Don't make three ffprobe calls. Make one and parse it.

# probe.sh
ffprobe -v error \
  -show_entries stream=index,codec_type,codec_name,profile,level,pix_fmt,width,height,avg_frame_rate,bit_rate \
  -show_entries format=duration,bit_rate,format_name \
  -of json input.mp4

Output looks like this:

{
  "streams": [
    {
      "index": 0,
      "codec_name": "h264",
      "codec_type": "video",
      "profile": "High",
      "level": 40,
      "pix_fmt": "yuv420p",
      "width": 1920,
      "height": 1080,
      "avg_frame_rate": "30000/1001"
    },
    {
      "index": 1,
      "codec_name": "aac",
      "codec_type": "audio"
    }
  ],
  "format": {
    "duration": "184.320000",
    "bit_rate": "8412160",
    "format_name": "mov,mp4,m4a,3gp,3g2,mj2"
  }
}

Wrap it:

# gate/probe.py
import json, subprocess

def probe(path: str) -> dict:
    cmd = [
        "ffprobe", "-v", "error",
        "-show_entries",
        "stream=index,codec_type,codec_name,profile,level,pix_fmt,"
        "width,height,avg_frame_rate,bit_rate",
        "-show_entries", "format=duration,bit_rate,format_name",
        "-of", "json", path,
    ]
    out = subprocess.run(cmd, capture_output=True, text=True, check=True)
    return json.loads(out.stdout)

def first(streams, kind):
    return next((s for s in streams if s.get("codec_type") == kind), None)

💡 Tip: check=True will raise on a file ffprobe can't open at all. That's its own verdict (reject), so catch CalledProcessError at the call site rather than letting it kill your worker.

2. The predicates, in the order they bite

Order matters here, not for correctness but for debuggability. You want the failure reason to be the most fundamental thing wrong with the file.

# gate/rules.py
from dataclasses import dataclass, field

DELIVERABLE_CODECS = {"h264"}
DELIVERABLE_PIX_FMTS = {"yuv420p", "yuvj420p"}
DELIVERABLE_AUDIO = {"aac"}
MAX_LEVEL = 42          # H.264 level 4.2
MAX_BITRATE = 12_000_000  # above this, re-encode even if conformant

@dataclass
class Verdict:
    action: str                     # passthrough | remux | transcode | reject
    reasons: list = field(default_factory=list)
    rules_version: str = "2026.09.1"

def classify(info: dict, container_ok: bool, keyframes_regular: bool) -> Verdict:
    streams = info.get("streams", [])
    v = first(streams, "video")
    a = first(streams, "audio")
    reasons = []

    if v is None:
        return Verdict("reject", ["no video stream"])

    # 1. codec
    if v["codec_name"] not in DELIVERABLE_CODECS:
        reasons.append(f"codec {v['codec_name']} not deliverable")

    # 2. pixel format: 10-bit H.264 is real and browsers refuse it
    if v.get("pix_fmt") not in DELIVERABLE_PIX_FMTS:
        reasons.append(f"pix_fmt {v.get('pix_fmt')} not 8-bit 4:2:0")

    # 3. level
    if v.get("level") and v["level"] > MAX_LEVEL:
        reasons.append(f"level {v['level']} above {MAX_LEVEL}")

    # 4. bitrate ceiling: conformant but wasteful
    br = int(info["format"].get("bit_rate") or 0)
    if br > MAX_BITRATE:
        reasons.append(f"bitrate {br} above ceiling")

    # 5. keyframe cadence: the one everybody forgets
    if not keyframes_regular:
        reasons.append("irregular keyframe interval, cannot segment cleanly")

    if reasons:
        return Verdict("transcode", reasons)

    # video is clean. is audio?
    if a and a["codec_name"] not in DELIVERABLE_AUDIO:
        return Verdict("remux", [f"audio {a['codec_name']} needs re-encode"])

    if not container_ok:
        return Verdict("remux", ["container or faststart fix needed"])

    return Verdict("passthrough", [])

3. The keyframe check (do not skip this one)

This is where naive passthrough implementations die. HLS and DASH cut segments on keyframes. A phone recording with a 10-second GOP, or an irregular scene-change-driven one, cannot be cut into 4-second segments without re-encoding.

Read the first couple hundred frames and look at the gaps:

# gate/keyframes.py
import subprocess, statistics

def keyframe_intervals(path: str, n_frames: int = 300) -> list[float]:
    cmd = [
        "ffprobe", "-v", "error", "-select_streams", "v:0",
        "-show_entries", "frame=pts_time,key_frame",
        "-of", "csv=p=0",
        "-read_intervals", f"%+#{n_frames}",
        path,
    ]
    out = subprocess.run(cmd, capture_output=True, text=True, check=True)
    times = []
    for line in out.stdout.strip().splitlines():
        parts = line.split(",")
        if len(parts) < 2:
            continue
        pts, is_key = parts[0], parts[1]
        if is_key == "1" and pts not in ("N/A", ""):
            times.append(float(pts))
    return [round(b - a, 3) for a, b in zip(times, times[1:])]

def keyframes_regular(path: str, target_segment: float = 4.0,
                      tolerance: float = 0.15) -> tuple[bool, str]:
    gaps = keyframe_intervals(path)
    if len(gaps) < 3:
        return False, "too few keyframes sampled"

    median = statistics.median(gaps)
    spread = max(gaps) - min(gaps)

    if spread > median * tolerance:
        return False, f"irregular GOP (median {median}s, spread {spread}s)"
    if target_segment % median > 0.05 and median % target_segment > 0.05:
        return False, f"GOP {median}s does not divide {target_segment}s segments"
    return True, f"regular GOP of {median}s"

Run it and you get something honest:

$ python3 -c "from gate.keyframes import keyframes_regular; print(keyframes_regular('phone.mp4'))"
(False, 'irregular GOP (median 2.0s, spread 4.867s)')

$ python3 -c "from gate.keyframes import keyframes_regular; print(keyframes_regular('screencap.mp4'))"
(True, 'regular GOP of 2.0s')

⚠️ Note: -read_intervals "%+#300" only samples the beginning of the file. A source can start regular and go irregular later. For high-value assets, sample a second window mid-file. For a first version, the head is enough, and it's the difference between a check that runs in 40ms and one that decodes the whole file.

4. Wire the three actions

Each verdict maps to a command. This is the whole payoff.

# gate/actions.py
import subprocess

def run_passthrough(src: str, dst: str):
    """Source becomes the top ladder rung as-is."""
    subprocess.run(["cp", src, dst], check=True)

def run_remux(src: str, dst: str, reencode_audio: bool):
    cmd = ["ffmpeg", "-y", "-i", src, "-c:v", "copy"]
    cmd += ["-c:a", "aac", "-b:a", "128k"] if reencode_audio else ["-c:a", "copy"]
    cmd += ["-movflags", "+faststart", dst]
    subprocess.run(cmd, check=True)

def run_transcode(src: str, dst: str):
    subprocess.run([
        "ffmpeg", "-y", "-i", src,
        "-c:v", "libx264", "-preset", "medium", "-crf", "21",
        "-pix_fmt", "yuv420p",
        "-g", "96", "-keyint_min", "96", "-sc_threshold", "0",
        "-c:a", "aac", "-b:a", "128k",
        "-movflags", "+faststart", dst,
    ], check=True)

Note the transcode path forces -pix_fmt yuv420p and a fixed GOP with -sc_threshold 0. That's not decoration: it guarantees the output passes your own gate, which matters when you re-run the pipeline later.

5. Put it together

# gate/__main__.py
import sys, json
from .probe import probe
from .keyframes import keyframes_regular
from .rules import classify
from .actions import run_passthrough, run_remux, run_transcode

def process(src: str, dst: str) -> dict:
    info = probe(src)
    regular, kf_note = keyframes_regular(src)
    container_ok = src.endswith(".mp4")   # replace with a real faststart check
    verdict = classify(info, container_ok, regular)

    if verdict.action == "passthrough":
        run_passthrough(src, dst)
    elif verdict.action == "remux":
        needs_audio = any("audio" in r for r in verdict.reasons)
        run_remux(src, dst, needs_audio)
    elif verdict.action == "transcode":
        run_transcode(src, dst)
    else:
        raise ValueError(f"rejected: {verdict.reasons}")

    return {
        "source": src,
        "action": verdict.action,
        "reasons": verdict.reasons,
        "keyframes": kf_note,
        "rules_version": verdict.rules_version,
    }

if __name__ == "__main__":
    print(json.dumps(process(sys.argv[1], sys.argv[2]), indent=2))

$ python3 -m gate uploads/phone-clip.mp4 out/phone-clip.mp4
{
  "source": "uploads/phone-clip.mp4",
  "action": "remux",
  "reasons": ["audio opus needs re-encode"],
  "keyframes": "regular GOP of 2.0s",
  "rules_version": "2026.09.1"
}

6. Store the verdict on the asset

Do not just log this. Put action and rules_version on the asset record.

In six months you'll lower MAX_BITRATE or add an AV1 rung, and you'll need to answer "which assets were passed through under the old rules?" Without the stamp, the answer is "re-process everything," which is the compute you built this to avoid.

ALTER TABLE assets
  ADD COLUMN ingest_action  text NOT NULL DEFAULT 'transcode',
  ADD COLUMN ingest_rules_version text NOT NULL DEFAULT 'legacy';

What's next

Two things to be honest about before you ship this.

Passthrough saves one rung, not the whole encode. ABR still needs 720p, 480p, 360p. What you're skipping is the top rung, which is the most expensive one, on some fraction of ingests. Real saving, smaller headline.

Measure before you build. Run steps 1 to 3 across a sample of what you've already ingested, skip the actions entirely, and just count verdicts:

for f in sample/*.mp4; do python3 -m gate.classify_only "$f"; done | \
  jq -r .action | sort | uniq -c

  41 transcode
  22 remux
  17 passthrough

If passthrough and remux are a rounding error for your traffic, you've spent an afternoon and learned something. If they aren't, you just found the cheapest infra win on your backlog.

From here, the next things worth adding are a real faststart check (ffprobe the moov atom position rather than trusting the extension), a mid-file keyframe sample, and rotation-metadata handling so sideways phone video doesn't get re-encoded just to be turned.

Add a DVR window to your live HLS stream without 404ing the scrubber

Mason K — Thu, 24 Sep 2026 12:21:25 +0000

TL;DR

We're adding rewind to a live HLS stream: configure a sliding DVR window in FFmpeg, expose it correctly in hls.js, build a seek bar that tracks the moving window, and add a go-live button that seeks to liveSyncPosition instead of the end of the seekable range (which stalls). Plus the storage-lifecycle check that prevents 404s inside your own advertised window.

"Can I go back to the start?" is the first request you get after shipping live. The segments are already on disk, so it feels like a config change. It is a config change, across three systems that have to agree, plus about 60 lines of player code.

Requirements: ffmpeg 7.0+, hls.js 1.6+, node 22.x.

ffmpeg -version | head -1
npm ls hls.js

1. Configure the packager window

The DVR window length is hls_time × hls_list_size. That's it. Everything else is bookkeeping.

# 10-minute sliding window: 6s segments × 100 entries
ffmpeg -i rtmp://localhost/live/stream \
  -c:v libx264 -preset veryfast -tune zerolatency \
  -g 120 -keyint_min 120 -sc_threshold 0 \
  -c:a aac -b:a 128k -ar 48000 \
  -f hls \
  -hls_time 6 \
  -hls_list_size 100 \
  -hls_flags delete_segments+independent_segments \
  -hls_delete_threshold 5 \
  -hls_segment_filename 'segments/seg_%05d.ts' \
  stream.m3u8

What each flag is doing:

Flag	Effect	Default
`hls_time 6`	target segment duration	2
`hls_list_size 100`	entries kept in the playlist	5
`delete_segments`	remove files once they leave the playlist	off
`hls_delete_threshold 5`	unreferenced segments kept before deletion	1
`independent_segments`	every segment starts on a keyframe	off

⚠️ That hls_list_size default of 5 is why so many first live deployments ship an accidental 30-second DVR nobody designed. If you never set it, you already have a DVR window. It's just very short.

-g 120 at 60fps gives a 2-second GOP, which divides evenly into 6-second segments. If your GOP doesn't divide your segment duration, segments drift off the target and your EXT-X-TARGETDURATION climbs.

Check the manifest:

$ head -8 stream.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:7
#EXT-X-MEDIA-SEQUENCE:412
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:6.000000,
segments/seg_00412.ts

EXT-X-MEDIA-SEQUENCE: 412 climbing on each refresh with the entry count staying at 100 means the sliding window is working.

2. Sliding window vs event playlist

There's a second shape, and it's a design decision, not a tuning knob.

# full-event DVR: append-only, nothing is deleted
ffmpeg -i rtmp://localhost/live/stream \
  ... -f hls -hls_time 6 -hls_playlist_type event stream.m3u8

-hls_playlist_type event emits #EXT-X-PLAYLIST-TYPE:EVENT and forces hls_list_size to 0, so the playlist contains every segment since the stream started.

	Sliding window	Event playlist
History	bounded	full
Manifest size	constant	grows all stream
Storage	bounded	grows all stream
Good for	24/7 channels	bounded broadcasts
Gotcha	history falls off	see below

⚠️ The event-playlist gotcha: some players refuse to seek in an EVENT playlist until #EXT-X-ENDLIST appears, which only happens when the stream ends. There's a long-running video.js issue on exactly this (videojs/video.js#8856) and the same pattern has shown up elsewhere. Test on your real target players, not just desktop Chrome, before you commit to EVENT.

For the rest of this build we use the sliding window.

3. The check nobody runs: does storage agree?

Your playlist advertises a seekable range. Three systems have to back it up, and they're configured by different people at different times:

Playlist: hls_time × hls_list_size = 600s
Origin disk: (hls_list_size + hls_delete_threshold) × hls_time = 630s
Object storage lifecycle: whatever the bucket policy says

If (3) is shorter than (1), your scrubber offers a range that 404s. Assert it in CI:

// scripts/check-dvr-consistency.mjs
const HLS_TIME = 6;
const HLS_LIST_SIZE = 100;
const HLS_DELETE_THRESHOLD = 5;
const STORAGE_TTL_SECONDS = 3600; // read this from your IaC, don't hardcode

const advertised = HLS_TIME * HLS_LIST_SIZE;
const onDisk = HLS_TIME * (HLS_LIST_SIZE + HLS_DELETE_THRESHOLD);

const problems = [];
if (advertised > onDisk) {
  problems.push(`playlist advertises ${advertised}s but origin keeps ${onDisk}s`);
}
if (advertised > STORAGE_TTL_SECONDS) {
  problems.push(`playlist advertises ${advertised}s but storage TTL is ${STORAGE_TTL_SECONDS}s`);
}

if (problems.length) {
  console.error("DVR window inconsistent:\n  " + problems.join("\n  "));
  process.exit(1);
}
console.log(`DVR window OK: ${advertised}s advertised, ${onDisk}s on disk`);

$ node scripts/check-dvr-consistency.mjs
DVR window OK: 600s advertised, 630s on disk

Nine lines of assertion. It catches the failure mode that support tickets are made of.

4. Read the window in hls.js

Now the player side. hls.js gives you the live edge and the seekable range; you have to turn that into UI.

// src/dvr.js
import Hls from 'hls.js';

export function attachDvr(video, src) {
  const hls = new Hls({
    // how far behind the live edge we sit, as a multiple of EXT-X-TARGETDURATION
    liveSyncDurationCount: 3,        // default is 3
    // liveSyncDuration: 18,         // seconds-based alternative; takes precedence
    backBufferLength: 90,
  });

  hls.loadSource(src);
  hls.attachMedia(video);

  const state = { hasDvr: false, windowSeconds: 0 };

  hls.on(Hls.Events.LEVEL_LOADED, (_evt, data) => {
    const details = data.details;
    state.hasDvr = details.live && details.totalduration > 30;
    state.windowSeconds = details.totalduration;
  });

  return { hls, state };
}

details.totalduration on a live level is the length of the current playlist, which is your DVR window. It moves as the window slides, so read it on every LEVEL_LOADED rather than caching it once.

5. A seek bar that tracks a moving target

The trap: on a live stream, video.duration is Infinity and seekable.start(0) increases over time. A normal <input type=range> bound to currentTime / duration does nothing useful.

Map the seekable range instead:

// src/dvr-ui.js
export function wireScrubber(video, hls, els) {
  const { scrubber, liveBadge, goLiveBtn, behindLabel } = els;

  function seekableRange() {
    if (!video.seekable.length) return null;
    return {
      start: video.seekable.start(0),
      end: video.seekable.end(video.seekable.length - 1),
    };
  }

  function render() {
    const range = seekableRange();
    if (!range) return;

    const span = range.end - range.start;
    if (span <= 0) return;

    // position within the window, 0..1
    const pos = (video.currentTime - range.start) / span;
    if (!scrubber.matches(':active')) {
      scrubber.value = String(Math.max(0, Math.min(1, pos)));
    }

    // how far behind the live edge are we?
    const behind = Math.max(0, hls.liveSyncPosition - video.currentTime);
    const atLive = behind < 10; // one-and-a-bit segments of slack

    liveBadge.classList.toggle('is-live', atLive);
    goLiveBtn.hidden = atLive;
    behindLabel.textContent = atLive ? 'LIVE' : `-${formatClock(behind)}`;
  }

  scrubber.addEventListener('input', () => {
    const range = seekableRange();
    if (!range) return;
    video.currentTime = range.start + Number(scrubber.value) * (range.end - range.start);
  });

  video.addEventListener('timeupdate', render);
  video.addEventListener('progress', render);
  return render;
}

function formatClock(seconds) {
  const s = Math.floor(seconds % 60).toString().padStart(2, '0');
  const m = Math.floor(seconds / 60);
  return `${m}:${s}`;
}

Two details worth calling out. We skip updating scrubber.value while the user is dragging (:active), otherwise timeupdate fights their thumb. And the live badge uses a tolerance, because being 3 seconds behind the edge is normal live playback, not rewind.

6. The go-live button (this is the one that stalls)

The obvious implementation is wrong:

// ❌ lands you at the newest byte with no buffer ahead, then stalls
video.currentTime = video.seekable.end(video.seekable.length - 1);

Use liveSyncPosition. hls.js computes it as the live edge minus a deliberate safety margin (liveSyncDurationCount × EXT-X-TARGETDURATION, default multiple of 3):

// ✅ src/go-live.js
export function goLive(video, hls) {
  const target = hls.liveSyncPosition;
  if (typeof target === 'number' && Number.isFinite(target)) {
    video.currentTime = target;
  } else if (video.seekable.length) {
    // fallback: back off from the edge ourselves
    const end = video.seekable.end(video.seekable.length - 1);
    video.currentTime = Math.max(0, end - 3 * (hls.levels[hls.currentLevel]?.details?.targetduration ?? 6));
  }
  if (video.paused) video.play();
}

💡 Tip: 3 to 4 segment durations behind the edge is where most deployments land. Closer and ordinary delivery jitter starves the buffer; further and you're needlessly behind. Tune liveSyncDurationCount, don't fight it in your seek code.

7. Handle the window sliding out from under a paused viewer

A viewer pauses inside the window, walks away, comes back 10 minutes later and presses play. Their position no longer exists.

// src/window-recovery.js
export function wireWindowRecovery(video, hls, onRecover) {
  function checkPlayhead() {
    if (!video.seekable.length) return;
    const start = video.seekable.start(0);
    if (video.currentTime < start) {
      // we fell out of the window while paused
      video.currentTime = start + 2; // small cushion past the edge
      onRecover?.();
    }
  }

  video.addEventListener('play', checkPlayhead);
  video.addEventListener('seeked', checkPlayhead);

  hls.on(Hls.Events.ERROR, (_e, data) => {
    if (data.details === Hls.ErrorDetails.FRAG_LOAD_ERROR && data.response?.code === 404) {
      // a segment we asked for is already gone; jump to the current window start
      checkPlayhead();
    }
  });
}

You want the visible recovery, not the silent one. Tell the viewer their position expired and you moved them, or they'll think the player jumped at random.

$ # simulate it: pause, wait past the window, press play
[dvr] playhead 412.8s fell behind window start 640.2s, recovering

Wrapping up

The whole feature is four decisions:

Window length: hls_time × hls_list_size, chosen on purpose. 30s for replaying a goal, 10min for a 24/7 channel, event playlist for a bounded broadcast.
Consistency: assert playlist ≤ origin ≤ storage TTL in CI.
Go live: liveSyncPosition, never seekable.end().
Recovery: detect a playhead that's fallen out of the window and move it visibly.

What's next: if you're on a 24/7 channel, look at whether a rolling VOD archive alongside the live window serves your use case better than a very long DVR (cheaper storage class, no manifest growth). And if you're targeting native, be aware that seeking inside a moving live window has its own long history on other platforms, including an ExoPlayer issue thread (google/ExoPlayer#87) that's worth reading before you assume parity with the web.

Build a video QC gate that catches black frames before your viewers do

Mason K — Thu, 24 Sep 2026 12:20:57 +0000

TL;DR

We're building a QC gate that runs after your transcode: tier 1 is ffprobe structural checks (no decode), tier 2 chains blackdetect + freezedetect + silencedetect into a single decode pass, and tier 3 is sampled SSIM. The part that makes it usable is the rules layer that turns raw detections into pass/warn/fail based on where in the timeline they land.

An encoder exiting 0 means a file was produced. It doesn't mean the file has picture in it. Black intros, frozen middles and silent audio all ship with a green build, and get found by a viewer weeks later.

Let's build the step that catches them. Requirements: ffmpeg 7.0+ and python 3.11+.

ffmpeg -version | head -1
ffmpeg -hide_banner -filters | grep -E "blackdetect|freezedetect|silencedetect"

 ... blackdetect       V->V       Detect video intervals that are (almost) black.
 ... freezedetect      V->V       Detects frozen video input.
 ... silencedetect     A->A       Detect silence.

1. Tier 1: structural checks, no decoding 🔍

Start here. It costs nothing and catches more than you'd expect.

# qc/structural.py
import json, subprocess
from dataclasses import dataclass

@dataclass
class Structural:
    duration: float
    video_streams: int
    audio_streams: int
    codecs: list[str]

def probe(path: str) -> Structural:
    cmd = [
        "ffprobe", "-v", "error",
        "-show_entries", "format=duration",
        "-show_entries", "stream=codec_type,codec_name",
        "-of", "json", path,
    ]
    data = json.loads(subprocess.run(cmd, capture_output=True, text=True,
                                     check=True).stdout)
    streams = data.get("streams", [])
    return Structural(
        duration=float(data["format"].get("duration") or 0),
        video_streams=sum(1 for s in streams if s["codec_type"] == "video"),
        audio_streams=sum(1 for s in streams if s["codec_type"] == "audio"),
        codecs=[s["codec_name"] for s in streams],
    )

def check_structural(output: str, source_duration: float, expect_audio: bool):
    s = probe(output)
    findings = []

    if s.video_streams == 0:
        findings.append(("fail", "no video stream in output"))
    if expect_audio and s.audio_streams == 0:
        findings.append(("fail", "expected audio, output has none"))

    drift = abs(s.duration - source_duration)
    if drift > max(2.0, source_duration * 0.02):
        findings.append(("fail", f"duration drift {drift:.1f}s "
                                 f"(out {s.duration:.1f}s vs src {source_duration:.1f}s)"))
    return findings

💡 Tip: missing audio is the single most common real-world hit here. It happens when the source puts audio on a non-first stream and your mapping assumed -map 0:a:0. Nothing else in your pipeline notices.

$ python3 -m qc.structural out/lesson-14.mp4 --source-duration 1842
fail: expected audio, output has none

One ffprobe call, sub-second, and you just caught a defect that would have shipped.

2. Tier 2: three detectors, one decode pass

The trick is chaining them so you decode once. -f null - means no output file is written at all.

ffmpeg -hide_banner -nostats -i out/lesson-14.mp4 \
  -vf "blackdetect=d=0.5:pic_th=0.98:pix_th=0.10,freezedetect=n=-60dB:d=2" \
  -af "silencedetect=n=-50dB:d=2" \
  -f null - 2> qc.log

The filters report to stderr, not stdout, and not as JSON. Parsing log lines is the integration path, annoying but stable:

[blackdetect @ 0x5f2a] black_start:0 black_end:22.4067 black_duration:22.4067
[freezedetect @ 0x5f2b] lavfi.freezedetect.freeze_start: 604.12
[freezedetect @ 0x5f2b] lavfi.freezedetect.freeze_duration: 11.3
[freezedetect @ 0x5f2b] lavfi.freezedetect.freeze_end: 615.42
[silencedetect @ 0x5f2c] silence_start: 0
[silencedetect @ 0x5f2c] silence_end: 1842.31 | silence_duration: 1842.31

Parse it:

# qc/detect.py
import re, subprocess
from dataclasses import dataclass

BLACK = re.compile(r"black_start:(?P<start>[\d.]+) black_end:(?P<end>[\d.]+) "
                   r"black_duration:(?P<dur>[\d.]+)")
FREEZE_START = re.compile(r"freeze_start:\s*(?P<start>[\d.]+)")
FREEZE_DUR = re.compile(r"freeze_duration:\s*(?P<dur>[\d.]+)")
SILENCE_START = re.compile(r"silence_start:\s*(?P<start>-?[\d.]+)")
SILENCE_DUR = re.compile(r"silence_duration:\s*(?P<dur>[\d.]+)")

@dataclass
class Event:
    kind: str      # black | freeze | silence
    start: float
    duration: float

def run_detectors(path: str, black_d=0.5, freeze_d=2.0, silence_d=2.0) -> list[Event]:
    cmd = [
        "ffmpeg", "-hide_banner", "-nostats", "-i", path,
        "-vf", f"blackdetect=d={black_d}:pic_th=0.98:pix_th=0.10,"
               f"freezedetect=n=-60dB:d={freeze_d}",
        "-af", f"silencedetect=n=-50dB:d={silence_d}",
        "-f", "null", "-",
    ]
    log = subprocess.run(cmd, capture_output=True, text=True).stderr
    events, pending = [], {}

    for line in log.splitlines():
        if m := BLACK.search(line):
            events.append(Event("black", float(m["start"]), float(m["dur"])))
        elif m := FREEZE_START.search(line):
            pending["freeze"] = float(m["start"])
        elif m := FREEZE_DUR.search(line):
            events.append(Event("freeze", pending.pop("freeze", 0.0), float(m["dur"])))
        elif m := SILENCE_START.search(line):
            pending["silence"] = max(0.0, float(m["start"]))
        elif m := SILENCE_DUR.search(line):
            events.append(Event("silence", pending.pop("silence", 0.0), float(m["dur"])))

    return events

⚠️ Note: silence_start can be reported as a small negative number at the head of a file. Clamp it to 0 or your position rules will misfire on exactly the case you care most about.

3. The rules layer (this is the actual product) ⚖️

Raw detections are not failures. A 1.5s black frame at 00:00 is a fade-in. A 22s black frame at 00:00 is a broken render. Same detector, opposite verdicts.

If you alert on every detection, the channel gets muted inside a month and you've built nothing.

# qc/rules.py
from dataclasses import dataclass, field
from .detect import Event

RULES_VERSION = "2026.09.1"
HEAD = 3.0   # seconds considered "the start"
TAIL = 3.0   # seconds considered "the end"

@dataclass
class Verdict:
    status: str = "pass"          # pass | warn | fail
    notes: list = field(default_factory=list)
    rules_version: str = RULES_VERSION

    def raise_to(self, level: str, note: str):
        order = {"pass": 0, "warn": 1, "fail": 2}
        if order[level] > order[self.status]:
            self.status = level
        self.notes.append(f"{level}: {note}")

def judge(events: list[Event], total_duration: float) -> Verdict:
    v = Verdict()

    for e in events:
        at_head = e.start <= HEAD
        at_tail = (e.start + e.duration) >= (total_duration - TAIL)
        spans_all = e.duration >= total_duration * 0.95

        if e.kind == "black":
            if at_head and e.duration <= 3:
                continue                                    # fade-in, fine
            if at_head:
                v.raise_to("fail", f"black intro {e.duration:.1f}s")
            elif at_tail and e.duration <= 5:
                continue                                    # fade-out, fine
            elif e.duration >= 5:
                v.raise_to("warn", f"black at {e.start:.0f}s for {e.duration:.1f}s")

        elif e.kind == "freeze":
            if at_tail:
                continue                                    # held last frame
            v.raise_to("fail", f"freeze at {e.start:.0f}s for {e.duration:.1f}s")

        elif e.kind == "silence":
            if spans_all:
                v.raise_to("fail", "audio silent for entire duration")
            elif e.duration >= 30:
                v.raise_to("warn", f"silence at {e.start:.0f}s for {e.duration:.1f}s")

    return v

Now the same detections produce useful output:

$ python3 -m qc out/lesson-14.mp4
{
  "status": "fail",
  "notes": [
    "fail: black intro 22.4s",
    "fail: freeze at 604s for 11.3s",
    "fail: audio silent for entire duration"
  ],
  "rules_version": "2026.09.1"
}

$ python3 -m qc out/promo-clip.mp4
{
  "status": "pass",
  "notes": [],
  "rules_version": "2026.09.1"
}

The second file also had a black detection (a 1.2s fade-in) and a tail freeze. It passed, correctly, because the rules know where those landed.

4. Tier 3: sampled reference comparison

SSIM against the source catches "the encode ran but produced garbage," and it costs two decodes. Don't put it in the synchronous path.

ffmpeg -hide_banner \
  -i out/lesson-14.mp4 -i src/lesson-14.mov \
  -lavfi "[0:v]scale=1280:720[a];[1:v]scale=1280:720[b];[a][b]ssim=stats_file=ssim.log" \
  -f null -

SSIM Y:0.981234 U:0.992011 V:0.991455 All:0.984772 (18.156204)

# qc/reference.py: run nightly or on sampled assets only
SSIM_FLOOR = 0.93

def check_ssim(all_score: float):
    if all_score < SSIM_FLOOR:
        return [("fail", f"SSIM {all_score:.3f} below floor {SSIM_FLOOR}")]
    return []

⚠️ Scale both inputs to a common resolution before comparing, or ssim errors out on mismatched dimensions. And pick the floor empirically from your own known-good output; a number copied from someone else's pipeline will either never fire or fire constantly.

5. Wire it into the pipeline

# qc/__main__.py
import sys, json
from .structural import check_structural, probe
from .detect import run_detectors
from .rules import judge

def gate(output: str, source_duration: float, expect_audio: bool = True) -> dict:
    # tier 1: fail fast, no decode
    structural = check_structural(output, source_duration, expect_audio)
    if any(level == "fail" for level, _ in structural):
        return {"status": "fail",
                "notes": [f"{l}: {m}" for l, m in structural],
                "tier": 1}

    # tier 2: one decode, three detectors
    info = probe(output)
    verdict = judge(run_detectors(output), info.duration)
    return {"status": verdict.status, "notes": verdict.notes,
            "rules_version": verdict.rules_version, "tier": 2}

if __name__ == "__main__":
    result = gate(sys.argv[1], float(sys.argv[2]))
    print(json.dumps(result, indent=2))
    sys.exit(1 if result["status"] == "fail" else 0)

Exiting nonzero on fail lets you drop this straight into a worker or a CI step.

6. Store the verdict with its rules version

ALTER TABLE assets
  ADD COLUMN qc_status text,
  ADD COLUMN qc_notes jsonb,
  ADD COLUMN qc_rules_version text;

This matters the first time you loosen a threshold. When a customer's library legitimately opens on slow fades and you relax the black-intro rule, you need to find everything that failed under the old version and re-judge it. Without the stamp, you re-run QC on your entire library.

The scarier direction is the same query in reverse: when you tighten a rule because something got through, you want the list of everything that passed under the looser version.

What's next

Run tier 1 against last week's output before you build anything else. One ffprobe call per asset, three assertions, a count at the end:

$ for f in out/*.mp4; do python3 -m qc.structural "$f" || echo "FAIL $f"; done | grep -c FAIL
7

Seven broken assets you're already serving is a better argument for this project than any design doc.

From there, the things worth adding are ebur128 for loudness (catches "audio exists but is inaudible," which silencedetect misses entirely), a check that every rendition in your ABR ladder has matching duration and keyframe positions, and per-tenant rule overrides so the customer with the slow fades gets their own thresholds instead of forcing you to loosen everyone's.

Build an audio description pipeline that actually fits the gaps

Mason K — Wed, 16 Sep 2026 13:40:31 +0000

TL;DR

We're building an audio description pipeline in Python: find the silent gaps with FFmpeg, compute a word budget per gap, generate descriptions constrained to that budget, render TTS, reject anything that overruns, and mux the result as a second HLS audio track. The overrun check is the step everyone skips and the one that makes the output usable.

Captions were easy because text sits in its own layer. Audio description has to go into the channel that already has the dialogue in it, so it's a packing problem, not a writing problem. A great 9-second description is a defect if the gap is 2.4 seconds.

That constraint drives the whole design. Let's build it.

What we're building

video.mp4
   ├─ 1. silence map        (ffmpeg silencedetect)  → gaps[]
   ├─ 2. budget per gap     (duration × speech rate) → max_words
   ├─ 3. frame sampling     (ffmpeg, mid-gap frames) → context
   ├─ 4. description        (constrained to max_words)
   ├─ 5. TTS + HARD CHECK   (reject if rendered > gap) → retry shorter
   ├─ 6. mix                (overlay narration on original audio)
   └─ 7. package            (second audio rendition in the HLS ladder)

Requirements: ffmpeg 6.0+ and python 3.11+.

ffmpeg -version | head -1
python3 --version

ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
Python 3.12.3

1. Find the gaps 🔍

silencedetect is the whole first step. It writes to stderr, not stdout, which catches people out:

ffmpeg -i video.mp4 -af silencedetect=noise=-30dB:d=1.2 -f null - 2>&1 | grep silence_

[silencedetect @ 0x14e004c30] silence_start: 3.264
[silencedetect @ 0x14e004c30] silence_end: 6.912 | silence_duration: 3.648
[silencedetect @ 0x14e004c30] silence_start: 19.04
[silencedetect @ 0x14e004c30] silence_end: 21.44 | silence_duration: 2.4

Two parameters to tune per content type. noise=-30dB is the threshold for what counts as silence, and you'll want it lower (more strict) for content with music beds, or you'll "find" gaps that are full of score. d=1.2 is the minimum duration, because a gap shorter than about a second can't hold a useful sentence anyway.

# src/gaps.py
import re
import subprocess
from dataclasses import dataclass

SILENCE_RE = re.compile(
    r"silence_start: (?P<start>[\d.]+)|silence_end: (?P<end>[\d.]+) \| silence_duration: (?P<dur>[\d.]+)"
)

@dataclass
class Gap:
    start: float
    end: float
    duration: float

def find_gaps(path: str, noise_db: int = -30, min_dur: float = 1.2) -> list[Gap]:
    proc = subprocess.run(
        ["ffmpeg", "-i", path, "-af",
         f"silencedetect=noise={noise_db}dB:d={min_dur}", "-f", "null", "-"],
        capture_output=True, text=True,
    )
    gaps, pending_start = [], None
    for m in SILENCE_RE.finditer(proc.stderr):
        if m.group("start"):
            pending_start = float(m.group("start"))
        elif m.group("end") and pending_start is not None:
            end = float(m.group("end"))
            gaps.append(Gap(pending_start, end, float(m.group("dur"))))
            pending_start = None
    return gaps

⚠️ Leave headroom. Starting narration the instant dialogue stops sounds wrong and clips on some decoders. Trim 150ms off each end of the gap before you budget it.

2. Turn duration into a word budget 📏

This is the step that makes the pipeline work, and it's just arithmetic:

# src/budget.py
from src.gaps import Gap

# Tune this for your TTS voice. Measure it, don't trust it:
# render 100 known-length sentences and divide.
WORDS_PER_SECOND = 2.5
EDGE_PADDING = 0.15  # seconds trimmed from each end

def word_budget(gap: Gap) -> int:
    usable = gap.duration - (EDGE_PADDING * 2)
    return max(0, int(usable * WORDS_PER_SECOND))

def usable_gaps(gaps: list[Gap], min_words: int = 4) -> list[tuple[Gap, int]]:
    scored = [(g, word_budget(g)) for g in gaps]
    return [(g, w) for g, w in scored if w >= min_words]

>>> gaps = find_gaps("video.mp4")
>>> len(gaps)
47
>>> len(usable_gaps(gaps))
31

Sixteen gaps were too short to say anything meaningful in. That's normal, and it's information: those are the moments where standard description cannot comply, and where WCAG's extended audio description criterion (1.2.7, Level AAA) exists precisely because the only fix is pausing the video. Log them, don't silently drop them.

💡 Tip: WORDS_PER_SECOND = 2.5 is a starting assumption, not a measured constant. Different voices and languages differ substantially. Measure yours and put the real number here.

3. Sample frames for visual context 🖼️

We need to know what's on screen during the gap, so grab a few frames from inside it:

# src/frames.py
import subprocess
from pathlib import Path
from src.gaps import Gap

def sample_frames(video: str, gap: Gap, out_dir: Path, n: int = 3) -> list[Path]:
    out_dir.mkdir(parents=True, exist_ok=True)
    paths = []
    step = gap.duration / (n + 1)
    for i in range(1, n + 1):
        ts = gap.start + (step * i)
        out = out_dir / f"{gap.start:.2f}_{i}.jpg"
        subprocess.run(
            ["ffmpeg", "-y", "-ss", f"{ts:.3f}", "-i", video,
             "-frames:v", "1", "-q:v", "3", str(out)],
            capture_output=True, check=True,
        )
        paths.append(out)
    return paths

-ss before -i seeks fast by jumping to the nearest keyframe. For a few frames of context that's fine; if you need frame-exact sampling, put -ss after -i and accept it'll be much slower.

4. Generate to the budget, not to a quality bar

Whatever model you use, the budget is the part of the prompt that matters. Put it in hard and put it first:

# src/describe.py
SYSTEM = """You write audio description for blind and low-vision viewers.

HARD CONSTRAINT: your output MUST be {max_words} words or fewer. This is not a
style preference. The narration plays in a {duration:.1f} second gap between
lines of dialogue and will be cut off if it runs long.

Rules:
- Describe only what a sighted viewer gains that the audio does not convey.
- Prioritise: character actions > scene changes > appearance > atmosphere.
- Use present tense. No "we see", no "the camera shows", no interpretation
  of emotion or intent.
- If nothing visually important happens here, output exactly: SKIP
"""

def build_prompt(gap, max_words, prior_descriptions):
    return SYSTEM.format(max_words=max_words, duration=gap.duration), {
        # Prior descriptions let the model stop re-introducing the same character.
        "already_described": prior_descriptions[-5:],
    }

Two details matter more than the model choice. SKIP must be a valid output, because a pipeline that fills all 31 gaps just because it can produces narration that's exhausting to listen to. And pass prior descriptions, or you get "a man in a blue coat" in gap 3 and again in gap 28. Models are weak on narrative continuity; carrying context is the cheap partial fix.

5. Render, then verify the duration 🎙️

This is the section the whole tutorial exists for. Do not trust the word budget. Render the audio and measure the actual file:

# src/render.py
import subprocess
from pathlib import Path

def audio_duration(path: Path) -> float:
    out = subprocess.run(
        ["ffprobe", "-v", "error", "-show_entries", "format=duration",
         "-of", "default=noprint_wrappers=1:nokey=1", str(path)],
        capture_output=True, text=True, check=True,
    )
    return float(out.stdout.strip())

def render_fitting(text: str, gap, out: Path, tts_fn, max_attempts: int = 3):
    """Render TTS and shrink the text until it fits the gap."""
    budget = gap.duration - 0.30  # edge padding
    current = text
    for attempt in range(max_attempts):
        tts_fn(current, out)
        actual = audio_duration(out)
        if actual <= budget:
            return {"ok": True, "text": current, "duration": actual, "attempts": attempt + 1}

        overrun = actual - budget
        keep = int(len(current.split()) * (budget / actual) * 0.9)
        print(f"  overrun {overrun:.2f}s, trimming to ~{keep} words")
        current = " ".join(current.split()[:keep])

    # Speeding up narration to force a fit makes it harder to understand.
    # Flag for human review instead.
    return {"ok": False, "text": current, "duration": audio_duration(out), "reason": "could not fit"}

python3 -m src.pipeline video.mp4

gap 03.26-06.91 (3.65s, budget 8 words)
  overrun 0.41s, trimming to ~7 words
  fit in 2 attempts: "She slides the envelope under the door."
gap 19.04-21.44 (2.40s, budget 5 words)
  fit in 1 attempt: "He notices the broken lock."
gap 44.10-45.80 (1.70s, budget 3 words)
  SKIP (nothing visually significant)
gap 61.22-63.02 (1.80s, budget 4 words)
  overrun 0.62s, trimming to ~3 words
  overrun 0.28s, trimming to ~2 words
  FLAGGED: could not fit -> review queue

24 described, 5 skipped, 2 flagged for review

Those 2 flagged items are the output you want. A pipeline that silently truncates mid-word, or time-compresses narration until it's unintelligible, produces a track that technically exists and nobody can use.

6. Mix and package as a second audio track 🔊

Build a narration bed, then overlay it on the original:

# place each clip at its gap start on a silent bed the length of the video
ffmpeg -f lavfi -t 1800 -i anullsrc=r=48000:cl=stereo \
  -i ad_003.wav -i ad_019.wav \
  -filter_complex "\
    [1:a]adelay=3414|3414[a1]; \
    [2:a]adelay=19190|19190[a2]; \
    [0:a][a1][a2]amix=inputs=3:duration=first:normalize=0[bed]" \
  -map "[bed]" narration_bed.wav

# duck the original under the narration, then mux as a described track
ffmpeg -i video.mp4 -i narration_bed.wav \
  -filter_complex "[0:a][1:a]sidechaincompress=threshold=0.05:ratio=8[ducked]; \
                   [ducked][1:a]amix=inputs=2:duration=first[out]" \
  -map 0:v -map "[out]" -c:v copy -c:a aac -b:a 128k \
  -metadata:s:a:0 title="Audio Description" \
  -metadata:s:a:0 language=eng \
  video_described.mp4

sidechaincompress ducks the original audio whenever narration plays, which is what a professional mix does. Without it the narration competes with the score.

Then it's another rendition in your HLS audio group:

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",NAME="English",LANGUAGE="en",DEFAULT=YES,AUTOSELECT=YES,URI="audio/en.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",NAME="English (Described)",LANGUAGE="en",CHARACTERISTICS="public.accessibility.describes-video",AUTOSELECT=NO,URI="audio/en-ad.m3u8"

⚠️ CHARACTERISTICS="public.accessibility.describes-video" is what tells players this is a description track rather than a second language. Without it, players won't surface it in their accessibility menu and iOS won't auto-select it for users who've enabled Audio Descriptions system-wide. Easy to omit, invisible when wrong.

7. Monitor it, or you'll lose it 📡

This is the operational failure nobody plans for. Nobody on your team uses the description track. Your smoke tests check that video plays. A packager change drops the second rendition from the manifest and you find out eighteen months later from a complaint.

# src/monitor.py
import m3u8  # pip install m3u8

DESCRIBES = "public.accessibility.describes-video"

def has_description_track(master_url: str) -> bool:
    playlist = m3u8.load(master_url)
    return any(
        m.type == "AUDIO" and DESCRIBES in (m.characteristics or "")
        for m in playlist.media
    )

def assert_described(master_url: str):
    if not has_description_track(master_url):
        raise AssertionError(f"description track missing from {master_url}")

Wire that into whatever already checks your manifests. An accessibility feature you aren't monitoring is an accessibility feature you used to have.

What's next

A human review queue is not optional. Automated description is production-viable across many content types with human review, and not reliable enough without it on complex narrative material. Build the queue as a first-class part of the pipeline.
Extended audio description (WCAG 1.2.7). For gaps that were too short, the compliant answer is a player that pauses, which is a different project. W3C's understanding doc starts there.
Test with actual screen reader users. Every metric here is a proxy for "is this useful", and proxies drift.

The regulatory deadlines are real (ADA Title II landed April 24, 2026 for larger public entities; the EU Accessibility Act has applied since June 2025), but the better reason to build this is that the pipeline above turns a specialist hour per video into a few minutes of review, and that's the difference between describing your top 50 videos and describing all of them. #a11y #video

Write a portability audit for your video provider before you need one

Mason K — Wed, 16 Sep 2026 13:40:12 +0000

TL;DR

We're building a small Node script that audits your video library for portability: can you get the source file back, is playback standard HLS, and how many places is the provider's ID buried. It writes a CSV. Run it once a quarter and you'll never be surprised by a migration estimate again.

Most of us pick a video API by reading pricing pages and running an upload benchmark. Nobody checks the exit until they're already trying to leave, at which point the answer is expensive and non-negotiable.

This is a small thing you can run today. It's a provider-agnostic script with a thin adapter per platform, and the worked example uses FastPix because its source-retention flag is explicit in the API, which makes for a clear demo. The pattern moves to Mux, Cloudflare Stream or api.video by writing a different adapter.

What we're checking

Three questions, in order of how much they'll cost you:

Source retention. Does the provider still hold your original upload, or only the transcoded renditions? If it's renditions-only, a migration re-encodes from output and you lose quality permanently.
Playback portability. Is the playback URL a standard HLS manifest any player can consume, or a proprietary embed?
ID sprawl. How many of your own tables carry the provider's media and playback IDs? Every one is a migration write path.

1. Project setup 🛠️

Node 20.x or newer, no dependencies beyond what ships with it.

mkdir video-portability-audit && cd video-portability-audit
npm init -y
npm pkg set type=module
node --version

v22.14.0

Credentials go in the environment, never in the file:

# .env.example : copy to .env, never commit .env
FASTPIX_TOKEN_ID=your_access_token_id
FASTPIX_SECRET_KEY=your_secret_key

⚠️ Note: the script only ever issues GET requests. It reads your library and writes a local CSV. It will not modify or delete anything.

2. The adapter interface

Every provider adapter answers the same three questions for one media ID. Keeping this interface narrow is what makes the script portable across platforms.

// src/adapter.js
/**
 * @typedef {Object} AuditResult
 * @property {string} mediaId
 * @property {boolean|null} sourceRetained   - can we get the original file back?
 * @property {boolean|null} standardHls      - is playback a plain .m3u8?
 * @property {string} playbackUrl
 * @property {string} notes
 */

export class Adapter {
  /** @returns {Promise<AuditResult>} */
  async audit(mediaId) {
    throw new Error('not implemented');
  }
}

3. The FastPix adapter

FastPix uses HTTP Basic auth with the Access Token ID as the username and the Secret Key as the password, so we can build the header with Buffer:

// src/providers/fastpix.js
import { Adapter } from '../adapter.js';

const API = 'https://api.fastpix.com/v1';

export class FastPixAdapter extends Adapter {
  constructor(tokenId, secretKey) {
    super();
    this.auth = 'Basic ' + Buffer.from(`${tokenId}:${secretKey}`).toString('base64');
  }

  async #get(path) {
    const res = await fetch(`${API}${path}`, {
      headers: { Authorization: this.auth, Accept: 'application/json' },
    });
    if (!res.ok) {
      throw new Error(`${res.status} ${res.statusText} on ${path}`);
    }
    return res.json();
  }

  async audit(mediaId) {
    // The create response returns `data` as an OBJECT, not an array.
    // Older docs showed an array; if you wrote a parser against those, it breaks here.
    const { data } = await this.#get(`/on-demand/${mediaId}`);

    const playbackId = data.playbackIds?.[0]?.id ?? null;
    const playbackUrl = playbackId
      ? `https://stream.fastpix.com/${playbackId}.m3u8`
      : '';

    return {
      mediaId,
      // sourceAccess is set at ingest time on the create call.
      // If it wasn't enabled then, the original is not retrievable now.
      sourceRetained: data.sourceAccess ?? null,
      standardHls: Boolean(playbackUrl.endsWith('.m3u8')),
      playbackUrl,
      notes: data.status ?? '',
    };
  }
}

💡 Tip: sourceAccess is a create-time decision. That's the important bit for this audit. If it defaulted off across your whole library, this script will tell you today rather than during a migration in 2028.

4. Feeding it media IDs

Here's a design choice worth explaining, because it looks like a limitation and isn't.

The script takes media IDs from your database, not from a list-everything API call. Two reasons. First, you already store these IDs, because you need them to render a page. Second, and more usefully: the IDs your application knows about are the ones that matter. An asset sitting in the provider's dashboard that nothing in your product references is not a migration problem. A row in your videos table is.

// src/ids.js
import { readFile } from 'node:fs/promises';

// One media ID per line. Generate it however you like, e.g.:
//   psql -At -c "select provider_media_id from videos where deleted_at is null" > ids.txt
export async function loadIds(path = 'ids.txt') {
  const raw = await readFile(path, 'utf8');
  return raw.split('\n').map((l) => l.trim()).filter(Boolean);
}

5. The runner, with concurrency and retries

Rate limits are the thing that turns a five-minute script into a two-hour one. A small concurrency cap plus backoff on 429 handles it:

// src/run.js
import { writeFile } from 'node:fs/promises';
import { loadIds } from './ids.js';
import { FastPixAdapter } from './providers/fastpix.js';

const CONCURRENCY = 4;

async function withRetry(fn, attempts = 4) {
  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      const is429 = String(err.message).startsWith('429');
      if (!is429 || i === attempts - 1) throw err;
      const waitMs = 2 ** i * 500;
      console.warn(`  rate limited, backing off ${waitMs}ms`);
      await new Promise((r) => setTimeout(r, waitMs));
    }
  }
}

async function pool(items, size, worker) {
  const results = [];
  const queue = [...items];
  const runners = Array.from({ length: size }, async () => {
    while (queue.length) {
      const item = queue.shift();
      results.push(await worker(item));
    }
  });
  await Promise.all(runners);
  return results;
}

const adapter = new FastPixAdapter(
  process.env.FASTPIX_TOKEN_ID,
  process.env.FASTPIX_SECRET_KEY,
);

const ids = await loadIds();
console.log(`auditing ${ids.length} assets...`);

const rows = await pool(ids, CONCURRENCY, async (id) => {
  try {
    return await withRetry(() => adapter.audit(id));
  } catch (err) {
    return { mediaId: id, sourceRetained: null, standardHls: null, playbackUrl: '', notes: `ERROR: ${err.message}` };
  }
});

const csv = [
  'media_id,source_retained,standard_hls,playback_url,notes',
  ...rows.map((r) =>
    [r.mediaId, r.sourceRetained, r.standardHls, r.playbackUrl, JSON.stringify(r.notes)].join(','),
  ),
].join('\n');

await writeFile('portability-audit.csv', csv);

const noSource = rows.filter((r) => r.sourceRetained === false).length;
const errors = rows.filter((r) => r.notes.startsWith('ERROR')).length;
console.log(`\ndone. ${rows.length} assets, ${noSource} without retrievable source, ${errors} errors`);
console.log('wrote portability-audit.csv');

Run it:

node --env-file=.env src/run.js

auditing 1204 assets...
  rate limited, backing off 500ms
  rate limited, backing off 1000ms

done. 1204 assets, 341 without retrievable source, 2 errors
wrote portability-audit.csv

That 341 is the number this whole script exists to produce. It's the size of the problem you didn't know you had.

6. The ID sprawl check 🔍

The third question doesn't need the API at all, just grep over your own codebase and a look at your schema. Provider IDs leak further than anyone expects:

# how many places does a provider ID appear?
rg -l --type-add 'src:*.{ts,tsx,js,jsx,sql,py,rb,go}' -t src \
   -e 'playbackId' -e 'playback_id' -e 'media_id' -e 'stream\.fastpix\.com' \
   | sort | uniq -c | sort -rn

Then the one people forget:

-- every column that could be holding a provider ID
SELECT table_name, column_name
FROM information_schema.columns
WHERE column_name ILIKE '%playback%'
   OR column_name ILIKE '%media_id%'
   OR column_name ILIKE '%video_id%'
ORDER BY table_name;

Count the rows. That's your migration write path. If provider IDs are also in emails you've already sent, in a cached CDN rule, or in URLs your customers bookmarked, add a redirect layer to the estimate.

💡 Tip: the fix for ID sprawl is cheap before you need it. Put your own UUID on every video, map it to the provider ID in exactly one table, and reference only your UUID everywhere else. A migration then touches one table instead of nine.

Porting the adapter to other providers

The interface is three fields, so a new adapter is usually under thirty lines:

Provider	Source retention	Playback	Bulk tooling
FastPix	`sourceAccess` on create	`stream.fastpix.com/<id>.m3u8`	Built-in batch migration tool
Mux	Check asset settings	Standard HLS	Documented framework, you build the workflow
Cloudflare Stream	Check asset settings	Standard HLS	Roll your own
api.video	Check asset settings	Standard HLS	Markets zero migration cost

I've deliberately left the middle column vague for three of the four, because the flag names differ and I'd rather you read the current reference than trust a table I wrote today. For the FastPix fields used above, the VOD API reference has the create-media body and the get-media response shape.

What's next

Two follow-ups worth doing:

Schedule it. Run the audit in CI monthly and fail the build if source_retained=false count increases. That turns a one-off report into a guardrail, and it's the version of this that actually keeps working after everyone forgets about it.
Add a restore test. Auditing that a source file should be retrievable is not the same as proving it is. Pull one source file per month and check the bytes. Backups you haven't restored are backups you don't have, and the same logic applies here.

If you run this against your own library, I'd be curious what your source_retained=false number comes out as. Mine was worse than I guessed.

Add casting to a custom HTML5 video player without breaking it

Mason K — Wed, 16 Sep 2026 13:39:41 +0000

TL;DR

We're adding a cast button to a custom <video> player using the Remote Playback API, with an AirPlay fallback for Safari. Then we fix the four bugs that make casting work on your laptop and fail on an actual TV: token expiry, CORS on the receiver, unreachable hosts, and treating prompt() as if it were a connection.

The thing to understand before writing any code: the TV does not run your player. It fetches your stream URL and plays it with its own stack. Your overlays, your quality caps, your analytics, your token refresh logic, none of it makes the trip.

So this tutorial is half API wiring and half making your stream survive without you.

1. The baseline player 🎥

Start with a plain element and our own controls. Nothing exotic:

<!-- index.html -->
<video id="player" playsinline controls></video>

<div class="controls">
  <button id="castBtn" hidden>Cast</button>
  <span id="castState"></span>
</div>

<script type="module" src="./src/player.js"></script>

// src/player.js
import Hls from 'hls.js'; // hls.js 1.5.x

const video = document.getElementById('player');
const MANIFEST = 'https://example.com/path/playback.m3u8';

if (video.canPlayType('application/vnd.apple.mpegurl')) {
  // Safari plays HLS natively, and this path is also what AirPlay uses.
  video.src = MANIFEST;
} else if (Hls.isSupported()) {
  const hls = new Hls();
  hls.loadSource(MANIFEST);
  hls.attachMedia(video);
}

⚠️ Note: that branch matters later. On Safari the element has a real src, which AirPlay can fling. With hls.js, MSE is feeding the element and video.src is a blob URL that means nothing to a TV. This is the single biggest source of confusion in casting code.

2. Feature detection, three ways

There's no one API. We check for each path and pick:

// src/cast-support.js
export function detectCastSupport(video) {
  return {
    // Standards-based path. Chromium-leaning; MDN lists it as limited availability.
    remotePlayback: 'remote' in video && typeof video.remote?.watchAvailability === 'function',

    // Safari's own AirPlay interface, predates the spec above.
    airplay: typeof window.WebKitPlaybackTargetAvailabilityEvent !== 'undefined',

    // Cast SDK, only if you've loaded the sender script and written a receiver app.
    castSdk: typeof window.cast !== 'undefined' && typeof window.chrome?.cast !== 'undefined',
  };
}

// quick check in the console
detectCastSupport(document.getElementById('player'))
// Chrome:  { remotePlayback: true,  airplay: false, castSdk: false }
// Safari:  { remotePlayback: false, airplay: true,  castSdk: false }
// Firefox: { remotePlayback: false, airplay: false, castSdk: false }

3. Wiring the Remote Playback API

Two calls do most of the work. watchAvailability tells us whether to show the button at all, and prompt opens the browser's own device picker.

// src/remote-playback.js
export function initRemotePlayback(video, btn, onState) {
  if (!('remote' in video)) return;

  // Show the button only when a device is actually reachable.
  video.remote
    .watchAvailability((available) => {
      btn.hidden = !available;
    })
    .catch(() => {
      // Some browsers can't watch continuously. Show the button and let
      // prompt() fail loudly instead of hiding the feature entirely.
      btn.hidden = false;
    });

  btn.addEventListener('click', async () => {
    try {
      await video.remote.prompt();
    } catch (err) {
      // AbortError = user dismissed the picker. Not an error worth surfacing.
      if (err.name !== 'AbortError') {
        console.error('cast failed', err);
        onState('error', err);
      }
    }
  });

  // These are the source of truth, NOT the click handler.
  video.remote.addEventListener('connecting', () => onState('connecting'));
  video.remote.addEventListener('connect', () => onState('connected'));
  video.remote.addEventListener('disconnect', () => onState('disconnected'));
}

The comment on those last three lines is the whole lesson of this tutorial, so let's make it explicit.

4. The state machine (this is the bug you'll ship) ⚠️

The naive version sets isCasting = true when the click handler resolves. That's wrong, because remote playback is asynchronous and can fail after it succeeds. The device drops off wifi. Someone's phone steals the TV. The receiver crashes.

Derive your UI from the connection state the browser reports:

// src/player.js (continued)
import { initRemotePlayback } from './remote-playback.js';

const btn = document.getElementById('castBtn');
const label = document.getElementById('castState');

let castState = 'disconnected';

function render() {
  label.textContent = {
    disconnected: '',
    connecting: 'Connecting to device...',
    connected: 'Playing on TV',
    error: 'Could not connect',
  }[castState];

  // Local controls are meaningless while the TV owns playback.
  document.querySelectorAll('.local-only').forEach((el) => {
    el.toggleAttribute('disabled', castState === 'connected');
  });
}

initRemotePlayback(video, btn, (state) => {
  castState = state;
  render();
});

// Belt and braces: video.remote.state is readable at any time.
setInterval(() => {
  if (video.remote?.state && video.remote.state !== castState) {
    castState = video.remote.state;
    render();
  }
}, 2000);

💡 Tip: once connected, treat the remote as the source of truth for playback position and paused state. You now have two states that can disagree, and the one on the television is the one your user is looking at.

5. AirPlay, Safari's way 🍎

Safari doesn't implement the API above. It has its own event and its own picker:

// src/airplay.js
export function initAirPlay(video, btn) {
  if (typeof window.WebKitPlaybackTargetAvailabilityEvent === 'undefined') return;

  video.addEventListener('webkitplaybacktargetavailabilitychanged', (e) => {
    btn.hidden = e.availability !== 'available';
  });

  btn.addEventListener('click', () => {
    video.webkitShowPlaybackTargetPicker();
  });

  video.addEventListener('webkitcurrentplaybacktargetiswirelesschanged', () => {
    console.log('airplay active:', video.webkitCurrentPlaybackTargetIsWireless);
  });
}

You can also let Safari handle it entirely with an attribute, which is what most sites do:

<video id="player" x-webkit-airplay="allow" playsinline controls></video>

6. Now fix the four bugs that only happen on real hardware

Everything above works on your laptop. Here's what breaks on an actual TV.

Token expiry on the wrong clock

You mint a short-lived signed playback token because that's correct security practice. The TV picks the stream up thirty seconds later and holds it for ninety minutes, then 403s on a manifest refresh. Works in your tab, dies on the device.

// src/playback-url.js
// Cast sessions outlive browser sessions. Mint a token scoped to the session type.
export async function getPlaybackUrl({ forCast = false } = {}) {
  const res = await fetch('/api/playback-token', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ ttlSeconds: forCast ? 60 * 60 * 4 : 60 * 15 }),
  });
  const { url } = await res.json();
  return url;
}

Swap the element's source before prompting:

btn.addEventListener('click', async () => {
  video.src = await getPlaybackUrl({ forCast: true });
  await video.remote.prompt();
});

⚠️ A longer TTL is a real security tradeoff, not a free fix. Scope the long-lived token to a single playback ID and a single device session, and keep the short TTL for normal browser playback.

CORS from an origin you forgot exists

The receiver is not your page. It's a different origin fetching your manifests and segments:

# verify from outside your app's origin
curl -I -H "Origin: https://not-your-site.example" \
  https://stream.example.com/abc123.m3u8

HTTP/2 200
access-control-allow-origin: *
access-control-allow-headers: range

If access-control-allow-origin is missing or pinned to your domain only, the TV gets blocked while your browser is fine.

Anything the TV can't resolve

Localhost, staging behind a VPN, an internal hostname. Your laptop resolves it, the TV on the guest network doesn't. Catch it in code instead of losing an afternoon:

// src/preflight.js
export function assertCastableUrl(url) {
  const u = new URL(url, location.href);
  const problems = [];
  if (u.protocol !== 'https:') problems.push('not https');
  if (['localhost', '127.0.0.1', '[::1]'].includes(u.hostname)) problems.push('localhost is not reachable from a TV');
  if (u.hostname.endsWith('.local') || u.hostname.endsWith('.internal')) problems.push('internal hostname');
  if (u.href.startsWith('blob:')) problems.push('blob URL from MSE, the TV cannot fetch this; use the real manifest URL');
  if (problems.length) console.warn(`[cast preflight] ${u.href}: ${problems.join(', ')}`);
  return problems.length === 0;
}

That last check is the hls.js trap from section 1. If you're feeding the element through MSE, video.src is a blob and there is nothing for the TV to fetch. Keep the real manifest URL around and assign it before casting.

The receiver's ABR isn't yours

Your bitrate cap for metered connections lived in hls.js config. The TV has never heard of it and will pull your top rendition. If that matters for cost or for a plan tier, the only lever you have left is the manifest itself: serve a cast-specific variant playlist with the renditions you're willing to send.

7. When to just turn it off

There's a property for this, and using it is a legitimate decision:

// DRM the receiver can't honour, overlay-critical content, or
// ad insertion that the cast path would silently skip.
video.disableRemotePlayback = true;

A disabled cast button with one line of copy explaining why beats a button that connects to a TV and then fails in a way the user blames on their wifi. "We support casting" is a claim with a testing burden attached.

Wrapping up

The test that predicts all of this: put your manifest URL in a bare <video> tag on a blank page, with none of your JavaScript anywhere, walk away for an hour, and see whether it's still playing. If it is, casting is nearly free, and so are smart TV apps and embedded webviews later. If it isn't, fix that first, because the cast button is only the first surface where it shows.

Two things worth reading next: MDN's Remote Playback API page has the current compatibility table, which is the part of this that changes; and if you need real control over what the TV renders, the Google Cast SDK's receiver application is the only path that gives it to you, at the cost of maintaining a second app.

What's your cast failure rate? If you're not measuring it separately from normal playback failures, it's probably higher than you think, because these bugs are invisible from the browser. #webdev #video