DEV Community: Mason K

Build a conformance gate that skips the encode when the upload is already fine

Mason K — Thu, 24 Sep 2026 12:21:46 +0000

TL;DR

We're building a conformance gate in Python: probe an upload once with ffprobe, run it through ordered predicates, and get back one of three verdicts (passthrough, remux, transcode). The gate includes the keyframe-cadence check that most passthrough implementations forget, which is why they break at segmentation time.

Most upload pipelines transcode every file by reflex. But a lot of what lands in a UGC pipeline is already H.264 8-bit 4:2:0 straight off a phone, and re-encoding it costs compute and adds a second generation of loss for very little gain.

The fix isn't "stop encoding." It's classifying each file so you do the cheapest correct operation. Let's build the classifier.

Requirements: ffmpeg 8.x (7.x is fine too) and python 3.11+.

ffmpeg -version | head -1
python3 --version

1. One probe call, everything we need

Don't make three ffprobe calls. Make one and parse it.

# probe.sh
ffprobe -v error \
  -show_entries stream=index,codec_type,codec_name,profile,level,pix_fmt,width,height,avg_frame_rate,bit_rate \
  -show_entries format=duration,bit_rate,format_name \
  -of json input.mp4

Output looks like this:

{
  "streams": [
    {
      "index": 0,
      "codec_name": "h264",
      "codec_type": "video",
      "profile": "High",
      "level": 40,
      "pix_fmt": "yuv420p",
      "width": 1920,
      "height": 1080,
      "avg_frame_rate": "30000/1001"
    },
    {
      "index": 1,
      "codec_name": "aac",
      "codec_type": "audio"
    }
  ],
  "format": {
    "duration": "184.320000",
    "bit_rate": "8412160",
    "format_name": "mov,mp4,m4a,3gp,3g2,mj2"
  }
}

Wrap it:

# gate/probe.py
import json, subprocess

def probe(path: str) -> dict:
    cmd = [
        "ffprobe", "-v", "error",
        "-show_entries",
        "stream=index,codec_type,codec_name,profile,level,pix_fmt,"
        "width,height,avg_frame_rate,bit_rate",
        "-show_entries", "format=duration,bit_rate,format_name",
        "-of", "json", path,
    ]
    out = subprocess.run(cmd, capture_output=True, text=True, check=True)
    return json.loads(out.stdout)

def first(streams, kind):
    return next((s for s in streams if s.get("codec_type") == kind), None)

💡 Tip: check=True will raise on a file ffprobe can't open at all. That's its own verdict (reject), so catch CalledProcessError at the call site rather than letting it kill your worker.

2. The predicates, in the order they bite

Order matters here, not for correctness but for debuggability. You want the failure reason to be the most fundamental thing wrong with the file.

# gate/rules.py
from dataclasses import dataclass, field

DELIVERABLE_CODECS = {"h264"}
DELIVERABLE_PIX_FMTS = {"yuv420p", "yuvj420p"}
DELIVERABLE_AUDIO = {"aac"}
MAX_LEVEL = 42          # H.264 level 4.2
MAX_BITRATE = 12_000_000  # above this, re-encode even if conformant

@dataclass
class Verdict:
    action: str                     # passthrough | remux | transcode | reject
    reasons: list = field(default_factory=list)
    rules_version: str = "2026.09.1"

def classify(info: dict, container_ok: bool, keyframes_regular: bool) -> Verdict:
    streams = info.get("streams", [])
    v = first(streams, "video")
    a = first(streams, "audio")
    reasons = []

    if v is None:
        return Verdict("reject", ["no video stream"])

    # 1. codec
    if v["codec_name"] not in DELIVERABLE_CODECS:
        reasons.append(f"codec {v['codec_name']} not deliverable")

    # 2. pixel format: 10-bit H.264 is real and browsers refuse it
    if v.get("pix_fmt") not in DELIVERABLE_PIX_FMTS:
        reasons.append(f"pix_fmt {v.get('pix_fmt')} not 8-bit 4:2:0")

    # 3. level
    if v.get("level") and v["level"] > MAX_LEVEL:
        reasons.append(f"level {v['level']} above {MAX_LEVEL}")

    # 4. bitrate ceiling: conformant but wasteful
    br = int(info["format"].get("bit_rate") or 0)
    if br > MAX_BITRATE:
        reasons.append(f"bitrate {br} above ceiling")

    # 5. keyframe cadence: the one everybody forgets
    if not keyframes_regular:
        reasons.append("irregular keyframe interval, cannot segment cleanly")

    if reasons:
        return Verdict("transcode", reasons)

    # video is clean. is audio?
    if a and a["codec_name"] not in DELIVERABLE_AUDIO:
        return Verdict("remux", [f"audio {a['codec_name']} needs re-encode"])

    if not container_ok:
        return Verdict("remux", ["container or faststart fix needed"])

    return Verdict("passthrough", [])

3. The keyframe check (do not skip this one)

This is where naive passthrough implementations die. HLS and DASH cut segments on keyframes. A phone recording with a 10-second GOP, or an irregular scene-change-driven one, cannot be cut into 4-second segments without re-encoding.

Read the first couple hundred frames and look at the gaps:

# gate/keyframes.py
import subprocess, statistics

def keyframe_intervals(path: str, n_frames: int = 300) -> list[float]:
    cmd = [
        "ffprobe", "-v", "error", "-select_streams", "v:0",
        "-show_entries", "frame=pts_time,key_frame",
        "-of", "csv=p=0",
        "-read_intervals", f"%+#{n_frames}",
        path,
    ]
    out = subprocess.run(cmd, capture_output=True, text=True, check=True)
    times = []
    for line in out.stdout.strip().splitlines():
        parts = line.split(",")
        if len(parts) < 2:
            continue
        pts, is_key = parts[0], parts[1]
        if is_key == "1" and pts not in ("N/A", ""):
            times.append(float(pts))
    return [round(b - a, 3) for a, b in zip(times, times[1:])]

def keyframes_regular(path: str, target_segment: float = 4.0,
                      tolerance: float = 0.15) -> tuple[bool, str]:
    gaps = keyframe_intervals(path)
    if len(gaps) < 3:
        return False, "too few keyframes sampled"

    median = statistics.median(gaps)
    spread = max(gaps) - min(gaps)

    if spread > median * tolerance:
        return False, f"irregular GOP (median {median}s, spread {spread}s)"
    if target_segment % median > 0.05 and median % target_segment > 0.05:
        return False, f"GOP {median}s does not divide {target_segment}s segments"
    return True, f"regular GOP of {median}s"

Run it and you get something honest:

$ python3 -c "from gate.keyframes import keyframes_regular; print(keyframes_regular('phone.mp4'))"
(False, 'irregular GOP (median 2.0s, spread 4.867s)')

$ python3 -c "from gate.keyframes import keyframes_regular; print(keyframes_regular('screencap.mp4'))"
(True, 'regular GOP of 2.0s')

⚠️ Note: -read_intervals "%+#300" only samples the beginning of the file. A source can start regular and go irregular later. For high-value assets, sample a second window mid-file. For a first version, the head is enough, and it's the difference between a check that runs in 40ms and one that decodes the whole file.

4. Wire the three actions

Each verdict maps to a command. This is the whole payoff.

# gate/actions.py
import subprocess

def run_passthrough(src: str, dst: str):
    """Source becomes the top ladder rung as-is."""
    subprocess.run(["cp", src, dst], check=True)

def run_remux(src: str, dst: str, reencode_audio: bool):
    cmd = ["ffmpeg", "-y", "-i", src, "-c:v", "copy"]
    cmd += ["-c:a", "aac", "-b:a", "128k"] if reencode_audio else ["-c:a", "copy"]
    cmd += ["-movflags", "+faststart", dst]
    subprocess.run(cmd, check=True)

def run_transcode(src: str, dst: str):
    subprocess.run([
        "ffmpeg", "-y", "-i", src,
        "-c:v", "libx264", "-preset", "medium", "-crf", "21",
        "-pix_fmt", "yuv420p",
        "-g", "96", "-keyint_min", "96", "-sc_threshold", "0",
        "-c:a", "aac", "-b:a", "128k",
        "-movflags", "+faststart", dst,
    ], check=True)

Note the transcode path forces -pix_fmt yuv420p and a fixed GOP with -sc_threshold 0. That's not decoration: it guarantees the output passes your own gate, which matters when you re-run the pipeline later.

5. Put it together

# gate/__main__.py
import sys, json
from .probe import probe
from .keyframes import keyframes_regular
from .rules import classify
from .actions import run_passthrough, run_remux, run_transcode

def process(src: str, dst: str) -> dict:
    info = probe(src)
    regular, kf_note = keyframes_regular(src)
    container_ok = src.endswith(".mp4")   # replace with a real faststart check
    verdict = classify(info, container_ok, regular)

    if verdict.action == "passthrough":
        run_passthrough(src, dst)
    elif verdict.action == "remux":
        needs_audio = any("audio" in r for r in verdict.reasons)
        run_remux(src, dst, needs_audio)
    elif verdict.action == "transcode":
        run_transcode(src, dst)
    else:
        raise ValueError(f"rejected: {verdict.reasons}")

    return {
        "source": src,
        "action": verdict.action,
        "reasons": verdict.reasons,
        "keyframes": kf_note,
        "rules_version": verdict.rules_version,
    }

if __name__ == "__main__":
    print(json.dumps(process(sys.argv[1], sys.argv[2]), indent=2))

$ python3 -m gate uploads/phone-clip.mp4 out/phone-clip.mp4
{
  "source": "uploads/phone-clip.mp4",
  "action": "remux",
  "reasons": ["audio opus needs re-encode"],
  "keyframes": "regular GOP of 2.0s",
  "rules_version": "2026.09.1"
}

6. Store the verdict on the asset

Do not just log this. Put action and rules_version on the asset record.

In six months you'll lower MAX_BITRATE or add an AV1 rung, and you'll need to answer "which assets were passed through under the old rules?" Without the stamp, the answer is "re-process everything," which is the compute you built this to avoid.

ALTER TABLE assets
  ADD COLUMN ingest_action  text NOT NULL DEFAULT 'transcode',
  ADD COLUMN ingest_rules_version text NOT NULL DEFAULT 'legacy';

What's next

Two things to be honest about before you ship this.

Passthrough saves one rung, not the whole encode. ABR still needs 720p, 480p, 360p. What you're skipping is the top rung, which is the most expensive one, on some fraction of ingests. Real saving, smaller headline.

Measure before you build. Run steps 1 to 3 across a sample of what you've already ingested, skip the actions entirely, and just count verdicts:

for f in sample/*.mp4; do python3 -m gate.classify_only "$f"; done | \
  jq -r .action | sort | uniq -c

  41 transcode
  22 remux
  17 passthrough

If passthrough and remux are a rounding error for your traffic, you've spent an afternoon and learned something. If they aren't, you just found the cheapest infra win on your backlog.

From here, the next things worth adding are a real faststart check (ffprobe the moov atom position rather than trusting the extension), a mid-file keyframe sample, and rotation-metadata handling so sideways phone video doesn't get re-encoded just to be turned.

Add a DVR window to your live HLS stream without 404ing the scrubber

Mason K — Thu, 24 Sep 2026 12:21:25 +0000

TL;DR

We're adding rewind to a live HLS stream: configure a sliding DVR window in FFmpeg, expose it correctly in hls.js, build a seek bar that tracks the moving window, and add a go-live button that seeks to liveSyncPosition instead of the end of the seekable range (which stalls). Plus the storage-lifecycle check that prevents 404s inside your own advertised window.

"Can I go back to the start?" is the first request you get after shipping live. The segments are already on disk, so it feels like a config change. It is a config change, across three systems that have to agree, plus about 60 lines of player code.

Requirements: ffmpeg 7.0+, hls.js 1.6+, node 22.x.

ffmpeg -version | head -1
npm ls hls.js

1. Configure the packager window

The DVR window length is hls_time × hls_list_size. That's it. Everything else is bookkeeping.

# 10-minute sliding window: 6s segments × 100 entries
ffmpeg -i rtmp://localhost/live/stream \
  -c:v libx264 -preset veryfast -tune zerolatency \
  -g 120 -keyint_min 120 -sc_threshold 0 \
  -c:a aac -b:a 128k -ar 48000 \
  -f hls \
  -hls_time 6 \
  -hls_list_size 100 \
  -hls_flags delete_segments+independent_segments \
  -hls_delete_threshold 5 \
  -hls_segment_filename 'segments/seg_%05d.ts' \
  stream.m3u8

What each flag is doing:

Flag	Effect	Default
`hls_time 6`	target segment duration	2
`hls_list_size 100`	entries kept in the playlist	5
`delete_segments`	remove files once they leave the playlist	off
`hls_delete_threshold 5`	unreferenced segments kept before deletion	1
`independent_segments`	every segment starts on a keyframe	off

⚠️ That hls_list_size default of 5 is why so many first live deployments ship an accidental 30-second DVR nobody designed. If you never set it, you already have a DVR window. It's just very short.

-g 120 at 60fps gives a 2-second GOP, which divides evenly into 6-second segments. If your GOP doesn't divide your segment duration, segments drift off the target and your EXT-X-TARGETDURATION climbs.

Check the manifest:

$ head -8 stream.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:7
#EXT-X-MEDIA-SEQUENCE:412
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:6.000000,
segments/seg_00412.ts

EXT-X-MEDIA-SEQUENCE: 412 climbing on each refresh with the entry count staying at 100 means the sliding window is working.

2. Sliding window vs event playlist

There's a second shape, and it's a design decision, not a tuning knob.

# full-event DVR: append-only, nothing is deleted
ffmpeg -i rtmp://localhost/live/stream \
  ... -f hls -hls_time 6 -hls_playlist_type event stream.m3u8

-hls_playlist_type event emits #EXT-X-PLAYLIST-TYPE:EVENT and forces hls_list_size to 0, so the playlist contains every segment since the stream started.

	Sliding window	Event playlist
History	bounded	full
Manifest size	constant	grows all stream
Storage	bounded	grows all stream
Good for	24/7 channels	bounded broadcasts
Gotcha	history falls off	see below

⚠️ The event-playlist gotcha: some players refuse to seek in an EVENT playlist until #EXT-X-ENDLIST appears, which only happens when the stream ends. There's a long-running video.js issue on exactly this (videojs/video.js#8856) and the same pattern has shown up elsewhere. Test on your real target players, not just desktop Chrome, before you commit to EVENT.

For the rest of this build we use the sliding window.

3. The check nobody runs: does storage agree?

Your playlist advertises a seekable range. Three systems have to back it up, and they're configured by different people at different times:

Playlist: hls_time × hls_list_size = 600s
Origin disk: (hls_list_size + hls_delete_threshold) × hls_time = 630s
Object storage lifecycle: whatever the bucket policy says

If (3) is shorter than (1), your scrubber offers a range that 404s. Assert it in CI:

// scripts/check-dvr-consistency.mjs
const HLS_TIME = 6;
const HLS_LIST_SIZE = 100;
const HLS_DELETE_THRESHOLD = 5;
const STORAGE_TTL_SECONDS = 3600; // read this from your IaC, don't hardcode

const advertised = HLS_TIME * HLS_LIST_SIZE;
const onDisk = HLS_TIME * (HLS_LIST_SIZE + HLS_DELETE_THRESHOLD);

const problems = [];
if (advertised > onDisk) {
  problems.push(`playlist advertises ${advertised}s but origin keeps ${onDisk}s`);
}
if (advertised > STORAGE_TTL_SECONDS) {
  problems.push(`playlist advertises ${advertised}s but storage TTL is ${STORAGE_TTL_SECONDS}s`);
}

if (problems.length) {
  console.error("DVR window inconsistent:\n  " + problems.join("\n  "));
  process.exit(1);
}
console.log(`DVR window OK: ${advertised}s advertised, ${onDisk}s on disk`);

$ node scripts/check-dvr-consistency.mjs
DVR window OK: 600s advertised, 630s on disk

Nine lines of assertion. It catches the failure mode that support tickets are made of.

4. Read the window in hls.js

Now the player side. hls.js gives you the live edge and the seekable range; you have to turn that into UI.

// src/dvr.js
import Hls from 'hls.js';

export function attachDvr(video, src) {
  const hls = new Hls({
    // how far behind the live edge we sit, as a multiple of EXT-X-TARGETDURATION
    liveSyncDurationCount: 3,        // default is 3
    // liveSyncDuration: 18,         // seconds-based alternative; takes precedence
    backBufferLength: 90,
  });

  hls.loadSource(src);
  hls.attachMedia(video);

  const state = { hasDvr: false, windowSeconds: 0 };

  hls.on(Hls.Events.LEVEL_LOADED, (_evt, data) => {
    const details = data.details;
    state.hasDvr = details.live && details.totalduration > 30;
    state.windowSeconds = details.totalduration;
  });

  return { hls, state };
}

details.totalduration on a live level is the length of the current playlist, which is your DVR window. It moves as the window slides, so read it on every LEVEL_LOADED rather than caching it once.

5. A seek bar that tracks a moving target

The trap: on a live stream, video.duration is Infinity and seekable.start(0) increases over time. A normal <input type=range> bound to currentTime / duration does nothing useful.

Map the seekable range instead:

// src/dvr-ui.js
export function wireScrubber(video, hls, els) {
  const { scrubber, liveBadge, goLiveBtn, behindLabel } = els;

  function seekableRange() {
    if (!video.seekable.length) return null;
    return {
      start: video.seekable.start(0),
      end: video.seekable.end(video.seekable.length - 1),
    };
  }

  function render() {
    const range = seekableRange();
    if (!range) return;

    const span = range.end - range.start;
    if (span <= 0) return;

    // position within the window, 0..1
    const pos = (video.currentTime - range.start) / span;
    if (!scrubber.matches(':active')) {
      scrubber.value = String(Math.max(0, Math.min(1, pos)));
    }

    // how far behind the live edge are we?
    const behind = Math.max(0, hls.liveSyncPosition - video.currentTime);
    const atLive = behind < 10; // one-and-a-bit segments of slack

    liveBadge.classList.toggle('is-live', atLive);
    goLiveBtn.hidden = atLive;
    behindLabel.textContent = atLive ? 'LIVE' : `-${formatClock(behind)}`;
  }

  scrubber.addEventListener('input', () => {
    const range = seekableRange();
    if (!range) return;
    video.currentTime = range.start + Number(scrubber.value) * (range.end - range.start);
  });

  video.addEventListener('timeupdate', render);
  video.addEventListener('progress', render);
  return render;
}

function formatClock(seconds) {
  const s = Math.floor(seconds % 60).toString().padStart(2, '0');
  const m = Math.floor(seconds / 60);
  return `${m}:${s}`;
}

Two details worth calling out. We skip updating scrubber.value while the user is dragging (:active), otherwise timeupdate fights their thumb. And the live badge uses a tolerance, because being 3 seconds behind the edge is normal live playback, not rewind.

6. The go-live button (this is the one that stalls)

The obvious implementation is wrong:

// ❌ lands you at the newest byte with no buffer ahead, then stalls
video.currentTime = video.seekable.end(video.seekable.length - 1);

Use liveSyncPosition. hls.js computes it as the live edge minus a deliberate safety margin (liveSyncDurationCount × EXT-X-TARGETDURATION, default multiple of 3):

// ✅ src/go-live.js
export function goLive(video, hls) {
  const target = hls.liveSyncPosition;
  if (typeof target === 'number' && Number.isFinite(target)) {
    video.currentTime = target;
  } else if (video.seekable.length) {
    // fallback: back off from the edge ourselves
    const end = video.seekable.end(video.seekable.length - 1);
    video.currentTime = Math.max(0, end - 3 * (hls.levels[hls.currentLevel]?.details?.targetduration ?? 6));
  }
  if (video.paused) video.play();
}

💡 Tip: 3 to 4 segment durations behind the edge is where most deployments land. Closer and ordinary delivery jitter starves the buffer; further and you're needlessly behind. Tune liveSyncDurationCount, don't fight it in your seek code.

7. Handle the window sliding out from under a paused viewer

A viewer pauses inside the window, walks away, comes back 10 minutes later and presses play. Their position no longer exists.

// src/window-recovery.js
export function wireWindowRecovery(video, hls, onRecover) {
  function checkPlayhead() {
    if (!video.seekable.length) return;
    const start = video.seekable.start(0);
    if (video.currentTime < start) {
      // we fell out of the window while paused
      video.currentTime = start + 2; // small cushion past the edge
      onRecover?.();
    }
  }

  video.addEventListener('play', checkPlayhead);
  video.addEventListener('seeked', checkPlayhead);

  hls.on(Hls.Events.ERROR, (_e, data) => {
    if (data.details === Hls.ErrorDetails.FRAG_LOAD_ERROR && data.response?.code === 404) {
      // a segment we asked for is already gone; jump to the current window start
      checkPlayhead();
    }
  });
}

You want the visible recovery, not the silent one. Tell the viewer their position expired and you moved them, or they'll think the player jumped at random.

$ # simulate it: pause, wait past the window, press play
[dvr] playhead 412.8s fell behind window start 640.2s, recovering

Wrapping up

The whole feature is four decisions:

Window length: hls_time × hls_list_size, chosen on purpose. 30s for replaying a goal, 10min for a 24/7 channel, event playlist for a bounded broadcast.
Consistency: assert playlist ≤ origin ≤ storage TTL in CI.
Go live: liveSyncPosition, never seekable.end().
Recovery: detect a playhead that's fallen out of the window and move it visibly.

What's next: if you're on a 24/7 channel, look at whether a rolling VOD archive alongside the live window serves your use case better than a very long DVR (cheaper storage class, no manifest growth). And if you're targeting native, be aware that seeking inside a moving live window has its own long history on other platforms, including an ExoPlayer issue thread (google/ExoPlayer#87) that's worth reading before you assume parity with the web.

Build a video QC gate that catches black frames before your viewers do

Mason K — Thu, 24 Sep 2026 12:20:57 +0000

TL;DR

We're building a QC gate that runs after your transcode: tier 1 is ffprobe structural checks (no decode), tier 2 chains blackdetect + freezedetect + silencedetect into a single decode pass, and tier 3 is sampled SSIM. The part that makes it usable is the rules layer that turns raw detections into pass/warn/fail based on where in the timeline they land.

An encoder exiting 0 means a file was produced. It doesn't mean the file has picture in it. Black intros, frozen middles and silent audio all ship with a green build, and get found by a viewer weeks later.

Let's build the step that catches them. Requirements: ffmpeg 7.0+ and python 3.11+.

ffmpeg -version | head -1
ffmpeg -hide_banner -filters | grep -E "blackdetect|freezedetect|silencedetect"

 ... blackdetect       V->V       Detect video intervals that are (almost) black.
 ... freezedetect      V->V       Detects frozen video input.
 ... silencedetect     A->A       Detect silence.

1. Tier 1: structural checks, no decoding 🔍

Start here. It costs nothing and catches more than you'd expect.

# qc/structural.py
import json, subprocess
from dataclasses import dataclass

@dataclass
class Structural:
    duration: float
    video_streams: int
    audio_streams: int
    codecs: list[str]

def probe(path: str) -> Structural:
    cmd = [
        "ffprobe", "-v", "error",
        "-show_entries", "format=duration",
        "-show_entries", "stream=codec_type,codec_name",
        "-of", "json", path,
    ]
    data = json.loads(subprocess.run(cmd, capture_output=True, text=True,
                                     check=True).stdout)
    streams = data.get("streams", [])
    return Structural(
        duration=float(data["format"].get("duration") or 0),
        video_streams=sum(1 for s in streams if s["codec_type"] == "video"),
        audio_streams=sum(1 for s in streams if s["codec_type"] == "audio"),
        codecs=[s["codec_name"] for s in streams],
    )

def check_structural(output: str, source_duration: float, expect_audio: bool):
    s = probe(output)
    findings = []

    if s.video_streams == 0:
        findings.append(("fail", "no video stream in output"))
    if expect_audio and s.audio_streams == 0:
        findings.append(("fail", "expected audio, output has none"))

    drift = abs(s.duration - source_duration)
    if drift > max(2.0, source_duration * 0.02):
        findings.append(("fail", f"duration drift {drift:.1f}s "
                                 f"(out {s.duration:.1f}s vs src {source_duration:.1f}s)"))
    return findings

💡 Tip: missing audio is the single most common real-world hit here. It happens when the source puts audio on a non-first stream and your mapping assumed -map 0:a:0. Nothing else in your pipeline notices.

$ python3 -m qc.structural out/lesson-14.mp4 --source-duration 1842
fail: expected audio, output has none

One ffprobe call, sub-second, and you just caught a defect that would have shipped.

2. Tier 2: three detectors, one decode pass

The trick is chaining them so you decode once. -f null - means no output file is written at all.

ffmpeg -hide_banner -nostats -i out/lesson-14.mp4 \
  -vf "blackdetect=d=0.5:pic_th=0.98:pix_th=0.10,freezedetect=n=-60dB:d=2" \
  -af "silencedetect=n=-50dB:d=2" \
  -f null - 2> qc.log

The filters report to stderr, not stdout, and not as JSON. Parsing log lines is the integration path, annoying but stable:

[blackdetect @ 0x5f2a] black_start:0 black_end:22.4067 black_duration:22.4067
[freezedetect @ 0x5f2b] lavfi.freezedetect.freeze_start: 604.12
[freezedetect @ 0x5f2b] lavfi.freezedetect.freeze_duration: 11.3
[freezedetect @ 0x5f2b] lavfi.freezedetect.freeze_end: 615.42
[silencedetect @ 0x5f2c] silence_start: 0
[silencedetect @ 0x5f2c] silence_end: 1842.31 | silence_duration: 1842.31

Parse it:

# qc/detect.py
import re, subprocess
from dataclasses import dataclass

BLACK = re.compile(r"black_start:(?P<start>[\d.]+) black_end:(?P<end>[\d.]+) "
                   r"black_duration:(?P<dur>[\d.]+)")
FREEZE_START = re.compile(r"freeze_start:\s*(?P<start>[\d.]+)")
FREEZE_DUR = re.compile(r"freeze_duration:\s*(?P<dur>[\d.]+)")
SILENCE_START = re.compile(r"silence_start:\s*(?P<start>-?[\d.]+)")
SILENCE_DUR = re.compile(r"silence_duration:\s*(?P<dur>[\d.]+)")

@dataclass
class Event:
    kind: str      # black | freeze | silence
    start: float
    duration: float

def run_detectors(path: str, black_d=0.5, freeze_d=2.0, silence_d=2.0) -> list[Event]:
    cmd = [
        "ffmpeg", "-hide_banner", "-nostats", "-i", path,
        "-vf", f"blackdetect=d={black_d}:pic_th=0.98:pix_th=0.10,"
               f"freezedetect=n=-60dB:d={freeze_d}",
        "-af", f"silencedetect=n=-50dB:d={silence_d}",
        "-f", "null", "-",
    ]
    log = subprocess.run(cmd, capture_output=True, text=True).stderr
    events, pending = [], {}

    for line in log.splitlines():
        if m := BLACK.search(line):
            events.append(Event("black", float(m["start"]), float(m["dur"])))
        elif m := FREEZE_START.search(line):
            pending["freeze"] = float(m["start"])
        elif m := FREEZE_DUR.search(line):
            events.append(Event("freeze", pending.pop("freeze", 0.0), float(m["dur"])))
        elif m := SILENCE_START.search(line):
            pending["silence"] = max(0.0, float(m["start"]))
        elif m := SILENCE_DUR.search(line):
            events.append(Event("silence", pending.pop("silence", 0.0), float(m["dur"])))

    return events

⚠️ Note: silence_start can be reported as a small negative number at the head of a file. Clamp it to 0 or your position rules will misfire on exactly the case you care most about.

3. The rules layer (this is the actual product) ⚖️

Raw detections are not failures. A 1.5s black frame at 00:00 is a fade-in. A 22s black frame at 00:00 is a broken render. Same detector, opposite verdicts.

If you alert on every detection, the channel gets muted inside a month and you've built nothing.

# qc/rules.py
from dataclasses import dataclass, field
from .detect import Event

RULES_VERSION = "2026.09.1"
HEAD = 3.0   # seconds considered "the start"
TAIL = 3.0   # seconds considered "the end"

@dataclass
class Verdict:
    status: str = "pass"          # pass | warn | fail
    notes: list = field(default_factory=list)
    rules_version: str = RULES_VERSION

    def raise_to(self, level: str, note: str):
        order = {"pass": 0, "warn": 1, "fail": 2}
        if order[level] > order[self.status]:
            self.status = level
        self.notes.append(f"{level}: {note}")

def judge(events: list[Event], total_duration: float) -> Verdict:
    v = Verdict()

    for e in events:
        at_head = e.start <= HEAD
        at_tail = (e.start + e.duration) >= (total_duration - TAIL)
        spans_all = e.duration >= total_duration * 0.95

        if e.kind == "black":
            if at_head and e.duration <= 3:
                continue                                    # fade-in, fine
            if at_head:
                v.raise_to("fail", f"black intro {e.duration:.1f}s")
            elif at_tail and e.duration <= 5:
                continue                                    # fade-out, fine
            elif e.duration >= 5:
                v.raise_to("warn", f"black at {e.start:.0f}s for {e.duration:.1f}s")

        elif e.kind == "freeze":
            if at_tail:
                continue                                    # held last frame
            v.raise_to("fail", f"freeze at {e.start:.0f}s for {e.duration:.1f}s")

        elif e.kind == "silence":
            if spans_all:
                v.raise_to("fail", "audio silent for entire duration")
            elif e.duration >= 30:
                v.raise_to("warn", f"silence at {e.start:.0f}s for {e.duration:.1f}s")

    return v

Now the same detections produce useful output:

$ python3 -m qc out/lesson-14.mp4
{
  "status": "fail",
  "notes": [
    "fail: black intro 22.4s",
    "fail: freeze at 604s for 11.3s",
    "fail: audio silent for entire duration"
  ],
  "rules_version": "2026.09.1"
}

$ python3 -m qc out/promo-clip.mp4
{
  "status": "pass",
  "notes": [],
  "rules_version": "2026.09.1"
}

The second file also had a black detection (a 1.2s fade-in) and a tail freeze. It passed, correctly, because the rules know where those landed.

4. Tier 3: sampled reference comparison

SSIM against the source catches "the encode ran but produced garbage," and it costs two decodes. Don't put it in the synchronous path.

ffmpeg -hide_banner \
  -i out/lesson-14.mp4 -i src/lesson-14.mov \
  -lavfi "[0:v]scale=1280:720[a];[1:v]scale=1280:720[b];[a][b]ssim=stats_file=ssim.log" \
  -f null -

SSIM Y:0.981234 U:0.992011 V:0.991455 All:0.984772 (18.156204)

# qc/reference.py: run nightly or on sampled assets only
SSIM_FLOOR = 0.93

def check_ssim(all_score: float):
    if all_score < SSIM_FLOOR:
        return [("fail", f"SSIM {all_score:.3f} below floor {SSIM_FLOOR}")]
    return []

⚠️ Scale both inputs to a common resolution before comparing, or ssim errors out on mismatched dimensions. And pick the floor empirically from your own known-good output; a number copied from someone else's pipeline will either never fire or fire constantly.

5. Wire it into the pipeline

# qc/__main__.py
import sys, json
from .structural import check_structural, probe
from .detect import run_detectors
from .rules import judge

def gate(output: str, source_duration: float, expect_audio: bool = True) -> dict:
    # tier 1: fail fast, no decode
    structural = check_structural(output, source_duration, expect_audio)
    if any(level == "fail" for level, _ in structural):
        return {"status": "fail",
                "notes": [f"{l}: {m}" for l, m in structural],
                "tier": 1}

    # tier 2: one decode, three detectors
    info = probe(output)
    verdict = judge(run_detectors(output), info.duration)
    return {"status": verdict.status, "notes": verdict.notes,
            "rules_version": verdict.rules_version, "tier": 2}

if __name__ == "__main__":
    result = gate(sys.argv[1], float(sys.argv[2]))
    print(json.dumps(result, indent=2))
    sys.exit(1 if result["status"] == "fail" else 0)

Exiting nonzero on fail lets you drop this straight into a worker or a CI step.

6. Store the verdict with its rules version

ALTER TABLE assets
  ADD COLUMN qc_status text,
  ADD COLUMN qc_notes jsonb,
  ADD COLUMN qc_rules_version text;

This matters the first time you loosen a threshold. When a customer's library legitimately opens on slow fades and you relax the black-intro rule, you need to find everything that failed under the old version and re-judge it. Without the stamp, you re-run QC on your entire library.

The scarier direction is the same query in reverse: when you tighten a rule because something got through, you want the list of everything that passed under the looser version.

What's next

Run tier 1 against last week's output before you build anything else. One ffprobe call per asset, three assertions, a count at the end:

$ for f in out/*.mp4; do python3 -m qc.structural "$f" || echo "FAIL $f"; done | grep -c FAIL
7

Seven broken assets you're already serving is a better argument for this project than any design doc.

From there, the things worth adding are ebur128 for loudness (catches "audio exists but is inaudible," which silencedetect misses entirely), a check that every rendition in your ABR ladder has matching duration and keyframe positions, and per-tenant rule overrides so the customer with the slow fades gets their own thresholds instead of forcing you to loosen everyone's.

Build an audio description pipeline that actually fits the gaps

Mason K — Wed, 16 Sep 2026 13:40:31 +0000

TL;DR

We're building an audio description pipeline in Python: find the silent gaps with FFmpeg, compute a word budget per gap, generate descriptions constrained to that budget, render TTS, reject anything that overruns, and mux the result as a second HLS audio track. The overrun check is the step everyone skips and the one that makes the output usable.

Captions were easy because text sits in its own layer. Audio description has to go into the channel that already has the dialogue in it, so it's a packing problem, not a writing problem. A great 9-second description is a defect if the gap is 2.4 seconds.

That constraint drives the whole design. Let's build it.

What we're building

video.mp4
   ├─ 1. silence map        (ffmpeg silencedetect)  → gaps[]
   ├─ 2. budget per gap     (duration × speech rate) → max_words
   ├─ 3. frame sampling     (ffmpeg, mid-gap frames) → context
   ├─ 4. description        (constrained to max_words)
   ├─ 5. TTS + HARD CHECK   (reject if rendered > gap) → retry shorter
   ├─ 6. mix                (overlay narration on original audio)
   └─ 7. package            (second audio rendition in the HLS ladder)

Requirements: ffmpeg 6.0+ and python 3.11+.

ffmpeg -version | head -1
python3 --version

ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
Python 3.12.3

1. Find the gaps 🔍

silencedetect is the whole first step. It writes to stderr, not stdout, which catches people out:

ffmpeg -i video.mp4 -af silencedetect=noise=-30dB:d=1.2 -f null - 2>&1 | grep silence_

[silencedetect @ 0x14e004c30] silence_start: 3.264
[silencedetect @ 0x14e004c30] silence_end: 6.912 | silence_duration: 3.648
[silencedetect @ 0x14e004c30] silence_start: 19.04
[silencedetect @ 0x14e004c30] silence_end: 21.44 | silence_duration: 2.4

Two parameters to tune per content type. noise=-30dB is the threshold for what counts as silence, and you'll want it lower (more strict) for content with music beds, or you'll "find" gaps that are full of score. d=1.2 is the minimum duration, because a gap shorter than about a second can't hold a useful sentence anyway.

# src/gaps.py
import re
import subprocess
from dataclasses import dataclass

SILENCE_RE = re.compile(
    r"silence_start: (?P<start>[\d.]+)|silence_end: (?P<end>[\d.]+) \| silence_duration: (?P<dur>[\d.]+)"
)

@dataclass
class Gap:
    start: float
    end: float
    duration: float

def find_gaps(path: str, noise_db: int = -30, min_dur: float = 1.2) -> list[Gap]:
    proc = subprocess.run(
        ["ffmpeg", "-i", path, "-af",
         f"silencedetect=noise={noise_db}dB:d={min_dur}", "-f", "null", "-"],
        capture_output=True, text=True,
    )
    gaps, pending_start = [], None
    for m in SILENCE_RE.finditer(proc.stderr):
        if m.group("start"):
            pending_start = float(m.group("start"))
        elif m.group("end") and pending_start is not None:
            end = float(m.group("end"))
            gaps.append(Gap(pending_start, end, float(m.group("dur"))))
            pending_start = None
    return gaps

⚠️ Leave headroom. Starting narration the instant dialogue stops sounds wrong and clips on some decoders. Trim 150ms off each end of the gap before you budget it.

2. Turn duration into a word budget 📏

This is the step that makes the pipeline work, and it's just arithmetic:

# src/budget.py
from src.gaps import Gap

# Tune this for your TTS voice. Measure it, don't trust it:
# render 100 known-length sentences and divide.
WORDS_PER_SECOND = 2.5
EDGE_PADDING = 0.15  # seconds trimmed from each end

def word_budget(gap: Gap) -> int:
    usable = gap.duration - (EDGE_PADDING * 2)
    return max(0, int(usable * WORDS_PER_SECOND))

def usable_gaps(gaps: list[Gap], min_words: int = 4) -> list[tuple[Gap, int]]:
    scored = [(g, word_budget(g)) for g in gaps]
    return [(g, w) for g, w in scored if w >= min_words]

>>> gaps = find_gaps("video.mp4")
>>> len(gaps)
47
>>> len(usable_gaps(gaps))
31

Sixteen gaps were too short to say anything meaningful in. That's normal, and it's information: those are the moments where standard description cannot comply, and where WCAG's extended audio description criterion (1.2.7, Level AAA) exists precisely because the only fix is pausing the video. Log them, don't silently drop them.

💡 Tip: WORDS_PER_SECOND = 2.5 is a starting assumption, not a measured constant. Different voices and languages differ substantially. Measure yours and put the real number here.

3. Sample frames for visual context 🖼️

We need to know what's on screen during the gap, so grab a few frames from inside it:

# src/frames.py
import subprocess
from pathlib import Path
from src.gaps import Gap

def sample_frames(video: str, gap: Gap, out_dir: Path, n: int = 3) -> list[Path]:
    out_dir.mkdir(parents=True, exist_ok=True)
    paths = []
    step = gap.duration / (n + 1)
    for i in range(1, n + 1):
        ts = gap.start + (step * i)
        out = out_dir / f"{gap.start:.2f}_{i}.jpg"
        subprocess.run(
            ["ffmpeg", "-y", "-ss", f"{ts:.3f}", "-i", video,
             "-frames:v", "1", "-q:v", "3", str(out)],
            capture_output=True, check=True,
        )
        paths.append(out)
    return paths

-ss before -i seeks fast by jumping to the nearest keyframe. For a few frames of context that's fine; if you need frame-exact sampling, put -ss after -i and accept it'll be much slower.

4. Generate to the budget, not to a quality bar

Whatever model you use, the budget is the part of the prompt that matters. Put it in hard and put it first:

# src/describe.py
SYSTEM = """You write audio description for blind and low-vision viewers.

HARD CONSTRAINT: your output MUST be {max_words} words or fewer. This is not a
style preference. The narration plays in a {duration:.1f} second gap between
lines of dialogue and will be cut off if it runs long.

Rules:
- Describe only what a sighted viewer gains that the audio does not convey.
- Prioritise: character actions > scene changes > appearance > atmosphere.
- Use present tense. No "we see", no "the camera shows", no interpretation
  of emotion or intent.
- If nothing visually important happens here, output exactly: SKIP
"""

def build_prompt(gap, max_words, prior_descriptions):
    return SYSTEM.format(max_words=max_words, duration=gap.duration), {
        # Prior descriptions let the model stop re-introducing the same character.
        "already_described": prior_descriptions[-5:],
    }

Two details matter more than the model choice. SKIP must be a valid output, because a pipeline that fills all 31 gaps just because it can produces narration that's exhausting to listen to. And pass prior descriptions, or you get "a man in a blue coat" in gap 3 and again in gap 28. Models are weak on narrative continuity; carrying context is the cheap partial fix.

5. Render, then verify the duration 🎙️

This is the section the whole tutorial exists for. Do not trust the word budget. Render the audio and measure the actual file:

# src/render.py
import subprocess
from pathlib import Path

def audio_duration(path: Path) -> float:
    out = subprocess.run(
        ["ffprobe", "-v", "error", "-show_entries", "format=duration",
         "-of", "default=noprint_wrappers=1:nokey=1", str(path)],
        capture_output=True, text=True, check=True,
    )
    return float(out.stdout.strip())

def render_fitting(text: str, gap, out: Path, tts_fn, max_attempts: int = 3):
    """Render TTS and shrink the text until it fits the gap."""
    budget = gap.duration - 0.30  # edge padding
    current = text
    for attempt in range(max_attempts):
        tts_fn(current, out)
        actual = audio_duration(out)
        if actual <= budget:
            return {"ok": True, "text": current, "duration": actual, "attempts": attempt + 1}

        overrun = actual - budget
        keep = int(len(current.split()) * (budget / actual) * 0.9)
        print(f"  overrun {overrun:.2f}s, trimming to ~{keep} words")
        current = " ".join(current.split()[:keep])

    # Speeding up narration to force a fit makes it harder to understand.
    # Flag for human review instead.
    return {"ok": False, "text": current, "duration": audio_duration(out), "reason": "could not fit"}

python3 -m src.pipeline video.mp4

gap 03.26-06.91 (3.65s, budget 8 words)
  overrun 0.41s, trimming to ~7 words
  fit in 2 attempts: "She slides the envelope under the door."
gap 19.04-21.44 (2.40s, budget 5 words)
  fit in 1 attempt: "He notices the broken lock."
gap 44.10-45.80 (1.70s, budget 3 words)
  SKIP (nothing visually significant)
gap 61.22-63.02 (1.80s, budget 4 words)
  overrun 0.62s, trimming to ~3 words
  overrun 0.28s, trimming to ~2 words
  FLAGGED: could not fit -> review queue

24 described, 5 skipped, 2 flagged for review

Those 2 flagged items are the output you want. A pipeline that silently truncates mid-word, or time-compresses narration until it's unintelligible, produces a track that technically exists and nobody can use.

6. Mix and package as a second audio track 🔊

Build a narration bed, then overlay it on the original:

# place each clip at its gap start on a silent bed the length of the video
ffmpeg -f lavfi -t 1800 -i anullsrc=r=48000:cl=stereo \
  -i ad_003.wav -i ad_019.wav \
  -filter_complex "\
    [1:a]adelay=3414|3414[a1]; \
    [2:a]adelay=19190|19190[a2]; \
    [0:a][a1][a2]amix=inputs=3:duration=first:normalize=0[bed]" \
  -map "[bed]" narration_bed.wav

# duck the original under the narration, then mux as a described track
ffmpeg -i video.mp4 -i narration_bed.wav \
  -filter_complex "[0:a][1:a]sidechaincompress=threshold=0.05:ratio=8[ducked]; \
                   [ducked][1:a]amix=inputs=2:duration=first[out]" \
  -map 0:v -map "[out]" -c:v copy -c:a aac -b:a 128k \
  -metadata:s:a:0 title="Audio Description" \
  -metadata:s:a:0 language=eng \
  video_described.mp4

sidechaincompress ducks the original audio whenever narration plays, which is what a professional mix does. Without it the narration competes with the score.

Then it's another rendition in your HLS audio group:

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",NAME="English",LANGUAGE="en",DEFAULT=YES,AUTOSELECT=YES,URI="audio/en.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",NAME="English (Described)",LANGUAGE="en",CHARACTERISTICS="public.accessibility.describes-video",AUTOSELECT=NO,URI="audio/en-ad.m3u8"

⚠️ CHARACTERISTICS="public.accessibility.describes-video" is what tells players this is a description track rather than a second language. Without it, players won't surface it in their accessibility menu and iOS won't auto-select it for users who've enabled Audio Descriptions system-wide. Easy to omit, invisible when wrong.

7. Monitor it, or you'll lose it 📡

This is the operational failure nobody plans for. Nobody on your team uses the description track. Your smoke tests check that video plays. A packager change drops the second rendition from the manifest and you find out eighteen months later from a complaint.

# src/monitor.py
import m3u8  # pip install m3u8

DESCRIBES = "public.accessibility.describes-video"

def has_description_track(master_url: str) -> bool:
    playlist = m3u8.load(master_url)
    return any(
        m.type == "AUDIO" and DESCRIBES in (m.characteristics or "")
        for m in playlist.media
    )

def assert_described(master_url: str):
    if not has_description_track(master_url):
        raise AssertionError(f"description track missing from {master_url}")

Wire that into whatever already checks your manifests. An accessibility feature you aren't monitoring is an accessibility feature you used to have.

What's next

A human review queue is not optional. Automated description is production-viable across many content types with human review, and not reliable enough without it on complex narrative material. Build the queue as a first-class part of the pipeline.
Extended audio description (WCAG 1.2.7). For gaps that were too short, the compliant answer is a player that pauses, which is a different project. W3C's understanding doc starts there.
Test with actual screen reader users. Every metric here is a proxy for "is this useful", and proxies drift.

The regulatory deadlines are real (ADA Title II landed April 24, 2026 for larger public entities; the EU Accessibility Act has applied since June 2025), but the better reason to build this is that the pipeline above turns a specialist hour per video into a few minutes of review, and that's the difference between describing your top 50 videos and describing all of them. #a11y #video

Write a portability audit for your video provider before you need one

Mason K — Wed, 16 Sep 2026 13:40:12 +0000

TL;DR

We're building a small Node script that audits your video library for portability: can you get the source file back, is playback standard HLS, and how many places is the provider's ID buried. It writes a CSV. Run it once a quarter and you'll never be surprised by a migration estimate again.

Most of us pick a video API by reading pricing pages and running an upload benchmark. Nobody checks the exit until they're already trying to leave, at which point the answer is expensive and non-negotiable.

This is a small thing you can run today. It's a provider-agnostic script with a thin adapter per platform, and the worked example uses FastPix because its source-retention flag is explicit in the API, which makes for a clear demo. The pattern moves to Mux, Cloudflare Stream or api.video by writing a different adapter.

What we're checking

Three questions, in order of how much they'll cost you:

Source retention. Does the provider still hold your original upload, or only the transcoded renditions? If it's renditions-only, a migration re-encodes from output and you lose quality permanently.
Playback portability. Is the playback URL a standard HLS manifest any player can consume, or a proprietary embed?
ID sprawl. How many of your own tables carry the provider's media and playback IDs? Every one is a migration write path.

1. Project setup 🛠️

Node 20.x or newer, no dependencies beyond what ships with it.

mkdir video-portability-audit && cd video-portability-audit
npm init -y
npm pkg set type=module
node --version

v22.14.0

Credentials go in the environment, never in the file:

# .env.example : copy to .env, never commit .env
FASTPIX_TOKEN_ID=your_access_token_id
FASTPIX_SECRET_KEY=your_secret_key

⚠️ Note: the script only ever issues GET requests. It reads your library and writes a local CSV. It will not modify or delete anything.

2. The adapter interface

Every provider adapter answers the same three questions for one media ID. Keeping this interface narrow is what makes the script portable across platforms.

// src/adapter.js
/**
 * @typedef {Object} AuditResult
 * @property {string} mediaId
 * @property {boolean|null} sourceRetained   - can we get the original file back?
 * @property {boolean|null} standardHls      - is playback a plain .m3u8?
 * @property {string} playbackUrl
 * @property {string} notes
 */

export class Adapter {
  /** @returns {Promise<AuditResult>} */
  async audit(mediaId) {
    throw new Error('not implemented');
  }
}

3. The FastPix adapter

FastPix uses HTTP Basic auth with the Access Token ID as the username and the Secret Key as the password, so we can build the header with Buffer:

// src/providers/fastpix.js
import { Adapter } from '../adapter.js';

const API = 'https://api.fastpix.com/v1';

export class FastPixAdapter extends Adapter {
  constructor(tokenId, secretKey) {
    super();
    this.auth = 'Basic ' + Buffer.from(`${tokenId}:${secretKey}`).toString('base64');
  }

  async #get(path) {
    const res = await fetch(`${API}${path}`, {
      headers: { Authorization: this.auth, Accept: 'application/json' },
    });
    if (!res.ok) {
      throw new Error(`${res.status} ${res.statusText} on ${path}`);
    }
    return res.json();
  }

  async audit(mediaId) {
    // The create response returns `data` as an OBJECT, not an array.
    // Older docs showed an array; if you wrote a parser against those, it breaks here.
    const { data } = await this.#get(`/on-demand/${mediaId}`);

    const playbackId = data.playbackIds?.[0]?.id ?? null;
    const playbackUrl = playbackId
      ? `https://stream.fastpix.com/${playbackId}.m3u8`
      : '';

    return {
      mediaId,
      // sourceAccess is set at ingest time on the create call.
      // If it wasn't enabled then, the original is not retrievable now.
      sourceRetained: data.sourceAccess ?? null,
      standardHls: Boolean(playbackUrl.endsWith('.m3u8')),
      playbackUrl,
      notes: data.status ?? '',
    };
  }
}

💡 Tip: sourceAccess is a create-time decision. That's the important bit for this audit. If it defaulted off across your whole library, this script will tell you today rather than during a migration in 2028.

4. Feeding it media IDs

Here's a design choice worth explaining, because it looks like a limitation and isn't.

The script takes media IDs from your database, not from a list-everything API call. Two reasons. First, you already store these IDs, because you need them to render a page. Second, and more usefully: the IDs your application knows about are the ones that matter. An asset sitting in the provider's dashboard that nothing in your product references is not a migration problem. A row in your videos table is.

// src/ids.js
import { readFile } from 'node:fs/promises';

// One media ID per line. Generate it however you like, e.g.:
//   psql -At -c "select provider_media_id from videos where deleted_at is null" > ids.txt
export async function loadIds(path = 'ids.txt') {
  const raw = await readFile(path, 'utf8');
  return raw.split('\n').map((l) => l.trim()).filter(Boolean);
}

5. The runner, with concurrency and retries

Rate limits are the thing that turns a five-minute script into a two-hour one. A small concurrency cap plus backoff on 429 handles it:

// src/run.js
import { writeFile } from 'node:fs/promises';
import { loadIds } from './ids.js';
import { FastPixAdapter } from './providers/fastpix.js';

const CONCURRENCY = 4;

async function withRetry(fn, attempts = 4) {
  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      const is429 = String(err.message).startsWith('429');
      if (!is429 || i === attempts - 1) throw err;
      const waitMs = 2 ** i * 500;
      console.warn(`  rate limited, backing off ${waitMs}ms`);
      await new Promise((r) => setTimeout(r, waitMs));
    }
  }
}

async function pool(items, size, worker) {
  const results = [];
  const queue = [...items];
  const runners = Array.from({ length: size }, async () => {
    while (queue.length) {
      const item = queue.shift();
      results.push(await worker(item));
    }
  });
  await Promise.all(runners);
  return results;
}

const adapter = new FastPixAdapter(
  process.env.FASTPIX_TOKEN_ID,
  process.env.FASTPIX_SECRET_KEY,
);

const ids = await loadIds();
console.log(`auditing ${ids.length} assets...`);

const rows = await pool(ids, CONCURRENCY, async (id) => {
  try {
    return await withRetry(() => adapter.audit(id));
  } catch (err) {
    return { mediaId: id, sourceRetained: null, standardHls: null, playbackUrl: '', notes: `ERROR: ${err.message}` };
  }
});

const csv = [
  'media_id,source_retained,standard_hls,playback_url,notes',
  ...rows.map((r) =>
    [r.mediaId, r.sourceRetained, r.standardHls, r.playbackUrl, JSON.stringify(r.notes)].join(','),
  ),
].join('\n');

await writeFile('portability-audit.csv', csv);

const noSource = rows.filter((r) => r.sourceRetained === false).length;
const errors = rows.filter((r) => r.notes.startsWith('ERROR')).length;
console.log(`\ndone. ${rows.length} assets, ${noSource} without retrievable source, ${errors} errors`);
console.log('wrote portability-audit.csv');

Run it:

node --env-file=.env src/run.js

auditing 1204 assets...
  rate limited, backing off 500ms
  rate limited, backing off 1000ms

done. 1204 assets, 341 without retrievable source, 2 errors
wrote portability-audit.csv

That 341 is the number this whole script exists to produce. It's the size of the problem you didn't know you had.

6. The ID sprawl check 🔍

The third question doesn't need the API at all, just grep over your own codebase and a look at your schema. Provider IDs leak further than anyone expects:

# how many places does a provider ID appear?
rg -l --type-add 'src:*.{ts,tsx,js,jsx,sql,py,rb,go}' -t src \
   -e 'playbackId' -e 'playback_id' -e 'media_id' -e 'stream\.fastpix\.com' \
   | sort | uniq -c | sort -rn

Then the one people forget:

-- every column that could be holding a provider ID
SELECT table_name, column_name
FROM information_schema.columns
WHERE column_name ILIKE '%playback%'
   OR column_name ILIKE '%media_id%'
   OR column_name ILIKE '%video_id%'
ORDER BY table_name;

Count the rows. That's your migration write path. If provider IDs are also in emails you've already sent, in a cached CDN rule, or in URLs your customers bookmarked, add a redirect layer to the estimate.

💡 Tip: the fix for ID sprawl is cheap before you need it. Put your own UUID on every video, map it to the provider ID in exactly one table, and reference only your UUID everywhere else. A migration then touches one table instead of nine.

Porting the adapter to other providers

The interface is three fields, so a new adapter is usually under thirty lines:

Provider	Source retention	Playback	Bulk tooling
FastPix	`sourceAccess` on create	`stream.fastpix.com/<id>.m3u8`	Built-in batch migration tool
Mux	Check asset settings	Standard HLS	Documented framework, you build the workflow
Cloudflare Stream	Check asset settings	Standard HLS	Roll your own
api.video	Check asset settings	Standard HLS	Markets zero migration cost

I've deliberately left the middle column vague for three of the four, because the flag names differ and I'd rather you read the current reference than trust a table I wrote today. For the FastPix fields used above, the VOD API reference has the create-media body and the get-media response shape.

What's next

Two follow-ups worth doing:

Schedule it. Run the audit in CI monthly and fail the build if source_retained=false count increases. That turns a one-off report into a guardrail, and it's the version of this that actually keeps working after everyone forgets about it.
Add a restore test. Auditing that a source file should be retrievable is not the same as proving it is. Pull one source file per month and check the bytes. Backups you haven't restored are backups you don't have, and the same logic applies here.

If you run this against your own library, I'd be curious what your source_retained=false number comes out as. Mine was worse than I guessed.

Add casting to a custom HTML5 video player without breaking it

Mason K — Wed, 16 Sep 2026 13:39:41 +0000

TL;DR

We're adding a cast button to a custom <video> player using the Remote Playback API, with an AirPlay fallback for Safari. Then we fix the four bugs that make casting work on your laptop and fail on an actual TV: token expiry, CORS on the receiver, unreachable hosts, and treating prompt() as if it were a connection.

The thing to understand before writing any code: the TV does not run your player. It fetches your stream URL and plays it with its own stack. Your overlays, your quality caps, your analytics, your token refresh logic, none of it makes the trip.

So this tutorial is half API wiring and half making your stream survive without you.

1. The baseline player 🎥

Start with a plain element and our own controls. Nothing exotic:

<!-- index.html -->
<video id="player" playsinline controls></video>

<div class="controls">
  <button id="castBtn" hidden>Cast</button>
  <span id="castState"></span>
</div>

<script type="module" src="./src/player.js"></script>

// src/player.js
import Hls from 'hls.js'; // hls.js 1.5.x

const video = document.getElementById('player');
const MANIFEST = 'https://example.com/path/playback.m3u8';

if (video.canPlayType('application/vnd.apple.mpegurl')) {
  // Safari plays HLS natively, and this path is also what AirPlay uses.
  video.src = MANIFEST;
} else if (Hls.isSupported()) {
  const hls = new Hls();
  hls.loadSource(MANIFEST);
  hls.attachMedia(video);
}

⚠️ Note: that branch matters later. On Safari the element has a real src, which AirPlay can fling. With hls.js, MSE is feeding the element and video.src is a blob URL that means nothing to a TV. This is the single biggest source of confusion in casting code.

2. Feature detection, three ways

There's no one API. We check for each path and pick:

// src/cast-support.js
export function detectCastSupport(video) {
  return {
    // Standards-based path. Chromium-leaning; MDN lists it as limited availability.
    remotePlayback: 'remote' in video && typeof video.remote?.watchAvailability === 'function',

    // Safari's own AirPlay interface, predates the spec above.
    airplay: typeof window.WebKitPlaybackTargetAvailabilityEvent !== 'undefined',

    // Cast SDK, only if you've loaded the sender script and written a receiver app.
    castSdk: typeof window.cast !== 'undefined' && typeof window.chrome?.cast !== 'undefined',
  };
}

// quick check in the console
detectCastSupport(document.getElementById('player'))
// Chrome:  { remotePlayback: true,  airplay: false, castSdk: false }
// Safari:  { remotePlayback: false, airplay: true,  castSdk: false }
// Firefox: { remotePlayback: false, airplay: false, castSdk: false }

3. Wiring the Remote Playback API

Two calls do most of the work. watchAvailability tells us whether to show the button at all, and prompt opens the browser's own device picker.

// src/remote-playback.js
export function initRemotePlayback(video, btn, onState) {
  if (!('remote' in video)) return;

  // Show the button only when a device is actually reachable.
  video.remote
    .watchAvailability((available) => {
      btn.hidden = !available;
    })
    .catch(() => {
      // Some browsers can't watch continuously. Show the button and let
      // prompt() fail loudly instead of hiding the feature entirely.
      btn.hidden = false;
    });

  btn.addEventListener('click', async () => {
    try {
      await video.remote.prompt();
    } catch (err) {
      // AbortError = user dismissed the picker. Not an error worth surfacing.
      if (err.name !== 'AbortError') {
        console.error('cast failed', err);
        onState('error', err);
      }
    }
  });

  // These are the source of truth, NOT the click handler.
  video.remote.addEventListener('connecting', () => onState('connecting'));
  video.remote.addEventListener('connect', () => onState('connected'));
  video.remote.addEventListener('disconnect', () => onState('disconnected'));
}

The comment on those last three lines is the whole lesson of this tutorial, so let's make it explicit.

4. The state machine (this is the bug you'll ship) ⚠️

The naive version sets isCasting = true when the click handler resolves. That's wrong, because remote playback is asynchronous and can fail after it succeeds. The device drops off wifi. Someone's phone steals the TV. The receiver crashes.

Derive your UI from the connection state the browser reports:

// src/player.js (continued)
import { initRemotePlayback } from './remote-playback.js';

const btn = document.getElementById('castBtn');
const label = document.getElementById('castState');

let castState = 'disconnected';

function render() {
  label.textContent = {
    disconnected: '',
    connecting: 'Connecting to device...',
    connected: 'Playing on TV',
    error: 'Could not connect',
  }[castState];

  // Local controls are meaningless while the TV owns playback.
  document.querySelectorAll('.local-only').forEach((el) => {
    el.toggleAttribute('disabled', castState === 'connected');
  });
}

initRemotePlayback(video, btn, (state) => {
  castState = state;
  render();
});

// Belt and braces: video.remote.state is readable at any time.
setInterval(() => {
  if (video.remote?.state && video.remote.state !== castState) {
    castState = video.remote.state;
    render();
  }
}, 2000);

💡 Tip: once connected, treat the remote as the source of truth for playback position and paused state. You now have two states that can disagree, and the one on the television is the one your user is looking at.

5. AirPlay, Safari's way 🍎

Safari doesn't implement the API above. It has its own event and its own picker:

// src/airplay.js
export function initAirPlay(video, btn) {
  if (typeof window.WebKitPlaybackTargetAvailabilityEvent === 'undefined') return;

  video.addEventListener('webkitplaybacktargetavailabilitychanged', (e) => {
    btn.hidden = e.availability !== 'available';
  });

  btn.addEventListener('click', () => {
    video.webkitShowPlaybackTargetPicker();
  });

  video.addEventListener('webkitcurrentplaybacktargetiswirelesschanged', () => {
    console.log('airplay active:', video.webkitCurrentPlaybackTargetIsWireless);
  });
}

You can also let Safari handle it entirely with an attribute, which is what most sites do:

<video id="player" x-webkit-airplay="allow" playsinline controls></video>

6. Now fix the four bugs that only happen on real hardware

Everything above works on your laptop. Here's what breaks on an actual TV.

Token expiry on the wrong clock

You mint a short-lived signed playback token because that's correct security practice. The TV picks the stream up thirty seconds later and holds it for ninety minutes, then 403s on a manifest refresh. Works in your tab, dies on the device.

// src/playback-url.js
// Cast sessions outlive browser sessions. Mint a token scoped to the session type.
export async function getPlaybackUrl({ forCast = false } = {}) {
  const res = await fetch('/api/playback-token', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ ttlSeconds: forCast ? 60 * 60 * 4 : 60 * 15 }),
  });
  const { url } = await res.json();
  return url;
}

Swap the element's source before prompting:

btn.addEventListener('click', async () => {
  video.src = await getPlaybackUrl({ forCast: true });
  await video.remote.prompt();
});

⚠️ A longer TTL is a real security tradeoff, not a free fix. Scope the long-lived token to a single playback ID and a single device session, and keep the short TTL for normal browser playback.

CORS from an origin you forgot exists

The receiver is not your page. It's a different origin fetching your manifests and segments:

# verify from outside your app's origin
curl -I -H "Origin: https://not-your-site.example" \
  https://stream.example.com/abc123.m3u8

HTTP/2 200
access-control-allow-origin: *
access-control-allow-headers: range

If access-control-allow-origin is missing or pinned to your domain only, the TV gets blocked while your browser is fine.

Anything the TV can't resolve

Localhost, staging behind a VPN, an internal hostname. Your laptop resolves it, the TV on the guest network doesn't. Catch it in code instead of losing an afternoon:

// src/preflight.js
export function assertCastableUrl(url) {
  const u = new URL(url, location.href);
  const problems = [];
  if (u.protocol !== 'https:') problems.push('not https');
  if (['localhost', '127.0.0.1', '[::1]'].includes(u.hostname)) problems.push('localhost is not reachable from a TV');
  if (u.hostname.endsWith('.local') || u.hostname.endsWith('.internal')) problems.push('internal hostname');
  if (u.href.startsWith('blob:')) problems.push('blob URL from MSE, the TV cannot fetch this; use the real manifest URL');
  if (problems.length) console.warn(`[cast preflight] ${u.href}: ${problems.join(', ')}`);
  return problems.length === 0;
}

That last check is the hls.js trap from section 1. If you're feeding the element through MSE, video.src is a blob and there is nothing for the TV to fetch. Keep the real manifest URL around and assign it before casting.

The receiver's ABR isn't yours

Your bitrate cap for metered connections lived in hls.js config. The TV has never heard of it and will pull your top rendition. If that matters for cost or for a plan tier, the only lever you have left is the manifest itself: serve a cast-specific variant playlist with the renditions you're willing to send.

7. When to just turn it off

There's a property for this, and using it is a legitimate decision:

// DRM the receiver can't honour, overlay-critical content, or
// ad insertion that the cast path would silently skip.
video.disableRemotePlayback = true;

A disabled cast button with one line of copy explaining why beats a button that connects to a TV and then fails in a way the user blames on their wifi. "We support casting" is a claim with a testing burden attached.

Wrapping up

The test that predicts all of this: put your manifest URL in a bare <video> tag on a blank page, with none of your JavaScript anywhere, walk away for an hour, and see whether it's still playing. If it is, casting is nearly free, and so are smart TV apps and embedded webviews later. If it isn't, fix that first, because the cast button is only the first surface where it shows.

Two things worth reading next: MDN's Remote Playback API page has the current compatibility table, which is the part of this that changes; and if you need real control over what the TV renders, the Google Cast SDK's receiver application is the only path that gives it to you, at the cost of maintaining a second app.

What's your cast failure rate? If you're not measuring it separately from normal playback failures, it's probably higher than you think, because these bugs are invisible from the browser. #webdev #video

Check whether hls.js is actually using a worker (most ESM setups aren't)

Mason K — Thu, 10 Sep 2026 06:55:59 +0000

TL;DR

If you import Hls from 'hls.js', you are probably getting the ESM build, and the ESM build does not bundle the transmuxer worker. Transmuxing runs on your main thread until you set workerPath. We are going to verify which mode you are in, fix it, and add a long-task observer so you can tell main-thread stalls apart from network stalls.

Two short facts before any code. hls.js 1.4 introduced the ESM build (dist/hls.mjs), and that build ships the worker as a separate file rather than inlining it. And Chromium has supported MediaSource inside dedicated workers since Chrome 108, which is a different thing that we will get to at the end. Everything here was checked against hls.js 1.7.x.

1. Find out what you are currently running 🔎

Do not guess. Run this in the console on a page where a video is playing:

// paste in DevTools console during playback
performance.getEntriesByType('resource')
  .filter(e => e.name.includes('worker'))
  .map(e => e.name);

Empty array means no worker file was ever fetched. Then check DevTools → Sources → Threads (Chrome) or the Debugger's worker list (Firefox). If the only thread listed is the main one, hls.js is transmuxing inline.

You can also ask the library directly:

// after new Hls(...)
console.log(hls.config.workerPath);        // null if you never set it
console.log(hls.config.enableWorker);      // true by default, which is misleading

⚠️ Note: enableWorker: true is the default and it stays true even when no worker can be created. It means "use a worker if one is available", not "a worker is running". This is the single biggest source of false confidence here.

2. Wire up `workerPath` 🛠️

The pattern is: get a real URL for hls.js/dist/hls.worker.js, pass it as workerPath. The syntax differs per bundler.

Vite / Rollup:

// src/player.ts
import Hls from 'hls.js';
import workerUrl from 'hls.js/dist/hls.worker.js?url';

const hls = new Hls({
  workerPath: workerUrl,
});

webpack 5:

// src/player.js
import Hls from 'hls.js';

const workerUrl = new URL(
  'hls.js/dist/hls.worker.js',
  import.meta.url
).toString();

const hls = new Hls({ workerPath: workerUrl });

Next.js (app router, client component):

// app/components/Player.tsx
'use client';

import { useEffect, useRef } from 'react';
import Hls from 'hls.js';

export default function Player({ src }: { src: string }) {
  const videoRef = useRef<HTMLVideoElement>(null);

  useEffect(() => {
    const video = videoRef.current;
    if (!video || !Hls.isSupported()) return;

    const hls = new Hls({
      workerPath: new URL(
        'hls.js/dist/hls.worker.js',
        import.meta.url
      ).toString(),
    });

    hls.loadSource(src);
    hls.attachMedia(video);
    return () => hls.destroy();
  }, [src]);

  return <video ref={videoRef} controls playsInline />;
}

If you copy nothing else, copy the verification step, because a wrong path fails silently:

hls.on(Hls.Events.MANIFEST_PARSED, () => {
  const gotWorker = performance
    .getEntriesByType('resource')
    .some(e => e.name.includes('hls.worker'));
  console.log('[hls] worker active:', gotWorker);
});

# what you want to see
[hls] worker active: true

3. Add a long-task observer so stalls stop being ambiguous 📊

A buffer stall tells you playback ran dry. It does not tell you whether the bytes were late or the thread was busy. PerformanceObserver with longtask closes that gap: it reports every task that held the main thread for more than 50ms.

// src/instrumentation/main-thread.js
const recentLongTasks = [];

const observer = new PerformanceObserver((list) => {
  for (const entry of list.getEntries()) {
    recentLongTasks.push({ start: entry.startTime, dur: entry.duration });
  }
  // keep the last 10 seconds only
  const cutoff = performance.now() - 10_000;
  while (recentLongTasks.length && recentLongTasks[0].start < cutoff) {
    recentLongTasks.shift();
  }
});

observer.observe({ type: 'longtask', buffered: true });

export function blockedMsInLast(windowMs = 3000) {
  const cutoff = performance.now() - windowMs;
  return recentLongTasks
    .filter(t => t.start >= cutoff)
    .reduce((sum, t) => sum + t.dur, 0);
}

Now tag every stall with it:

// src/player.js
import { blockedMsInLast } from './instrumentation/main-thread.js';

hls.on(Hls.Events.ERROR, (_evt, data) => {
  if (data.details !== Hls.ErrorDetails.BUFFER_STALLED_ERROR) return;

  analytics.track('video_stall', {
    blocked_ms_3s: blockedMsInLast(3000),
    buffer_len: hls.media
      ? hls.media.buffered.length && hls.media.buffered.end(0) - hls.media.currentTime
      : 0,
    level: hls.currentLevel,
  });
});

# a main-thread stall
video_stall { blocked_ms_3s: 1180, buffer_len: 0.2, level: 3 }

# a network stall
video_stall { blocked_ms_3s: 0, buffer_len: 0.1, level: 3 }

Those two rows deserve completely different tickets. Right now most teams have them in the same bucket.

💡 Tip: longtask is not supported in Safari. Feature-detect with PerformanceObserver.supportedEntryTypes.includes('longtask') and fall back to counting requestAnimationFrame gaps if you need cross-browser coverage.

4. Reproduce it on purpose 🧪

Before you trust the numbers, prove the instrument moves. Add a deliberate main-thread hog and watch the stall rate climb:

// DO NOT SHIP. Reproduction only.
setInterval(() => {
  const end = performance.now() + 120;   // block for 120ms
  while (performance.now() < end) { /* spin */ }
}, 500);

Run your player with that on, once with workerPath unset and once with it set. The version without a worker degrades noticeably sooner, because transmuxing is queued behind the same blocking task.

Then throttle CPU in DevTools (Performance → CPU → 4x or 6x slowdown) rather than throttling the network. Most player testing throttles the network and never touches the CPU, which is why this class of bug reaches production.

5. What the worker still does not fix

Worth being precise about, because it changes what you do next.

Stage	Runs where (hls.js, worker configured)
Segment fetch	Network thread
Demux + remux	Worker
`SourceBuffer.appendBuffer()`	Main thread
Decode + render	Browser media pipeline

The hls.js worker is a transmuxer worker. The MediaSource still lives on the main thread, so appends still contend with your app.

The platform-level answer is MSE in a dedicated worker: construct the MediaSource inside the worker, get a handle, and pass it to the <video> element. Chromium enabled this by default in Chrome 108 (Opera 94 followed). Firefox and Safari have not shipped it. Plain main-thread MSE is universal (Firefox 42+, Safari 8+), the worker variant is not, and no mainstream HLS library drives it end to end today. So treat it as a Chromium-only enhancement you might reach for in a custom player, not as something to design your architecture around.

Related config worth knowing while you are in here: preferManagedMediaSource (added in hls.js 1.5.0) controls whether hls.js picks ManagedMediaSource over MediaSource on platforms exposing both. Set it to false if you specifically want the classic path.

What's next

Two follow-ups, in the order I would do them.

Ship the long-task tag behind a sampling flag (1% of sessions is plenty) and look at the split after a week. If a meaningful share of your stalls have blocked time in front of them, your next video-quality win is a front-end performance ticket, not a CDN ticket.
Go after the biggest offender on the page. Usually a list rendering under the player, an unvirtualised feed, or a JSON parse on the response of whatever loads next. Moving that to a worker helps video even though it has nothing to do with video.

If you want to go deeper on the platform side, the hls.js API docs cover workerPath and enableWorker in full, and the MDN Media Source Extensions page has the current worker-support picture. And if you find that your stalls are genuinely network-side after all, that is a good outcome too: you just stopped guessing.

Build a React Native video upload that survives backgrounding, app kills, and cellular

Mason K — Thu, 10 Sep 2026 06:55:23 +0000

TL;DR

A fetch() upload dies when the OS suspends your app, and your JS context is recreated with no memory of it. We are going to build an upload that is a persisted record plus a state machine, hand the actual transfer to a native background session, and reconcile against the server when the app comes back. About 150 lines.

Versions this was written against: Expo SDK 54 (React Native 0.81) and SDK 55 (React Native 0.83). Relevant because SDK 54 is the last release with legacy architecture support and SDK 55 is New Architecture only, so any native upload library you pick needs New Architecture support.

The failure we are fixing 📱

// the version in every tutorial
const res = await fetch(uploadUrl, { method: 'PUT', body: file });
setProgress(100);

Background the app mid-transfer and this stops existing. No error, no rejection, no catch block. iOS suspends the process; Android may reclaim it. When the user returns, your JS runtime is fresh and that promise never settles.

Test it yourself before reading further, because it is more convincing than any explanation:

# start a large upload, then:
#  1. swipe to home
#  2. lock the phone
#  3. wait 60s
#  4. reopen the app
# most implementations show a frozen progress bar forever

1. Know what the platforms will and will not do

Neither OS lets JavaScript keep a transfer alive. The constraints are worth internalising because they dictate the design.

Platform	What continues in the background	The catch
iOS	Background `URLSession` upload task	Must upload from a file on disk. Data/stream bodies are not supported in background sessions. Completion-handler convenience APIs are unavailable; use the delegate API.
Android	Foreground service (`dataSync`)	Service type required when targeting Android 14+. Android 15 caps `dataSync` and `mediaProcessing` at 6 hours per 24 hours, then calls `Service.onTimeout()`. Past the budget, starting one throws `ForegroundServiceStartNotAllowedException`.

Two consequences fall straight out:

Write the body to a file first. Even if your source is already a file, normalise the path, because iOS needs a stable URL it can read after your process is gone.
Start the upload from a user tap. Google's guidance is explicit: a dataSync service started from direct user interaction gets the full window once the app backgrounds. The budget resets when the user next foregrounds the app.

⚠️ Note: Service.onTimeout(int, int) does not exist on Android 14 or below. On those versions you get cached with no callback, so do not build recovery logic that depends on the timeout firing.

2. Persist the upload before any bytes move 💾

This is the whole trick. The record outlives the process; the promise does not.

// src/uploads/store.ts
import * as SQLite from 'expo-sqlite';

export type UploadState =
  | 'pending'
  | 'uploading'
  | 'uploaded'      // bytes are on the server
  | 'processing'    // server is transcoding
  | 'ready'
  | 'failed';

const db = await SQLite.openDatabaseAsync('uploads.db');

await db.execAsync(`
  CREATE TABLE IF NOT EXISTS uploads (
    id           TEXT PRIMARY KEY,
    local_uri    TEXT NOT NULL,
    upload_url   TEXT NOT NULL,
    asset_id     TEXT,
    state        TEXT NOT NULL,
    bytes_sent   INTEGER DEFAULT 0,
    bytes_total  INTEGER DEFAULT 0,
    updated_at   INTEGER NOT NULL
  );
`);

export async function createUpload(row: {
  id: string; localUri: string; uploadUrl: string; bytesTotal: number;
}) {
  await db.runAsync(
    `INSERT INTO uploads (id, local_uri, upload_url, state, bytes_total, updated_at)
     VALUES (?, ?, ?, 'pending', ?, ?)`,
    [row.id, row.localUri, row.uploadUrl, row.bytesTotal, Date.now()]
  );
}

export async function setState(id: string, state: UploadState, patch: object = {}) {
  const cols = Object.keys(patch);
  const setSql = ['state = ?', 'updated_at = ?', ...cols.map(c => `${c} = ?`)].join(', ');
  await db.runAsync(
    `UPDATE uploads SET ${setSql} WHERE id = ?`,
    [state, Date.now(), ...cols.map(c => (patch as any)[c]), id]
  );
}

export async function unfinished() {
  return db.getAllAsync<{ id: string; asset_id: string | null; state: UploadState }>(
    `SELECT * FROM uploads WHERE state IN ('pending','uploading','uploaded','processing')`
  );
}

3. Hand the transfer to a native background session 🚚

expo/fetch is the modern Expo upload path, but it does not support background sessions. (The older FileSystem.uploadAsync did have a background session type; it is legacy, deprecated as of SDK 54, and had known failures on larger files.) So for anything that must survive backgrounding, you go native, either with a community module or your own.

// src/uploads/transfer.ts
import Upload from 'react-native-background-upload';
import { setState } from './store';

export async function startUpload(row: {
  id: string; localUri: string; uploadUrl: string;
}) {
  await setState(row.id, 'uploading');

  const taskId = await Upload.startUpload({
    url: row.uploadUrl,
    path: row.localUri.replace('file://', ''),  // iOS wants a bare path
    method: 'PUT',
    type: 'raw',
    field: 'file',
    // keeps the transfer alive when the app is backgrounded
    notification: {
      enabled: true,
      autoClear: true,
      onProgressTitle: 'Uploading video',
    },
    // Android: ties into a dataSync foreground service
    android: { notificationChannel: 'uploads' },
  });

  Upload.addListener('progress', taskId, (data) => {
    setState(row.id, 'uploading', {
      bytes_sent: Math.round((Number(data.progress) / 100) * 1),
    });
  });

  Upload.addListener('completed', taskId, () => setState(row.id, 'uploaded'));
  Upload.addListener('error', taskId, () => setState(row.id, 'failed'));
  Upload.addListener('cancelled', taskId, () => setState(row.id, 'failed'));

  return taskId;
}

💡 Tip: check the module's New Architecture status before you commit to it. From React Native 0.82 the opt-out was removed, and Expo SDK 55 (RN 0.83) is New Architecture only. A library that has not migrated is a dead end, not a temporary inconvenience.

4. Reconcile on relaunch, and let the server win 🔄

Every listener you registered is gone after a process kill. So on boot, do not trust local state; ask the server.

// src/uploads/reconcile.ts
import { unfinished, setState } from './store';

export async function reconcileUploads() {
  const rows = await unfinished();

  for (const row of rows) {
    if (!row.asset_id) {
      // never got far enough to have a server-side asset: retry from scratch
      await setState(row.id, 'pending');
      continue;
    }

    const res = await fetch(`/api/assets/${row.asset_id}`);
    if (!res.ok) continue;

    const asset = await res.json();
    // server states -> local states
    const map: Record<string, any> = {
      waiting_for_upload: 'pending',
      uploaded: 'processing',
      processing: 'processing',
      ready: 'ready',
      errored: 'failed',
    };
    await setState(row.id, map[asset.status] ?? 'failed');
  }
}

Call it once, early:

// app/_layout.tsx
useEffect(() => { reconcileUploads(); }, []);

# typical output after a kill mid-upload
[reconcile] 2 unfinished uploads
[reconcile] a3f1… server says 'processing' -> local 'processing'
[reconcile] 91cd… no asset_id -> requeued as 'pending'

5. Stop calling 100 percent "done" ✅

Bytes delivered is not video playable. Keep processing as a real, visible state driven by a webhook or a poll, and only move to ready when the server says the asset is playable.

// src/uploads/ui.ts
export function label(state: UploadState, pct: number) {
  switch (state) {
    case 'pending':    return 'Waiting to upload';
    case 'uploading':  return `Uploading ${pct}%`;
    case 'uploaded':   return 'Upload complete, preparing video';
    case 'processing': return 'Preparing video';
    case 'ready':      return 'Ready';
    case 'failed':     return 'Upload failed, tap to retry';
  }
}

That one change removes a whole category of support tickets, because "preparing video" is a thing users will wait through and a stuck 100% is not.

The test that matters 🧪

1. Airplane mode off, start a 200MB upload
2. Background the app at ~30%
3. Lock the phone for 2 minutes
4. Toggle airplane mode on and off
5. Force-quit the app
6. Reopen

Correct behaviour: the upload either resumed in the background or the record came back as pending and requeued. Incorrect behaviour: a progress bar frozen at 30%, or a duplicate asset on the server.

What's next

Move to a resumable protocol (tus or S3 multipart) so step 4 continues instead of restarting. On cellular this is the difference between a feature that works and one users abandon.
Instrument from the server, not the client. Client-reported success rates only count the uploads that already worked, since the failures happen in a process that no longer exists. Count assets that reached ready against assets that were created.
If you are on Expo and considering writing the native module yourself, Expo's own engineering post on building a video upload module with Expo Modules is the closest thing to a reference implementation.

Build a repeatable banding test with FFmpeg (and find the ladder rung that breaks first)

Mason K — Thu, 10 Sep 2026 06:55:04 +0000

TL;DR

Banding arguments go in circles because everyone tests on different footage. We are going to generate a synthetic gradient, encode it four ways (naive 8-bit, 8-bit debanded, 10-bit filter graph with explicit dither, and 10-bit AV1), and build a small harness so you can run it on your own ladder. Everything runs locally with FFmpeg, no assets to download.

Written against FFmpeg 7.x/8.x. Check yours with ffmpeg -version.

1. Generate a worst-case clip 🎨

Real footage has sensor noise, and that noise dithers away banding by accident. To see the problem clearly you want a clip with none:

# 10 seconds of a slow horizontal gradient, 1080p, no noise
ffmpeg -f lavfi -i "gradients=s=1920x1080:c0=0x101020:c1=0x404860:\
type=linear:nb_colors=2:speed=0.01:d=10" \
  -pix_fmt yuv420p10le -c:v ffv1 \
  master.mkv

# what you should see
Output #0, matroska, to 'master.mkv':
  Stream #0:0: Video: ffv1, yuv420p10le, 1920x1080, 25 fps

We keep the master lossless and 10-bit so every later step is the only lossy thing happening. Dark blues are deliberate: banding is worst in the low end, which is also where nobody looks.

💡 Tip: keep a second master made from real footage (a sunset, a studio backdrop, a fade to black). The synthetic clip finds the problem; real footage tells you whether the fix survives contact with grain.

2. Encode it four ways 🛠️

A. The naive 8-bit encode

ffmpeg -i master.mkv \
  -vf "format=yuv420p" \
  -c:v libx264 -profile:v high -preset medium -crf 23 \
  -pix_fmt yuv420p out_a_naive.mp4

This is what most ladders do. It is the baseline.

B. Debanding in an 8-bit graph

ffmpeg -i master.mkv \
  -vf "format=yuv420p,gradfun=strength=1.2:radius=16" \
  -c:v libx264 -profile:v high -preset medium -crf 23 \
  -pix_fmt yuv420p out_b_gradfun8.mp4

gradfun fits the gradient that should be in the flat region and dithers it back in. strength is both the maximum change per pixel and the flat-region detection threshold (range 0.51 to 64, default 1.2). radius is the neighbourhood used for the fit (range 8 to 32, default 16); larger is smoother but the filter gets more reluctant near detail.

⚠️ FFmpeg's own docs say gradfun is not recommended before lossy compression, because the compression loses the dither and the bands come back. That is exactly what this command does, on purpose, so you can see it happen.

C. 10-bit filter graph with an explicit dither down

ffmpeg -i master.mkv \
  -vf "format=yuv420p10le,gradfun=strength=1.5:radius=24,\
format=yuv420p" \
  -sws_dither error_diffusion \
  -c:v libx264 -profile:v high -preset medium -crf 23 \
  -tune grain \
  -x264-params aq-mode=3:aq-strength=1.0 \
  -pix_fmt yuv420p out_c_10bitchain.mp4

Three separate things are doing work here:

The filter graph runs at 10 bits, so gradfun has somewhere to put intermediate values instead of re-quantising at every stage.
The step back to 8 bits is deliberate, with error_diffusion dithering, rather than an implicit truncation.
-tune grain plus a higher aq-mode stops x264 from smoothing away the low-amplitude noise you just paid for. This is where most of the win is, and it is the step people skip.

D. 10-bit AV1

ffmpeg -i master.mkv \
  -vf "format=yuv420p10le" \
  -c:v libsvtav1 -preset 6 -crf 32 \
  -pix_fmt yuv420p10le out_d_av1_10bit.mp4

No debanding filter at all. This removes the quantisation problem instead of concealing it.

3. Compare them without squinting 📊

Eyeballing gradients is unreliable. Amplify the differences instead:

# blow up the low end so steps become obvious
ffmpeg -i out_a_naive.mp4 \
  -vf "eq=contrast=6.0,format=rgb24" \
  -frames:v 1 vis_a.png

for f in out_a_naive out_b_gradfun8 out_c_10bitchain out_d_av1_10bit; do
  ffmpeg -y -i "$f.mp4" -vf "eq=contrast=6.0" -frames:v 1 "vis_${f}.png"
done

Then stack them for a single side-by-side:

ffmpeg -i vis_out_a_naive.png -i vis_out_b_gradfun8.png \
       -i vis_out_c_10bitchain.png -i vis_out_d_av1_10bit.png \
  -filter_complex "[0][1]hstack[t];[2][3]hstack[b];[t][b]vstack" \
  comparison.png

And check what each one cost you:

for f in out_*.mp4; do
  printf "%-28s %8s KiB\n" "$f" "$(( $(stat -c%s "$f" 2>/dev/null || stat -f%z "$f") / 1024 ))"
done

# example shape of the output (your numbers will differ)
out_a_naive.mp4               ...... KiB
out_b_gradfun8.mp4            ...... KiB
out_c_10bitchain.mp4          ...... KiB
out_d_av1_10bit.mp4           ...... KiB

I am not printing my numbers here on purpose. They depend on your FFmpeg build, your CRF, and your source, and a number copied from someone else's blog post is how bad ladder decisions get made. Run it on your own footage.

4. The 10-bit trap on the web 🚧

Reasonable instinct, wrong execution:

# DO NOT ship this to a browser audience
ffmpeg -i master.mkv -c:v libx264 -pix_fmt yuv420p10le out_high10.mp4

That produces H.264 High 10. It is a real profile and x264 will make it happily, and much of the installed base cannot hardware-decode it, if it plays at all. Mass-market H.264 means 8-bit High profile. Shipping High 10 turns a cosmetic complaint into a playback failure plus a hot phone.

Codec	10-bit in a mainstream profile	Safe for web delivery at 10-bit
H.264 / AVC	High 10 (niche)	❌
HEVC	Main 10	✅ where HEVC is supported
VP9	Profile 2	✅
AV1	Main (8/10-bit)	✅ practical open-web route

If you already ship an AV1 rendition for bitrate reasons, making it the 10-bit one is close to free.

How much of your audience that covers is a measurement, not an assumption. One published dataset (WebCodecs Fundamentals, over a million devices, 2026) puts AV1 and HEVC decode between them at 99.73% of sessions. Treat that as their number, check it against your own player telemetry before planning a ladder around it.

5. Point the harness at your real ladder

The useful output of all this is not "which encode looked best on a synthetic gradient." It is which rung of your ladder breaks first. Loop your own ladder over the master:

# ladder.txt: one "height crf maxrate bufsize" per line
while read -r h crf maxrate bufsize; do
  ffmpeg -y -i master.mkv \
    -vf "scale=-2:${h},format=yuv420p" \
    -c:v libx264 -preset medium -crf "$crf" \
    -maxrate "$maxrate" -bufsize "$bufsize" \
    -pix_fmt yuv420p "rung_${h}.mp4"
done < ladder.txt

It is almost always a lower rung: a large flat sky at a bitrate that was chosen while looking at a scene with motion in it. That rung is where your complaint came from, and it is the cheapest one to fix, usually by moving -tune grain and a higher aq-strength onto just those renditions rather than raising bitrate across the ladder.

Wrapping up

Four things to take away:

Banding is a bit depth problem that lossy encoding exposes, not a bitrate problem.
A debanding filter in an 8-bit graph before a lossy encode mostly gets undone.
High bit depth in the filter graph plus an explicit dither plus grain-preserving encoder settings is the combination that survives.
10-bit H.264 is not a web delivery format. AV1 is.

What's next: wire the comparison step into CI so a ladder change that reintroduces banding fails the build, and add a real-footage master alongside the synthetic one. If you want to go further on the filter side, the FFmpeg filter docs cover deband (which averages detected flat pixels, detection range defaults to 16) as an alternative to gradfun, and it behaves differently enough on real grain to be worth a fifth column in your comparison.

Audit what every rung of your HLS ladder costs per viewer-hour

Mason K — Fri, 04 Sep 2026 12:53:45 +0000

TL;DR

Your bitrate ladder is a price list, and ABR buys the most expensive rung it can. We'll write a ~60-line Node script that fetches any HLS master playlist and prints what each rendition costs per viewer-hour, then look at two fixes: reshaping the ladder with FFmpeg, and moving delivery to per-minute pricing where bitrate drift can't inflate the bill.

Delivery bills usually grow for a boring reason: the average delivered bitrate crept up and nobody noticed, because nobody prices the ladder. Let's price the ladder.

Tools for this build: Node 22 (any Node 18+ works, we only need global fetch), FFmpeg 7.0+, and any HLS stream you can reach, including your own.

1. Grab a master playlist 🎯

Every HLS stream starts from a master playlist that declares each rendition and its BANDWIDTH in bits per second. That attribute is the whole audit surface:

curl -s https://your-cdn.example.com/v1/master.m3u8 | head -20

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=6221600,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"
1080p_high/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4521600,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"
1080p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3021600,RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.2"
720p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1521600,RESOLUTION=854x480,CODECS="avc1.64001e,mp4a.40.2"
480p/index.m3u8

BANDWIDTH is the peak declared rate, so this audit is a worst-case price. That's the right case to price, because on good networks ABR sits on your top rung all day.

2. The audit script 🛠️

// ladder-audit.js
// Usage: node ladder-audit.js <master.m3u8 URL> <cost per GB in USD>

const [, , url, costPerGBArg] = process.argv;
const COST_PER_GB = Number(costPerGBArg ?? 0.085); // your CDN's blended $/GB

const res = await fetch(url);
if (!res.ok) {
  console.error(`Fetch failed: ${res.status} ${res.statusText}`);
  process.exit(1);
}
const manifest = await res.text();

const rungs = [];
const lines = manifest.split("\n");
for (let i = 0; i < lines.length; i++) {
  if (!lines[i].startsWith("#EXT-X-STREAM-INF:")) continue;
  const attrs = lines[i].slice("#EXT-X-STREAM-INF:".length);
  const bandwidth = Number(/BANDWIDTH=(\d+)/.exec(attrs)?.[1] ?? 0);
  const resolution = /RESOLUTION=([\dx]+)/.exec(attrs)?.[1] ?? "audio-only";
  rungs.push({ bandwidth, resolution, uri: lines[i + 1]?.trim() });
}

console.log(`\nLadder audit for ${url}`);
console.log(`Assumed delivery cost: $${COST_PER_GB}/GB\n`);
console.table(
  rungs
    .sort((a, b) => b.bandwidth - a.bandwidth)
    .map((r) => {
      const gbPerHour = (r.bandwidth * 3600) / 8 / 1e9;
      return {
        resolution: r.resolution,
        mbps: (r.bandwidth / 1e6).toFixed(2),
        "GB/viewer-hour": gbPerHour.toFixed(2),
        "$/viewer-hour": (gbPerHour * COST_PER_GB).toFixed(3),
      };
    })
);

Run it against your stream with the per-GB rate from your last invoice:

node ladder-audit.js https://your-cdn.example.com/v1/master.m3u8 0.085

Ladder audit for https://your-cdn.example.com/v1/master.m3u8
Assumed delivery cost: $0.085/GB

┌─────────┬────────────┬────────┬────────────────┬────────────────┐
│ (index) │ resolution │ mbps   │ GB/viewer-hour │ $/viewer-hour  │
├─────────┼────────────┼────────┼────────────────┼────────────────┤
│ 0       │ '1920x1080'│ '6.22' │ '2.80'         │ '0.238'        │
│ 1       │ '1920x1080'│ '4.52' │ '2.03'         │ '0.173'        │
│ 2       │ '1280x720' │ '3.02' │ '1.36'         │ '0.116'        │
│ 3       │ '854x480'  │ '1.52' │ '0.68'         │ '0.058'        │
└─────────┴────────────┴────────┴────────────────┴────────────────┘

💡 Tip: to turn this into a blended number, weight each row by the share of watch time it serves. Most player analytics (hls.js LEVEL_SWITCHED events, or your QoE tool's rendition report) can give you that distribution.

3. Read the output like a bill 💸

Two questions to ask of the table:

Why does the top rung exist? If 6.2 Mbps and 4.5 Mbps 1080p rungs look identical for your content, the top one costs you 37 percent more per viewer-hour for nothing. Someone should be able to defend every rung in one sentence.
Is one ladder serving very different content? A screencast and a sports clip through the same ladder means one of them is overpaying. Fixed ladders price for your most complex content and bill you for it on your simplest.

4. Fix A: reshape the ladder with FFmpeg

The DIY fix is per-title thinking: cap each rung by quality, not just bitrate. CRF encoding with a maxrate ceiling gets you most of the way without a full VMAF pipeline:

# 1080p rung: quality-targeted, bitrate-capped
ffmpeg -i input.mp4 \
  -c:v libx264 -crf 21 -preset slow \
  -maxrate 4500k -bufsize 9000k \
  -vf scale=1920:1080 \
  -c:a aac -b:a 128k \
  1080p.mp4

Simple content lands well under the cap and you pocket the difference; complex content hits maxrate and keeps its headroom. The honest cost: doing this properly across a catalog means trial encodes, quality scoring, a queue, and monitoring. It's a pipeline, not a flag. Budget maintenance time for it like any other service you run.

5. Fix B: change the pricing model

The other fix is structural. Managed platforms mostly bill delivery per minute watched, not per GB, which makes bitrate drift financially impossible: the meter is watch time. Cloudflare Stream is $1 per 1,000 minutes delivered at any resolution. FastPix lists about $0.00096 per minute at 1080p, roughly 5.8 cents per viewer-hour, and does context-aware encoding by default, so the per-title work from Fix A is the encoder's job instead of yours. Encoding itself is free on its standard plan. Mux and api.video follow the same per-minute delivery pattern, so this section's flow works across all of them.

Getting a video into a context-aware pipeline is one call:

# Create a media from a URL (Basic auth: Access Token ID / Secret Key)
curl -X POST https://api.fastpix.com/v1/on-demand \
  -u "$FASTPIX_TOKEN_ID:$FASTPIX_SECRET" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [{ "type": "video", "url": "https://static.fastpix.com/fp-sample-video.mp4" }],
    "accessPolicy": "public"
  }'

{
  "success": true,
  "data": {
    "id": "0a8e6d7f-...",
    "status": "Created",
    "playbackIds": [{ "id": "9f2b1c4d-...", "accessPolicy": "public" }]
  }
}

Wait for the video.media.ready webhook, then play https://stream.fastpix.com/<playbackId>.m3u8 and point the audit script from step 2 at the master playlist it serves. Comparing the rendition bitrates you get against the fixed ladder you had is the whole evaluation, on your own content, in an afternoon. The VOD API reference covers the create-media options if you want to pin maxResolution or quality.

For calibration: in one published benchmark on a 177 MB file over 4G, FastPix's context-aware output averaged around 700 Kbps where fixed-ladder platforms delivered 1.8 to 2.5 Mbps, with zero rebuffering across all platforms tested; the same report scored Mux best on viewer experience thanks to faster cold startup. Run your own numbers; that's what the script is for.

⚠️ Note: FastPix is API-first. There's no drag-and-drop CMS included, so a non-developer content team will need a front-end built on the API.

What's next

Wire the watch-time distribution into the script and compute a real blended $/viewer-hour, then put it on a dashboard next to watch time. It should have an owner.
If you go the DIY route, the next problem is quality scoring at scale: VMAF in your encode CI, per-title thresholds, and a regression suite for encoder upgrades.

The number the script prints is rarely the number people expect. That surprise is the point.

Encode your first VVC clip with vvenc 1.14, then watch every browser refuse to play it

Mason K — Fri, 04 Sep 2026 12:53:23 +0000

TL;DR

H.266/VVC is the codec behind Brazil's TV 3.0 broadcast standard, and the open-source encoder (VVenC 1.14) is now fast enough to try on a laptop. We'll encode a real clip, mux it into MP4, and then test playback across browsers, FFmpeg, and desktop players to map exactly where VVC does and doesn't work in 2026. Spoiler: your <video> tag says no.

Why bother, if browsers don't play it? Because VVC just got its first massive real-world deployment (Brazilian broadcast TV mandates it, layered with MPEG-5 LCEVC), and "I've actually run the encoder" beats "I read a press release" in every architecture discussion you'll have about codecs this year.

What we're using: VVenC 1.14 (released January 2026), FFmpeg 7.1+ (for a native VVC decoder; FFmpeg 8.0 works too), and optionally GPAC's MP4Box.

1. Get the encoder 🛠️

VVenC is Fraunhofer HHI's open VVC encoder, BSD-3-Clause-Clear licensed. On macOS, Homebrew carries it; everywhere else the cmake build is quick:

# macOS
brew install vvenc

# or build from source (Linux/macOS)
git clone https://github.com/fraunhoferhhi/vvenc.git
cd vvenc
make install-release
# binaries land in bin/release-static/

Check it:

vvencapp --version

vvencapp: Fraunhofer VVC Encoder ver. 1.14.0

2. Prepare a test clip 🎬

vvencapp eats raw YUV or y4m. Take any 10 to 20 second 1080p clip and unwrap it with FFmpeg:

# clip.mp4 -> raw y4m (this file will be big, that's normal)
ffmpeg -i clip.mp4 -t 15 -pix_fmt yuv420p10le -strict -1 clip_10bit.y4m

💡 Tip: VVC's sweet spot assumes 10-bit. Use yuv420p10le rather than 8-bit so you're testing the codec the way broadcast deploys it.

3. Encode with vvencapp

vvencapp --preset medium -i clip_10bit.y4m --qp 32 -o clip.266

You'll see per-frame progress and a summary along these lines (numbers are from my run on a laptop, yours will differ):

vvencapp [info]: start encoding...
POC    30 TId: 0 (B-SLICE, QP 35)
...
vvenc [info]: stats summary: frames= 375, bitrate= 2861 kbps
vvenc [info]: Total Time: 214.5 sec [user], 3.5 fps

Two things to notice. First, --preset medium on a modern laptop runs at single-digit fps for 1080p. That's dramatically better than the reference software (Fraunhofer quotes speedups of 20x to 2,400x for VVenC over VTM depending on preset), but it's still far from realtime. Second, try --preset faster and watch fps climb while the bitrate for the same QP rises; that's the whole encoder trade-off surface in two runs.

For a size comparison that's actually yours, encode the same y4m with x265 and SVT-AV1 and put the outputs side by side:

ffmpeg -i clip_10bit.y4m -c:v libx265 -crf 24 -preset slow clip_h265.mp4
ffmpeg -i clip_10bit.y4m -c:v libsvtav1 -crf 32 -preset 6 clip_av1.mp4
ls -lh clip.266 clip_h265.mp4 clip_av1.mp4

I'm deliberately not printing my byte counts here: run it on your own footage, because content complexity dominates codec comparisons and your clip is the only benchmark that matters for you.

4. Mux it into MP4 📦

A raw .266 bitstream won't go far. Recent FFmpeg and GPAC both know how to put VVC in MP4:

# FFmpeg 7.1+
ffmpeg -i clip.266 -c copy clip_vvc.mp4

# or GPAC
MP4Box -add clip.266 -new clip_vvc.mp4

5. Inspect what you made 🔬

Before playing anything, confirm the file is what you think it is. ffprobe from a recent FFmpeg reads VVC fine:

ffprobe -hide_banner clip_vvc.mp4

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'clip_vvc.mp4':
  Duration: 00:00:15.00, start: 0.000000, bitrate: 2894 kb/s
  Stream #0:0: Video: vvc (vvc1 / 0x31637676), yuv420p10le(tv),
    1280x720, 30 fps, 30 tbr, 15360 tbn

Two details worth noticing in that output. The sample entry is vvc1, the MP4 signaling browsers would need to recognize before anything else could happen; and the pixel format survived as yuv420p10le, confirming you exercised the 10-bit path broadcast actually uses. If you ever need to debug VVC support claims on a device, this vvc1 string is what its media capabilities API has to acknowledge.

# bonus: count how the encoder spent its frames
ffprobe -v error -select_streams v:0 -show_entries frame=pict_type \
  -of csv clip_vvc.mp4 | sort | uniq -c

That little histogram of I/P/B frames is the GOP structure you configured, verified from the bitstream rather than assumed.

6. The playback tour, from success to failure

FFmpeg's native decoder (works):

ffplay clip_vvc.mp4

FFmpeg has shipped a native VVC decoder in recent releases, and current VLC builds play VVC too. Desktop players are fine.

Browsers (do not work): drop this in a console or a test page:

// vvc-support-check.js
const v = document.createElement("video");
console.log("VVC :", v.canPlayType('video/mp4; codecs="vvc1.1.6.L93.B0"') || "(empty = no)");
console.log("AV1 :", v.canPlayType('video/mp4; codecs="av01.0.08M.08"') || "(empty = no)");
console.log("HEVC:", v.canPlayType('video/mp4; codecs="hvc1.1.6.L93.B0"') || "(empty = no)");

VVC : (empty = no)
AV1 : probably
HEVC: maybe   # hardware-dependent

As of mid-2026 no major browser ships VVC decode. Chrome, Firefox, and Edge decode AV1 natively, and AV1 hardware decode has landed across Apple, Qualcomm, and MediaTek chips. That asymmetry is the entire story of why your web ladder is H.264 + AV1 (+ HEVC for some Apple paths) and not VVC.

7. So where does VVC actually run? 📡

Environment	VVC in 2026	Why
Browsers	No support in any major browser	Patent-pool ambiguity, AV1 already shipped royalty-free
Broadcast TV (Brazil TV 3.0 / DTV+)	Mandated, on air since Aug 2025	Standard requires VVC + LCEVC decode in new sets
Smart TV SoCs	Arriving in new chipsets	Pulled along by broadcast mandates
FFmpeg / VLC	Decode works	Native decoder in recent releases
Your encode box	Works today via VVenC / libvvenc	BSD-licensed encoder, practical speeds

The interesting technical wrinkle in Brazil's stack is MPEG-5 LCEVC: not a standalone codec but an enhancement layer over a base codec. V-Nova's TV 3.0 trial figures claim 4K HDR at 8.08 Mbps with VVC+LCEVC versus 12.54 Mbps for VVC alone (vendor numbers, take accordingly). The layered idea is worth knowing even if you never touch VVC, because it can ride on codecs browsers already play.

What's next

Point the same workflow at LCEVC and see the enhancement-layer model up close (V-Nova publishes SDKs and the TV 3.0 write-up at v-nova.com).
Watch chipset spec sheets, not browser release notes, if you ship connected-TV apps. TV SoCs are where VVC decode will reach your users first.
Keep your ladder boring: H.264 floor, AV1 where advertised. The experiment you just ran is exactly why.

Sources: VVenC releases, V-Nova TV 3.0, Streaming Media, State of Streaming Codecs 2026.

Build a live latency HUD with ffmpeg, a burned-in clock, and hls.js

Mason K — Fri, 04 Sep 2026 12:52:58 +0000

TL;DR

We'll measure live stream latency two honest ways at once: a UTC clock burned into the video (true glass-to-glass) and EXT-X-PROGRAM-DATE-TIME math surfaced by hls.js (hls.latency). You'll end up with a local test stream, a player page with a latency HUD, and a clear picture of where your seconds actually go.

Most teams quote their protocol's brochure latency ("HLS is ~30s, LL-HLS is single digits") and have never measured their own. Let's fix that locally in about twenty minutes, with tools you already have: FFmpeg 7.0+ and hls.js 1.6+.

1. Generate a live stream with a clock burned in 🕐

The oldest trick in streaming QA is still the best: put the current time inside the video. Any latency measurement scheme can lie to you; pixels can't.

# make-stream.sh
mkdir -p out
ffmpeg -f lavfi -i "testsrc2=size=1280x720:rate=30" \
  -vf "drawtext=text='UTC %{gmtime\:%H\\\:%M\\\:%S}':fontsize=64:fontcolor=white:box=1:boxcolor=black@0.6:x=(w-tw)/2:y=h-th-40" \
  -c:v libx264 -preset veryfast -tune zerolatency -g 60 \
  -f hls \
  -hls_time 2 \
  -hls_list_size 6 \
  -hls_flags delete_segments+program_date_time \
  out/live.m3u8

The two flags that matter for this build:

-hls_flags +program_date_time writes an EXT-X-PROGRAM-DATE-TIME tag per segment, which is what player-side latency math needs.
-g 60 pins the GOP to 2 seconds at 30fps, matching -hls_time 2. Segment duration can never be shorter than your GOP; this pairing is a latency decision, not a detail.

Sanity-check the playlist while it runs:

cat out/live.m3u8

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:2
#EXT-X-PROGRAM-DATE-TIME:2026-08-31T09:14:32.640+0000
#EXTINF:2.000000,
live0042.ts
...

Serve the folder (segments and the player page from the same origin keeps CORS quiet):

python3 -m http.server 8000 -d out

2. The player page with a latency HUD 📺

<!-- out/player.html -->
<!doctype html>
<video id="v" controls muted autoplay style="width:640px"></video>
<div id="hud" style="font:16px monospace; padding:8px"></div>

<script src="https://cdn.jsdelivr.net/npm/hls.js@1.6"></script>
<script>
  const video = document.getElementById("v");
  const hud = document.getElementById("hud");

  const hls = new Hls({ liveSyncDurationCount: 3 }); // default-ish: 3 segments back
  hls.loadSource("/live.m3u8");
  hls.attachMedia(video);

  setInterval(() => {
    // 1) hls.js's own PDT-derived distance from the live edge
    const reported = Number.isFinite(hls.latency) ? hls.latency.toFixed(1) : "n/a";

    // 2) manual check: wall clock minus the PDT of what's on screen
    let manual = "n/a";
    if (hls.playingDate) {
      manual = ((Date.now() - hls.playingDate.getTime()) / 1000).toFixed(1);
    }

    hud.textContent =
      `hls.latency: ${reported}s | manual PDT delta: ${manual}s | ` +
      `buffer: ${(video.buffered.length ? video.buffered.end(video.buffered.length - 1) - video.currentTime : 0).toFixed(1)}s`;
  }, 1000);
</script>

Open http://localhost:8000/player.html and you have three numbers updating live:

hls.latency: hls.js's estimate of how far you are behind the live edge, derived from PROGRAM-DATE-TIME.
manual PDT delta: the same idea computed by hand from hls.playingDate, so you can see there's no magic.
buffer: how much insurance the player is holding. Watch how it moves with the other two.

3. Now read the pixels 👀

Here's the payoff. Compare the burned-in UTC clock in the video against your machine's actual clock (date -u in a terminal next to the player):

date -u +%H:%M:%S   # run repeatedly, or: while true; do date -u; sleep 1; done

On my run, the burned-in clock trailed wall time by about 8 seconds while hls.latency said about 6. Both numbers are correct. They answer different questions:

The pixel clock measures glass-to-glass: everything from the frame being drawn, through encode, segmentation, and buffering, to render.
The PDT math measures from wherever the timestamp was stamped, which here is FFmpeg's segmenter. Encode-side delay upstream of the stamp is invisible to it.

⚠️ Note: this gap gets bigger in production. Hosted platforms typically stamp PROGRAM-DATE-TIME at ingest; Mux, for example, documents that its PDT-based latency metric reads about one second lower than true glass-to-glass for exactly this reason. Neither number is a lie. Know which one your dashboard shows.

💡 Tip: both methods assume clocks agree. If the encoder box and the viewer device disagree by two seconds of NTP drift, every number above silently absorbs it. On a single laptop you're safe; across real infrastructure, check chronyc tracking (or your platform's equivalent) before trusting sub-second readings.

4. Turn the knobs and watch the budget move 🎛️

The HUD makes latency cause-and-effect visible in a way no doc can. Try each of these and watch the numbers:

Segment duration: rerun with -hls_time 6 and -g 180. Latency roughly triples. This is the classic ~30s HLS default experience: three 6-second segments of sync distance plus buffer.
Player sync target: change liveSyncDurationCount: 3 to 1. hls.js parks you closer to the edge; rebuffer risk rises accordingly. That's the whole latency/stability trade in one line.
Pause for 30 seconds, then resume. You resume where you paused; your latency now includes your coffee. Real viewers do this constantly, which is why fleet-wide latency is a distribution, not a number.

Knob	Where it lives	Typical cost
GOP length	encoder	seconds, sets the floor for segment size
Segment duration x sync count	packager + player	the bulk of classic HLS latency
Buffer target	player	whatever it wants, silently
Viewer behavior (pause, join late)	humans	unbounded

5. Ship it as a beacon 📡

The HUD proves the math on one machine. The production version is the same three numbers sampled quietly and sent home, because fleet latency is a distribution and your desk is its most flattering sample:

// latency-beacon.js -- add next to the HUD code
function sampleLatency() {
  if (!Number.isFinite(hls.latency)) return;
  const payload = JSON.stringify({
    t: Date.now(),
    latency: +hls.latency.toFixed(2),
    target: hls.targetLatency ?? null,
    buffered: video.buffered.length
      ? +(video.buffered.end(video.buffered.length - 1) - video.currentTime).toFixed(2)
      : 0,
    // whatever session/stream IDs your analytics already use:
    sessionId: window.SESSION_ID,
  });
  navigator.sendBeacon("/telemetry/latency", payload);
}

setInterval(sampleLatency, 15_000);
window.addEventListener("pagehide", sampleLatency); // last word on the way out

sendBeacon survives tab closes and costs nothing on the render path. Fifteen-second sampling is plenty; you're charting a distribution, not tracing frames. On the backend, keep p50 and p95 per stream, and put them on the same dashboard as rebuffer rate so every latency conversation happens next to its trade-off.

💡 Tip: also record why a session drifted when you can. A paused: true flag on samples taken within a minute of a pause event separates "our pipeline got slower" from "viewers make coffee," and those need very different meetings.

What's next

Chart the percentiles. Once the beacon lands, p95 live latency across real viewers is the number to watch; it's where the spoiler complaints live.
Go low-latency for real. Partial segments and blocking playlist reload (LL-HLS) can bring the packager's share down to low single digits; the HUD you just built works unchanged and will tell you honestly whether your config delivers it.
Watch both curves. Any latency win you take, verify against rebuffer rate. They're the same dial, viewed from opposite sides.

The whole point of the HUD is that "we're about five seconds" stops being a vibe. It becomes a number you watched move.

DEV Community: Mason K

Build a conformance gate that skips the encode when the upload is already fine

TL;DR

1. One probe call, everything we need

2. The predicates, in the order they bite

3. The keyframe check (do not skip this one)

4. Wire the three actions

5. Put it together

6. Store the verdict on the asset

What's next

Add a DVR window to your live HLS stream without 404ing the scrubber

TL;DR

1. Configure the packager window

2. Sliding window vs event playlist

3. The check nobody runs: does storage agree?

4. Read the window in hls.js

5. A seek bar that tracks a moving target

6. The go-live button (this is the one that stalls)

7. Handle the window sliding out from under a paused viewer

Wrapping up

Build a video QC gate that catches black frames before your viewers do

TL;DR

1. Tier 1: structural checks, no decoding 🔍

2. Tier 2: three detectors, one decode pass

3. The rules layer (this is the actual product) ⚖️

4. Tier 3: sampled reference comparison

5. Wire it into the pipeline

6. Store the verdict with its rules version

What's next

Build an audio description pipeline that actually fits the gaps

TL;DR

What we're building

1. Find the gaps 🔍

2. Turn duration into a word budget 📏

3. Sample frames for visual context 🖼️

4. Generate to the budget, not to a quality bar

5. Render, then verify the duration 🎙️

6. Mix and package as a second audio track 🔊

7. Monitor it, or you'll lose it 📡

What's next

Write a portability audit for your video provider before you need one

TL;DR

What we're checking

1. Project setup 🛠️

2. The adapter interface

3. The FastPix adapter

4. Feeding it media IDs

5. The runner, with concurrency and retries

6. The ID sprawl check 🔍

Porting the adapter to other providers

What's next

Add casting to a custom HTML5 video player without breaking it

TL;DR

1. The baseline player 🎥

2. Feature detection, three ways

3. Wiring the Remote Playback API

4. The state machine (this is the bug you'll ship) ⚠️

5. AirPlay, Safari's way 🍎

6. Now fix the four bugs that only happen on real hardware

Token expiry on the wrong clock

CORS from an origin you forgot exists

Anything the TV can't resolve

The receiver's ABR isn't yours

7. When to just turn it off

Wrapping up

Check whether hls.js is actually using a worker (most ESM setups aren't)

TL;DR

1. Find out what you are currently running 🔎

2. Wire up workerPath 🛠️

3. Add a long-task observer so stalls stop being ambiguous 📊

4. Reproduce it on purpose 🧪

5. What the worker still does not fix

What's next

Build a React Native video upload that survives backgrounding, app kills, and cellular

TL;DR

The failure we are fixing 📱

1. Know what the platforms will and will not do

2. Persist the upload before any bytes move 💾

3. Hand the transfer to a native background session 🚚

4. Reconcile on relaunch, and let the server win 🔄

2. Wire up `workerPath` 🛠️