A short video can decode correctly while its captions are wrong. The useful engineering split is to check file identity, timeline boundaries, audio preservation and visual behavior separately. Here is one concrete 24 fps caption pipeline, including the limits of its checks.
Generated with AI from Pawpi’s caption code and production records.
Keep frame coordinates explicit
The examined vertical clip retains source frames [94, 475), using an exclusive end. Its length is 475 − 94 = 381 frames, or 15.875 seconds. The first retained story frame becomes output frame zero. An opening of 94 frames at 24 fps is 3.916666… seconds; rounding that to 3.9 before rebasing introduces avoidable error.
For a cue wholly inside this single continuous retained interval:
source_start = 94
source_end_exclusive = 475
short_frames = source_end_exclusive - source_start
assert short_frames == 381
def rebase_frame(source_frame):
if not source_start <= source_frame < source_end_exclusive:
raise ValueError("Frame is outside the retained range")
return source_frame - source_start
assert rebase_frame(124) == 30
This is a small illustrative calculation, not the full production editor. Cues crossing a cut need explicit trimming or resegmentation. Multiple retained ranges need a piecewise map. Do not apply this one subtraction indiscriminately.
Preserve measured word positions
The first cue in the actual timing file has an output interval [30, 59). Its four word onsets are 30, 35, 42 and 47. The source metadata retains alignment methods and source timestamps. The renderer requires nonempty word timing, matching displayed tokens, ordered onsets and speech ends within each cue. It also refuses to silently shrink a phrase beyond its two-line layout limit.
The existing implementation uses Python assertions for these guards. That is a limitation: Python optimization can disable assertions. A production invocation must not enable optimization; a reusable validator should use explicit exceptions for input validation. These records and guards do not prove that the alignment sounds correct.
Compare audio separately from the MP4
The clean and captioned video necessarily have different whole-file hashes. To check that adding captions preserved the compressed audio payload, the existing renderer copies the audio stream and hashes the selected audio stream with FFmpeg:
ffmpeg -v error -i clean.mp4 -map 0:a:0 -c copy -f hash -hash sha256 -
ffmpeg -v error -i captioned.mp4 -map 0:a:0 -c copy -f hash -hash sha256 -
Here, clean.mp4 and captioned.mp4 are descriptive placeholders for the two actual files. Rerunning these commands on the project files produced the same audio hash:
SHA256=c800875ad8cd4e355a2b1c11fba320cc36f0faa88009a20d1c9a87242e4a8399
That supports unchanged selected audio bytes across these two files. It does not prove perfect audiovisual synchronization, source-to-master editing accuracy, or identical container timestamps.
Decode and inspect the resulting stream
The captioned file also passed a complete decode:
ffmpeg -v error -i captioned.mp4 -f null -
ffprobe -v error -select_streams v:0 -show_entries stream=width,height,nb_frames,avg_frame_rate,duration -of json captioned.mp4
The observed video was 1080×1920 at 24/1 fps, with 381 frames and a 15.875-second duration. The timing file contained eight cues and 28 words. File hashes for the clean clip, captioned clip, timing JSON and SRT matched their existing manifest entries.
The renderer uses a transparent caption strip, reveals each word at its onset, and positions the whole phrase consistently. It quantizes overlay time to the nearest frame before choosing a cue-specific vertical position; this avoids a decimal-time comparison choosing the default position at a boundary.
Keep the remaining review visible
A successful decode and matching hashes cannot tell you whether text covers an eye or whether a word appears when it is heard. Frame samples were inspected for this note; a new complete listening review was not performed. The checks support reproducibility and audio preservation, while motion, placement and listening remain distinct review tasks. No performance improvement or production-time saving is claimed.
References: FFmpeg stream selection and stream copy, FFmpeg hash muxer, Python assert statement.
Top comments (1)
Checking file identity separately from timeline boundaries is the right split — a captioned MP4 will never hash-equal the clean one, and an audio-payload check won't catch a word that's early by two frames. Two practical hardening bits: don't leave those timing guards as Python
asserts (as you noted,-Owill swallow them), and visually sample word-onsets around each retained-range boundary, because a piecewise map that's off by one frame only shows up there. Disclosure: I build Vidmoat, a hosted AI video editor where captions live as text clips on a real timeline you can still adjust before the MP4, so I care about the same "the file is not the review" idea. The exclusive-end frame math in the post is the kind of thing that's easy to get almost right.