Three things that broke when I moved video compression into the browser

I run a video compressor that works entirely in the browser.
No upload, no server, nothing leaves the machine. Two of the three bugs below
only showed up when I stopped reading code and started timing things, so I want
to write them down while the numbers are still in front of me.

The pipeline has three paths. If the file already meets the target, copy the
encoded samples and encode nothing. If the browser can decode the container,
transcode with WebCodecs. Otherwise fall back to ffmpeg.wasm, which is roughly
an order of magnitude slower. mediabunny handles the demux and remux.

AVI to Matroska with -c copy produces a 1151 byte file

The fast path needs mediabunny to be able to read the container. It cannot read
AVI, but plenty of AVI files carry H.264 inside, so the plan was to rewrap
losslessly and stay on the fast path:

ffmpeg -i in.avi -map 0:v:0 -map 0:a? -c copy \
       -avoid_negative_ts make_zero out.mkv

That fails. Not slowly, not subtly:

[matroska] Timestamps are unset in a packet for stream 0
[matroska] Can't write packet with unknown timestamp
[out#0/matroska] Error muxing a packet

AVI has no per-packet timestamps. It stores a fixed frame rate plus an index and
lets the demuxer work them out. Matroska requires timestamps on every block, so
the copy has nothing to write. What comes out is a 1151 byte header with zero
clusters, which is a valid Matroska file containing no media.

-avoid_negative_ts does not help, and it took me longer than it should have to
see why: the timestamps are not negative, they are absent. Two different
problems that produce similar looking errors.

The fix is one flag on the input side:

ffmpeg -fflags +genpts -i in.avi -map 0:v:0 -map 0:a? -c copy \
       -avoid_negative_ts make_zero out.mkv

+genpts synthesises presentation timestamps from the stream's frame rate.
Output went from 1151 bytes to 55.8 MB on a 20 second 1080p30 clip, all 600
frames decoding cleanly.

Then I measured it against a control. Same geometry, same duration, same target,
but Xvid instead of H.264 so it takes the ffmpeg path:

H.264 AVI, rewrap then WebCodecs   7.5 s    53.3 MB to 12.3 MB
Xvid AVI, ffmpeg.wasm             53.4 s    48.2 MB to 12.5 MB

Output sizes within 2 percent, wall clock 7x apart. Both numbers come from
headless Chromium with software encoding, so a real machine with a hardware
encoder should do better than this.

One more thing worth doing: ffmpeg.wasm resolves with the exit code instead of
rejecting, and a muxer that dies mid run still leaves a readable file behind. A
truncated remux looks exactly like a complete one unless you check.

Every AAC track defeats mediabunny's copy path, by design

The passthrough case is supposed to touch nothing. Give mediabunny an empty
video config and it copies encoded samples straight through, since
forceTranscode defaults to false.

It worked for video and not for audio. A 3.3 MB source came back at 3.6 MB, so
I parsed the output boxes:

source   moov 33,948   mdat 3,432,287
output   moov 17,879   mdat 3,788,908

The moov got smaller. The growth was all in mdat. Per track:

avc1   900 samples   3,071,520 bytes
mp4a  1295 samples     717,380 bytes

Video was byte identical to the source. Audio was double: 717 KB against the
source's 360 KB, about 191 kbps from a 96 kbps original.

The reason is in mediabunny's copy conditions. The fast path requires, among
other things, !needsTrimming, where needsTrimming is
firstTimestamp < startTimestamp. I asked the library what it saw:

video codec: avc   firstTimestamp: 0
audio codec: aac   firstTimestamp: -0.023219954648526078

Negative. And 0.0232 seconds at 44100 Hz is 1024 samples, which is exactly one
AAC frame. That is encoder priming delay, and every AAC track produced by any
normal encoder has it. So the audio copy path is not occasionally unavailable,
it is never available.

You cannot fix this from the outside. Passing a bitrate to keep the size down
forces a transcode by itself, because !trackOptions.bitrate is one of the
copy conditions. Passing nothing lets the library re-encode at its own default.

I went with a floor instead: if a passthrough would return more bytes than it
received, and the source is already MP4, hand back the source untouched.
Verified byte exact, 3,466,275 in and 3,466,275 out where it used to be
3,806,815. For a file that was already close to optimal, "nothing needed doing"
is a more honest answer than a 10 percent larger file.

The multithreaded ffmpeg core costs you your ad revenue

@ffmpeg/core-mt is 4 to 8 times faster than the single threaded core. It needs
SharedArrayBuffer, which needs cross origin isolation, which means sending
COOP and COEP headers.

COEP: require-corp breaks third party embeds that do not opt in. On a site
funded by AdSense that is not a technical tradeoff, it is a revenue decision.
I stayed single threaded and put the effort into not reaching for ffmpeg at all,
which is what the rewrap path above is for.

What I would take away from this

Both real bugs were invisible to code review and to unit tests. The AVI failure
had a passing test suite and a plausible looking implementation sitting on top
of it. What found it was dragging an actual AVI file into an actual browser and
noticing that a 54 MB input had produced 1151 bytes.

If you want to try it on something awkward, the two cases I get asked about most
have their own pages: compress video for Discord
if you are fighting the 10 MB free tier limit, and
compress video to 10 MB if the
cap is the whole problem. Both run on your own machine.