iOS Safari can't decode your .mov, and the reason is 2 bytes deep in the container

Our tool transcribes audio in the browser — Whisper running locally via transformers.js, no upload. It worked fine, until analytics showed something too clean to be a coincidence:

.mov uploads on mobile failed 100% of the time. Not 90%. Every single one. Desktop had never reported a single .mov failure.

This is what I found, and how it got fixed without pulling in ffmpeg.wasm or WebCodecs.

The 30-second reproduction

I took one AAC audio track and put it in two containers — same encoder, same bytes for the audio itself, only the wrapper differs:

const buf = await file.arrayBuffer();
await new AudioContext().decodeAudioData(buf);

On an iPhone 17 Pro simulator (iOS 18.7 / Safari 26.5):

File	iOS Safari	Chromium
sample.mov (ftyp qt)	EncodingError: Decoding failed	OK
sample.mp4 (ftyp isom)	OK	OK

So it isn't the codec. It's the container.

The obvious fix that doesn't work

First instinct: it's the brand in the ftyp box. Patch qt → isom, four bytes, done.

It still fails. I'm writing this down so nobody else burns an afternoon on it. The ftyp brand is not what Safari looks at. The difference lives inside moov.

The actual root cause

Dig down to moov → trak → mdia → minf → stbl → stsd — the sample description that tells the decoder how the audio is encoded. Both files carry an mp4a entry. They are not the same mp4a entry:

QuickTime writes:                  MP4 expects:
  version       = 1        <—        version       = 0
  compressionID = -2 (fffe) <—       compressionID = 0
  + 16 bytes of v1 extension <—      (absent)
  esds wrapped in a 'wave' box <—    esds is a direct child
  extra 'chan' channel layout        (absent)

iOS Safari's decodeAudioData only accepts a version 0 audio sample entry. Chromium accepts both — which is exactly why desktop never saw this and mobile never survived it.

That version field is a uint16. Two bytes decide whether the file plays.

The fix: rebuild the container, don't touch the codec

Since the audio bitstream is already valid AAC, nothing needs to be re-encoded. The job is pure byte plumbing: extract the audio samples, write a fresh audio-only MP4 with a well-formed stsd, and hand it back to the browser's native decodeAudioData.

That last part matters. Because we never decode audio ourselves, we don't need WebCodecs — and therefore don't inherit AudioDecoder's iOS 17+ floor — and we don't ship a multi-megabyte ffmpeg.wasm to fix a container-level problem.

.mov ──► scan top-level boxes to find moov (read box headers only, never touch mdat)
      ──► parse the audio track's stsd / stts / stsc / stsz / stco|co64
      ──► sliding window (8 MB) to copy out audio samples
      ──► rebuild ftyp + moov (stsd forced to version 0) + mdat
      ──► decodeAudioData

The sample entry we emit is deliberately boring:

box('mp4a',
  zeros(6),
  u16(1),  // data_reference_index
  u16(0),  // version — the only one iOS Safari accepts
  u16(0),  // revision
  u32(0),  // vendor
  u16(channels),
  u16(16), // samplesize
  u16(0),  // compressionID — QuickTime writes -2; MP4 says 0
  u16(0),  // packetsize
  u32(Math.min(sampleRate, 65535) * 65536), // 16.16 fixed point
  esds     // pulled out of QuickTime's 'wave' wrapper if needed
)

One parsing subtlety worth stealing: the esds descriptor sits as a direct child of mp4a in MP4, but QuickTime buries it inside a wave box. So the lookup scans all siblings first, then recurses — otherwise a wave-nested esds can shadow the one you actually want.

Version 1 entries also carry 16 extra bytes before their child boxes, and version 2 carries 36 with the real sample rate in a float64. Get those offsets wrong and you'll parse garbage as a box header:

const version = view.getUint16(base + 8);
let childStart = base + 28;
if (version === 1) childStart = base + 28 + 16;
else if (version === 2) { sampleRate = view.getFloat64(base + 32); childStart = base + 64; }

The bug we weren't looking for

The old code started with this:

const buf = await file.arrayBuffer();  // 161 MB average on mobile

Mobile video files in our logs average 161.9 MB — and 99% of those bytes are video frames we don't need for a transcript. Reading the whole file into memory, then decoding it into an equally long PCM buffer, is a great way to get killed by iOS Safari's memory limits.

Locating boxes by reading only their 8–16 byte headers, then copying samples through an 8 MB sliding window, changes the peak from whole file to 8 MB + the audio itself:

File	Native decode	After extraction	Extract time	Decode after
sample.mov (0.10 MB / 5s)	FAIL	0.08 MB	16 ms	OK
hevc.mov (HEVC video + AAC)	FAIL	0.12 MB	4 ms	OK
huge5.mov (158.6 MB / 900s)	FAIL	13.88 MB	378 ms	OK (476 ms)
audio.m4a	OK	0.08 MB	3 ms	OK (no regression)
sample.mp4	OK	0.08 MB	3 ms	OK (no regression)

The huge5.mov row is the one that mattered: 158.6 MB is the real-world mobile average, the old path read all of it and still threw EncodingError, the new path produces a decoded 15-minute audio buffer in 854 ms total. Decoded output was verified non-silent (peak > 0) and length-accurate (900.023s → 900.1s), because "it returned a buffer" is not the same as "it worked".

Fail backwards, never sideways

Every step returns null the moment reality stops matching the assumptions, and the caller falls back to the original direct-decode path:

Input	Result
Fragmented MP4 (empty_moov)	null — sample tables live in moof, assumption broken
PCM audio track in .mov	null — only mp4a is handled
.webm	never enters this path
.mov with no audio track	null — no soun track found

This is the rule that kept the change safe to ship: a module that exists to rescue guaranteed failures must never make a working case worse. Every null is a case where the old path was already going to run anyway.

What I'd take away from this

• When a format works in one browser and not another, suspect the container before the codec. Codec support is well-documented; container tolerance is not.
• ffmpeg.wasm is not the default answer. If the bitstream is already valid, you have a plumbing problem, and plumbing is a few hundred lines of DataView.
• Reading a whole file to decode it is a bug on mobile, even when it doesn't crash today.
• And afterwards: delete the warning banner. We used to tell mobile users "large .mov files may fail". Leaving that up after the fix would be worse than never having shown it — the file size no longer correlates with any real risk, because we only ever move the audio bytes now.

---

This shipped in TranscriptSnap, a browser-local transcription tool — Whisper runs on your machine via transformers.js, files never leave the device, which is also why a container quirk in Safari became our problem instead of a server's. The full write-up with the on-device measurements lives here.

Comments

[Kane Lim]

your post is interesting
I would like to get to know you better and discuss about your post. Would you please contact me? t_g_@kanelim1997

[Deland]

Thanks for reading. Let's keep it in this thread though - that way anyone else who hits the same .mov failure finds the answer too. Which part are you curious about, the stsd parsing or the sliding-window extraction?

[Kane Lim]

stsd parsing