ffmpeg.wasm: "memory access out of bounds" is usually not about memory (and 4 more slideshow gotchas)

I spent three iterations getting a browser-only "photos to video" tool working on top of ffmpeg.wasm. Everything runs client-side, nothing is uploaded. Most of the time went into five things that the docs and error messages don't tell you. Sharing them in case they save you an evening.

If you want to see the result, it is a free photo slideshow maker that builds the MP4 in your tab.

1. memory access out of bounds is just how an abort looks from outside

When the ffmpeg core crashes, the error that reaches JavaScript is RuntimeError: memory access out of bounds. It looks like an out-of-memory problem. In my case it never was. Any abort inside the core is reported this way.

The real reason is only in the engine's own log. Attach a log handler before you run anything:

ffmpeg.on('log', ({ message }) => console.debug('[ffmpeg]', message))

Without that, you are guessing. With it, the next two items took minutes instead of hours.

2. The image2 muxer needs -update 1 for a single image

If you write one frame to disk (for example to extract a poster), image2 treats the output name as a sequence pattern and fails. Add -update 1:

ffmpeg -i in.mp4 -frames:v 1 -update 1 poster.jpg

3. Prepare frames on a canvas, not in ffmpeg

My first version pushed every photo through ffmpeg for scaling and letterboxing. After fixing 1 and 2 the core still crashed while writing images. Moving frame preparation to canvas turned out to be better on every axis:

• one ffmpeg command instead of N+1;
• noticeably faster;
• createImageBitmap(file, { imageOrientation: 'from-image' }) applies the EXIF rotation for you, so phone photos are not sideways;
• every frame has the same pixel format, so the concat step needs one codec only.

4. The "repeat the last file" trick can hurt

A classic recipe against a truncated final frame in the concat demuxer is to list the last file twice. In this build it gave the last photo double time: three frames at 2 s came out as 8 s instead of 6 s. Without the duplicate I measured 5.97 s and the last frame was held fully. Measure your own build before you copy a recipe.

5. Audio: apad plus -shortest, not just -shortest

If you add a music track, plain -shortest cuts the whole video to the length of the track (a 6 s slideshow with 3 s of music becomes 3 s). Pad the audio with silence and then let -shortest trim a long track to the video:

-af apad -shortest

Bonus: remux instead of re-encode when you can

For a format converter the biggest win was skipping the encode. When the source codec fits the target container, -c copy only rewrites the container. In my measurements a remux took about 0.14 s against 1.41 s for a re-encode on a small clip, and in the deployed video format converter it is roughly 0.6-0.7 s against 2.9-5.6 s. I still re-encode HEVC on purpose, because an MP4 with HEVC does not open everywhere.

What trips you up most with ffmpeg.wasm? I am curious whether others saw the same "out of bounds" masking.