Handling Files Over 2GB with WebAssembly in the Browser

At RedPandaCompress people routinely drop 4-8GB video files into the browser and expect them to compress or convert without ever touching a server. No upload, no queue, no "your file is processing, check back later" email. Everything happens client-side, in WebAssembly.

The catch: WebAssembly was never really designed for that.

The wall you hit first

A wasm module's linear memory is one contiguous, growable ArrayBuffer. In practice you hit trouble long before the theoretical 4GB (32-bit) ceiling:

• Every growth step needs a new buffer big enough to hold the old contents plus the increase, allocated contiguously, before the old one is freed. That doubles your peak requirement right at the moment you're already low on headroom.
• Browsers cap single ArrayBuffer allocations well under 4GB in practice, and you'll see it as a plain RangeError: Array buffer allocation failed — not an out-of-memory crash, just a hard refusal.
• None of this is optional if your naive approach is "read the whole file, hand the bytes to the wasm module." A 3GB input file, loaded into a wasm-visible buffer, is already most of the way to that wall — before you've decoded a single frame.

So the fix isn't "increase the memory limit." It's making sure the whole file never has to live in one contiguous wasm-addressable buffer in the first place — at any stage: input, processing, or output.

Input: don't copy the file in, mount it

The obvious approach — read the File object into an ArrayBuffer with file.arrayBuffer() and write it into the wasm filesystem (MEMFS) — means the entire input now exists twice: once as a JS ArrayBuffer, once copied into the wasm heap. For an 8GB file that's an 8GB tax before any real work starts.

ffmpeg.wasm (and Emscripten's FS layer generally) supports mounting a filesystem backed directly by a Blob/File, instead of copying bytes into MEMFS:

await ffmpeg.createDir("/data");
await ffmpeg.mount(FFFSType.WORKERFS, { blobs: [{ name: "input.mp4", data: file }] }, "/data");

await ffmpeg.exec(["-i", "/data/input.mp4", "-c:v", "copy", "output.mp4"]);

await ffmpeg.unmount("/data");

WORKERFS reads lazily, straight from the browser's own File/Blob (which is typically backed by disk, not RAM) whenever ffmpeg actually asks for a byte range. The file is never copied into the wasm heap wholesale. This one change is what makes "8GB input" a non-event instead of an immediate wall.

Output: don't collect it in memory either

The other direction has the same trap. If ffmpeg's output lands in wasm's MEMFS and you then read it out as a single Uint8Array to build a Blob, you've reintroduced the exact problem you just solved on the input side — the whole output now has to exist as one contiguous buffer before you can hand it back to the page.

The fix is symmetric: stream it out in chunks instead of pulling one giant buffer at the end. Read the output progressively and assemble it as Blob parts:

const chunks = [];
const reader = response.body.getReader();
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  chunks.push(value);
}
const blob = new Blob(chunks, { type: "video/mp4" });

Chrome (and most modern browsers) can spill large Blobs to disk rather than holding them entirely in RAM, so building the result as many small parts — instead of one ArrayBuffer the size of the whole output — sidesteps the same allocation ceiling on the way out.

For genuinely huge jobs: bound the work, not just the memory

Even with lazy input and streamed output, a single ffmpeg invocation over the entire file still has to hold its own working state proportional to what it's processing. For very large compress jobs we split the source into bounded segments, run each one through its own short-lived ffmpeg pass, and concatenate the results — rather than asking one wasm instance to eat the whole file in one go. It's the same principle one level up: cap how much any single unit of work has to hold in memory at once, regardless of how large the total job is.

The takeaway

There's no single trick that makes ">2GB in WebAssembly" work. It's the same discipline applied at every boundary: never let the whole file — input, intermediate state, or output — become one contiguous thing that has to live entirely inside wasm-addressable memory at once. Mount instead of copy. Stream instead of collect. Segment instead of one giant pass.

This is the first of what I expect will be a few posts on the weirder corners of shipping a real, no-upload media tool as a client-side wasm app — happy to go deeper on any of this if people are curious.