Processing Video in the Browser with ffmpeg.wasm: What Works and What Doesn't

Running FFmpeg inside the browser sounds like a dream. No upload, no server bill, no queue. The user picks a file, your page compresses it, done.

ffmpeg.wasm makes this possible, and for small jobs it works well. But when I pushed it toward real-world video files, I hit a series of walls. This post covers a minimal working example, the limits you should expect, and a simple way to decide between client-side and server-side processing.

Why process video in the browser at all?

There are good reasons:

• Privacy: the file never leaves the user's device.
• Cost: your servers don't pay for CPU time or bandwidth.
• Latency for small files: no upload round trip.

And there are good reasons not to, which we'll get to.

A minimal example

Install the packages (this uses the 0.12 API):

npm install @ffmpeg/ffmpeg @ffmpeg/util

Then load the core and run a compression command:

import { FFmpeg } from "@ffmpeg/ffmpeg";
import { fetchFile, toBlobURL } from "@ffmpeg/util";

const ffmpeg = new FFmpeg();

async function load() {
  const baseURL = "https://unpkg.com/@ffmpeg/core@0.12.6/dist/umd";
  await ffmpeg.load({
    coreURL: await toBlobURL(`${baseURL}/ffmpeg-core.js`, "text/javascript"),
    wasmURL: await toBlobURL(`${baseURL}/ffmpeg-core.wasm`, "application/wasm"),
  });
}

async function compress(file) {
  await ffmpeg.writeFile("input.mp4", await fetchFile(file));

  await ffmpeg.exec([
    "-i", "input.mp4",
    "-c:v", "libx264",
    "-crf", "28",
    "-preset", "veryfast",
    "-c:a", "aac",
    "-b:a", "96k",
    "output.mp4",
  ]);

  const data = await ffmpeg.readFile("output.mp4");
  return new Blob([data.buffer], { type: "video/mp4" });
}

Hook it up to a file input, call load() once, then compress(file) for each upload. You can also listen for progress:

ffmpeg.on("progress", ({ progress }) => {
  console.log(`${Math.round(progress * 100)}%`);
});

The commands are the same FFmpeg arguments you'd use on the command line, so everything you already know about CRF and presets carries over.

What works well

• Short clips and small files. Screen recordings of a few seconds to a minute, short social clips, and previews are realistic.
• Simple operations. Trimming, extracting audio, converting between common formats, and light compression.
• Privacy-sensitive tools. When "your file never leaves your device" is a selling point, this is a strong fit.

What doesn't work well

1. Memory

The whole input file has to be written into the in-browser virtual file system, and the output has to live there too, on top of whatever FFmpeg needs to process frames. Large inputs can exhaust memory or hit WebAssembly limits, and the failure mode is often a tab crash or an opaque error. The project's own documentation notes a file size ceiling in the low gigabytes, so multi-gigabyte videos are not a realistic target.

2. Speed

WebAssembly is fast, but it's not native FFmpeg. Single-threaded builds are noticeably slower than a local install, and libx264 at slower presets can take a long time even on short videos. A few practical consequences:

• Use faster presets (veryfast, superfast) if you must run in the browser.
• Show real progress to the user, or they will assume the page froze.
• Expect mobile devices to struggle more than desktops.

3. Multi-threading needs special headers

The multi-threaded core relies on SharedArrayBuffer, which browsers only expose on cross-origin isolated pages. That means your server must send:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

These headers can break third-party embeds, analytics scripts, and iframes on the same page. For a standalone tool page that's fine. For a page inside a larger site, it can be painful.

4. Bundle and startup cost

The WASM core is several megabytes. Users pay that download cost before the first operation. Load it lazily, only when the user actually starts a job, and cache it.

5. Codec and feature coverage

The default builds don't include every codec and filter you'd have in a full native FFmpeg. If you need something specific (certain hardware-oriented encoders, niche filters), check what the core ships with before committing.

Test it on your own files

Before you decide, benchmark it with files that look like your users' files, not toy samples. A simple test matrix I'd run:

File	Size	Resolution	Time in browser	Time with native FFmpeg
Short clip	~20 MB	1080p	measure	measure
Medium video	~200 MB	1080p	measure	measure
Large video	~1 GB+	4K	measure	measure

[Fill this in with your own numbers. The shape of the results matters more than any single figure: browser time grows quickly with file size, and eventually the job fails.]

Client-side vs. server-side: a decision guide

Question	Lean client-side	Lean server-side
Typical file size	Under a few hundred MB	Large, or unpredictable
Privacy requirement	Files must not leave the device	Acceptable to upload
User devices	Modern desktops	Mobile, low-end devices
Throughput	Occasional use	Frequent or batch jobs
Speed expectations	Patient users	Users expect fast results
Codec needs	Common formats	Anything FFmpeg supports
Page constraints	Standalone tool page	Embedded in a complex site

A hybrid approach also works well: try client-side for small files, and fall back to a server for anything above a threshold.

What about the server side?

If you decide to go server-side, you have a few options:

1. Run FFmpeg yourself on a worker queue (containers, background jobs, object storage for uploads). You get full control, but also the ops burden: scaling, timeouts, disk cleanup, security around untrusted input files.
2. Use a managed video API. Less ops, but usage-based pricing.
3. Don't build it at all. If your users just need to shrink or convert a file, point them to an existing tool. For example, VideoBee's video compressor runs server-side and supports large single files (up to 10 GB), and there's a video converter for format changes. (Full disclosure: I work on VideoBee.)

For a product where video processing is the core feature, you'll likely end up building your own pipeline. For a product where it's a side feature, buying or linking out is usually cheaper than maintaining it.

Takeaways

• ffmpeg.wasm is great for small, privacy-sensitive, occasional jobs.
• It struggles with large files, mobile devices, and heavy encoding.
• Multi-threading requires cross-origin isolation headers, which can affect the rest of your page.
• Benchmark with realistic files before you commit.
• For big files, move processing to a server or use an existing service.

Have you shipped something with ffmpeg.wasm? I'd love to hear where you hit limits, especially on mobile. Let me know in the comments.