Media processing in web applications used to be a clear backend responsibility. If you needed to trim audio, convert formats, or extract metadata, you installed FFmpeg on a server and moved on. With WebAssembly becoming mature, that boundary has started to blur. Libraries like ffmpeg.wasm now allow FFmpeg to run directly in the browser, which sounds ideal but only until you try to integrate it into a real React application with a custom build system.
This post is not based on demos or documentation alone. It reflects what actually happens when you try to use FFmpeg and ffmpeg.wasm with React 18 and a custom Gulp setup, and the kinds of issues you only discover once things break.
Native FFmpeg vs ffmpeg.wasm: Same API, Different Reality
Native FFmpeg runs outside the browser. It has full access to disk, memory, and CPU, and it supports streaming inputs naturally. In Node.js, it usually looks something like this:
import { exec } from "child_process";
exec(
"ffmpeg -i input.mp3 -acodec pcm_s16le output.wav",
(err) => {
if (err) {
console.error("FFmpeg failed", err);
}
}
);
This approach is fast, predictable, and boring in the best possible way.
ffmpeg.wasm, however, runs inside the browser sandbox. There is no filesystem in the traditional sense, no streaming, and everything happens in memory. Even though the API feels familiar, the constraints are very different and those constraints surface quickly in real apps.
Why I Tried ffmpeg.wasm in the First Place
I was working on a project where video and audio compression had to be done on the client side. The reason was simple: media files were small (maximum around 2 MB), which made browser-side processing feasible.
The goal was to process media directly in the browser. Audio and video files were fetched from an API, transformed on the client, and then played back or exported without sending them back to the server. This reduced backend load and avoided privacy concerns.
The React app already used React 18, but the build system was Gulp-based instead of Webpack or Vite, mainly because the web app was also used inside an Electron setup. That single detail turned out to be one of the biggest sources of friction.
Gulp Does Not Handle WASM Automatically
Most ffmpeg.wasm examples assume your bundler knows how to load WebAssembly and workers (Create React App or Vite). Older Gulp setups, especially legacy ones which do not.
My first attempt resulted in FFmpeg failing silently, with the worker never initializing.
The fix was manual but unavoidable. I had to explicitly copy the FFmpeg core files into the build output.
// gulpfile.js
gulp.src("node_modules/@ffmpeg/core/dist/*")
.pipe(gulp.dest("dist/ffmpeg"));
Once copied, FFmpeg had to be loaded using absolute paths instead of relying on bundler magic.
import { FFmpeg } from "@ffmpeg/ffmpeg";
const ffmpeg = new FFmpeg();
await ffmpeg.load({
corePath: "/ffmpeg/ffmpeg-core.js",
wasmPath: "/ffmpeg/ffmpeg-core.wasm",
});
At this point, FFmpeg finally loaded but it still wasn’t stable.
Cross-Origin Isolation: The Invisible Requirement
The next issue was harder to diagnose. FFmpeg would sometimes work, sometimes crash, and sometimes fail with errors related to SharedArrayBuffer. The root cause had nothing to do with React or Gulp.
ffmpeg.wasm requires cross-origin isolation.
Without these headers, browser memory features required by FFmpeg are unavailable:
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Once these headers were added at the proxy or server level, FFmpeg stopped failing randomly and started behaving consistently. This requirement alone makes ffmpeg.wasm unsuitable for environments where you don’t control response headers.
Working with Files in Browser Memory
Unlike native FFmpeg, ffmpeg.wasm does not stream files. Everything is written into an in-memory filesystem. Even for small files, this feels unusual at first.
Fetching a file and writing it into FFmpeg looks like this:
const response = await fetch(audioUrl);
const buffer = await response.arrayBuffer();
await ffmpeg.writeFile("input.mp3", new Uint8Array(buffer));
Running FFmpeg commands is familiar, but the output also stays in memory:
await ffmpeg.exec([
"-i", "input.mp3",
"-ac", "1",
"output.wav"
]);
const output = await ffmpeg.readFile("output.wav");
const blob = new Blob([output.buffer], { type: "audio/wav" });
const url = URL.createObjectURL(blob);
This works well for short audio clips. As files grow larger, memory pressure increases rapidly.
Browser Memory Limits Are Not Theoretical
Once file sizes crossed a certain threshold, the browser tab would freeze or crash. This wasn’t a bug in the code—it was a limitation of running FFmpeg inside a browser sandbox. Everything happens in memory, and there is no efficient way to process large files incrementally.
This was the point where it became clear that ffmpeg.wasm is best suited for lightweight processing, not heavy media workloads.
React 18 and the Double Initialization Trap
React 18's Strict Mode intentionally runs effects twice in development. This exposed another subtle issue: FFmpeg was being initialized twice, causing worker conflicts and inconsistent state.
The fix was to ensure FFmpeg was created only once, outside React's normal render flow.
const ffmpegRef = useRef<FFmpeg | null>(null);
if (!ffmpegRef.current) {
ffmpegRef.current = new FFmpeg();
}
By treating FFmpeg more like a singleton runtime than a UI dependency, the instability disappeared.
When ffmpeg.wasm Is the Right Tool
After working through these issues, the strengths of ffmpeg.wasm became clear. It works well when files are small, when privacy matters, and when you want instant client-side feedback. Tasks like audio trimming, format conversion, and preview generation are good fits.
It is not designed for large videos, long-running jobs, or anything that resembles a production transcoding pipeline.
Why Native FFmpeg Still Dominates Serious Workloads
Native FFmpeg remains the better choice for anything heavy or critical. It handles large files without memory issues, supports streaming inputs, and runs independently of the UI. In production systems, it is still the most reliable option.
In practice, the best architecture is often hybrid: use ffmpeg.wasm for small, interactive client-side tasks, and fall back to native FFmpeg on the backend for real processing.
Final Thoughts
Using FFmpeg in modern React applications is less about the API and more about understanding the environment. Build tools, browser security policies, memory constraints, and React's lifecycle all shape what is possible.
With a custom Gulp setup, you should expect manual configuration. It is also important to ensure that legacy projects keep their Gulp and Node.js versions updated so that modern JavaScript features used by newer libraries are supported.
With ffmpeg.wasm, you should expect limits. And when reliability matters, native FFmpeg is still the safest choice.
Client-side media processing is powerful but only when you choose the right tool for the right job.
Top comments (1)
This was insightful. God bless developers who are still building for legacy setups 🙏