DEV Community: Tutu

ffmpeg.wasm to WebCodecs: the 12x Gap, What Broke, and What Still Can't Move

Tutu — Fri, 23 Oct 2026 15:03:44 +0000

When we built the local-processing mode of our media toolbox (meicut.com) — 20+ tools that trim, compress, convert, and filter video and audio entirely in the browser — the first version bet everything on ffmpeg.wasm. It felt like the obvious choice. Within weeks, the hot paths moved to WebCodecs hardware encoding, ffmpeg.wasm survives as a fallback plus the tools that sit at hard boundaries — audio processing, speed change, AVI/GIF output, reverse — and we have a precise map of where each technology's real boundaries are.

This is the full story: why we started with WASM, the three technology-inherent problems that forced the migration, how the WebCodecs pipeline works, and the decision framework we now use to route each tool.

Why we chose ffmpeg.wasm first (and it was the right call)

Our backend already processed everything server-side with the ffmpeg CLI. That made ffmpeg.wasm almost unfairly attractive:

1. Zero redesign of processing semantics. Our server code was full of carefully-tuned filter graphs — -ss/-t input-side seeking for cuts, even-dimension rounding for crops, maskedmerge + geq three-plane masks for mosaic regions. With ffmpeg.wasm, the browser executor could reuse these line by line. Local and cloud outputs were identical by construction.

2. Complete capability coverage. ffmpeg handles every container, codec, and filter we needed: atempo for audio speed, MP3/FLAC encoding, AVI/GIF muxing, complex filter chains. WebCodecs, by contrast, is a codec primitive — it encodes and decodes raw frames. It doesn't demux your MKV, it doesn't resample your audio, and it definitely doesn't do WSOLA time-stretching.

3. 28 tools, one UI. Because the processing semantics came for free, we could share the entire frontend between cloud and local modes and ship every tool at once.

We knew WebCodecs existed. We also knew its caveats: uneven browser support, no MP3/FLAC encoders, and opaque hardware encoder behavior. Betting on WASM first let us ship. That sequencing mattered — if we'd started with WebCodecs, we'd have spent weeks just on demuxing before shipping a single tool.

Problem 1: Slow, and it sounded like a jet engine

ffmpeg.wasm's x264 is software encoding compiled to WASM — no SIMD in practice, no hardware acceleration. Compressing a high-resolution 2.5K clip made laptop fans spin up like we were mining crypto. Competitors using WebCodecs hardware encoding were silent and an order of magnitude faster. Our benchmark: same 10.5 MB / 720p / 10 s file, medium quality, median of 3 runs on the same machine:

Path	Time (median of 3)	Output
ffmpeg.wasm (soft x264, incl. 32 MB engine download)	26.5 s	1.27 MB
WebCodecs (hardware H.264)	2.3 s	5.78 MB

(The output sizes differ because the two paths use different rate-control strategies — more on that below. Both were visually fine; the point is the 12× speed gap and the fan noise.)

For a free tool whose entire pitch is "it just works in your browser", CPU-bound software encoding is a UX ceiling you can't design around.

Problem 2: Outputs that came out bigger

Pure CRF rate control targets quality, not size. Feed it an already-lean source (3.30 Mbps) and CRF 28 happily produces a larger file (3.91 Mbps). Users compress a video to make it smaller; an output that grows is a bug report waiting to happen, no matter how defensible the encoding theory is.

CRF-driven inflation is the flavor we first hit, and it is squarely a WASM-path failure. But "outputs that come out bigger" is not exclusive to WASM: the hardware encoder in WebCodecs uses target-bitrate semantics and will happily spend the whole budget — without a per-pixel ceiling, a low-res "smaller" encode can land at 5 Mbps just because the target said so. The dual ceiling in the migration section is what stops that on both paths.

Problem 3: The 32 MB engine bootstrap is a reliability surface

The WASM engine is a ~32 MB download (core JS + WASM + worker), fetched on first use. Around that download sits an entire bootstrap chain — progress reporting, caching, ff.load() initialization, pthread startup for the multi-threaded core — and we learned that every link in that chain can fail silently:

Our progress denominator was an estimate (34,135,000 bytes); the real download was 32,849,385 bytes — 96.2% of it. So the bar froze at "96%" even on healthy loads, and users reasonably read that as a hang. (Fix: exact byte counts per engine file, updated every time the core package is bumped.)
On some machines, ff.load() never resolved — no error, no rejection, just silence. We added a 60 s watchdog that terminates the attempt and falls back to the single-threaded core.
We ultimately traced one production-wide outage of this bootstrap to a server configuration bug of our own making — a missing HTTP response header that Chrome requires before it will construct a module worker. But the larger lesson stood: the engine bootstrap is infrastructure, with the failure modes of infrastructure — while WebCodecs has no bootstrap at all. The browser already contains everything it needs.

That contrast is what turned "WASM is slow" into "the hot paths should not depend on a downloadable engine at all".

The WebCodecs migration

The pipeline

input file
  → mediabunny (demux; we use it for muxing too — MP4 with faststart)
  → VideoDecoder (hardware) → optional per-frame canvas processing
  → VideoEncoder (hardware H.264)
  → audio: copied passthrough when container-compatible, transcoded otherwise

First interaction runs immediately — no warmup, no download progress, no load watchdog.

hwOrSw: hardware first, always a way back

Every migrated tool is wrapped in the same pattern:

async function hwOrSw<T>(hw: () => Promise<T>, sw: () => Promise<T>): Promise<T> {
  try {
    return await hw()
  } catch (e) {
    if (e instanceof LocalCancelledError) throw e // user cancel: never "fall back" into more work
    console.warn('[local] hardware path failed, falling back to wasm:', e)
    return sw()
  }
}

Any failure in the hardware path — unsupported codec, decode error, encode error — silently lands on the proven WASM executor. A localStorage flag forces all-software for production debugging.

The stall watchdog

Individual files can freeze hardware decoders/encoders silently. We carried over the lesson from the engine-load watchdog: bounded waits, never infinite hangs. If progress doesn't advance for 30 seconds, we abandon the hardware attempt and fall back.

Rate control: the dual ceiling that stopped inflation

Hardware encoders use target bitrate semantics — they spend the whole budget, unlike CRF which backs off on easy content. And we needed a guarantee: output must be smaller than input. Two caps, take the minimum:

target = min(
  sourceVideoBitrate × qualityRatio,      // 0.8 / 0.6 / 0.4 — guarantees smaller
  outputPixels × fps × bpp                 // quality ceiling: 0.2 / 0.15 / 0.1
)

The first term kills the reverse-inflation bug. The second is the subtle one: without a bits-per-pixel ceiling, a 480p "smaller" compression would cheerfully burn 5 Mbps of hardware-encoder budget because the target said so.

Preflight with real dimensions

H.264 level support varies with resolution/bitrate/framerate. Asking VideoEncoder.isConfigSupported() about a hypothetical config misses machines that encode 480p fine but choke on 4K. We probe with the actual output dimensions before committing to the hardware path.

The honest hard-boundary list: what stays on WASM

A full audit of every tool produced the list WebCodecs advocates rarely publish — what it genuinely cannot do today:

Tool / capability	Why WebCodecs can't	Stays on
Audio speed change	`atempo` is WSOLA time-stretching; WebCodecs has no audio processing at all	WASM
MP3 / FLAC export	Browser `AudioEncoder` has no MP3 or FLAC encoders; output contracts would break	WASM
Volume adjust, audio convert, audio cut	Same MP3/FLAC encoder gap	WASM
Reverse video	Needs every frame in memory to play backwards; large inputs OOM the tab	WASM
AVI / GIF output	mediabunny has no muxer for either container	WASM
Most video filters	ffmpeg `eq` uses offset semantics; CSS/canvas uses multiply — visually different results. Only three presets (grayscale/sepia/invert) are pixel-equivalent	WASM for the rest
Crop with pan / custom background	Our canvas center-crop covers the common cases; edge semantics differ	WASM
Multi-segment cuts with fades	Crossfade filter graphs have no streaming equivalent	WASM

Everything else — compress, mute, rotate, flip, format conversion (MP4/MOV/MKV/WebM), single-segment trim, center crop, mosaic blur, sketch effect — runs on hardware.

The decision framework we ended up with

Is the capability expressible as decode → optional canvas → encode?
├─ No (audio processing, exotic containers, complex filter graphs)
│    → ffmpeg.wasm, and that's final
└─ Yes
     → Does the browser support it with THESE dimensions/codec?
          ├─ Yes → WebCodecs hardware path
          │         (30 s stall watchdog, any error → wasm)
          └─ No  → ffmpeg.wasm

Three principles held up across the whole migration:

Bounded waits everywhere. Both paths have watchdogs (60 s engine load, 30 s stall). Silent infinite hangs are the defining failure mode of in-browser media processing; every wait must have a ceiling and a fallback.
Fallbacks are architecture, not error handling. hwOrSw isn't a try/catch — it's the product. Users on odd hardware get slower results, never failed ones.
Match the technology to the capability, not the other way round. WASM-first got us coverage and correctness; WebCodecs-second got us speed where it matters. Neither was a mistake — the mistake would have been treating either one as the whole answer.

The result: the tools people actually hammer (compress, convert, trim) run at hardware speed with no engine download, the long tail still works everywhere ffmpeg works, and the 32 MB WASM engine only ships to browsers that genuinely need it.

This pipeline powers the local-processing mode at meicut.com, where you can try both paths on your own files.

Trimming Videos Is a Trade-off: copy-cut, re-encode, or filter-graph

Tutu — Thu, 15 Oct 2026 15:10:13 +0000

Part of the Meicut Engineering series — how we build an in-browser media toolkit.

ffmpeg -i in.mp4 -ss 60 -c copy -t 30 out.mp4 is everywhere. So is -i in.mp4 -c:v libx264 .... And so is -filter_complex "...concat...". Each one is presented as the way to trim a video. In practice there is no single best way: three mechanisms, three different trade-offs, and the right one depends entirely on your constraints. This post lays out all three, says which we chose and why under our constraints, and is open to better answers.

Approach 1: `-c copy` — the lossless shortcut, and its price

What it does: it tells ffmpeg not to decode or re-encode anything. It copies compressed packets and rewrites container metadata. For MP4/MOV, that includes the timestamps and possibly an edit list that tells players where to start.

What you get:

Bit-identical output (no pixels touched)
Orders of magnitude faster than any encode, since the bottleneck is I/O, not CPU

What it costs:

Not frame-accurate. A stream copy cannot cut between encoded frames; the cut snaps to the nearest keyframe. Your start point is accurate only to within one group-of-pictures (GOP): typically a fraction of a second to a couple of seconds, depending on -g/gop_size.
B-frame and IDR hazards. If the cut lands mid-GOP on a source with B-frames, the first output packet may not be an independently-decodable IDR frame, and a player can glitch or show garbage at the head.
Seek-position-dependent timestamp metadata. With output-side seeking (-ss after -i), copy-cut depends on the timestamp metadata the muxer writes, whose player support is inconsistent. Input-side seeking (-ss before -i) is the more robust copy-cut form, but by construction it lands on a keyframe. Either way, you trade precision for speed.
Codec/container compatibility. The compressed stream has to be legal inside the target container; exotic codecs or non-standard pairings won't stream-copy.

Where it shines: dropping an intro/outro, extracting one coarse segment, batch-trimming files with consistent parameters, when "second-precision, zero-loss, near-instant" beats "frame-perfect." It is the engine behind most "instant / lossless trim" buttons, and it is a useful trade, just one that gives up frame accuracy.

Approach 2: re-encode with input seeking — frame-accurate and portable

What it does: -ss before -i (input seeking) locates the region quickly at the demux level, then a full re-encode produces the segment.

What you get:

Frame-accurate, but only because we re-encode. Input seeking alone does not buy you frame precision. With -c copy, input seeking still lands on a keyframe. The precision comes from decoding from the nearest keyframe and then starting to write exactly at the requested timestamp. That chain holds only on the re-encode path, which is the boundary from Approach 1.
Container-portable and filter-capable. You can re-target resolution, codec, bitrate, and apply effects (fades, etc.) in the same pass.
Clean output. No edit-list or mid-GOP artifacts to worry about.

What it costs:

Slower, by definition: you encode every output frame.
A second-generation quality loss, even if mild at a reasonable CRF (CRF 23 is our default preset family, as discussed in Part 1: Video Compression at Product Scale).
Needs a working encode pipeline (codec, preset tuning, compatibility flags like -pix_fmt yuv420p).

Where it shines: any time the cut point must be precise: a transition, a single-frame edit, trimming then merging, or any transform on top of the trim.

Approach 3: filter-graph splicing — one pass, multiple segments

What it does: a single -filter_complex pass trims every segment (trim/atrim), resets timestamps (setpts=PTS-STARTPTS), and concatenates them (concat), all in one ffmpeg process.

What you get:

One process, no intermediate files. Nothing to write, clean up, or keep in sync.
Frame-accurate, timestamp-continuous joins at the filter level (frame-accurate, not "sample-accurate", which is an audio term). Fades compose naturally into the same graph.

What it costs:

The whole input is decoded to feed the trims. Because trim runs in the filter layer, ffmpeg decodes the input once and streams it through every segment's trim. The decode cost is O(file length) for all segments combined, not per-segment. A deep start in a long file still wastes CPU relative to a demux-level seek, but it is not multiplied by the segment count.
A complex graph (multi-input, transitions) is more to get wrong and harder to debug.

One adjacent mechanism worth naming separately, because it is for merging rather than trimming: the concat demuxer (-f concat) joins pre-existing files that already share stream parameters. It is fast and avoids re-encoding, but it demands parameter consistency. Resolution and framerate mismatches are the obvious breakers; the subtler killer is timebase mismatch, which ffprobe does not surface visibly and which tends to show up later as progressive audio/video drift rather than a hard failure.

A comparison, side by side

	Re-encode?	Frame accuracy	Speed	Quality	Best for
copy-cut	No	keyframe-level (seconds)	fastest	bit-identical	coarse trims, intro/outro removal, consistent batches
input-seek + re-encode	Yes	frame-level	slower (you encode)	second-generation, CRF-controlled	precise cuts, cross-container, add effects
filter-graph splicing	Yes	frame-level	full decode	second-generation, clean joins	multi-segment, fades, one-process workflow

Speed: filter-graph splicing depends on segment count and file length; against a single re-encode it is comparable.

Our choice, and why — not a universal answer

Under our constraints, frame accuracy, multi-segment joins, fades, and cross-container output are all non-negotiable product requirements, on both a browser-local path and a server path. So we currently do:

Single-segment trim: input seeking (-ss before -i) + re-encode. Fast to locate, frame-accurate to cut, portable output.
Multi-segment trim: one -filter_complex pass (trim/atrim + setpts + concat). Single process, clean joins, fades included.

We did not pick -c copy as the default. Not because copy-cut is wrong; it is genuinely the right tool for a large class of "quick, coarse, lossless" trims. The framing is: in a product that promises frame-accurate cuts, a fast path that quietly snaps your cut point to a keyframe is a bug; in a coarse-trim workflow, that same snapping is a feature. Our product is the former, so copy-cut cannot be our default.

We also own the cost: re-encoding is slower and introduces a second generation; the quality side is usually a non-issue, because CRF 23's second-generation loss is invisible to the eye in most consumer scenarios. The speed side is the real one. On a long video, the encode step can take from seconds to minutes depending on the device and whether hardware encoding is available. That is exactly why we reserve the copy-cut fast path as a candidate for the future, rather than claiming one approach fits everyone.

An invitation, not a conclusion

Trimming a video is not a single correct command. It is a series of trade-offs between precision, speed, losslessness, storage, and the transforms you need to compose. Our implementation serves a specific product: precise, multi-segment cuts. If your use case differs, say you only need second-precision and care most about zero loss and speed, copy-cut may well be your right answer. I'd like to hear which approach you reach for and why; we are still evaluating where a copy-cut fast path might be worth adding alongside our re-encode default.

The trimmer behind this decision is part of the online toolkit at meicut.com — cut video — runs in the browser, no install required; files auto-delete after processing.

Meicut Engineering series:

Part 1: Video Compression at Product Scale
Part 2: Trimming Videos Is a Trade-off — copy-cut, re-encode, or filter-graph
Part 3: "Convert Format" Is a Misnomer — containers vs codecs
Part 4: Subtitles: soft vs burned, and the font problem

Meicut Engineering #1: Video Compression at Product Scale

Tutu — Tue, 06 Oct 2026 17:50:13 +0000

Part of the Meicut Engineering series — how we build an in-browser media toolkit.

The encoding command — ffmpeg -i in.mp4 -c:v libx264 -crf 23 out.mp4 — is well documented. What is rarely discussed is the engineering that turns this single command into a service: where encoding runs, how long-running jobs behave, how concurrency is bounded, and how output is matched to the user's stated goal. This post covers those decisions.

1. Where encoding runs

The first architectural decision is the execution environment. Three options exist for a web media tool:

Approach	Execution	Cost profile	Constraints
Server-side ffmpeg	Encode on infrastructure	Upload → encode → download	Requires upload; full ffmpeg capability
ffmpeg.wasm	In-browser (WebAssembly)	No upload, strong privacy	≈ an order of magnitude slower than native (community benchmarks); input+output held in a WASM heap capped near 2–4GB
WebCodecs	In-browser (raw codecs)	No upload	Most control; most implementation work

Browser-native execution (ffmpeg.wasm / WebCodecs) is attractive for privacy — files never leave the device. It is, however, constrained by encode throughput and memory ceiling: a 500MB input can exceed the WASM heap and crash the tab. Server-side execution reliably handles large files and the full filter set (fonts for subtitles, multi-file merges).

We therefore implemented server-side encoding first, deferring browser-native processing. The ordering principle — support the hardest case before the common case — recurs throughout the product.

2. Task pipeline

With encoding server-side, "compress video" becomes a pipeline:

upload → validate → queue → encode → store → downloadable link

Three constraints follow.

Long-tail latency. A short clip encodes in seconds; a 4K 20-minute input at a slow preset runs for minutes. The pipeline must be asynchronous — each request returns a job ID, the client polls, and progress reflects pipeline state rather than a synthetic spinner. A wall-clock cap per job, with a clear path to re-encode at a faster preset, replaces silent failure.

Concurrency. If every request spawned an ffmpeg process, a burst of 4K uploads would exhaust CPU and starve smaller jobs. We bound concurrency by job size and worker pool: small jobs take a fast lane; heavy encodes run under a concurrency limit, so no single input or burst dominates. A queue with two priority tiers sufficed for our volume.

Batch as the same pipeline, reused. Compression is rarely one file at a time. Many users need to compress multiple videos at once — batch compression is the common request behind any "video compressor" tool. Under the hood it is not a separate system: a user submits many files, each becomes a job in the same queue — sharing the concurrency bound, priority tiers, and per-file validation. Progress and results are tracked per file, and a failed item does not abort the batch; it is reported alongside the successful ones. The single-job pipeline, reused across a list of inputs.

Input validation. Users upload arbitrary inputs — truncated files, unusual containers, audio mislabeled as video. ffmpeg tolerates much until it does not; a mid-encode crash is poor UX. We probe inputs first (ffprobe) and return a readable diagnostic before committing to an encode.

3. Encoding parameters

A single CRF value applied to every input optimizes for no use case. We classify each job before encoding:

Intended use determines the parameter set: email attachment → aggressive size target with downscaling permitted; social → balanced 1080p; archive → keep resolution at high quality.
Content type matters: screen recordings (static) and concert footage (motion-heavy) require different bitrate budgets under the same preset.

Output is CRF with a size-aware backstop: quality-targeted, but if the result exceeds the size implied by the use case, it is re-encoded once at a stricter CRF. Checking the outcome against the goal, rather than the parameter, catches most poor encodes.

4. Playback compatibility

Encoding is only half the task; guaranteeing playback is the other. Three flags are non-negotiable in a product:

ffmpeg -i in.mp4 -c:v libx264 -preset medium -crf 23 -pix_fmt yuv420p -movflags +faststart -c:a aac -b:a 128k -ar 44100 out.mp4

-pix_fmt yuv420p — required for broad player and browser support; 4:4:4 or 10-bit input plays on a subset of devices.
-movflags +faststart — moves the moov atom forward so playback starts before download completes, reduces perceived load time on web.
Explicit audio parameters (aac 128k 44100) — avoid container defaults; inaudible or silent tracks are the most common audio support issue.

5. Privacy as a design constraint

The stated promise — files are used only for the task and auto-deleted — is implemented as a pipeline property rather than a toggle. Files are written to short-TTL storage and destroyed after the job, with a scheduled cleanup as a backstop in case a process terminates abnormally. Default behavior, not user-remembered settings, is what keeps the promise reliable.

Conclusion

Video compression as a product is the pipeline surrounding the encode: execution environment, long-job behavior, bounded concurrency, and output matched to the user's goal. The command itself is the smallest part; the value is in the decisions that make it reliable at scale.

The pipeline described here is implemented in the online video compressor — no install required; files auto-delete after processing.

Meicut Engineering series:

#1: Video Compression at Product Scale
#2: Trimming Videos Without Re-encoding (when -c copy is and isn't viable)
#3: "Convert Format" Is a Misnomer — containers vs codecs
#4: Subtitles: soft vs burned, and the font problem

DEV Community: Tutu

ffmpeg.wasm to WebCodecs: the 12x Gap, What Broke, and What Still Can't Move

Why we chose ffmpeg.wasm first (and it was the right call)

Problem 1: Slow, and it sounded like a jet engine

Problem 2: Outputs that came out bigger

Problem 3: The 32 MB engine bootstrap is a reliability surface

The WebCodecs migration

The pipeline

hwOrSw: hardware first, always a way back

The stall watchdog

Rate control: the dual ceiling that stopped inflation

Preflight with real dimensions

The honest hard-boundary list: what stays on WASM

The decision framework we ended up with

Trimming Videos Is a Trade-off: copy-cut, re-encode, or filter-graph

Approach 1: -c copy — the lossless shortcut, and its price

Approach 2: re-encode with input seeking — frame-accurate and portable

Approach 3: filter-graph splicing — one pass, multiple segments

A comparison, side by side

Our choice, and why — not a universal answer

An invitation, not a conclusion

Meicut Engineering #1: Video Compression at Product Scale

1. Where encoding runs

2. Task pipeline

3. Encoding parameters

4. Playback compatibility

5. Privacy as a design constraint

Conclusion

Approach 1: `-c copy` — the lossless shortcut, and its price