DEV Community

Mason K
Mason K

Posted on

Convert video in the browser with WebCodecs (no ffmpeg.wasm)

#javascript #tutorial #video #webdev

TL;DR

We build a browser-side video pipeline with WebCodecs plus Mediabunny as the container layer: read a file's real metadata without downloading it all, transcode and resize it, write an HLS ladder client-side, and feature-detect codecs so you know when to fall back to a server. No WebAssembly build step.

Everyone's first answer to client-side video is ffmpeg.wasm, and it works, but it runs software codecs inside a sandbox while the machine's hardware decoder sits idle. WebCodecs exposes that hardware decoder and encoder to JavaScript. What it does not give you is containers: no MP4 parsing, no playlist writing. That is the gap we are filling.

1. Set up the project 🛠️ 

npm create vite@latest browser-video-pipeline -- --template vanilla-ts
cd browser-video-pipeline
npm install mediabunny
Enter fullscreen mode Exit fullscreen mode 
added 1 package in 1s
Enter fullscreen mode Exit fullscreen mode

One package, zero dependencies, MPL-2.0 licensed. You need an environment that runs ECMAScript 2021 or later, and TypeScript 5.7+ if you want the types. Tree-shaking matters here: the project's own size table puts reading MP4 at 16.0 kB min+gzip, writing MP4 at 17.3 kB, reading every supported format at 30.0 kB, and the whole library with everything at 69.6 kB.

2. Open a file and read what is actually in it 

// src/inspect.ts
import { Input, ALL_FORMATS, BlobSource } from 'mediabunny';

export async function inspect(file: File) {
  const input = new Input({
    formats: ALL_FORMATS,
    source: new BlobSource(file),
  });

  const format = await input.getFormat();
  const duration = await input.computeDuration();
  const video = await input.getPrimaryVideoTrack();
  const audio = await input.getPrimaryAudioTrack();

  return {
    container: format.name,
    mimeType: await input.getMimeType(),
    duration,
    video: video && {
      codec: await video.getCodec(),
      width: video.getDisplayWidth(),
      height: video.getDisplayHeight(),
      rotation: video.getRotation(),
      decodable: await video.canDecode(),
    },
    audio: audio && {
      codec: await audio.getCodec(),
      sampleRate: audio.getSampleRate(),
      channels: audio.getNumberOfChannels(),
    },
  };
}
Enter fullscreen mode Exit fullscreen mode 
{
  container: 'QuickTime File Format',
  duration: 92.4,
  video: { codec: 'avc', width: 1920, height: 1080, rotation: 0, decodable: true },
  audio: { codec: 'aac', sampleRate: 48000, channels: 2 }
}
Enter fullscreen mode Exit fullscreen mode

💡 Tip: new Input(...) performs zero reads. Nothing is fetched or parsed until you ask a question, and then only the byte ranges needed to answer it. Swap BlobSource for UrlSource and you can show real durations for a remote library without downloading the files.

Other sources available: BufferSource, UrlSource, ReadableStreamSource, CustomPathedSource, and FilePathSource for server runtimes.

3. Feature-detect before you promise anything ⚠️

This is the step people skip and then ship a broken Safari build. Codec support is the browser's, and the docs say outright that it cannot be guaranteed by the library.

// src/capabilities.ts
import { canEncodeVideo, canEncodeAudio, getFirstEncodableVideoCodec } from 'mediabunny';

export async function plan() {
  const videoCodec = await getFirstEncodableVideoCodec(['av1', 'vp9', 'avc']);
  const canAac = await canEncodeAudio('aac');

  if (!videoCodec) return { mode: 'server' as const, reason: 'no encodable video codec' };
  if (!canAac) return { mode: 'server' as const, reason: 'no AAC encoder in this browser' };

  return { mode: 'client' as const, videoCodec };
}
Enter fullscreen mode Exit fullscreen mode 
# Chrome 141, macOS
{ mode: 'client', videoCodec: 'av1' }

# a browser without an AAC encoder
{ mode: 'server', reason: 'no AAC encoder in this browser' }
Enter fullscreen mode Exit fullscreen mode

⚠️ Note: AAC encoding is genuinely missing in some browsers, which is awkward given how much of the world's audio is AAC. There are extension packages for the gaps WebCodecs leaves: @mediabunny/aac-encoder, @mediabunny/mp3-encoder, @mediabunny/flac-encoder, @mediabunny/prores, @mediabunny/ac3, @mediabunny/dts, @mediabunny/truehd.

4. Transcode and resize

The conversion API is three objects: an input, an output, and a Conversion between them.

// src/convert.ts
import {
  Input, Output, Conversion, ALL_FORMATS,
  BlobSource, Mp4OutputFormat, BufferTarget,
} from 'mediabunny';

export async function toMp4(file: File, height = 720) {
  const input = new Input({ formats: ALL_FORMATS, source: new BlobSource(file) });

  const output = new Output({
    format: new Mp4OutputFormat(),
    target: new BufferTarget(),
  });

  const conversion = await Conversion.init({
    input,
    output,
    video: [{ codec: 'avc', height }],
    audio: [{ codec: 'aac' }],
  });

  await conversion.execute();
  return new Blob([output.target.buffer!], { type: 'video/mp4' });
}
Enter fullscreen mode Exit fullscreen mode

Swap Mp4OutputFormat for WebMOutputFormat or WavOutputFormat and the rest is unchanged. BufferTarget holds the result in memory; use StreamTarget with a writable stream if the output is large enough that you would rather not.

The documented conversion operations are transmuxing, transcoding, resizing, rotation, cropping, resampling and trimming. The cheap win is transmuxing: a phone MOV that is already H.264 and AAC can be rewritten into MP4 without re-encoding at all.

5. Write an HLS ladder in the browser

HLS reading and writing shipped in v1.42.0, and this is the part that turns a demo into a pipeline. It costs roughly 30 kB of extra bundle.

// src/to-hls.ts
import {
  Input, Output, Conversion, ALL_FORMATS, BlobSource,
  HlsOutputFormat, MpegTsOutputFormat, PathedTarget,
} from 'mediabunny';

export async function toHls(file: File, writeFile: (path: string, data: Uint8Array) => Promise<void>) {
  const input = new Input({ formats: ALL_FORMATS, source: new BlobSource(file) });

  const output = new Output({
    format: new HlsOutputFormat({ segmentFormat: new MpegTsOutputFormat() }),
    target: new PathedTarget(async (path, data) => writeFile(path, data)),
  });

  const conversion = await Conversion.init({
    input,
    output,
    video: [
      { codec: 'avc', height: 1080 },
      { codec: 'avc', height: 720 },
      { codec: 'avc', height: 480 },
    ],
    audio: [{ codec: 'aac' }],
  });

  await conversion.execute();
}
Enter fullscreen mode Exit fullscreen mode 
# the paths PathedTarget hands you
master.m3u8
video0/playlist.m3u8
video0/segment0.ts
video1/playlist.m3u8
...
Enter fullscreen mode Exit fullscreen mode

⚠️ Check the import names against the HLS guide before you paste this. The docs' HLS examples elide their import lines, so the identifiers above are taken from usage rather than copied from an import statement.

What HLS support covers: VOD and live, reading encrypted and DRM-protected streams, every segment format (MPEG-TS, CMAF, fMP4, ADTS, MP3, WAV), byte-range segments via #EXT-X-BYTERANGE, #EXT-X-PROGRAM-DATE-TIME, mid-stream discontinuities, and I-frame playlists. Pass live: true to HlsOutputFormat for a rolling playlist.

What it does not cover, from the project's own list:

Not supported Impact
Subtitles (WebVTT, CEA-608/708) Captions workflows stay server-side
ID3v2 metadata extraction No in-band metadata reading
Writing encrypted segments Client-side DRM packaging is out
Low-latency HLS Normal-latency live only
Built-in ABR, analytics, CMCD Your player's job, not the library's

And the sentence to remember: it is not an HLS player. Playback is still hls.js or the native element.

6. Know when to give up and use a server 

// src/pipeline.ts
const p = await plan();
if (p.mode === 'server') {
  await uploadForServerTranscode(file);   // your existing path
} else {
  const mp4 = await toMp4(file, 720);
  await upload(mp4);                       // 40 MB instead of 900 MB
}
Enter fullscreen mode Exit fullscreen mode

Reasons to stay on the server, honestly: you need identical output for every user regardless of browser; you need subtitles read out of the source; you need encrypted HLS; you need LL-HLS; you need ProRes without shipping an extension package. There is a @mediabunny/server package for Node, Bun and Deno with its own hardware-accelerated coders and multithreading if you want the same API on both sides, which is a nice way to avoid writing the pipeline twice.

7. About those benchmark numbers

You will see them quoted, so know what they are. The project publishes figures measured on 2025-06-22 on a Ryzen 7600X with an RTX 4070 against a 691 MiB 1080p H.264 file: metadata extraction at 862 ops/s versus ffmpeg.wasm's 1.83, and convert-to-WebM-with-resize at 804 frames/s versus ffmpeg.wasm's 12.0. Those are the maintainer's own numbers on one desktop, from before HLS support landed, and there is no native FFmpeg CLI column. Directionally the gap is real, because one path uses the GPU and the other does not. Measure on your own target devices before you quote a figure to anyone.

What's next

Two things I would build next on this base. First, move the conversion into a worker and report progress to the UI; the library's docs do not discuss Web Workers, so verify behaviour before relying on it. Second, pair it with VideoSampleSink or CanvasSink to pull frames for client-side thumbnails, which removes another server round trip.

If you want the read side in more depth, the reading guide covers the sinks and the lazy-loading model properly. #webdev #javascript

Top comments (0)