Reversing a video in the browser without running out of memory

Playing a video backwards sounds trivial: decode the frames, flip the order, encode again. In a browser tab it isn't, because a decoded 1080p frame is about 8 MB of RGBA. A 15 second clip at 30 fps is 450 frames, so "just keep them all" means roughly 3.7 GB of memory. The tab dies long before that.

We built a reverse video tool that runs entirely client side, with no upload, and this is how it stays inside a fixed memory budget. The code uses Mediabunny for demuxing and muxing and WebCodecs for the actual decode and encode.

The constraint: you can only decode forwards

Video codecs such as H.264 store most frames as differences from earlier frames. You can start decoding only at a keyframe and you have to go forwards from there. So "give me the last frame first" really means "decode the whole group of pictures that ends with it, then hand them back in reverse".

That gives the basic plan:

1. Read the packet metadata only (timestamps and which packets are keyframes). No decoding yet, so this is fast.
2. Cut the timeline into segments at keyframes.
3. Walk the segments from the last to the first. Decode one segment forwards, sort its frames newest first, and feed them to the encoder.
4. Throw the segment away and move on.

Memory now depends on the longest segment, not on the length of the clip.

Step 1: find the keyframes without decoding

const packets = new EncodedPacketSink(videoTrack);
const times = [], keys = [];
for await (const p of packets.packets(undefined, undefined, { metadataOnly: true })) {
  times.push(p.timestamp);
  if (p.type === "key") keys.push(p.timestamp);
}
times.sort((a, b) => a - b);
keys.sort((a, b) => a - b);

metadataOnly skips reading the packet data, so this pass is quick even for a 3 minute file.

Step 2: segments with a frame budget

Keyframe spacing is up to the encoder that made the file. A phone may put one every second; x264's default can be one every 250 frames. A segment that long would blow the budget again, so each keyframe segment is split further by a frame count:

const FRAME_BUDGET = 256 * 1024 * 1024;              // about 256 MB of decoded frames
const maxFrames = Math.max(8, Math.floor(FRAME_BUDGET / (width * height * 4)));

At 1280 x 720 that's 69 frames per segment, at 1920 x 1080 it's 32. Splitting inside a keyframe group means some frames get decoded more than once (the decoder has to start at the keyframe again), which costs time but keeps memory flat. We'll take slower over crashed.

Step 3: decode a segment, emit it backwards

for (let i = segs.length - 1; i >= 0; i--) {
  const [a, b] = segs[i];
  const frames = [];
  for await (const c of canvasSink.canvases(a, b)) {
    if (c.timestamp < boundary - 1e-6) frames.push(c);   // a frame at the edge can show up twice
  }
  frames.sort((x, y) => y.timestamp - x.timestamp);
  for (const f of frames) {
    let ts = end - (f.timestamp + f.duration);           // mirror the timestamp
    if (ts <= lastOut) ts = lastOut + 1e-4;              // keep output timestamps strictly increasing
    lastOut = ts;
    const sample = new VideoSample(f.canvas, { timestamp: Math.max(0, ts), duration: f.duration });
    await videoSource.add(sample);
    sample.close();
  }
  if (frames.length) boundary = Math.min(boundary, frames[frames.length - 1].timestamp);
}

Two details bit us:

• Duplicate edge frames. The frame shown at a segment's start can be yielded by both neighbouring segments. Tracking the lowest timestamp already emitted (boundary) and skipping anything at or after it fixed a visible stutter.
• Timestamp collisions. Mirroring end - (t + duration) can produce two equal timestamps when durations are rounded. Encoders reject that, so each output timestamp is nudged to be strictly later than the previous one.

Audio: the easy part, done differently

Audio is tiny next to video. Three minutes of 48 kHz stereo as 32-bit floats is about 70 MB, so we decode the whole track, reverse each channel array in place and feed it to the encoder in one second chunks, interleaved with the video segments so the muxer gets both streams in time order.

Users get three choices: reverse the sound with the picture, keep the original sound playing forwards, or drop it. Reversed speech is gibberish (often the joke), so "keep sound" matters for music clips.

Picking the output codec

const mp4 = await canEncodeVideo("avc", { width, height });
// H.264 + AAC in MP4 where the browser can encode it, otherwise VP9 + Opus in WebM

Chrome, Edge and Safari encode H.264, so most people get an MP4. A browser without an H.264 encoder falls back to WebM instead of failing.

What it costs in practice

We ran a 15 second, 1920 x 1080, 30 fps test clip (5.31 MB MP4) through the tool in Chrome on a Windows laptop:

Output	Time	Output size
720p	8.3 s	7.06 MB
1080p	18.4 s	15.0 MB

The output is bigger than the input because every frame is re-encoded at a high quality setting with a keyframe every 2 seconds, while the source was compressed harder. For sharing, 720p is the sweet spot: twice as fast and you won't see the difference on a phone.

We cap clips at 3 minutes and 1080p. The frame budget keeps memory bounded, but the time grows with length, and a 10 minute reverse in a background tab is a bad experience.

If you want to try it on your own clip, the tool is the reverse video tool on WoowTools. Everything above happens in the tab; the file never leaves the machine.

Takeaways

• Decode forwards, emit backwards, one bounded segment at a time.
• Size segments by bytes, not by keyframes alone.
• Watch for duplicate edge frames and equal timestamps when you mirror time.
• Audio can be handled whole; video never can.