I leaked 1,500 video frames on purpose and Chromium's memory barely moved

My side project walks through a video clip one frame at a time in the browser. Before going further I wanted to know what a mistake looks like, so I made one on purpose. I decoded 1,500 frames, kept every one of them, never released any, and watched the browser's memory. In Chromium 149 it went up by 138 MB. That was less than the run where I did everything right.

Some words first. A video file stores compressed data, and decoding means turning that data back into full pictures. In the browser VideoDecoder does this and hands you each picture as a VideoFrame. A decoded 1920×1080 frame is big: allocationSize() reported 3,110,400 bytes for one, about 3.1 MB. You are supposed to call close() on a frame when you are done, which tells the browser it can have that memory back. My broken version skipped that line and pushed every frame into an array.

const held = [];
const dec = new VideoDecoder({
  output: (frame) => {
    held.push(frame);    // broken on purpose: never closed
    // frame.close();    // the well-behaved version
  },
  error: e => console.error(e),
});

The clip is an example video of waves from Wikimedia Commons. I re-encoded it to H.264 using ffmpeg 7.1 and looped it end to end until it was 60 seconds long, 25 fps, 1,500 frames, so it is a stitched test clip and not a real long recording. mp4box.js 0.5.2 splits the file and feeds the decoder. I ran it on a 16 GB Apple M4 Mac in Chromium 149, Chrome 154, Firefox 151 and WebKit 26.5 (macOS), with a fresh browser for every run. By "process memory" I mean the resident memory of all the browser's processes added together, read from ps.

Look at the two short blue bars. With 1,500 frames held, Chromium 149 grew by 138 MB and Chrome 154 by 149 MB. When I closed every frame straight away, Chromium grew by 186 MB, so going by this number alone the leaky version looks like the tidy one. Firefox 151 is the only bar that looks like what I expected, 2,291 MB against 163 MB when frames were closed. WebKit has no bar because its web process was outside what my ps reading could see.

So where did the frames go? It reminded me of the fridge in my dorm. My own shelf looked empty, because I had put everything on the shared shelf. On macOS there is a system-wide count of graphics buffers, called IOSurface, which are picture-sized blocks of memory. The count is for the whole system, not for one app. You can read that count from a terminal.

ioclasscount IOSurface       # graphics buffers, whole system
ps -axo pid=,ppid=,rss=      # per process, add up the browser's tree

This chart is the first one flipped. Chromium 149 added 1,508 graphics buffers, Chrome 154 and WebKit 26.5 added 1,510 each, which lines up with the 1,500 frames I was holding. Firefox added 7. With frames closed right away, Chromium added 12. So the frames were there the whole time, on a counter I hadn't been reading. Why Firefox keeps them in its own process and the other three don't, I haven't looked into.

I also tried reading total system memory, and it was a much worse ruler. In the Chromium run it peaked 2,694 MB above the starting level, but other apps were open on the machine and the same scenario could land more than 1 GB apart between two runs. The buffer count was the steady one, so that is the number I trust here. One more thing I tried was dropping the frames without closing them and without keeping a reference. Six seconds later Chromium still showed 1,508 extra buffers. WebKit was down to 32 and printed a console warning about a frame destroyed without being closed. I only waited those six seconds.

To see what a finished frame-by-frame pipeline does on the same counter, I used ImgIng (https://imging.ai/) and ran its video matting tool in Chromium 149 on the same 1,500-frame clip, with the compatibility model option (RVM MobileNetV3) and a solid colour background. The tool is marked Beta, and its notes say desktop Chrome and Edge are supported first. I sampled every 2 seconds, 104 samples in total, and the buffer count stayed between −2 and 79 above the starting level without climbing as the job went on. The clip is only waves, with no subject to cut out, so this says something about memory and nothing about matting quality.

These two screenshots use a different clip. The waves have no person to cut out, so the screen stayed blank, and I tried 8 seconds of a street guitarist instead (video by Isak Zvegelj, Wikimedia Commons, CC BY 3.0). Top is the clip as imported, bottom is with the background swapped for white. The numbers above are still from the wave clip.

What I do now is run ioclasscount IOSurface in a terminal before and after a test and compare the two counts, and I stopped treating a flat process number as proof that I closed everything. All of this is one Mac. IOSurface is a macOS name, and I have not checked where frames end up on Windows, Linux or a phone, or whether hardware and software decoding differ. The wave footage is by David Shay (דוד שי) on Wikimedia Commons, CC BY-SA 4.0.