You set up a VideoDecoder, and its output callback hands you a VideoFrame, which is one decoded picture from the video. You draw it, the function returns, and nothing points at the frame any more. Anywhere else in JavaScript that would be the end of it, because the garbage collector, the part of the engine that frees objects nobody can reach, cleans up later. I lead a small front-end group and could not say whether that holds for video frames, so I measured it.
The clip was an example video I found online, a shot of waves from Wikimedia Commons. I transcoded it to H.264 with ffmpeg 7.1 and looped it end to end into a 60-second 1920×1080 file, so it is 1,500 frames of repeating content, not a long recording. mp4box.js 0.5.2 pulled the compressed samples out and I fed them to the decoder. One decoded frame of this clip reports 3,110,400 bytes from allocationSize(), about 3.1 MB. The machine was an Apple M4 with 16 GB of memory running macOS.
I wrote the output callback three ways
const stash = [];
const videoDecoder = new VideoDecoder({
output: (img) => {
if (variant === 'close') img.close(); // release it now
if (variant === 'keep') stash.push(img); // never release
// 'drop': no close(), no reference, leave it to GC
},
error: error => console.error(error),
});
For each variant I started a fresh browser, decoded all 1,500 frames, then left the page alone for 6 seconds. The reading I trust most is a system-wide count of graphics buffers. A graphics buffer is a block of memory the operating system hands out for pixel data, called an IOSurface on macOS, and I counted how many existed above the starting level.
The top three bars are the three variants above: 12 extra buffers when each frame is closed right away, 1,508 when every frame is kept, and 1,508 again when nothing is kept and nothing is closed.
Dropping the reference held as many buffers as keeping it
In Chromium 149 the drop variant matched the keep variant exactly, 1,508, and the count was still there after the 6 seconds I watched. The misleading part is the browser's own process memory. It grew by 138 MB when I kept the frames and by 0 MB when I dropped them, while memory use across the whole system peaked 2,694 MB and 3,316 MB higher. Those system-wide figures are noisy. The same scenario can differ by more than 1 GB between two runs, so I read them only as "a few gigabytes". Chrome 154 gave the same picture, with 46, 1,510 and 1,589 extra buffers for close, keep and drop.
Firefox 151 showed it in a different place. The graphics buffer count barely moved (5, 7 and 7), and the browser's process memory grew instead: 163 MB when closing, 2,291 MB when keeping, 2,221 MB when dropping. Dropping again came out about the same as keeping.
Only WebKit took the frames back
WebKit 26.5 (macOS) was the exception. Closing gave 19 extra buffers and keeping gave 1,510, but dropping ended at 32, and the console printed "A VideoFrame was destroyed without having been closed explicitly". I did not look into why the engines differ, and I did not wait longer than 6 seconds, so I can't say whether the other three reclaim the frames later.
The release call goes where the work on a frame ends
If the work is synchronous, that is the last line of the output callback. If each frame takes a while, the release moves to the end of that work, and then the number of frames waiting their turn starts to matter. I tried a slow consumer that draws each frame to a small canvas, reads one pixel, waits 10 ms and then closes it. When I handed the decoder all the compressed data at once, 1,421 frames were waiting at the peak and the buffer count went up by 1,488, although every frame did get closed. When I held back the input, the peak was 17 waiting frames and 29 extra buffers.
while (videoDecoder.decodeQueueSize > 8 || waiting.length > 4) {
await wait(2);
}
videoDecoder.decode(packet);
decodeQueueSize counts chunks the decoder has not decoded yet, and waiting is my list of decoded frames not yet processed.
My slow consumer is only a stand-in, so I also took the same reading on a finished frame-by-frame pipeline. I used ImgIng (https://imging.ai/), a browser-based image and video tool, and ran the same clip through its video matting, a Beta feature aimed first at desktop Chrome and Edge. Over the 1,500 frames in Chromium 149, the extra buffer count stayed between −2 and 79 and did not climb as the job went on. The clip is waves with no subject to cut out, so this tells me about memory and nothing about matting quality.
The preview above is from a second clip, because the wave clip has no subject to matte: 8 seconds of a street guitarist, video by Isak Zvegelj on Wikimedia Commons (CC BY 3.0). Source frame on the left, matted preview on the right. I did not repeat the buffer readings on it.
All of this is one machine. IOSurface is a macOS term, and I did not test where frames live on other systems or separate hardware decoding from software decoding. In my own code I now search for every place a VideoFrame is received and follow each path to a close(), early returns and error branches included. Then I check with a system-level reading, since in Chromium and Chrome the process memory showed almost nothing.
The waves clip is by David Shay (דוד שי) on Wikimedia Commons, CC BY-SA 4.0, and the looped file is my transcode of it.
Top comments (0)