Why Chromium Was Ignoring My GPU — And How I Boosted Performance from 4fps to 58fps

📝 Originally published (in Japanese) at forge.workstyle.tech.

Building a 24/7 AI Avatar Live Streaming System with Headless Chromium

I was working on a live streaming system where an AI avatar (a 3D VRM model) could speak, react to comments, and stream continuously—all without human intervention. The core of this system was a pipeline that rendered a 3D scene using headless Chromium, captured the video, and streamed it via RTMP.

This turned out to be a minefield of pitfalls. Even with a GPU installed, rendering would fall back to CPU. The "safety net" of reducing resolution didn’t work. GPU rendering would succeed, but the image would be broken. Capturing video that worked on CPU would turn black the moment it ran on GPU. All of these issues shared a common trait: no errors were thrown, and all metrics looked normal—making them especially tricky to debug.

This article walks through the sequence of stumbles I actually encountered while building this system. Here are the four key takeaways up front:

1. CPU rendering (SwiftShader) won’t cut it for 3D avatars — and the "fallback to lower resolution" safety net doesn’t work
2. To actually render on GPU, you need a 3-set combo — full-build Chromium, ICD registration, and launch flags
3. Choosing ANGLE’s backend can break the image even when FPS is the same — gl-egl and vulkan aren’t equivalent
4. canvas.captureStream() breaks on GPU — switch to CDP’s Page.startScreencast instead

In the same scene, CPU rendering gave about 4 FPS, while properly configured GPU rendering delivered 57–58 FPS — over a 10x difference. Below is the full story of how I got there.

---

The Plan to "Stream Using Only CPU" Collapsed

My initial design relied solely on CPU rendering. No GPU needed, which was great for cost and scalability. The design doc even included a fallback plan: "If it’s too heavy, drop to 540p / 24 FPS."

Chromium includes SwiftShader, a CPU-based WebGL implementation. Even without a GPU, WebGL runs. And when I tested a simple triangle, it hit 60 FPS. That gave me a false sense of security.

The problem appeared when rendering the actual VRM avatar. It dropped to 4 FPS — the same browser, same CPU, same resolution, but 15x slower.

A Cheap Test: Resolution Sweep

When you hit a heavy rendering workload, one of the first things I do is sweep resolution and measure FPS. It takes five minutes and gives huge insight. Here’s what I saw:

Resolution	FPS
720p	~4
540p	~4
360p	~4

The fallback plan in the design doc was useless from the start. If halving the pixel count doesn’t improve FPS, the bottleneck isn’t fragment processing.

Here’s how to interpret the results:

Observation	Bottleneck	Possible Fix
FPS scales with resolution	Fragment processing (pixel fill)	Lower resolution, simplify shaders, reduce post-effects
FPS doesn’t change with resolution	Geometry / CPU processing	Reduce vertex count, reduce draw calls, cut bone/physics, use GPU

In this case, it was the latter. In a VRM scene, these operations don’t depend on output resolution:

• Skinning: Vertex transformation per bone. Depends on vertex and bone count, not output pixels
• Spring Bone: Physics simulation for hair/clothing. Runs on CPU every frame
• MToon Material: Extra render pass for outlines in toon shading. Increases draw calls

Reducing output pixels doesn’t reduce any of these. So even at 360p, it stayed at 4 FPS.

"Fallback Plans" Should Be Based on Proportionality

My mistake was writing a fallback without verifying what scales with what. The plan said “drop to 540p if heavy,” but I never confirmed whether resolution was the bottleneck.

Better fallback options might have been:

• Drop FPS (30 → 15): Works even if geometry-bound. But quality drop is more noticeable than resolution
• Simplify the scene (fewer spring bones, fewer bones, disable outlines): Effective but harms character fidelity. Hard to predict impact
• Use GPU: Guaranteed. But introduces cost and availability issues

Ultimately, I chose the third option and rented GPU-enabled cloud instances (RTX-class). Once running on GPU, the same scene hit 57–58 FPS. From 4 to 57 FPS — no room for debate.

Lesson in one line: When writing fallback plans in design docs, always include what you’re reducing and what it’s proportional to. If you can’t, it’s not a plan — it’s a wish.

---

Even with a GPU Assigned, Chromium Renders on CPU

Okay, so I decided to use GPU. Then the real fun began.

I spun up a GPU-enabled container, passed --gpus all, launched headless Chromium, and confirmed nvidia-smi was working. Yet WebGL still wasn’t rendering on GPU. It silently fell back to SwiftShader. No errors, no warnings — you only notice when you measure FPS.

Turns out, to actually render on GPU, all three of these must be true. Miss any one, and it drops to CPU:

1. Full-build Chromium (default headless shell doesn’t include GPU path)
2. EGL ICD registration (e.g., 10_nvidia.json + libglvnd)
3. Launch flags (--use-gl=angle --use-angle=vulkan --ignore-gpu-blocklist --no-sandbox)

1. Default Headless Shell Doesn’t Render on GPU

When launching Chromium programmatically, if you don’t specify, it often starts the lightweight headless shell. Great for CI DOM testing, but it lacks the GPU rendering path.

You must explicitly use the full build:

const browser = await chromium.launch({
  channel: 'chromium',   // ← Use full build, not headless shell
  args: [ /* see flags below */ ],
});

I wasted a lot of time here. Everything looked fine: browser launched, page loaded, WebGL context created. But it was rendering on CPU the whole time.

2. EGL ICD Registration Is Required for GPU Detection

Having GPU visibility in the container ≠ having graphics driver entry points registered. The latter uses ICD (Installable Client Driver) files.

• EGL: /usr/share/glvnd/egl_vendor.d/10_nvidia.json
• Vulkan: Files under /usr/share/vulkan/icd.d/

If these are missing, Chromium assumes “NVIDIA EGL not available” and falls back to SwiftShader. In my first validation, the Vulkan ICD directory was empty.

Also, libglvnd (GL vendor-neutral dispatch layer) is required. You can install it directly, but in practice, installing diagnostic tools pulls it in:

RUN apt-get update && apt-get install -y \
      mesa-utils vulkan-tools \   # pulls in libglvnd + useful tools
      xvfb \
    && rm -rf /var/lib/apt/lists/*

In the container, set NVIDIA_DRIVER_CAPABILITIES=all. The default (compute,utility) doesn’t mount graphics-related libraries into the container. “GPU is visible but not rendering” is a classic symptom of this.

3. Launch Flags

--no-sandbox
--use-gl=angle
--use-angle=vulkan          # or gl-egl
--ignore-gpu-blocklist

--use-angle has two options: gl-egl and vulkan. Both gave the same FPS (57–58), but image quality differed. I ended up choosing vulkan (explained later).

--ignore-gpu-blocklist prevents Chromium from self-censoring: “This driver setup is known to have issues, so don’t use GPU.” In containerized environments, this flag is almost always needed.

How to Confirm Success

The worst thing you can do is assume “it launched, so it’s fine.” This stack lies by showing success when it’s not.

Read GL_RENDERER. Get the actual renderer name from within the page:

const gl = document.createElement('canvas').getContext('webgl');
const ext = gl.getExtension('WEBGL_debug_renderer_info');
console.log(gl.getParameter(ext.UNMASKED_RENDERER_WEBGL));

• CPU fallback: SwiftShader, Google SwiftShader, etc.
• Success: Strings containing NVIDIA or adapter name

I made this log output automatically on startup. Even if it works once, image updates can break it, so continuous verification is key.

Measure real FPS. Track requestAnimationFrame intervals over tens of seconds. The results were clear:

State	VRM Scene FPS
SwiftShader (CPU)	~4
GPU rendering success	57–58

Over 10x difference — no ambiguity.

The Order of Stumbles

1. Launched with GPU container → looked fine → actually SwiftShader
2. Added flags → no change → ICD missing
3. Added ICD → no change → headless shell
4. Switched to full build → 57 FPS

I kept tweaking flags, but the real issue spanned three layers: flags, driver registration, and browser build. Each had to be fixed one by one. If I were to do this again, I’d first create a minimal probe page that outputs GL_RENDERER to quickly isolate the layer. Debugging speed increases 10x.

---

Same FPS, but the Image Is Broken — ANGLE Backend Matters

GPU rendering worked. FPS was 57. Capture was flowing. Yet the stream showed the avatar’s head flickering and flashing.

Turns out, switching --use-angle from gl-egl to vulkan fixed it. Nothing else changed.

As shown earlier, both backends gave identical FPS:

Backend	VRM Scene FPS
--use-angle=gl-egl	57–58
--use-angle=vulkan	57–58

Performance identical. Initially, I thought “either is fine, pick the first one that works” and chose gl-egl. That was a mistake.

Symptom: Breakage Invisible to Metrics

In the stream, only the avatar’s head (face and hair) flickered. Frame rate was stable, so it wasn’t dropped frames. Specific material rendering was unstable per frame.

VRM uses MToon for toon shading, which adds extra render passes (e.g., outlines by extruding backfaces). The drawing order and depth handling differ subtly between ANGLE backends, causing this instability.

The key point: this bug didn’t show up in any of our metrics.

• FPS: normal (57–58)
• GPU usage: normal
• Console errors: none
• WebGL context loss: none
• Captured video duration: real-time

It was only visible to the human eye.

Isolation Test

I changed only the backend, keeping everything else constant:

--use-gl=angle --use-angle=gl-egl   → flickering head
--use-gl=angle --use-angle=vulkan   → no flicker

Same image, same page, same GPU, same flags — only --use-angle changed. The flicker appeared/disappeared cleanly, confirming the cause. I later confirmed this in production streams and baked vulkan as a required flag.

Lesson: "Correctness" of Rendering Can’t Be Measured by Performance

This changed two things in my operations:

1. Always visually inspect rendered output after rendering changes. CI metrics (FPS, error count, process survival) don’t guarantee rendering correctness. As seen here, all metrics can be green while the image is broken. Now, whenever we touch the renderer, we capture a screenshot or short clip and review it. “It runs” ≠ “it renders correctly.”

2. Treat seemingly equivalent options as non-equivalent by default. Just because gl-egl and vulkan have the same FPS doesn’t mean they’re interchangeable. If you can’t prove they’re equal, assume they’re not. Pick one as default, document it, and only change it with justification.

Note: ANGLE backend behavior depends on environment (driver version, GPU generation, container libraries). Saying “vulkan is always correct” is wrong. The right conclusion is: inspect your own environment visually and decide.

---

Once on GPU, Capture Turned Black

With correct rendering achieved, the final hurdle was capture.

To stream rendered video, the obvious choice is canvas.captureStream(). It gives a MediaStream from a canvas, ready for recording or broadcasting. It worked fine in CPU mode. But when I switched to GPU rendering, it broke.

• Frames turned black
• Rendering flickered
• Eventually stopped

Same code, same page, only GPU added. Paradoxically, improving the environment broke it.

What Was Happening

The issue was negotiation failure in GPU buffer format (gfx::BufferFormat). Chromium supports multiple buffer formats for GPU-to-CPU transfer. In our container environment, none of the preferred formats were usable.

In CPU mode, this path didn’t exist — no GPU buffers were involved. By moving to GPU, we hit a new code path that failed.

Interestingly, unstable rendering also affected capture in CPU mode. When recording the screen, after 90 seconds, only 6 seconds of video were recorded. The main thread was contending between rendering, readback (ReadPixels), and encoding, causing frames to drop from the timeline. Slow rendering doesn’t just hurt FPS — it corrupts output duration. In live streaming, platforms may stop recognizing the stream as live.

While some issues can be fixed with flags, I decided not to go deeper for two reasons:

1. Highly environment-dependent — no guarantee it won’t break again on the next host
2. Live streaming can’t afford to fail. I didn’t want to rely on a fragile, host-specific workaround in production

The Alternative: CDP’s Page.startScreencast

Instead, I switched to Chrome DevTools Protocol (CDP) Page.startScreencast, which is what DevTools uses for remote inspection. It continuously sends screenshots of the page.

The new pipeline:

Chromium
  └ CDP: Page.startScreencast (JPEG frames stream in)
       └ Node.js paces at 30 FPS and writes to named pipe
            └ ffmpeg: mux MJPEG (video) + FIFO (audio) → RTMP

Unlike captureStream() — which tries to pass GPU textures directly into the media pipeline — screencast returns images. It’s less efficient, but removes the unstable dependency on buffer format negotiation. In practice, this stabilized real-time output on both CPU and GPU hosts.

Bonus: HTML Overlays Work Out of the Box

An unexpected benefit: captureStream() only captures the canvas, but screencast captures the whole page.

In live streaming, we overlay comments, donation banners, and responses as HTML/CSS. With canvas capture, these had to be drawn in WebGL — complex text rendering, wrapping, animations, fonts. All custom.

With screencast, normal HTML/CSS overlays appear directly in the stream.

<!-- This renders directly in the stream -->
<div class="telop">Thanks for the comment!</div>

We can style the telop with CSS, preview in the browser, and deploy. The workaround we chose to avoid a constraint ended up increasing expressive power.

Implementation Notes

1. You must pace frames yourself. Screencast sends frames “as fast as possible,” so intervals aren’t constant. On the Node side, I throttle to 30 FPS before feeding to ffmpeg. Skipping this causes variable frame rate and A/V sync issues.

2. Audio must merge via a separate path. Video is a stream of JPEGs, so audio goes through a separate named pipe into ffmpeg. You must decide which timeline to use as zero point. We start writing video when the first audio chunk arrives.

3. Use separate pipe paths for multi-process setups. If multiple streams run on one host, fixed-path FIFOs collide. Use port numbers or IDs to differentiate paths.

---

Summary

Here’s a recap of the pitfalls I hit while building a 24/7 AI avatar streaming pipeline using headless Chromium:

• CPU rendering (SwiftShader) is unusable for 3D avatars. Simple WebGL hits 60 FPS, but VRM drops to 4 FPS. Since the bottleneck is geometry/CPU-bound, resolution reduction fallback doesn’t help. Fallback plans must specify what scales with what
• Headless Chromium silently falls back to CPU even with GPU present. Success requires three things: full-build Chromium (channel: 'chromium'), ICD registration (EGL/Vulkan + libglvnd), and launch flags. Confirm success using GL_RENDERER string and real FPS. Never trust “it launched”
• ANGLE’s gl-egl and vulkan can produce different images at the same FPS. Symptoms (e.g., flickering MToon materials) won’t show in metrics. Always visually inspect output after rendering changes
• canvas.captureStream() breaks on GPU due to buffer format negotiation failures. Switch to CDP’s Page.startScreencast for stability. Bonus: whole-page HTML overlays become possible. Handle pacing, audio merging, and path collisions manually

The common thread: GPU-involved setups don’t fail loudly — they degrade silently or break in subtle ways. Slow performance, green metrics but broken images, or new breakage when improving the environment. None of these show up if you assume “it launched, so it’s fine.”

The fastest path is to build verification probes upfront: a page that outputs GL_RENDERER, a resolution sweep, and visual inspection of output. That’s what ultimately saved the most time.