How Many AI Avatars Can One GPU Handle? Real-World Test Reveals 4 Avatars at ¥7,600 Each per Month

📝 Originally published (in Japanese) at forge.workstyle.tech.

Building an Unmanned System for 3D Avatar Live Streaming

We're developing an unmanned system where 3D avatars automatically handle live streaming. The system boots up a cloud GPU pod at the scheduled start time, the renderer assembles and streams the video, and then the pod is discarded when the segment ends. Since there's no human oversight, three factors directly impact the success of the business and service quality: "how many avatars can run simultaneously," "how the system recovers from failures," and "how quickly it starts up."

These questions couldn't be answered through estimates alone. Renting a GPU for a few hours costs only a few hundred yen. In this article, we'll share three stories of how we measured and designed the system, following the structure of "stumbling block → cause → solution."

• Capacity: How many avatars can run on a single GPU? The answer is 4, at a monthly cost of ¥7,600 per avatar. However, the bottleneck wasn't the GPU.
• Reliability: Despite using the same image, some hosts crashed every 60 seconds. We implemented a mechanism to automatically switch to a different host.
• Startup Speed: Reduced the time from pod startup to streaming start from 4 minutes to 95 seconds.

These three aspects seem independent but are actually interconnected. Faster startup enabled practical host switching, and understanding capacity allowed us to set prices. Let's dive into each one.

1. How Many Avatars Can Run on a Single GPU? - Measured Result: 4

The first number we desperately needed was "how many avatars can run on a single GPU." Without this, we couldn't determine pricing, and without pricing, we couldn't assess the business viability.

Estimates were useless, so we measured it. Here are the results:

Item	Measured Value
GPU	RTX 4000 Ada (Community type, $0.28/hour)
Simultaneous Streams	4 avatars maintaining 720p30 in real-time (Recorded segment: 89 seconds / 89 seconds)
GPU Usage	26%
Bottleneck	CPU (16 vCPU side saturated first)
Estimated Upper Limit	5–6 avatars

And the pricing:

Operation Mode	Monthly Cost per Avatar
24/7 Streaming	Approximately ¥7,600
8-hour Daily Schedule	Approximately ¥2,500

Judging Success by "Actual Segment Length," Not FPS

A common mistake in measuring simultaneous execution is focusing solely on FPS. For streaming, this is insufficient. You need to check if the recorded segment length matches the actual time.

The reason is simple: when rendering fails, the pipeline doesn't "stutter" but skips time. We experienced a case where a 90-second animation recorded only 6 seconds. The FPS logs looked fine, but the output was truncated.

So, we set the success criteria as:

Run N avatars simultaneously for 89 seconds,
All output files must have an actual length of 89 seconds.

With 4 avatars, all files were 89 seconds / 89 seconds. This confirmed that "4 avatars can run simultaneously." We also verified that the screen capture rate was 33fps.

Surprisingly, the GPU Was Underutilized

Running 4 avatars resulted in 26% GPU usage. This means the GPU had more than triple the capacity. The bottleneck was the CPU (16 vCPU). The breakdown explains why:

Process	Uses
3D Scene Rendering	GPU
Frame Extraction	CPU / Transfer
H.264 Encoding	CPU (if software encoding)
Audio Mixing and Muxing	CPU
RTMP Streaming	CPU / Network

The GPU only handles rendering, while the rest of the streaming pipeline relies on the CPU. Assuming "we're renting a GPU" leads to focusing on GPU specs, but the actual limiting factor was the number of vCPUs.

This observation suggests another improvement: using a hardware encoder (NVENC) would free up CPU resources, potentially increasing the number of avatars. When choosing a GPU, "NVENC availability" should be a criterion.

Implementation Notes for Sharing Hosts

When packing multiple avatars into one host, we made one implementation change.

The renderer originally sent audio from the page to ffmpeg via a named pipe (fifo). If this path is shared across processes, host sharing fails. The second avatar would grab the same pipe, causing audio interference. We solved this by making the path unique per port number.

/tmp/audio.fifo            → Not shareable
/tmp/audio-<port>.fifo     → Unique per avatar

Shared resources like temporary files, fixed ports, lock files, and cache directories become issues when sharing hosts. Identifying these beforehand makes measurements smoother.

How We Calculated Pricing

The calculation is straightforward:

$0.28/hour × 720 hours/month = $201.6/month (per GPU)
$201.6 ÷ 4 avatars = $50.4/avatar ≈ ¥7,600/avatar (at ¥150/USD)

Operational hours significantly impact costs. At 24/7 operation, it's ¥7,600 per avatar, but our system schedules streams and automatically terminates pods afterward. With 8 hours daily, it's ¥2,500 per avatar—a threefold difference.

Running streams during off-peak hours when no viewers are present is simply wasting money. The key learning was treating this as a scheduling problem ("which time slots to use?") rather than a necessity for constant operation.

Often-Overlooked Costs

Besides GPU hourly rates, consider these:

• Outbound Bandwidth: Streaming at 2.5Mbps uses ~27GB/day. Clouds with egress fees (~$0.09/GB) add ~¥360/day or ¥10,000/month. GPU-specialized clouds often include bandwidth, creating a cost gap.
• LLM/TTS Generation Costs: A 24/7 AI streamer continuously runs inference. If using external TTS/LLM APIs, costs may exceed the renderer. We opted to use our in-house inference platform.
• Spot/Preemptible Instances: These are cheaper but risky for live streaming due to potential interruptions. Not suitable for long productions.

2. Same Image, But Hosts Crashed Every 60 Seconds

Even with known capacity, a single GPU might not be stable. Using affordable community-type GPUs (where individuals or businesses rent out excess GPU capacity), we encountered this issue:

Run	Host	Behavior
run3	Host A	No disconnections for 10 minutes
run4	Host B	Crashed every 60–150 seconds → recovered → crashed again

The image, settings, and code were identical. Only the physical host differed.

Initially, we suspected our code and checked recovery logs, but eventually concluded it wasn't our fault. The solution was to switch hosts automatically.

Why It's Problematic - Recovery Issues Make Symptoms Worse

In live streaming, this issue isn't resolved by simple restarts. Each renderer recovery:

• Reloads the page and reconnects to the server
• Causes the avatar to repeat greetings upon reconnection
• Misses the closing remarks meant for the end of the program

Viewers see a stream where the avatar reintroduces itself every minute. The more robust the recovery, the stranger the symptoms, making it a tricky problem.

Design: Two-Layer Host Switching

We implemented a two-component solution. One alone wasn't enough.

Layer 1: Renderer Self-Reports Failure (Time-Window Burst Detection)

The renderer already had self-recovery for crashes (e.g., ffmpeg or page crashes). We added a time window to recovery counts:

If recoveries exceed 4 within a 600-second window,
Terminate the process with exit code 1.

The key is not using cumulative counts. In long streams, even healthy hosts recover a few times nightly. Cumulative counters would flag healthy long streams. We focus on "concentrated failures in a short time" using a sliding window.

Self-termination seems counterintuitive but is the most reliable way to signal "this instance is faulty" to higher layers.

Layer 2: Scheduler Monitors Pod Health and Switches Hosts

The scheduler, managing stream lifecycles, now monitors pod status:

• Periodically checks pod status
• Detects EXITED (including Layer 1 self-termination) or API 404 responses (pod disappeared)
• Upon detection, recreates the pod on a different host (up to 3 retries)
• Excludes pods already in closing sequences

Treating 404s as "disappearance" rather than errors was crucial. Treating them as exceptions would trigger retry loops. Deleted pods also return 404s on deletion requests, which is harmless (the pod is already gone). While logs show ERROR, it can be ignored.

Validation: Manually Trigger Failure and Verify Recovery

After implementation, we tested by manually deleting a pod during a stream:

Pod deletion
  → Detected disappearance in 32 seconds
  → Recreated pod on a different host
  → Renderer started, stream resumed
  → Program ended automatically, pod discarded

Without failure injection, recovery code often remains untested. Crude failure methods are fine; manually triggering failures is reliable. Here, deleting the pod via API mimicked a faulty host.

Remaining Constraints - Host Switching Relies on Speed

Honestly, it's not perfect. Switching hosts mid-stream can cause the stream to end prematurely. YouTube automatically terminates streams on disconnection, sometimes closing before the new pod starts. Since we don't recreate stream keys, the program ends.

The solution is restarting faster than the stream closes. This ties into the next section. Host switching is only practical with fast startup. We reduced startup time to 95 seconds by baking everything into the renderer image. Some cases still fail.

Decision Guidelines

Community-type GPUs are cheap ($0.24–0.28/hour) but inconsistent. We use them as follows:

Use Case	Choice
Development experiments, short tests	Community type (cost-effective)
Production streams, long tests	Secure type (operated by businesses, more consistent)

Regardless of choice, implement host switching. Even secure types fail; frequency differs.

3. Reducing Startup Time from 4 Minutes to 95 Seconds - Eliminating "Setup at Startup"

Host switching effectiveness depends on startup speed. Initially, startup to streaming took ~4 minutes. This delay directly impacted program starts and caused silence during host switches. Using a custom image reduced this to 95 seconds. Here's what we changed.

Initial Approach: Official Image + Startup Setup

For quick development, we used this setup:

1. Start pod with Playwright's official image
2. Fetch setup script from external source at container startup
3. Script installs packages via apt (Vulkan tools, Japanese fonts, ffmpeg, etc.)
4. Configures GPU driver ICD settings
5. Downloads and extracts app archive, runs npm install
6. Starts renderer

This worked during development. No image rebuilds were needed for code changes, speeding up iteration. During GPU experimentation, this agility was invaluable. Issues arose when moving to production.

Three Problems with Startup Setup

1. Simply Slow

Apt index updates, package downloads, and npm dependency resolution happen every time. A 4-minute delay is unacceptable for a streaming service.

2. Increased External Dependencies

Each startup connects to apt mirrors, npm registries, and our asset server. If any service is slow, streaming delays. Our reliability depended on external services' performance.

An incident occurred when our asset server's nginx, configured with worker_processes auto, spawned 196 workers, hitting memory limits and preventing stream starts (containers with auto use the host's core count). Components added for streaming ended up blocking it.

3. Occasional Write Failures at Startup

This was tricky. Writing to the filesystem (overlayfs) immediately after container startup sometimes failed, affecting ICD settings. Failures meant GPU rendering wouldn't initialize, and the startup would continue with CPU rendering.

Chromium falling back to CPU rendering shows no errors, resulting in "streams starting with choppy video." Like the "time skips" issue in Chapter 1, retries were added but didn't address the root cause.

Solution: Bake Everything into the Image

We created a custom image, moving all startup tasks to build time:

FROM mcr.microsoft.com/playwright:v1.54.0-noble

RUN apt-get update && apt-get install -y \
      ffmpeg \
      fonts-noto-cjk \        # Japanese fonts (prevents tofu text)
      vulkan-tools mesa-utils \
      xvfb \
    && rm -rf /var/lib/apt/lists/*

# Bake NVIDIA EGL/Vulkan ICD settings (prevents startup write failures)
COPY icd/ /usr/share/

COPY app/ /app/
RUN cd /app && npm ci --omit=dev

Now, startup only involves pulling the image and starting the process. Pod creation to YouTube live transition takes 95 seconds—a 2.5x improvement from ~4 minutes.

Trade-offs and Handling

Honestly, baking has trade-offs. Initial image pulls are heavy. Playwright's official image is large, and adding more increases size. On uncached hosts, pulls can take minutes (we observed 9 minutes even with the old method).

Thus, 95 seconds assumes a cached host. First pulls take longer, which must be estimated honestly.

We chose baking because time variance is reduced. Startup setups depend on multiple external services, causing unpredictability. Image pulls are a one-time, cacheable dependency. Consistent slowness is easier to manage than occasional slowness.

Unexpected Benefits

• No need for an asset server (eliminating nginx accident risks)
• Shorter startup logs, faster debugging (previously, apt and npm outputs buried errors)
• Fixed versions ("Worked last week, not today" issues disappear. Startup apt implicitly uses the latest versions)

The last point is most significant. Startup setups leave reproducibility to chance. While it appeared as a speed problem, it was rooted in reproducibility and dependencies.

Summary - Measure Before Designing

We determined three key metrics through measurement:

Capacity

• Judge host sharing by actual segment length, not FPS (rendering failures skip time)
• RTX 4000 Ada ($0.28/hour) handles 4 avatars at 720p30, with an estimated upper limit of 5–6
• Bottleneck is CPU, not GPU (26% GPU usage). Include NVENC availability in GPU selection
• Pricing: ¥7,600/avatar for 24/7, ¥2,500/avatar for 8 hours daily. Operational hours impact costs threefold

Reliability

• Community GPUs may have faulty hosts. Don't over-blame your code; consider hardware variance early
• Two-layer solution: Renderer detects bursts in a time window and self-terminates; scheduler recreates on a different host
• Treat API 404s as "disappearance." Manually trigger failures to test recovery

Startup Speed

• Startup setups (apt/npm/asset fetches) are powerful for development but cause slowness, external dependencies, and occasional write failures in production
• Baking everything into the image reduced time from ~4 minutes to 95 seconds. Trade-off is heavier initial pulls, but consistent slowness is preferable
• Baking improves reproducibility, not just speed. Avoid worker_processes auto in containers

These aspects are interconnected. Faster startup enabled practical host switching, which allowed using affordable community GPUs for longer, and measuring on these GPUs provided pricing data. Cloud resources aren't uniform. Design for instance replacement, not repair—and measure capacity, startup time, and pricing with actual tests. It's the most cost-effective approach.