Building Virmii: From a Character Image to a Browser-Tracked VTuber

You have a character design. You want to see it move, react, and appear on stream.

Getting there involves several separate jobs: creating a 3D model, preparing its rig and expressions, setting up tracking, and connecting the result to OBS.

I’m building Virmii, an AI VTuber avatar studio that brings those steps into one workspace.

The goal is to give creators a more direct path from a character reference to an avatar they can actually use.

Character artwork featured on Virmii.

Watch the demo

Here’s a look at Virmii in action:

Start with an image—or bring your own avatar

Virmii connects five stages of the avatar workflow:

1. Upload a character reference. Start with a clear image of the character you want to create.
2. Generate a 3D model. Use AI to turn the reference into a textured model.
3. Prepare the avatar. Follow rigging and VRM preparation, then preview the result.
4. Try browser face tracking. Use your camera to test supported movement and expressions.
5. Connect to OBS. Add the private live-stage URL to your streaming setup.

If you already have a compatible VRM, you can upload it and start with previewing and tracking.

You can also keep the completed VRM for compatible external tools.

Expressions depend on the model

A character’s personality often comes through in small details: a blink, a glance, or the way its mouth moves while you speak.

Those movements need support in the avatar itself. Facial tracking cannot make every model behave identically.

Virmii checks the model’s capabilities so you can understand what it supports. Avatars with a suitable facial setup can support blinking and lip-sync; more limited models have a different range of movement.

An expression illustration from the Virmii website. Actual tracking capability depends on the avatar’s facial setup.

How the browser connects to OBS

For developers, one interesting part of the project is the connection between tracking and rendering.

The tracking pipeline uses MediaPipe. Where browser support allows it, camera frames are processed in a dedicated worker, allowing inference to run independently of the page’s animation loop. A main-thread fallback handles other environments.

The tracking page sends compact face-motion packets to the OBS stage over a WebRTC DataChannel.

The connection prefers a direct peer-to-peer route. When network conditions prevent that, TURN provides a fallback.

This lets the tracking page focus on reading motion while the OBS stage renders the character.

A generated model still needs preparation

Image-to-3D generation is one stage in a larger pipeline.

The result needs a usable rig, facial preparation, output validation, and a way to reach the renderer. Behind Virmii’s interface:

• Convex manages backend state and asynchronous job progress.
• Cloudflare R2 stores model files and related assets.
• A Blender-based pipeline handles avatar preparation.
• MediaPipe and WebRTC connect camera-driven motion to the live stage.

One engineering concern is defining what “complete” means.

A job returning a file does not tell us whether that file is ready to become an avatar. The pipeline checks the model’s structure and validates the output before recording a completed version.

That work is less visible than generation, but it matters every time someone opens an avatar and expects it to move.

Try it with your own character

If you’re exploring a virtual identity for a coding stream, developing an original character, or looking for a browser workflow for an existing VRM, I’d love for you to try Virmii.

Bring a clear character reference or a compatible avatar, explore the workflow, and tell me where you get stuck.

Try Virmii →

Which part of setting up a VTuber has been hardest for you: creating the model, getting expressions working, or connecting everything to your stream?