I render videos frame by frame with Three.js in headless Chrome. Here is why

Every short video I made cost me an hour in an editor for less than a minute of output. The editing was never the creative part. It was dragging the same cuts to the same words, again and again.

So I asked a different question: what if the video is just data? A script, a list of who says what, and the timings of the words. Everything else, the camera, the captions, the characters moving their mouths, is a rendering problem. Rendering problems I can solve with code.

That became PolyCast, a small open-source tool I just released. Write a script as JSON, and it renders a low-poly 3D podcast video: two characters in a studio, generated voices, lip-sync, and camera cuts and captions cued to the exact word.

How the pipeline works

It runs in this order:

1. The script. A JSON file with the dialogue lines, who says each line, and camera cues. That file is the whole video, nothing else is edited by hand.
2. The voices. Each line is turned into audio with Cartesia. If you do not have a Cartesia key, it falls back to your computer's free built-in voice, so the whole thing can run at zero cost.
3. The timings. Once the audio exists, the tool knows when every word is spoken. Lip-sync is driven by the real audio, and the camera cuts, the screens in the studio and the captions are all cued to those same timings.
4. The render. The scene is Three.js running in headless Chrome. Instead of recording the screen in real time, it steps the scene forward frame by frame and captures each frame, then assembles the MP4.

Why frame by frame

Screen recording drops frames and drifts. If the machine stutters, your video stutters. Capturing frame by frame is slower, but the output is deterministic: the same script always produces the same video, frame for frame. For a tool that other people will run on machines I cannot see, that mattered more to me than render speed.

It also works as a Claude Code skill

This is the part I end up using daily. PolyCast can be added to Claude Code as a skill. I describe the video I want in plain words, and it writes the script, generates the voices, directs the scene and hands me the finished MP4. The 44 second demo on the repo was made exactly like that. I never opened an editor.

Try it

PolyCast is free and MIT licensed: https://github.com/mahamsultana/polycast

The demo video, a frame grid and the full guide are in the repo. If you try it, I would like to know one thing: is JSON the right interface for a video script, or would you rather write it in Markdown? That is the decision I am least sure about.