Video-Coder: The Open-Source Agent That Scripts a 3D Scene in Blender Before Making Your Video

AI video generators are brilliant artists and terrible physicists: balls float, objects melt, and motion ignores gravity. A research team from six institutions has a different idea — do not ask the video model to imagine the scene; build the scene first, in code. Released on 9 November 2026, Video-Coder is an open-source agent that writes Blender Python code to construct a dynamic 3D scene, renders it as a draft, and only then hands over to the video generator.

How Video-Coder works

1. Stage one — script the scene: the agent iteratively writes, executes and refines Blender Python programs that assemble a 3D scene — objects, positions, movement — and renders a draft video from it.
2. Stage two — guide the generator: that draft gives the downstream video generator explicit structure and motion signals: what is where, and how it should move, instead of leaving physics to be guessed from a text prompt.

The team describes Video-Coder as the first attempt to train a code-driven video generation agent — the scene is a program, so it can be checked, corrected and re-run, which a pure text prompt never can.

The results: +7 on physics, +24 on video quality

Applied to the MiniMax H3 video backbone, the team reports:

• PhyGenBench (physical plausibility): a gain of roughly 7 points.
• VBench-2.0 (overall video quality): a gain of roughly 24 points.

The honest caveats: these are the research team's own reported numbers, measured on a single backbone (MiniMax H3). They show the approach works on that setup; how it transfers to other generators is exactly what the open-source release lets others test.

Who built it

Video-Coder comes from a six-institution collaboration: the University of Central Florida, the University of Würzburg, Microsoft Research, Tsinghua University, UNC-Chapel Hill and UW-Madison. The full title of the work is "Video-Coder: Reinforcing Agentic Coding for Video Generation" — the agent was trained with supervised fine-tuning plus agentic reinforcement learning, rewarding both code that runs and video that looks right.

It is open source — data, models and code

The team has fully open-sourced the project: training data, models and code are public, and the system is reproducible. The project's Hugging Face hub is the training-data repository at huggingface.co/datasets/Video-Coder/Train-Data, which links onward to the project page, the paper and the code.

How to try it (as far as the public release supports)

1. Start at the Hugging Face page: open Video-Coder/Train-Data — this is the project's official data hub.
2. Look at the training data: the repository holds two JSON files — sft_data.json for the supervised fine-tuning stage and rl_data.json for the reinforcement-learning stage (about 2.73 GB in total) — free to download and inspect.
3. Follow the links on that page: the dataset card links to the project page, the paper and the code repository — the code is where the inference pipeline lives.
4. Bring the right tools: the agent drives Blender through Python, and its output conditions a separate video generator (the paper's experiments used MiniMax H3) — so this is a developer/research setup, not a one-click consumer app.

Bottom line: Video-Coder's bet is that the cheapest way to fix AI video physics is not a bigger video model, but an executable 3D draft in front of the one we already have. The benchmark gains say the bet is worth testing — and because everything is open, anyone can.

Sources: Emergent, "Video-Coder Agent Scripts 3D Scenes Before Video Gen", 9 November 2026 — emergent.sh; Video-Coder project data page, Hugging Face — huggingface.co/datasets/Video-Coder/Train-Data. Benchmark figures are the research team's own reported results.

Originally published on Ai Game Changer Pro.