For 32 days I ran an experiment: a small "virtual studio" of AI bots building a mobile tower-defense game in Unity, every day, on a home PC without a graphics card (16 GB RAM, free API keys).
The setup
- A Node.js "manager" and 5 bots, each with its own tools (design, code, 3D assets, test, and a writer for the daily diary)
- A Unity Editor plugin listening on 127.0.0.1: create objects and scenes, write C# scripts, compile, enter Play Mode, read the console, take screenshots
- Models: free Gemini / Groq keys, optional local Ollama
What actually mattered
- Verified tools. Every tool reports what really happened (did it compile? is the object in the scene?), not what the model hoped.
- A judge. A separate model call that closes a goal only when the evidence exists. Without it, the bots "finished" things that didn't exist.
- Look before you act. Bots must read the scene or the log before changing them. This alone cut a lot of blind mistakes.
- Memory of outcomes. Every task and its result is logged; daily lessons come from failures; the log can be exported as a fine-tuning dataset.
-
Quota survival. Several providers, model discovery via
/models, resting on HTTP 429, a daily token budget.
What still fails
- Big refactors across many files
- Anything that needs taste: UI layout, level feel
- Long chains of work when the free quota is low
Domain reloads will bite you
If your editor bridge is an HttpListener inside Unity, a script compile reloads the domain and kills it mid-request. Stop the listener in beforeAssemblyReload, let the client retry until the bridge is back, and treat cut-off answers as "dropped", not "failed".
Try it yourself
I packaged a clean skeleton of this system: a manager, a judge, memory, 4 starter bots and the Unity plugin. It's an early starter kit (English, Unity 2022.3 tested, Unreal part experimental), meant to be developed further:
Happy to answer questions about the architecture in the comments.
Written by my AI assistant from the experiment's logs and my notes.

Top comments (0)