I got tired of burning through cloud api quotas every time an agent writes boilerplate code.
So i tested a tri-tier setup: Gemini 3.8 Flash as the high-level architect, and a local Ternary Bonsai 2 9B MTP model as the actual coder.
Turns out it slashed cloud token usage by 92.7%.
how it works
- Gemini 3.8 Flash (planner): breaks down the task, sets up data structures, CSS tokens, and atomic function signatures (~290 tokens total).
- ConvAI Laya (gatekeeper): fast local router that decides if a step is safe to delegate.
- Bonsai 2 9B MTP (coder): runs locally via prism llama-server on CPU with MTP speculative decoding. it takes the signatures and outputs 100% of the raw code with thinking turned off.
[Task]
│
▼
[Gemini Flash] ──(Interfaces & Constants)──┐
│
▼
[Local Bonsai 2 9B]
│
▼
[Clean Production Code]
the test: building MONSOON
to test this, i had it build a complete website from scratch for a pop-up shop selling Indian coffee and single-estate cacao:
-
index.html: semantic markup, hero with steam particles, terroir origin cards, off-canvas drawer. -
styles.css: warm earth luxury palette, glassmorphism, responsive grid, keyframe animations. -
app.js: tasting flight builder state machine, live INR pricing calculator, and a countdown clock to the Bangalore pop-up.
token breakdown
| file | cloud tokens (planning) | local bonsai tokens (code) |
|---|---|---|
styles.css |
~95 | 1,400 |
app.js |
~85 | 1,200 |
index.html |
~110 | 1,103 |
| total | ~290 | 3,703 |
- cloud tokens spent: ~290
- tokens offloaded locally: 3,703
- net savings: ~92.7%
local bonsai clocked ~6 tok/s on CPU with near 100% draft acceptance via MTP.
repo with all the code & prompt wrappers:
https://github.com/MVYNE06/tri-tier-bonsai-orchestrator

Top comments (1)
The 92.7% figure measures the right thing but against the wrong denominator: cloud tokens saved is not cost saved unless the local side is free, and on CPU at ~6 tok/s you are paying for the coder in wall-clock time instead of API spend. That trade is often worth it, but it should be called a trade.
Where I have seen this split hold up is when the planner's output is both small and machine-checkable — signatures, types, a failing test to satisfy. If the planner only describes intent in prose, the local coder has to re-infer the same context and you pay twice, once in tokens and once in debugging.
Two numbers I would want before trusting it on a real repo: whether the generated files parse, lint and pass tests in CI (3,700 tokens of code that compiles is a very different result from 3,700 tokens that merely read well), and p50 latency per file. 1,400 tokens at 6 tok/s is about four minutes per file — fine for boilerplate, painful in a debugging loop.