AlfaCan Defence Group — the team behind Alfa-Coder
Most AI coding tools are one model in a chat. For the past months we've been building something else: a small engineering team where every seat is an AI model, and no seat is allowed to check its own work.
An Architect writes the design. Code implements it, splitting the work across parallel agents. A Breaker attacks the result like an adversary. A Skeptic checks which of the Breaker's findings are real. A Reviewer turns the confirmed ones into a fix plan. A Validator signs off only with evidence for every acceptance criterion. A Curator watches every stage for drift from the spec. Each seat runs on a model from a different lab. Before any code, the seats hold a meeting: each answers independently, they argue only where they disagree, and a human approves the plan.
One rule matters more than the rest, and the program enforces it, not a prompt: Code cannot hand over a stage until the build is green and the tests pass.
This week it built an OAuth and webhook core for a publishing service — about 1,800 lines, 46 tests against a real Postgres — in roughly 90 minutes, and fixed four bugs nobody had asked it to look for, including an SQL type error that would have returned a 500 on every new connection.
Getting there broke in ways we didn't expect. Here are the ones worth sharing.
1. The reviewer that kept saying the same thing
The Curator reads each stage's work and raises notes: "requirement X is not met", "this number isn't in the spec". It kept raising the same note again and again in slightly different words, and the stage looped on it.
The model wasn't the problem. We were feeding it only the number and title of its last 40 notes — no status, no answer from the stage, no objection. It honestly didn't know the note had been fixed. It also had no way to see deleted lines, so a fix that removed code looked like no change at all.
We gave it the full history of its notes, with statuses and answers, and real diffs with removed lines. Then we added duplicate detection: a note in other words is not a new note, and one note gets at most two rounds.
That fix created the opposite bug. "Migration 0019 is not applied" and "Migration 0020 is not applied" now looked like the same note, and the second one disappeared silently. So did "no tests for the payment block" versus "no tests for the delivery block". Similarity on headlines is not enough: numbers, identifiers, paths and HTTP methods have to tell notes apart, and anything that is only probably a repeat must never be dropped silently. It goes to the human once, as one question at hand-over: "these look like earlier notes — give them to the stage, or take them off?"
It took four rounds of adversarial review on that one module before nothing slipped through in either direction.
2. The thinking model that rejected its own agents
Code runs parallel agents for reading and writing. The first time we turned them on in production, every agent died on its second request with HTTP 400: "The reasoning_content in the thinking mode must be passed back to the API."
The main Code seat worked fine against the same model. The difference turned out to be one line. After tool results, the main loop always appends a block of environment details — so to the model, the next request is a new turn. The agent loop sent the tool results and nothing else, so the model treated it as the continuation of its own turn and wanted its reasoning back.
We checked it against the live API in two requests: without a trailing line, 400; with one, 200. The fix is a short "Step N of M" line after tool results, and keeping the reasoning in the agent's turn for providers that need it back.
The uncomfortable part: we had a test for exactly this request shape, and it locked in the broken shape, because it ran against mocks. Now every release runs the agents live on every model the seats use before it ships.
3. "I approve the plan, but…"
The human approves the plan with a fixed phrase. People write more than the phrase, so we accepted the phrase plus an explanation. That opened a hole: "I approve the plan, because it looks fine, but do stage 2 on Redis" counted as an approval, and the amendment was lost.
We spent five review rounds on a list of words that mark an amendment — "but", "only", "instead", "later" and so on. Every round, the reviewer found a new phrasing that got through: "because ok with Redis instead of Postgres", "because tests on the client", "because ok (stage 2 on Redis)". Any open vocabulary can be assembled into an amendment.
The version that held: after the approval phrase, only a short reason from a closed list of whole phrases counts — "all good", "the plan is clear", "no issues", "it's urgent". Anything else is an amendment, and the program asks again. A false approval launches work nobody asked for; an extra question costs one reply.
4. Our own gateway was spending 30× what the client asked
A gateway sits between the editor and the model providers. It capped every answer at 32,768 tokens and filled the cap in when a request didn't set one. When we raised the cap, an adversarial review found that the gateway was rewriting budgets the client had set: the Curator asks for at most 4,096 tokens, and the provider received 131,072. On a looping model, that is thirty times the intended cost and time.
Rule we now enforce: the smallest valid budget the client sent wins, the cap only lowers it, and the cap is filled in only when the client set nothing.
The same review caught that a 131,072-token answer can't finish inside a 25-minute turn at roughly 45 tokens a second. The turn would end in a cut-off stream and a fully paid failure. The real maximum is what fits in the deadline.
5. When a provider goes down for an hour
One afternoon a provider started answering 500 to everything for about an hour, and the seat running on it stopped. So each external model now has a spare: on a provider failure, or when all its seats are busy for more than 15 seconds, the request goes to a model from another lab — before the first byte of the answer, never in the middle of a stream.
The first version of the circuit breaker that stops hammering a dead provider had its own bug: it counted our queue — seats busy with long turns — as a provider failure. At peak load it would have sent everything to the spare while the main model sat idle, halving throughput. Our own load is not their outage.
What we took from it
- Most of the bugs were not in the models. They were in what we showed the models, and in how we read their answers.
- Every fix to a fuzzy rule created a new edge, so each one went through an adversarial cycle: an attacker model, a skeptic model, fixes, and a control round, until a round found nothing serious.
- Tests against mocks prove the code matches our assumptions about a provider, not the provider. Probe live.
Try it
Alfa-Coder runs as a VS Code extension or in a browser workspace. You pay for working hours, not tokens, and time spent waiting on a model provider isn't billed.
We're looking for developers to break it. There are 10 free hours: register at https://ai-studio.alfa-can.com/alfa-coder and press the test button on your dashboard. We'd value blunt feedback — what broke, what annoyed you, what you'd never use. We'll be in the comments.
Top comments (0)