I Added More AI Agents to the Problem. Nothing Changed.

I built one agent and multi-agent versions, put them through the same tests, and learned what actually mattered.

Part 12 findings of an experiment: building an LLM-powered support agent with deterministic boundaries. The companion repo contains the full code.

---

"We considered multi-agent and decided against the complexity" is the most self-satisfied sentence in software architecture. It's also unfalsifiable, which is why it's so popular.

So I built the thing I was going to claim I didn't need. A triage agent that routes. A refund specialist owning the order tools and the approval gate. A knowledge specialist answering from the corpus. A coordinator holding them together.

Both versions implement the same interface, so the eval suite grades them without knowing which is which.

// AgentTeam: the coordinator, in full
public AgentRun run(EvalScenario scenario) {
    AgentSession session = new AgentSession(scenario.customerId());
    List<ScoredArticle> retrieved = knowledgeBase.search(Query.of(scenario.message()));
    Handoff handoff = triage.route(scenario.message());

    if (handoff.lane() == Lane.KNOWLEDGE) {
        Answer answer = knowledgeSpecialist.answer(retrieved);
        return new AgentRun(Classification.unclassified(), null,
                answer.reply(), answer.fromKnowledge(), retrieved);
    }

    Result result = refundSpecialist.handle(session, handoff);
    var classification = new Classification(
            ActionType.PROCESS_REFUND, handoff.orderId(), "requested via chat");
    return new AgentRun(classification, result, result.detail(), false, retrieved);
}

triage.route calls the same IntentClassifier the single agent calls, and refundSpecialist.handle calls the same scoping, eligibility and gate objects. That reuse is the experiment being fair, not the experiment being rigged: change either one and the diff would measure my rewriting instead of the architecture.

The result

$ ./gradlew architectureComparison

PROPERTY             BASELINE  CANDIDATE    CHANGE
safety                  1.000      1.000    +0.000
gate-outcome            1.000      1.000    +0.000
intent-accuracy         0.875      0.875    +0.000
groundedness            1.000      1.000    +0.000
answered                0.667      0.667    +0.000

FIXED   (0)
BROKEN  (0)

Not a single scenario changes verdict. Not one property moves by a thousandth.

What it cost, counted from the source: one production type became five, 91 lines of code became 127, one orchestration hop became two. On an LLM-backed stack, where each agent makes its own model call, one request would become at least two.

Why it came out flat

The routing decision is the intent classification. That already existed — the single agent has been doing it since post 6.

And once routed, the specialists call the same scoping check, the same policy engine, and the same risk gate, in the same order. Not because I copied the code, but because that order is a business requirement. You cannot evaluate eligibility before you know the order is the customer's, and you cannot propose a refund before you know it's eligible.

Splitting the caller changed who invokes the boundary. It didn't change what the boundary does — and the boundary is where every guarantee in this system lives.

flowchart LR
    subgraph SA["Single agent"]
        direction LR
        M1["Message"] --> C1["Classify, retrieve"] --> B1["Scope, eligibility, gate"]
    end
    subgraph TM["Agent team"]
        direction LR
        M2["Message"] --> T["Triage<br/>(the same classify)"] --> SP["Specialist"] --> B2["Scope, eligibility, gate<br/>(the same objects)"]
    end
    SA --> R["Identical on all<br/>5 eval properties"]
    TM --> R
    classDef step fill:#eef2f6,stroke:#8fa3b8,color:#24313f
    classDef same fill:#ecf2ed,stroke:#93b39d,color:#3d5344
    class M1,C1,M2,T,SP step
    class B1,B2,R same
    style SA fill:#f7f9fb,stroke:#c5d1dc,color:#24313f
    style TM fill:#f7f9fb,stroke:#c5d1dc,color:#24313f

What multi-agent actually buys

I'm not arguing the pattern is useless. I use it daily in a different context: coding agents that delegate to subagents. One runs a broad search while another reads a diff, each with its own context window and tool set.

There, the split pays for itself immediately. The work is genuinely parallel, the contexts are genuinely separate, and a subagent burning through 40 files costs the parent nothing.

None of those conditions hold here. One customer message, one lane, sub-second work, one small tool set. The delegation would be a handoff with nothing to hand off.

The dishonest version of this post shows a strawman team and declares victory. So, plainly: what I built is the structural version of multi-agent. Separate responsibilities, a handoff, a coordinator.

It isn't agents that each make model calls and negotiate at runtime. That variant buys real things — per-role prompts, per-role tools, parallel execution. It also doubles the model calls, adds latency, and introduces a failure mode I don't have today: two agents disagreeing about what the customer wants.

What it wouldn't do is move the deterministic boundary.

What would change my mind

Four things, and they're in the ADR so I can be held to them:

• A second and third action type with genuinely disjoint tool sets, where one agent's tool list stops fitting in a reviewable prompt
• Work that can run in parallel and is slow enough for latency to matter
• A reason to run different models per role, for cost or capability
• Any eval run where the team beats the single agent on a property

That last one is the real safeguard. MultiAgentEquivalenceTest runs both architectures on every build and asserts the difference is zero. The day it fails, this decision gets reopened by a test rather than by an argument.

The team stays in the repo — wired, tested, and not the default. Deleting it would turn evidence back into taste, and the whole point was to have grounds for the claim.

---

What's the architecture you rejected, and can you still run it?

Comments

[Jo Do]

"Unfalsifiable, which is why it's so popular" - ouch, and true. Building the multi-agent version just to test the claim is the honest way to settle it. The eval-blind interface trick (the suite grades without knowing which is which) is the piece most comparisons skip, and it is what makes your null result mean something: same tests, same grader, no storytelling. Routing complexity has to earn its keep against exactly this.