Luhui Dev

Posted on Sep 4 Originally published at luhuidev.com

The Bumps and Bruises of Building Multi-Agent Systems

#ai #agents #luhuidev

🙋‍
I’m Luhui Dev, a developer who has been breaking down Agent engineering and exploring how AI can be applied in education.
I focus on Agent Harness, LLM application engineering, AI for Math, and the productization of education SaaS.

Introduction

When you first build a Multi-Agent system, its form is very appealing.

One agent plans, another executes, and a third reviews. Add a researcher, programmer, tester, and supervisor, and the console fills with tasks, messages, and tool calls. Several agents running at once looks impressively professional.

But the deeper I went, the clearer the other side became: the work is split, but responsibility is not; messages are sent, but information is not actually acted on; the review flow finishes, but the result is still unusable.

With one agent, failures are usually legible. It misunderstood the request, a tool call failed, or it wrote incorrect code. You can normally trace one execution path back to the cause. Multi-Agent systems distribute the failure across task decomposition, context transfer, shared state, role permissions, and acceptance criteria. A mundane failure at the end may have started ten turns earlier.

After building Plan-Exec flows, adding Supervisors to execution agents, and following experiments from Anthropic, Google, and Cursor, I have settled on one view:

Multi-Agent does not turn one smart person into a company. It turns one model's problems into a set of organizational problems.

Here are seven practical lessons.

1. Five named roles do not mean a task can be divided into five parts

The easiest thing to do in Multi-Agent design is define roles: a product manager understands the request, an architect designs the solution, a programmer writes code, a tester validates it, and a CEO makes the final decision.

It reads well because it copies the shape of a human organization. In execution, it often does not work that way.

While designing a Plan-Exec workflow, I saw a Planner produce a seemingly complete task list while the Executor had no idea what verifiable artifact each step should produce. One task said “analyze the geometric relationships in the image”; another said “draw the figure from the analysis.” But nobody specified whether the first agent should return prose, a structured relation graph, or executable geometry DSL. The first agent wrote an analysis; the second looked at the image again. The division of labor added only one more act of retelling.

In an analysis of seven Multi-Agent systems and more than two hundred execution traces, the Berkeley team found that about 41.77% of failures were specification and system-design failures: agents disregarding requirements, ineffective roles, repeated steps, lost context, and ambiguous completion.[1]

The issue is not that role names lack detail. Human roles come with accumulated experience, responsibility boundaries, professional standards, and shared defaults. Putting “you are a senior test engineer” in a system prompt adds linguistic context; it does not create those institutions.

Roles are narrative. Interfaces are engineering. If a subtask has no clear input, output, completion criterion, and failure state, it should not be handed to another agent.

2. Apparently parallel tasks often hide an unbreakable chain of thought

Multi-Agent systems work well for research, and that is unsurprising. Investigating dozens of companies, searching several repositories for vulnerabilities, or collecting scattered facts from many pages can be split by object or direction. One unsuccessful branch does not break the others.

But Anthropic's Research team explicitly notes that tasks with extensive shared context and mutual dependencies are not a good fit for today's Multi-Agent systems. Coding is especially relevant: the truly independent pieces are usually fewer than they first appear.[2]

Parallelism can shorten the amount of work. It cannot shorten dependencies.

Google Research reached a similar conclusion across 180 agent configurations. On financial tasks that supported parallel analysis, centralized Multi-Agent systems outperformed one agent by 80.9%. On PlanCraft, which emphasizes continuous planning, every Multi-Agent architecture declined by 39% to 70%.[3]

Every handoff needs the current state to be explained again. One agent compresses its full judgment into an output; the next reconstructs the situation from that output. Continuous reasoning becomes fragments, and the system spends more tokens explaining itself than solving the task.

3. As agent count grows, duplicate work grows first

Multi-Agent logs create an easy illusion: agents search simultaneously, the Supervisor issues tasks, executors call tools in turns, and the interface continuously says “analyzing,” “validating,” and “improving.” The system looks busy.

Look closer and you often find similar search results, equivalent analyses, and project context repeatedly read by different agents. More tokens were spent, but not proportionally more information was produced.

MAST reports repeated steps as its most common individual failure, at 17.14%. One plain example: a user asked for the first ten tracks of a playlist. The Spotify agent could return all of them at once, but the orchestrator had ten exchanges and retrieved one track each time.[1]

This is usually not a missing model capability. It is a system that does not know the right granularity for allocating work.

There is another pattern in development. A Supervisor sees a wrong output and gives broad natural-language feedback. The Executor cannot locate a particular defect, so it reruns the entire task. The new result differs slightly, the Supervisor checks again, and neither agent repairs a known issue. They merely sample the same area repeatedly.

Some Multi-Agent systems are not collaborating. They are manufacturing reading material for one another.

4. Give a task pool to autonomous agents, and hard work often gets no owner

An attractive design lets equal agents inspect a shared task pool and decide what to do next. It matches the intuition of emergent collaboration: no fixed flow, just agents claiming work, updating state, and waiting for each other's output.

Cursor tried this in long-running coding experiments. Agents coordinated through shared files and locks intended to prevent duplicate claiming. Agents held locks for a long time, forgot to release them, exited while holding them, or edited files without taking them. With twenty agents, effective throughput was equivalent to only one to three agents; most time was spent waiting.

Removing pessimistic locking and switching to optimistic concurrency reduced file contention, but brought back ownership problems. Agents selected local, safe, easy changes; no one took on the difficult, risky core work. Small commits accumulated while the hardest part of the project did not move.[4]

This resembles diffusion of responsibility in human organizations. The difference is that an agent feels neither deadline pressure nor career consequences for avoiding difficult work.

A final delivery cannot emerge reliably from workers with no final owner.

5. Information posted to the group is not information known by the system

In Multi-Agent systems, a crucial fact often appears but has no effect on later actions.

MAST records a case where a Phone Agent discovered a special API requirement for username format but did not communicate that constraint explicitly to the Supervisor. The Supervisor did not ask, kept trying the wrong format, and the task failed.[1]

I see this regularly in real execution. A Supervisor notices a wrong point in a geometry figure but says only “the figure does not fully match the prompt,” so the execution agent adjusts line styling. An agent discovers a contradiction in the requirements but places it as an ordinary note at the end of its response; the next agent continues with the original plan.

Natural-language messages carry tone, priority, and an action cue for people. An agent receives only context. Unless the system explicitly distinguishes constraints, decisions, and blockers, it may not understand why a fact matters.

6. Copying one model ten times creates ten similar blind spots

We often assume that if one agent can be wrong, three agents voting must be safer; if an execution agent misses an issue, a Reviewer will find it. That logic requires their errors to be relatively independent.

When agents use the same model, similar prompts, and the same source material, that assumption often fails. Anthropic found that eighteen of thirty agents created exactly the same Git branch name. Multiple creative agents chose the same novel title without genre guidance. When asked to create an impressive project freely, more than half chose a ray tracer or self-hosting compiler.[5]

At one-task scale, homogeneity looks like repetition. At system scale, it can become congestion, a stampede, or a shared misjudgment.

Google's research also found that independently operating, uncoordinated agents could amplify error up to 17.2 times. An orchestrator reduced amplification, but it was still 4.4 times.

When several agents agree, do not treat their count as the number of independent pieces of evidence. They may have followed the same training bias and prompt anchor to the same wrong answer.

7. Agent debate often gives an error better reasons instead of correcting it

Some systems try to solve homogeneity through debate: one agent proposes an answer, another challenges it, then two or three rounds end in a vote.

It resembles peer review. In practice, the first agent may give a fluent but wrong explanation and the others continue from its premise. Later reasoning becomes richer, while the original error becomes more convincing.

A study of ten homogeneous agents debating for three rounds found majority adoption as high as 85.5%. Some agents that had initially answered correctly changed to the wrong answer after seeing peers' views. Compared with isolated self-correction, debate used 2.1 to 3.4 times as many tokens with equal or lower accuracy.[6]

Models are particularly good at continuing existing text. Once the first answer enters the shared context, later agents are not starting from a blank page. They may be reviewing it, or simply being influenced by it.

Debate without independent judgment easily becomes expensive mutual affirmation.