The most common advice for multi-agent systems: start with a team. CEO agent, CTO agent, sales agent, marketing agent. Looks great in diagrams.
Here's why it usually fails, and a better mental model.
The Org Chart Problem
When you build a "team" first, you end up with agents that have overlapping responsibilities, unclear handoffs, and no good answer to: "who actually does this?"
The CEO agent defers to the CTO agent. The CTO agent delegates back to the CEO. Both write status updates nobody reads. The sales agent and marketing agent both think they own "outreach." You have 5 agents producing 1 meeting's worth of real output.
It looks like an organisation but it works like a committee.
The Unix Pipeline Model
Unix got it right in the 70s: small tools that do one thing well, connected through explicit interfaces.
Instead of "sales agent" (vague), build:
- One agent that finds contacts matching a profile
- One agent that writes personalised outreach emails
- One agent that tracks response status and schedules follow-ups
Each does one thing. Each has a clear input and output. You can test each in isolation. When something breaks, you know which stage failed.
The "team" emerges from composition, not from building a team upfront.
The Practical Difference
Org chart approach: Define roles, prompt each agent with their role, hope they coordinate.
Pipeline approach: Define tasks, build agents around tasks, connect outputs explicitly.
The first produces agents that argue about ownership. The second produces agents that actually ship work.
Why? Because when you build an agent for "handle sales," you're encoding organizational ambiguity into the prompt. The agent has to guess which sub-task to pick first. Should it prospect? Should it close? Should it manage the pipeline? Three different jobs, one agent, no clear answer.
When you build an agent for "write personalised outreach email given a contact profile," there's no guessing. It reads the input, does the job, returns the output. Done.
The Problem With Mixing Layers
Most team-based agent architectures end up with agents at different abstraction levels:
- CEO agent: decides strategy
- Sales agent: executes sales tasks
- Tools: CLI commands, APIs
Now the CEO agent has to think about high-level strategy, but also understand when to invoke the Sales agent, and what format the Sales agent expects. The Sales agent has to understand what the CEO wants, but also manage the nuts-and-bolts of email sending.
Compare this to:
- Strategy layer: "find 10 contacts in fintech, write cold emails, track responses"
- Task agents: prospect agent, email agent, tracker agent
- Tools: CLI, APIs
Each layer has a clear job. The strategy layer doesn't know how emails are sent. The email agent doesn't know why it exists. Everything is composable.
When You Do Need a Team
Multi-agent hierarchy makes sense when you need:
- Parallel execution — tasks that genuinely run simultaneously and then merge (scrape 50 sites in parallel, then analyze results together)
- Specialised expertise — different models for different tasks (Claude 3.5 for reasoning, Claude 3 Haiku for light ops tasks)
- Supervision — a coordinator that monitors outputs and intervenes on failures
But the default should be: one agent, one job, clear interface. Add complexity only when you can articulate exactly why the simpler version can't work.
The Architecture That Ships
Our most reliable agent fleet is structured like this:
Input (command or event) → Routing agent (decides which pipeline to invoke) → Task agents (specialised, composable) → Merge layer (combines results) → Output (action or notification)
No CEO agent. No org chart. No role confusion.
When a task fails, we know exactly which agent failed. When we want to add a new capability, we write a new task agent and add it to a pipeline. When we need to optimize costs, we swap a Claude 3.5 agent for Haiku on low-complexity tasks.
The org chart is a reporting structure, not an execution model. Most agent builders confuse the two.
Start Simple, Add Complexity Only When Needed
Don't open with: "Here are my 5 agents and how they interact."
Open with: "Here's the work I need done. What's the simplest way to break it into steps?"
Then build one agent per step. Let the pipeline structure emerge.
You can always promote a pipeline to a supervised multi-agent system later. You can't easily refactor a broken org chart once it's 3 layers deep.
If you're building multi-agent systems, check out Mission Control OS — we've been running it in production for a year.
Top comments (0)