When a team shows me an AI feature that is slow, expensive or unpredictable, the cause is rarely the model. It is a decision made in the first hour and never revisited: workflow or agent, and if agent, one or several.
Whoever decides what runs next, your code or the model, sets what the system costs, how it fails and how you debug it. That is the argument of this issue, and the evidence for it comes from the teams who build agents for a living.
If your code decides, you have a workflow. If the model decides, you have an agent. If one model hands work to other models, you have a multi-agent system. Each one is a different bill, a different kind of failure and a different way to debug.
How we know
We read the primary write-ups from teams that build agents for a living, Anthropic, Cognition and Sierra, plus a peer-reviewed study of multi-agent failures, and quote each number in the sentence its source states it in.
Make the workflow your default and let the agent earn its place
Anthropic draws the line in two sentences. Workflows are systems where models and tools are orchestrated through predefined code paths [1]. Agents are systems where the model directs its own process and tool use [1]. The line is not about how clever the model is; it is about who holds the steering wheel.
A support ticket, handled as a workflow
- Classify the ticket: One model call labels it billing, bug or account
- Route by the label: A plain switch picks the next step, not the model
- Draft the reply: One model call writes it from the account's history
- Approve and send: A person reads the draft before it goes out
Every arrow in that track is code you wrote. You can unit test it, retry a single step and explain a bad reply to an auditor. The model does real work inside two of the boxes, but it never decides which box comes next.
Anthropic's advice is to find the simplest solution possible and only increase complexity when needed [1]. Their observation from customer work is that the most successful implementations use simple, composable patterns rather than complex frameworks [1]. A framework that hides the loop makes it easy to hand the model a decision you never meant to give it.
Most business automation belongs here: triaging an invoice, summarising a ticket, drafting a reply for approval, enriching a lead. The path is known before the request arrives, so there is nothing for a model to decide about the order of steps. Give it the judgement inside a step, and keep the order in code.
Every decision you hand the model is paid for in tokens
Anthropic measured what its own research feature spends. Agents typically use about 4× more tokens than chat, and multi-agent systems about 15× more [2]. Each extra decision is another call that rereads the context so far.
Tokens used, compared with one chat exchange [2]
- A chat exchange: baseline
- A single agent: 4×
- A multi-agent system: 15×
That spend buys something. In the same study, token usage by itself explained 80% of the variance in how well the system did [2]. More tokens meant more searching and better answers, on a research task where breadth is the whole point.
The trade is stated plainly: agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense [1]. For a ticket router that runs ten thousand times a day, it rarely does. For a research question an analyst asks once, it often does.
So price one request before you build. Count the model calls on the happy path, multiply by the context each one carries, and compare that with what the task is worth. If you cannot write that figure down, you are not ready to pick an architecture.
An agent that succeeds once has not shown it will succeed twice
Sierra built a benchmark of support conversations with simulated customers, real tools and real policies. All 12 agents they tested had difficulty, and even the best one stayed under 50% average success [3]. These were not toy tasks; they were the returns and booking changes a support team handles every day.
60% drop in the best agent's score when the same retail task had to succeed in eight tries instead of one. [3]
That drop is the number to worry about. The best agent fell to about 25% when the same retail task had to succeed on all eight tries [3]. A workflow that works on Monday works on Tuesday. An agent that works on Monday has rolled the dice once.
At Sierra, our experience in enabling real-world user-facing conversational agents has made one thing extremely clear: a robust measurement of agent performance and reliability is critical to their successful deployment.
Karthik Narasimhan, Head of research, Sierra [3]
Errors also compound. Anthropic notes that minor issues for traditional software can derail agents entirely, because each step builds on the state the last one left [2]. One wrong tool result early in a run becomes the premise for everything after it.
The fix is not a better prompt. It is a habit: log every trace from the first day, read them, and turn real failures into test cases. Hamel Husain, who has helped many teams ship products built on language models, names the cause of failure without hedging [4].
I’ve found that unsuccessful products almost always share a common root cause: a failure to create robust evaluation systems.
Hamel Husain, AI consultant and educator, Parlance Labs [4]
A second agent pays for itself only when the work splits cleanly
Multi-agent systems do win under the right conditions. Anthropic's multi-agent setup beat a single agent by 90.2% on its internal research eval [2]. Research splits well: many independent searches, each returning a short summary to a lead agent.
90.2% better than a single agent on Anthropic's internal research eval, where subagents search in parallel. [2]
The same team is clear about where it does not hold. Domains that require all agents to share the same context, or involve many dependencies between agents, are not a good fit today [2]. Most coding tasks have fewer truly parallel parts than research [2].
Cognition, which builds a coding agent, reached the same conclusion from the other side. Its principle is that actions carry implicit decisions, and conflicting decisions carry bad results [5]. The simplest way to keep decisions consistent is one single-threaded agent whose context is continuous [5].
While I’m optimistic about the long-term possibilities of agents collaborating with one another, it is evident that in 2025, running multiple agents in collaboration only results in fragile systems.
Walden Yan, Co-founder, Cognition [5]
A Berkeley-led team annotated more than 1600 traces across 7 multi-agent frameworks [6] and found 14 distinct ways they fail [6]. Their summary is blunt: despite the enthusiasm, gains on popular benchmarks are often minimal [6].
A useful test before you add an agent: write down what each one owns, what it hands back and what it never sees. If you cannot, you have one agent with extra steps and a larger bill.
Control is the design decision, and the model is a detail
Notice what none of these sources argue about: which model to use. They argue about control. Anthropic reserves agents for open-ended problems where it is difficult or impossible to predict the required number of steps, and where you cannot hardcode a fixed path [1].
That is a test you can run on a whiteboard: list the steps. If you can draw them, write them in code and call the model inside the boxes. If you cannot, hand the model the wheel and pay for traces, evals and a person approving anything that sends, pays or deletes.
The teams I see struggle usually got this backwards. They started with a team of agents because it looked like good engineering, then spent a month debugging handoffs. Start with the workflow, measure it, and let the evidence promote it to an agent one decision at a time.
What to do on Monday
- Write down who decides For your next AI feature, write one line per step saying whether your code, the model or a supervisor model picks what runs next.
- Build the workflow first Ship the version where code holds the order, and move a decision to the model only when a real case shows the fixed path cannot handle it.
- Price one request Estimate the model calls and context per run before you build, then measure it on real traffic and compare.
- Rerun the same task Run each test case several times and count how often it succeeds on every run, not on one.
- Keep a person on sends Anything that sends, pays or deletes waits for a human approval, whatever decides the steps before it.
Build it in the lab
For the feature you are building now, who decides what runs next: your code or the model?
Sources
- Anthropic, Building effective agents (2024)
- Anthropic, How we built our multi-agent research system (2025)
- Sierra, τ-Bench: Benchmarking AI agents for the real-world (2024)
- Hamel Husain, Your AI Product Needs Evals (2024)
- Cognition, Don’t Build Multi-Agents (2025)
- Cemri et al., Why Do Multi-Agent LLM Systems Fail? (2025)
Originally published at learnwithparam.com.
Top comments (0)