A single, well-designed LLM call almost always beats a multi-step agent chain on cost, latency, and reliability — unless the task genuinely requires the model to observe an intermediate result before deciding what to do next. Most "agent" features founders ask us to build don't need that. They need one good prompt with structured output.
Why chains are the default anyway
Multi-step chains are seductive because they mirror how a human would solve the problem: research, then draft, then review, then revise. Frameworks make it easy to wire up four LLM calls with a planner in front, and it demos well. We built exactly that shape ourselves, for our own cold outreach system: extract company facts, then summarize, then draft the email, then a separate "polish" pass — four hops because that's how a person would do it if you handed them the job one step at a time.
It's also where the shape breaks. Each hop is:
- A new place to lose context. Our "summarize" step only saw the extraction step's output, not the original scraped page — so by the time the email got drafted, it was working from a lossy summary of a summary.
- A new latency tax. Four sequential calls at 2-3 seconds each is 10+ seconds before a user sees anything, even before you add retries.
- A new failure surface. In our pipeline, the "polish" step regularly rewrote details the extraction step had gotten right, because it had no visibility into why the draft said what it said — it just saw a draft and a mandate to improve it.
Anthropic's own guidance on this is blunt: start with the simplest solution, and only add agentic complexity — multi-step planning, tool loops — when a single call with good context genuinely can't do the job (Building Effective Agents). That's not a hedge, it's the pattern that holds up in production.
What we learned running our own outreach engine
We run our own cold outreach system: it scrapes a prospect's site with a self-hosted Firecrawl instance and a local LLM, then writes one tailored email per company. Our first version split this into stages — extract company facts, then summarize, then draft the email, then a separate "polish" pass. It was slower, cost more in tokens per lead, and the polish step regularly overwrote details the extraction step had gotten right, because it had no visibility into why the draft said what it said.
We collapsed it into a single call: extract the facts and draft the email in one prompt, with the extraction fields specified as structured output before the email body. Cost per lead dropped, and quality went up, because the model held the full context — the actual scraped page, not a lossy summary of it — the whole time it was writing. The lesson generalizes past outreach: if your "agent" is really just transform-input-to-output with no branching decision in the middle, splitting it into steps is pure overhead.
A simple test before you chain anything
Ask whether the task has a decision point that depends on information the model doesn't have yet. If yes, you likely need at least one loop — the model has to act, see what happened, then decide the next action. If no, every step's input is fully knowable before the model runs, and it belongs in one call with a good prompt, or a deterministic code pipeline with the LLM doing one clearly-scoped part.
Examples that genuinely need iteration:
- An agent debugging code against a failing test suite (it must run the test to know what to fix next)
- An agent navigating a website where the next click depends on what's on the current page
- A support agent that needs to ask a clarifying question before it can act
Examples that don't, no matter how "agentic" they feel in the pitch deck:
- Summarizing a document and extracting structured fields
- Classifying and routing a support ticket
- Drafting an email, a spec, or a status update from known inputs
We cover the broader version of this decision — when a task is a workflow versus a true agent — in our breakdown of agents vs. workflows. If you're choosing between building this in-house and buying a platform, custom agent vs. chatbot platform covers that adjacent decision.
The cost math founders actually care about
Chains cost more in three ways that compound — our own outreach pipeline paid all three before we collapsed it:
- Token cost. Each call re-sends context (system prompt, prior outputs) that a single call would only pay for once. Our four-step version resent the scraped page's summary at every hop; the single-call version pays for the full page once.
- Retry cost. If any step in the chain fails or returns malformed output, you either retry that step (adding latency) or retry the whole chain (adding cost). When our "polish" step returned something malformed, the fix was to rerun it — but by then the extraction and draft steps had already run and been paid for. A single call fails once, not four times.
- Engineering cost. Our four-stage version needed orchestration code and state passing between extract, summarize, draft, and polish — code that disappeared entirely once we merged it into one prompt with structured output. That's ongoing maintenance surface, not just build cost.
If you're estimating what an AI feature will cost to run at volume, model both architectures against real sample inputs before committing — see our guide to LLM cost optimization for how to structure that estimate. We've also written about the diligence question that comes up once a vendor pitches you a multi-agent system: ask them to justify each hop, the way we describe in evals in AI vendor contracts.
Structured output does most of what "planning steps" claim to do
A lot of what multi-step chains are used for — extract this, then format that, then validate the other thing — is better solved with structured output: a JSON schema or function-calling spec the model fills in a single pass. Modern models are reliable at returning well-formed structured data when you specify the schema clearly, which removes the main reason teams reach for a "validation step" in the first place. Add a deterministic code-level check after the single call (does this field parse, is this value in range) instead of a second LLM call to check the first one's work.
Where we land
Default to one call. Give it the full context it needs, ask for structured output, and validate deterministically. Reach for a chain — or a real agent loop with tool use — only when the task has an actual decision point that depends on something the model can't know until it acts. That's a smaller category than most agent demos suggest, and it's the difference between an AI feature that's cheap and boring to run and one that quietly burns budget every time it re-reads its own output.
If you're scoping an AI feature and aren't sure which side of that line it falls on, let's talk.
Originally published on the Pykero blog.
Top comments (0)