- Book: Agents in Production — Building, Tracing, and Shipping Multi-Step AI You Can Trust
- Also by me: Observability for LLM Applications — the companion book in The AI Engineer's Library (2-book series)
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
On 12 June 2025, Walden Yan at Cognition published Don't Build Multi-Agents. His argument was flat: hand a task to one decision-maker with full context, and stop there. A day later, on 13 June, Anthropic published How we built our multi-agent research system, reporting that a lead agent spawning parallel subagents beat a single agent by a wide margin on their internal research evals.
Two credible teams. One day apart. Opposite conclusions. Read as universal rules, they contradict. Read as reports from two different problems, they agree completely. Cognition ships an autonomous software engineer that holds a repository in its head for hours. Anthropic ships a research feature that sweeps the open web for a user waiting on an answer. Different shapes, because the task decides the shape.
That is the whole point of this post. There are three shapes of agent system, and they sit on one axis: how much the model is allowed to decide on its own. Pick the least-powerful one that still solves your task.
The autonomy axis
Line the three shapes up from least to most autonomy:
- Fixed workflow. You write the control flow in code. The model fills specific slots. It never chooses what happens next.
- Fully autonomous loop. The model runs a loop, decides every step, and calls tools until it declares itself done. Your code holds no rails.
- Constrained autonomy. The model still decides each step, but inside walls your code sets: a step budget, a tool allowlist, hard stop conditions.
Most teams reach for shape 2 because it reads as the smart, modern option. Most teams need shape 3, and a surprising number need shape 1. The failure mode of picking too much autonomy is a loop that burns tokens for forty minutes and forgets what it was doing. The failure mode of picking too little is a pipeline that cannot handle the one input you did not anticipate. Match the shape to the task and both problems go away.
All code below uses the Anthropic SDK with Claude as the model. pip install anthropic, then ANTHROPIC_API_KEY in the environment.
Shape 1: the fixed workflow
A workflow is code that calls a model at fixed points. The steps are yours. The model classifies, extracts, or drafts inside a slot you built. Here is a support-ticket handler: classify, then branch, then draft.
import anthropic
client = anthropic.Anthropic()
MODEL = "claude-opus-4-8"
def classify(ticket: str) -> str:
r = client.messages.create(
model=MODEL,
max_tokens=16,
messages=[{
"role": "user",
"content": (
"Category for this ticket "
f"(billing/bug/other):\n{ticket}"
),
}],
)
return r.content[0].text.strip().lower()
def draft_reply(ticket: str, category: str) -> str:
r = client.messages.create(
model=MODEL,
max_tokens=512,
system=f"You reply to {category} tickets.",
messages=[{"role": "user", "content": ticket}],
)
return r.content[0].text
def handle(ticket: str) -> str:
category = classify(ticket)
if category == "billing":
return route_to_billing(ticket)
return draft_reply(ticket, category)
The if lives in Python, not in the model. You can trace every path. You can unit-test each step. There is one place to look when a reply comes out wrong. If your task is a known sequence of transforms over text, this is the right shape, and you can stop reading. Do not add a loop the task does not need.
Shape 2: the fully autonomous loop
Sometimes the sequence is not knowable in advance. The model has to read something, decide what to read next, and keep going until it has an answer. That is a loop where the model drives.
tools = [{
"name": "search_docs",
"description": "Search the internal docs.",
"input_schema": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
}]
def run_agent(question: str) -> str:
messages = [{"role": "user", "content": question}]
while True:
r = client.messages.create(
model=MODEL,
max_tokens=1024,
tools=tools,
messages=messages,
)
if r.stop_reason != "tool_use":
return next(
b.text for b in r.content
if b.type == "text"
)
messages.append(
{"role": "assistant", "content": r.content}
)
results = []
for b in r.content:
if b.type == "tool_use":
out = search_docs(**b.input)
results.append({
"type": "tool_result",
"tool_use_id": b.id,
"content": out,
})
messages.append(
{"role": "user", "content": results}
)
This works, and it is the pattern behind coding agents and research agents. Look at the exit condition. The loop ends when the model stops asking for tools. Nothing else. If the model decides the task is never finished, while True obliges it. You have handed the halting decision to a language model, and language models are sometimes wrong about whether they are done.
A loop like this is a while whose condition is evaluated by a model that can be wrong. In production that is how you get a runaway token bill and a trace nobody can read. The loop is not wrong for every task. It is wrong when you ship it without walls.
Shape 3: constrained autonomy, the middle you want
Keep the loop. Add the walls. Bound the steps, gate the tools, and give the runner a terminal condition it controls in code.
ALLOWED_TOOLS = {"search_docs"}
def run_bounded(question: str,
max_steps: int = 6) -> str:
messages = [{"role": "user", "content": question}]
for _ in range(max_steps):
r = client.messages.create(
model=MODEL,
max_tokens=1024,
tools=tools,
messages=messages,
)
if r.stop_reason != "tool_use":
return next(
b.text for b in r.content
if b.type == "text"
)
messages.append(
{"role": "assistant", "content": r.content}
)
results = []
for b in r.content:
if b.type != "tool_use":
continue
if b.name not in ALLOWED_TOOLS:
out = "Tool not permitted."
else:
out = dispatch(b.name, b.input)
results.append({
"type": "tool_result",
"tool_use_id": b.id,
"content": out,
})
messages.append(
{"role": "user", "content": results}
)
return "Stopped: step budget reached."
Three walls, all in code. for _ in range(max_steps) caps the loop no matter what the model wants. The allowlist rejects any tool call outside the set you approved, even if a prompt injection talks the model into trying. The step-budget return gives the runner a definite end that does not depend on the model saying it is done.
The model still chooses each step. It reasons, it picks queries, it decides when it has enough. You have not taken its judgment away. You have put a fence around the field. The same shape ports cleanly to the TypeScript SDK:
for (let step = 0; step < maxSteps; step++) {
const r = await client.messages.create({
model: "claude-opus-4-8",
max_tokens: 1024,
tools,
messages,
});
if (r.stop_reason !== "tool_use") {
return textFrom(r);
}
// gate tools, append results, continue
}
Picking the shape
Ask one question before you write any loop. What is the least autonomy that still solves this task?
If the steps are fixed, build the workflow. It is the cheapest thing to run, the easiest thing to trace, and the hardest thing to break. If the steps are open-ended, you need a loop, and the honest default is the bounded one: a step budget, a tool allowlist, a code-owned stop condition. The unbounded loop is a bounded loop with the safety removed. Reach for full autonomy only when you have measured that the walls cost you real capability, and even then, keep the token budget.
The Anthropic-Cognition debate was never about which shape is correct in general. It was two teams announcing which shape fit their task. Yours has a shape too. Find the least-powerful one that fits, and let the walls do the work the model should not be trusted to do alone.
The two books in The AI Engineer's Library map onto exactly this split. Agents in Production is the build side: the ReAct loop, the step and cost budgets, the human gates, and how to decide between these three shapes for a given task. Observability for LLM Applications is the operate side: tracing a bounded loop end to end, evaluating each step, and catching the runaway before it shows up on the bill. Building agents and knowing whether they work are two jobs, and these are the two books.
Top comments (0)