- Book: Agents in Production — Building, Tracing, and Shipping Multi-Step AI You Can Trust
- Also by me: Observability for LLM Applications — the companion book in The AI Engineer's Library (2-book series)
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
You ask an agent to "find a slot for the quarterly review with the Berlin and Lisbon teams next week, and send the invites." It looks at the request, calls list_rooms, grabs the first free room, books it for Monday 09:00 Berlin time (08:00 in Lisbon, before anyone is at their desk), and reports success.
The tool did exactly what it promised. The room was free. The bug is that the agent had no plan. It took the first action its pattern-matching surfaced and stopped there. A person handling this would write down the constraints first (two timezones, roughly 90 minutes, attendees in two offices), then look for rooms and times that satisfy all of them.
A plain ReAct loop does not do that. Thought, action, observation, repeat. For tasks that fit in three or four tool calls, that loop is enough. For anything longer, you want the model to plan before it acts, and to hand you its thinking in a shape your code can trust.
Why a plan beats prose
A block of prose that says "first I'll search, then I'll compute the total, then I'll draft the message" is not something your executor can act on. You would have to parse it, guess at the ordering, and hope the model kept its promise. A schema removes the guessing. The model emits a list of typed steps, each naming a tool and its arguments, and your code walks the list.
That single change buys you a lot. The expensive "understand the whole problem" reasoning happens once, on the planner call, instead of being re-derived on every turn of a ReAct loop. The plan itself becomes a concrete artifact you can log, diff, and show to a human before any side effect fires. And because each step declares its dependencies, you get a graph rather than a flat list, which means independent branches can run in parallel later without the model needing to know.
The pattern has a name in the literature: plan-then-execute, popularized as "plan-and-execute" and grounded in Xu et al.'s ReWOO paper (arXiv:2305.18323). You do not need a framework to use it.
The plan as a typed contract
Start with the schema. This is the contract between the planner and the executor. Pin anthropic==0.94.1 and pydantic==2.11.2.
from __future__ import annotations
import json
from typing import Literal
import anthropic
from pydantic import BaseModel, Field
client = anthropic.Anthropic()
PLANNER = "claude-opus-4-8"
MAX_REPLANS = 3
class Step(BaseModel):
id: int
description: str
tool: Literal["search", "calc", "write"]
args: dict = Field(default_factory=dict)
depends_on: list[int] = Field(
default_factory=list
)
class Plan(BaseModel):
goal: str
steps: list[Step]
Each step names a tool, carries its arguments, and declares which earlier steps it depends on. The depends_on field is what turns the list into a graph. MAX_REPLANS is the seatbelt: three re-plans per task absorbs flaky tools without letting the model spiral into an expensive loop.
Forcing structured output from the model
Anthropic's API has no response_format parameter. You get structured output by defining one tool whose input_schema is your Pydantic schema, then forcing that tool with tool_choice. The model's arguments land in the tool-use block, and Pydantic validates them.
PLAN_TOOL = {
"name": "emit_plan",
"description": "Emit the final execution plan.",
"input_schema": Plan.model_json_schema(),
}
def plan(goal: str, feedback: str | None = None) -> Plan:
prompt = (
"Produce a minimal plan for the goal. "
"Use the fewest steps that work.\n\n"
f"GOAL: {goal}"
)
if feedback:
prompt += (
f"\n\nPREVIOUS ATTEMPT FEEDBACK:\n{feedback}"
)
msg = client.messages.create(
model=PLANNER,
max_tokens=2048,
tools=[PLAN_TOOL],
tool_choice={
"type": "tool", "name": "emit_plan"
},
messages=[{"role": "user", "content": prompt}],
)
block = next(
b for b in msg.content if b.type == "tool_use"
)
return Plan.model_validate(block.input)
Forcing tool_choice to the tool guarantees one tool_use block. If the model's output does not fit the schema, model_validate raises. That is the behavior you want. A loud failure beats silent corrupt state feeding the rest of your pipeline.
Executing the plan
Execution is the boring part, which is the point. Walk the steps, feed each one the results of its dependencies, and dispatch to the real tool. Tool errors become strings so the re-planner can read them. Exceptions only escape when the re-plan budget runs out.
def run_tool(step: Step, ctx: dict[int, str]) -> str:
if step.tool == "search":
return search_api(step.args["query"])
if step.tool == "calc":
return str(eval_expr(step.args["expression"]))
if step.tool == "write":
return step.args["template"].format(**ctx)
raise ValueError(f"unknown tool: {step.tool}")
def execute(goal: str) -> dict[int, str]:
current = plan(goal)
results: dict[int, str] = {}
replans = 0
while True:
failed, error = None, None
for step in current.steps:
if step.id in results:
continue
ctx = {
i: results[i] for i in step.depends_on
}
try:
results[step.id] = run_tool(step, ctx)
except Exception as e:
failed = step
error = f"{type(e).__name__}: {e}"
break
if failed is None:
return results
replans += 1
if replans > MAX_REPLANS:
raise RuntimeError(
f"re-plan budget hit at step "
f"{failed.id}: {error}"
)
feedback = json.dumps({
"failed_step": failed.model_dump(),
"error": error,
"partial_results": {
str(k): v for k, v in results.items()
},
}, indent=2)
current = plan(goal, feedback=feedback)
Completed steps carry over in the results dict. A re-plan that keeps the same IDs for the successful prefix skips straight back to the failure point.
Re-planning when a step fails
The re-plan trigger is the except clause and nothing more. Catch the exception, stringify it, bundle the partial results, and ask the planner for a fresh Plan that accounts for what already worked. Most re-plans look almost identical to the original with one step swapped: a different search query, a retry against a backup endpoint. Occasionally the model realizes the original approach was wrong from the start and rewrites the whole thing. Both outcomes are fine, and both are logged as a new artifact you can diff against the previous plan.
The cost of this pattern is rigidity. If the world changes mid-plan, the plan is stale. That is what the re-plan budget is for. Set it low and fail loudly past it. A task that needs seven re-plans is not one the agent should be running unsupervised, and the loud failure is how you find that out before your bill does.
What the typed plan gives you
Cost drops because the reasoning-heavy call happens once. Observability improves because a plan is a JSON artifact a human can read in seconds, where a ReAct trace is a stream a human has to reconstruct by hand. You get early exits: if step three returns "user already registered," the executor can short-circuit the rest without another model round trip. And you get a natural pause point for human-in-the-loop approval, between drawing the plan and running any tool with side effects. Banks and hospitals will ask for exactly that split.
Raw ReAct is the floor. A typed plan with a re-plan budget is a cheap step up that pays for itself the first week a tool misbehaves in production.
If this was useful
Planning, decomposition, and schema-constrained output are three of the patterns that separate an agent that feels smart from one that feels stupid. Agents in Production walks through building them by hand, and Observability for LLM Applications covers the tracing and cost work that tells you whether your re-plan budget is set right. Both books sit in The AI Engineer's Library.
Top comments (0)