- Book: Agents in Production — Building, Tracing, and Shipping Multi-Step AI You Can Trust
- Also by me: Observability for LLM Applications — the companion book in The AI Engineer's Library (2-book series)
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
You reach for a framework on day one. LangGraph, CrewAI, the OpenAI Agents SDK, the Claude Agent SDK. Then the agent hangs in production, you open a trace, and you have no idea what the runner did between the model call and the tool result. The abstraction that saved you an afternoon of typing is now the thing standing between you and the bug.
Here is the fix. Write the loop yourself once, by hand, in about sixty lines. Every framework you will meet is a decoration on this loop. Once you have built it, you can read any of them.
The pattern has a name and a paper
In October 2022, Yao et al. published ReAct: Synergizing Reasoning and Acting in Language Models (arXiv:2210.03629). The idea is small enough to state in one sentence. An agent is a loop where a language model produces three things, in order, over and over, until it decides it is done:
- A Thought — the model reasoning about what to do next.
- An Action — a call to a tool that queries or changes the world outside the model.
- An Observation — whatever the tool returned, handed back as context for the next turn.
That is ReAct: Thought, Action, Observation, repeat. The original paper proved it on knowledge-lookup and text-game benchmarks, where interleaving reasoning with tool calls beat both pure chain-of-thought (which hallucinated facts and then reasoned on top of them) and pure tool-calling (which took one wrong step and never recovered).
The number to remember is not the benchmark score. It is the shape. Every production agent shipping today, whether it runs on Claude or GPT or Gemini, is a descendant of those three slots.
What the paper had to do, and what you no longer do
The 2022 driver was string surgery. The model was a completion model. You fed it a prompt ending in Thought 1: and let it generate. It wrote a thought, then a line starting with Action 1:, then a function-call-looking string like Search[Apple Remote]. Your code then had to detect the action line, stop generation, regex out the tool name and argument, run the tool, append Observation 1: ..., and resume.
If the model wrote search(Apple Remote) instead of Search[Apple Remote], the regex missed and the loop died. You could lose an afternoon to a missing bracket.
Nobody writes it that way now. Not because the pattern was wrong. It was right. The problem is that parsing a model's prose into tool calls is fragile, when the output should be structured data from the start.
What changed: native tool calling
By 2023 the frontier models spoke structured tool calls as a first-class output. You define a tool with a JSON Schema, the model returns a typed block naming the tool and its arguments, and the SDK parses it. No regex. No stop tokens. No brackets.
The mapping onto ReAct is exact. The Thought is the text block the model returns before it acts. The Action is a tool_use block with a validated input dict. The Observation is a tool_result block you feed back, referenced by id. Same three slots, a type system bolted on each one.
The loop, from scratch
Here is a full ReAct loop against claude-opus-4-8 with the bare anthropic SDK. One tool, one file, no framework. The messages list is the only state.
import anthropic
client = anthropic.Anthropic()
MODEL = "claude-opus-4-8"
tools = [
{
"name": "get_weather",
"description": "Current weather for a city.",
"input_schema": {
"type": "object",
"properties": {
"city": {"type": "string"},
},
"required": ["city"],
},
}
]
def run_tool(name, args):
# Real implementation goes here. Return a
# plain string the model can read.
if name == "get_weather":
return f"18C and clear in {args['city']}."
return f"Unknown tool: {name}"
def run_agent(user_input, max_steps=10):
messages = [{"role": "user", "content": user_input}]
for _ in range(max_steps):
resp = client.messages.create(
model=MODEL,
max_tokens=1024,
tools=tools,
messages=messages,
)
messages.append(
{"role": "assistant", "content": resp.content}
)
if resp.stop_reason != "tool_use":
return resp # model is done: end_turn
results = []
for block in resp.content:
if block.type != "tool_use":
continue
output = run_tool(block.name, block.input)
results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": output,
})
messages.append({"role": "user", "content": results})
return resp # hit the step cap
Read it against the three slots. The TextBlock in resp.content is the Thought. The ToolUseBlock is the Action. The tool_result you append is the Observation. The loop exits when stop_reason comes back as end_turn instead of tool_use. That is the whole engine.
Note the max_steps cap. A loop with no bound is a loop that can call the same tool forever and bill you for every turn. Sixty lines of agent still needs one number guarding it.
Two details the 2022 paper did not have
Parallel tool calls. When actions are independent, the model returns several tool_use blocks in one turn. Ask for the weather in Berlin, Paris, and London and you get three blocks back. You run them, then return one user message carrying three tool_result blocks, each referencing its own tool_use_id. The loop above already handles this — it iterates over every block in the response. The footgun: if your tools are not idempotent and the model fires send_invoice twice, you get two invoices. Keep parallel calls off for side-effecting tools until idempotency is handled.
Where the Thought lives. On a reasoning model the Thought can sit in three places: the visible text block, a hidden thinking block that is billed and often holds the real reasoning, or nowhere at all when the model jumps straight to a tool_use. If your tracing only logs visible text, you will stare at traces with an empty Thought column and wonder why the agent called delete_user. Capture all three.
Make the Observation readable
There is one failure mode that tells you whether you understand the loop. The agent works in dev, works in staging, then in production it calls the same tool nine times with slightly different arguments, each time getting a slightly different error, until the step cap fires.
That is not a framework bug. It is a T-A-O feedback problem. The Observations you handed back were unreadable to the model, so it kept guessing. The fix is one line in your tool wrapper: turn the raw exception into a plain-English error string.
def run_tool(name, args):
try:
return dispatch(name, args)
except Exception as exc:
# A sentence the model can act on beats a
# stack trace it cannot.
return f"Tool '{name}' failed: {exc}. Try again."
The model reads every Observation as input to its next decision. An honest "this failed, here is why" lets it change course. A truncated stack trace does not. This is exactly the kind of thing a framework hides from you, which is why you want to have felt it once yourself.
What a framework actually adds
Nothing to the loop. The engine stays Thought, Action, Observation. What frameworks add sits around it:
- Planning. A separate reasoning call up front produces an ordered plan, then a per-step ReAct loop executes each step. Cheaper, because the big reasoning call happens once, and easier to debug, because you have a plan to diff against behavior.
- Recovery. A Reflexion-style wrapper catches a failed run, has the model write a short critique of why it failed, and retries the step with that critique in context.
-
Memory. Summarization and trimming of the
messageshistory, because every Observation is re-sent on every turn and a long loop against a chatty tool bloats the context fast. - Tool selection. Once you pass roughly twenty tools, selection accuracy drops. Frameworks add retrieval over the tool catalog so the model only sees the relevant few.
Every one of these is a layer on top of the loop you just wrote. None of them replaces it. When something breaks at 3 a.m., you debug it by opening a trace and asking three questions in order: was the Thought reasonable, was the Action well-formed, was the Observation what the tool should have returned. Every bug lives in one of those three slots.
Where to start
Ship the middle layer first. A pure ReAct loop with a good set of tools, a sane step cap, and readable error strings carries most single-task work: answer questions from your docs, triage a ticket, pull yesterday's metrics into a standup post. Add planning and recovery when the loop stops being enough, which is usually past ten steps or when you need memory the message history cannot hold.
Build the sixty lines. Watch it hang once. You will know exactly which of the three slots it got stuck in, and no framework will ever be a black box to you again.
If you want the full version of this — the native tool-calling deep dive, the tracing that captures all three Thought locations, the planning and recovery layers, and the failure modes you will actually meet — that is what The AI Engineer's Library is for. Agents in Production covers building and shipping the loop and the layers on top; Observability for LLM Applications covers the tracing, evals, and cost work that keeps it honest once real traffic hits it.
Top comments (0)