A large language model, on its own, can only do one thing: emit text. It can't check your calendar, run a test suite, or refund a payment. So how did we get from "very good autocomplete" to systems that book travel and fix codebases?
The answer is almost embarrassingly simple: you wrap the model in a loop and give it tools. That's an agent. Everything else is detail — but the details are where production systems live or die.
What actually counts as an "agent"
There's a lot of hype-driven vagueness here, so let's use the definitions the labs themselves use.
OpenAI's working definition: an agent is a system that uses an LLM to control workflow execution. It decides the next step, recognizes when the task is complete, self-corrects, and can hand control back to a human. Crucially, things that embed an LLM but don't let it control the flow — a simple chatbot, a single-turn call, a sentiment classifier — are explicitly not agents.
Anthropic draws the same line differently: a workflow runs LLMs and tools through predefined code paths; an agent lets the model dynamically direct its own process and tool usage. The dividing line is who decides what happens next — your code, or the model.
Every agent is built from three components:
- Model — the LLM doing the reasoning and deciding.
- Tools — functions and APIs it can call to take action.
- Instructions — the guidelines and guardrails that define behavior.
Tools come in three flavors: data tools (query a database, read a PDF, search the web), action tools (send an email, update a CRM record), and orchestration tools (other agents, exposed as callable tools).
The agent loop (the "run")
The core mechanism is a loop — often called a run — that repeatedly calls the model until an exit condition is met. Each pass accumulates state in the context, so a decision made late in the run still carries the residue of every earlier step.
- Assemble context — system prompt + tool definitions + conversation history + any retrieved data.
- Call the model — it returns either a final answer or a request to call a tool.
- If it's a tool call, execute it — your harness (not the model) runs the actual function and captures the result.
- Feed the result back — append it to the context and loop.
- Exit — on a final answer, no tool calls, an error, or a max-turn cap.
Anthropic frames the same cycle as gather context → take action → verify work → repeat. That word verify matters: agents get "ground truth" from tool results and code execution at each step — real feedback from the environment, instead of reasoning in a vacuum. A coding agent that runs the tests after editing is doing exactly this.
Tool calling: the exact mechanics
This is the single most important mechanism to understand, because it's how a text model "does things." Walk through it carefully:
1. You declare tools as JSON Schema. Each tool is a name, a description, and a parameters schema, passed in the API's tools field. That description is not decoration — a vague one will misdirect the model entirely.
2. The model decides. With the default tool_choice: "auto", it may call zero, one, or several tools. When it decides a tool is needed, it interrupts its own prose and emits structured JSON arguments that match the schema — for example, a location argument with the value "Paris".
3. Your code executes. You parse the tool call (function name + JSON arguments), run the real function, and get a result. The model never executes anything itself. It only emits intentions; your harness does the work. This separation is also your security boundary.
4. You feed the result back. Append a message with role tool, the matching tool_call_id, and the result content — then call the model again so it can incorporate the output into its next step.
Structured outputs
To guarantee valid JSON (not just hope for it), strict mode constrains generation to the schema (additionalProperties: false, all fields required). Without it, JSON output is best-effort: you validate after generation and re-prompt on failure — a feedback loop, not a guarantee. For anything downstream-critical, use the constrained version.
Design the interface like a product
Anthropic makes a point worth stealing: the Agent–Computer Interface (ACI) deserves as much design care as a human UI. That includes error-proofing ("poka-yoke") — designing tool signatures so the model can't easily make a mistake. Clear names, tight schemas, and helpful error messages do more for reliability than any amount of clever prompting.
Think about it from the model's side: it's operating your system blind, through a keyhole of text. Good tools are self-describing, hard to misuse, and fail loudly with actionable messages.
Single agent first
One last, load-bearing piece of advice before the fancy stuff: start with a single agent. A single model in a loop with well-designed tools solves a surprising share of real problems. Reach for multiple agents only when specialization genuinely removes a bottleneck — because, as we'll see later in the series, more agents means distributed-systems complexity, not maturity.
Key takeaways
- An agent = an LLM + tools + instructions, run in a loop the model controls.
- The model only emits intentions; your harness executes them and feeds results back.
- Tools are declared as JSON Schema; strict mode makes outputs reliable.
- Treat the tool interface like a product surface — clarity beats cleverness.
Top comments (0)