Every AI agents tutorial walks you through the happy path. Build the loop, connect the tools, watch the agent complete the task. It works. You're impressed. You deploy it.
Then you find out on a Monday morning that the agent spent $340 over the weekend running a loop it couldn't exit because one of its tool calls kept returning a malformed response.
That's the 20% nobody covers.
Why tutorials skip this
The happy path is teachable in 15 minutes. The production concerns take weeks of running things in production to understand. Tutorial authors don't have your production environment, your tool reliability characteristics, or your actual usage patterns. They can show you the loop. They can't show you what happens when the loop breaks.
I've been following machine learning and agent research since 2013, through the Coursera era and through several production deployments of non-LLM ML systems before anyone was calling them "agents." What's different now is how accessible the building blocks are. What's the same is that deploying any system that runs autonomously requires you to answer the same operational questions you'd answer for any other production system: how do I know it's working, what happens when it fails, and how much is this going to cost?
Those questions don't change because the system is powered by a language model.
Observability
An agentic system is a distributed system. It makes sequential calls to external services (the model API, your tools, your databases), maintains state across those calls, and makes decisions that affect subsequent steps. You need telemetry on all of it.
The minimum instrumentation I'd put on any production agent:
Per-run: start time, end time, total token consumption (prompt + completion separately), number of tool calls, final status (completed / failed / interrupted), and the full trace of steps taken.
Per tool call: tool name, arguments (sanitized for secrets), response status, response latency, and token cost if applicable.
Per decision point: what the model decided to do next, what alternatives it considered (if your prompting surfaces this), and whether the decision matches expected patterns for this task type.
The full step trace is the most important piece. When an agent fails, the trace tells you where in the sequence it went wrong. Without it you're debugging from symptoms rather than from cause.
Most LLM SDKs give you token counts. You have to instrument the rest yourself. A simple structured logger writing to your existing observability stack is enough to start with.
const agentRun = {
runId: crypto.randomUUID(),
startTime: Date.now(),
steps: [],
tokenConsumption: { prompt: 0, completion: 0 },
status: 'running'
};
// After each model call:
agentRun.steps.push({
stepNumber: agentRun.steps.length + 1,
action: response.content[0].type, // 'tool_use' or 'text'
toolName: response.content[0].name ?? null,
tokenCost: { prompt: response.usage.input_tokens, completion: response.usage.output_tokens },
timestamp: Date.now()
});
agentRun.tokenConsumption.prompt += response.usage.input_tokens;
agentRun.tokenConsumption.completion += response.usage.output_tokens;
Fallback strategy
Three failure modes I've seen in production agentic systems, in order of frequency:
Tool call failure. The tool returns an error, a malformed response, or nothing. If the agent's loop doesn't have explicit handling for this, it either retries indefinitely, hallucinates a response, or stops with an unhelpful error. All three are worse than a clean fallback.
The fix is a retry limit with escalation logic. If a tool call fails twice, either fall back to an alternative approach or stop and surface the failure to a human. "Stop and surface" is the right answer more often than people expect.
async function callToolWithFallback(toolName, args, maxRetries = 2) {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
try {
return await tools[toolName](args);
} catch (error) {
if (attempt === maxRetries) {
// Log, alert, and return a structured failure
logger.error({ toolName, args, error, attempt });
return { success: false, error: error.message, requiresHumanReview: true };
}
await sleep(1000 * (attempt + 1)); // backoff
}
}
}
Reasoning loop. The agent keeps taking steps without converging on a result. This happens when the task is underspecified, when tool responses are ambiguous, or when the model gets into a pattern where each step looks reasonable but the sequence isn't making progress.
A step limit is the blunt instrument, and it's the right one. Set a maximum number of steps per run. When the limit is hit, stop, log the full trace, and surface it for human review. You can tune the limit up after you've seen what normal runs look like.
Context window saturation. Long-running agents accumulate context. Tool responses, previous steps, and reasoning traces all consume tokens. When you're approaching the context window limit, the model's reasoning quality degrades before you hit a hard error. Budget for this explicitly.
Cost management
Token costs compound quickly in agentic systems. Each step requires at minimum a full context injection: all previous steps, all tool definitions, the system prompt. A 10-step agent run at 2,000 tokens per step with claude-opus-4 pricing is not a trivial cost.
Set a budget per run. Not as a soft suggestion, as a hard limit that terminates the run and logs an alert.
const MAX_TOKENS_PER_RUN = 50000; // set based on your cost tolerance
if (agentRun.tokenConsumption.prompt + agentRun.tokenConsumption.completion > MAX_TOKENS_PER_RUN) {
agentRun.status = 'budget_exceeded';
throw new Error(`Run ${agentRun.runId} exceeded token budget. Terminating.`);
}
Also: choose the right model for each step. Not every step in an agent loop requires your most capable model. Tool selection and result summarization can often run on cheaper, faster models. Reserve the expensive reasoning for the steps that actually need it.
The production checklist I use
Before deploying any agentic workflow:
- Step limit defined and enforced
- Token budget defined and enforced
- All tool calls have retry logic with a maximum attempt count
- Full run trace logged to a searchable store
- Alerting on runs that exceed time, cost, or step thresholds
- At least one human review path for runs that fail or exceed thresholds
- Test cases for tool failure scenarios, not just the happy path
This is the same checklist I'd use for any autonomous process. The "agent" part doesn't change the operational requirements. It just makes it easier to forget them.
Top comments (0)