Imagine an AI agent handling customer support. A customer asks: “Can you refund my order?”
The agent checks the order, confirms that the purchase is eligible, calls the refund API, and gets a timeout. What does the agent know at this point? Not necessarily that the refund failed.
The request may have reached the payment provider, the refund may have succeeded, and only the response was lost. If the agent simply retries:
Agent → Refund API
↓
Timeout
↓
Agent → Refund API
you could end up issuing the same refund twice. The model didn't necessarily make a bad decision. The system gave it an unsafe loop.
This is why I think many agent failures become easier to understand when you stop looking at the model in isolation. At a high level, an agent is running a loop:
Observe → Think → Act → Observe → Think → Act
That loop sounds simple. Production systems are not.
The Loop Needs Boundaries
An agent needs to know more than what action to take next. It needs to know when to stop, what has already happened, what failed, and whether an action can safely be repeated.
Without those controls, relatively small problems can turn into serious failures. Consider an agent trying to update a customer's address. It calls the tool:
The operation succeeds, but the response is lost because of a network failure. The agent doesn't know whether the action succeeded.
If the system allows it to retry indefinitely, the agent could keep attempting the same operation. This isn't really a reasoning problem. It's a state and execution problem.
Infinite Loops Are Easier Than They Look
One of the simplest agent failures is also one of the most obvious:
Observe
↓
Think
↓
Act
↓
Tool fails
↓
Think
↓
Retry
↓
Tool fails
↓
Retry
↓
...
If nothing tells the agent that it has reached a failure condition, there's no reason for the loop to stop. That's why I'd put an explicit iteration limit around agent execution.
The exact number isn't important. The boundary is. An agent should never have unlimited authority to keep reasoning and acting simply because it hasn't reached the desired outcome.
Every Action Needs a State
Another common problem is losing state between iterations. Imagine an agent performing a deployment:
Build
→ Test
→ Deploy
→ Verify
The deployment succeeds, but the process crashes before recording that state. When the agent resumes, it sees: “Deployment not completed.” So it deploys again.
Now you have duplicated work or, depending on the operation, potentially a much bigger problem. This is why durable checkpoints matter. Instead of relying entirely on the model's conversational context, the system should persist important state:
If the process restarts, the system knows where it actually is. The model can reason from that state instead of trying to reconstruct history.
Tools Need Structured Responses
Tool design also has a huge impact on agent reliability. Compare these two responses: "Done."
and:
The second response gives the next iteration something concrete to reason about. This becomes particularly important when tools can return partial success, warnings, or ambiguous outcomes.
For example:
That's much more useful than simply returning: "Request failed."
The agent now knows that it should not automatically assume the operation didn't happen. Structured tool contracts make the loop easier to control.
Idempotency Is Critical
If an agent can retry actions, those actions need to be designed with retries in mind. For operations such as payments, refunds, emails, account changes, or provisioning, the system should be able to recognize duplicate requests.
For example:
If the same request is sent twice because of a timeout or retry, the backend can recognize that it's the same operation rather than treating it as a completely new one. This is especially important because agents operate in environments where failures are normal.
Networks fail. Services timeout. Processes restart. Responses get lost. The system has to be designed around those realities.
Success and Failure Should Be Explicit
Another pattern I like is explicit terminal states. Instead of letting the agent keep running until it "feels" finished, define states such as:
- RUNNING
- WAITING
- SUCCEEDED
- FAILED
- ESCALATED
Now the control system has something deterministic to enforce. For example, an agent might discover that it doesn't have enough information to complete a request. Instead of continuing to guess:
Missing information
↓
Ask user / escalate
↓
WAITING
Similarly, if a sensitive operation fails repeatedly:
Tool failure
↓
Retry
↓
Retry limit reached
↓
FAILED
The agent doesn't get to decide that it should retry forever. The system decides what happens when the boundary is reached.
The Model Decides. The System Controls.
This is the distinction I think is important when designing production agents. The model can decide: “I should call the refund tool.” But the system should decide:
- Is this tool allowed?
- Is the user authorized?
- Has this operation already happened?
- Can it safely be retried?
- How many attempts are allowed?
- What happens if the result is ambiguous?
- When should execution stop?
- When should a human take over?
That separation makes the architecture much safer and easier to debug. A useful mental model is:
The model provides reasoning. The surrounding system provides control. That's why I wouldn't start debugging an unreliable agent by immediately asking whether I need a better model. I'd first inspect the loop. Does it have iteration limits? Are actions idempotent? Is state durable? Are tool responses structured? Can the system distinguish failure from unknown outcome? Are checkpoints persisted? Are terminal states explicit? Can the agent escalate instead of endlessly retrying?
If those pieces are missing, replacing the model may simply give you a more capable system running the same broken loop. Agent reliability isn't just a model problem. It's a loop-design problem.
Top comments (0)