We spend a lot of time thinking about what an AI agent should do next.
Which tool should it call?
What information should it retrieve?
Should it ask another agent?
Should it retry?
But one of the hardest decisions is much simpler:
When should the agent stop?
An agent that stops too early may miss important evidence.
An agent that keeps going too long may waste time, increase cost, repeat the same work, or even make a good answer worse.
Imagine an incident investigation agent. It checks the latest deployment, service dependencies, known issues, historical incidents, and the current runbook.
At some point, it already has enough evidence to make a reasonable recommendation.
But if the agent keeps searching, it may find older incidents with similar symptoms, deprecated documentation, or unrelated edge cases. More information does not always improve the decision.
Sometimes it creates more confusion.
That means stopping should be part of the agent design, not an afterthought.
A simple approach is to use a maximum number of steps. For example, the agent can make at most eight tool calls before it must return a result.
That protects against infinite loops, but it does not answer the deeper question.
What if the correct answer is clear after three steps?
What if eight steps are not enough?
A better stopping condition is based on evidence sufficiency.
The agent should stop when it has enough reliable information to answer the question, when additional actions are unlikely to change the decision, or when the remaining uncertainty requires a human instead of more tool calls.
For example, if the agent finds that a deployment happened five minutes before an incident, the affected service depends on the changed component, a known issue matches the symptoms, and the current runbook recommends rollback, continuing to search may add very little value.
The system can stop and return its recommendation.
The opposite situation is just as important.
Suppose the agent checks deployments, dependencies, and monitoring data, but the evidence remains incomplete and conflicting.
The agent should not continue calling tools forever hoping that certainty will eventually appear.
At some point, the correct action is:
“I do not have enough evidence. Human investigation is required.”
That is also a successful stopping condition.
Repeated actions are another useful signal.
If an agent keeps calling the same search tool with slightly different queries and receives nearly identical results, the workflow may be stuck.
The application can detect repeated tool calls, unchanged results, or lack of progress and stop the loop.
This is where traditional software engineering becomes important.
The model may decide what action makes sense next, but application code should still enforce limits such as maximum steps, timeouts, repeated-action detection, cost budgets, and human escalation rules.
In other words, the model can reason about when it is done, but the surrounding system should make sure it actually stops.
I think this is an underrated part of agentic system design.
Autonomy is not only about giving an agent the ability to keep acting.
It is also about giving it a reliable way to know when further action is no longer useful.
A good AI agent should not just know what to do next.
It should also know when there should be no next step.
Top comments (0)