Large language models have become remarkably capable.
With enough parameters, data, and training, they can develop abilities that are difficult to predict from smaller systems. They can write software, interpret documents, use tools, plan multi-step tasks, and adapt to changing instructions. Watching a strong agent solve a complex problem from beginning to end can feel like a qualitative leap.
The demo is impressive.
Production is different.
The difficult part is no longer proving that an agent can complete a task once. The difficult part is making sure it behaves acceptably when the context is long, the tools are unreliable, the instructions are incomplete, and nobody is watching every step.
That is the agent’s last mile.
Capability is not the same as trust
An agent can be intelligent enough to produce a useful answer without being reliable enough to control a real workflow.
This distinction is easy to miss because successful demonstrations compress uncertainty. They usually have a clear goal, a clean environment, relevant context, working tools, and a human who knows when to intervene.
Production systems rarely have those conditions.
A customer support agent may receive an incomplete request. A coding agent may inherit a repository with contradictory documentation. A research agent may collect sources that disagree with each other. An operations agent may encounter a timeout after the external system has already completed the action.
In each case, the model may still produce a fluent response. Fluency is not evidence that the underlying state is correct.
The agent can sound certain while its view of the world is incomplete.
Long context creates a new reliability problem
Long context is powerful because it allows an agent to retain more instructions, documents, conversation history, and tool results.
But more context does not automatically produce more reliable reasoning.
A long context can contain stale facts, duplicated information, conflicting instructions, low-quality retrievals, intermediate mistakes, and assumptions that were never validated. The model must decide which parts matter, which parts are authoritative, and which parts should be ignored.
That is a difficult judgment.
When the context grows, the agent may also lose track of the origin of a claim. A suggestion from an earlier step can start to look like a confirmed fact. A failed tool call can be treated as if it succeeded. A speculative interpretation can become part of the working memory and influence every later decision.
The result is not always an obvious hallucination. More often, it is a plausible continuation built on a weak assumption.
This is what makes long-context failures dangerous. They may not look like failures until the output reaches the real world.
The last mile is a systems problem
The last mile of an agent is not solved by model intelligence alone.
It includes everything around the model:
- How tasks are scoped.
- Which tools the agent is allowed to call.
- What happens when a tool times out.
- How state is stored between steps.
- How outputs are validated.
- When the system pauses for approval.
- How errors are logged.
- Whether an action can be reversed.
- How a human can reconstruct what happened.
A highly capable model inside a weak system can still create unreliable outcomes.
In fact, a more capable model may increase the operational risk because people are more willing to trust it. When an agent performs well most of the time, its occasional failures become easier to overlook.
The goal is not to make the agent appear perfect. The goal is to make its failures bounded, visible, and recoverable.
Boring safeguards are still valuable
The most important production techniques are often less exciting than the model itself.
A system should verify that an action actually happened before reporting completion. If an agent creates a ticket, it should receive a confirmed ticket identifier. If it changes a file, the system should verify the file state. If it sends a request to an external service, it should distinguish between accepted, completed, failed, and unknown.
“Done” should be a state backed by evidence.
Actions should also be divided by risk. Reading information, drafting a response, changing a local setting, deleting data, and sending an external message should not all require the same level of autonomy.
Low-risk actions can run automatically. High-impact actions should require stronger validation, explicit approval, or both.
Other safeguards are equally unglamorous but important:
- Use strict schemas for tool inputs and outputs.
- Make operations idempotent whenever possible.
- Add timeouts and bounded retries.
- Save checkpoints before meaningful state changes.
- Keep permissions narrow.
- Separate planning from execution.
- Use deterministic validators for critical conditions.
- Escalate when the system cannot establish the current state.
- Preserve an audit trail of decisions and tool results.
These measures do not make the model smarter. They make the surrounding system more honest about what the model knows and what it does not know.
A reliable agent needs a floor
Every production agent should have a minimum behavioral contract.
The contract does not need to promise perfect accuracy. It needs to define what the system will never quietly do.
For example:
- It must not claim an action succeeded without confirmation.
- It must not invent missing information to complete a workflow.
- It must not silently continue after a critical tool failure.
- It must not perform irreversible actions without the required approval.
- It must preserve enough state for a human to recover the task.
- It must clearly distinguish facts, assumptions, and estimates.
- It must stop when the current state cannot be trusted.
These expectations are a reliability floor.
Without them, every team ends up relying on personal vigilance. Someone notices a strange result, someone checks the logs, someone manually fixes the state, and everyone hopes the same issue does not happen again.
That does not scale.
Human involvement should be designed, not improvised
Human oversight is sometimes presented as a temporary weakness that better models will eventually remove.
For many workflows, that is the wrong framing.
The better question is where human judgment has the highest value. A person may not need to approve every search query or formatting decision. They may need to approve a financial transfer, a production deployment, a customer-facing commitment, or a destructive change.
Good systems place human attention at the points where mistakes are expensive and difficult to reverse.
The agent should do the exploration, preparation, comparison, and repetitive work. The human should retain control over ambiguous decisions, high-impact actions, and exceptions that fall outside the system’s tested boundaries.
This is not a failure of autonomy. It is a sensible allocation of responsibility.
Production readiness means bounded risk
We should stop treating production readiness as a binary question: either an agent can be trusted completely, or it cannot be used.
Real systems are rarely built that way.
A production-ready agent may still make mistakes. The difference is that its mistakes are constrained by permissions, detected by checks, contained by boundaries, and recoverable through known procedures.
That standard is more realistic and more useful than demanding 100 percent trust.
The agent does not need to be right about everything. It needs to be reliable about what it reports, cautious about what it changes, and transparent about what remains uncertain.
The last mile is where trust is built
The emergence of new capabilities from large language models is genuinely impressive. Agents can now perform tasks that once required several specialized tools and a human coordinating every step.
But capability gets attention. Reliability earns adoption.
The last mile is the work of turning a powerful model into a dependable system. It involves conservative defaults, explicit boundaries, validation, fallbacks, observability, and clear expectations.
These safeguards may look boring beside a model that can reason across a million-token context or operate a complex application. They are still what allows that capability to enter production without making every user a full-time supervisor.
The future of agents will not be decided only by how much they can do.
It will also be decided by how safely they behave when they are wrong.
Top comments (2)
I've found the dangerous failures are the ones that complete the task while leaving the wrong side effect. I treat "done" as an evidence-backed claim from the system, including recipient, resulting state, and reversibility, not as the agent's final sentence.
"Done should be a state backed by evidence" is the line I would build the whole contract around. The hard edge case is an interrupted write: a timeout means the agent often cannot tell whether nothing happened or the side effect landed and only the receipt was lost.
That needs a first-class unknown state, not a retry disguised as recovery. Give each mutation a stable operation ID, read the destination before retrying, and preserve the original request plus expected effect so another worker can reconcile it. Idempotency prevents duplicates; reconciliation prevents the system from confidently reporting the wrong history.