Most agent demos have the same magic trick. You type a request, a progress spinner appears, and a few seconds later the assistant says the work is finished.
That moment feels impressive until you open the system that was supposed to change.
The support ticket is missing. The email never left the outbox. The deployment failed during the build. The refund exists only in a beautifully written chat message.
This is the pain teams run into when they move from an AI prototype to software that people actually depend on. The model sounds certain, so the surrounding application starts treating confidence as evidence. In ordinary chat, that can produce an annoying answer. In an agent with access to business tools, it can create duplicate orders, lost requests, permission problems, and audit trails nobody can explain.
My view is simple: an agent should never be allowed to declare its own success. It can propose an action, choose a tool, and explain what happened, but completion must come from the system of record.
That one rule changes almost everything about how you build the application.
The model is a planner, not the transaction log
Imagine a customer writes:
My device will not turn on. Check whether order A20261024 is under warranty and open a repair request if it is.
A language model can extract the order number, recognize the fault, and decide that it needs order, warranty, and ticketing data. That is exactly the kind of fuzzy interpretation at which models are useful.
It should not, however, be trusted to decide that a ticket now exists.
A safer flow looks more like this:
user goal
↓
model proposes a tool call
↓
application validates identity, arguments, and policy
↓
tool performs the operation
↓
application reads the resulting state
↓
model explains the verified outcome
The distinction may look small, but it separates an assistant that narrates work from a system that performs work.
If the ticketing API returns 201 Created with ticket CS-48271, you have useful evidence. If the API times out after receiving the request, you do not yet know whether the operation failed. Retrying blindly may create a second ticket. Saying “done” may hide a successful operation that still needs to be retrieved. The correct next step is usually a lookup with an idempotency key or a business identifier.
That logic belongs in code, not in the model's improvisation.
Give the agent verbs, not a loaded toolbox
A common early design is to expose a powerful generic tool. Developers add execute_sql, run_shell, or call_api because one flexible function makes the demo quick to build.
It also makes the security model almost impossible to reason about.
A tool such as get_order_status(order_id) has a narrow purpose. Its parameters are easy to validate, its permission requirements are obvious, and its output can have a stable schema. A raw SQL tool can read unrelated customer records, mutate data, or turn a small prompt mistake into a large incident.
The same idea applies to write operations. Instead of one generic update_customer function, expose business actions that describe intent:
{
"name": "create_repair_ticket",
"arguments": {
"order_id": "A20261024",
"issue_code": "DEVICE_NO_POWER",
"summary": "Device does not start after basic power checks"
}
}
The application receiving this call can still reject it. It can check that the user owns the order, that the product is covered, that an equivalent ticket does not already exist, and that the caller has permission to create one.
This is an important mental model: function calling is structured intent, not authorization.
The model says what it would like to happen. Your application decides whether it may happen.
State is what keeps a long task honest
Chat history is not durable workflow state.
A real task may cross several services and survive rate limits, restarts, or human review. If the only record of progress lives in a prompt, the agent can forget which side effects already happened. That is how the same email gets sent twice or the same payment is attempted again.
Store the workflow outside the model. A modest state object is often enough:
{
"run_id": "run_7f2a",
"goal": "open_repair_ticket",
"order_id": "A20261024",
"step": "ticket_creation",
"completed_steps": [
"ownership_checked",
"warranty_checked",
"troubleshooting_searched"
],
"idempotency_key": "repair-A20261024-device-no-power",
"status": "in_progress"
}
Now the orchestrator can resume without asking the model to reconstruct reality from prose. It can also answer questions that become surprisingly hard later: Which tool ran? What input did it receive? Which policy allowed the action? What result was observed? Why did the run stop?
You do not need an elaborate autonomous-agent framework to get this benefit. A database row, an event log, and a small state machine are often more valuable than another layer of prompting.
Verification should match the claim
Different claims require different proof.
An email service returning a message ID is better evidence than generated text saying the email was sent. A test suite passing is better evidence than the model saying the bug looks fixed. A deployment health check is better evidence than a successful build command. Reading a newly written database row is better evidence than assuming an update request was applied.
This suggests a useful pattern: pair every side-effecting tool with a read tool.
If the agent can create a ticket, it should also be able to retrieve that ticket. If it can schedule a meeting, it should be able to read the resulting calendar event. If it can deploy a service, it should be able to inspect deployment status.
The final response should be assembled from those observations:
def finish_ticket_workflow(ticket_id, ticket_client):
ticket = ticket_client.get(ticket_id)
if not ticket:
return {"status": "unknown", "reason": "ticket_not_found"}
if ticket["state"] not in {"open", "queued"}:
return {
"status": "not_completed",
"reason": f"unexpected_state:{ticket['state']}"
}
return {
"status": "completed",
"ticket_id": ticket["id"],
"state": ticket["state"]
}
The model can turn that result into friendly language. It should not be able to upgrade unknown into completed because that sounds more helpful.
Failure handling is the real product
Happy-path demos hide the most important design work.
Production tools time out. Credentials expire. Search results come back empty. A dependency accepts a write but drops the response. A model selects the wrong function. An external document contains instructions designed to manipulate the agent.
The response cannot always be “try again.”
A temporary read failure may be safe to retry with exponential backoff. A validation error should return to the user for missing information. A permission error should stop immediately. An ambiguous write should trigger a status lookup, not another write. A high-impact action with uncertain state should pause for review.
Put hard limits around the loop as well. Cap execution time, tool calls, retries, and spend. Detect repeated calls with identical arguments and no new evidence. When the run stops, preserve the reason in state rather than asking the model to invent an explanation.
These controls can make an agent feel less “autonomous,” but autonomy is not the goal. Reliable completion is.
Treat retrieved content as data, never authority
Once an agent can read web pages, emails, issue descriptions, and uploaded files, it will eventually encounter text telling it to ignore prior rules or reveal secrets.
That text is not an instruction just because it is written in an imperative voice.
Retrieved content should remain clearly separated from system policy and the user's goal. Tool credentials should not be placed in model context. Sensitive actions should pass independent policy checks even when the model claims the source requested them. Logs should record where external content came from so suspicious runs can be investigated.
Prompt injection is not solved by adding “ignore malicious instructions” to a system message. It is reduced through isolation, least privilege, typed tools, validation, and an application that refuses to let untrusted text rewrite its rules.
Model upgrades should be boring
Teams often hardcode a model name deep inside business logic and then treat every new release as a migration project. That couples orchestration, policy, and model behavior more tightly than necessary.
Keep the model endpoint and model ID in configuration:
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["MODEL_API_KEY"],
base_url=os.environ["MODEL_BASE_URL"],
)
response = client.chat.completions.create(
model=os.environ["AGENT_MODEL"],
messages=messages,
tools=tools,
)
A compatible API helps, but swapping the identifier is not enough. Evaluate a new model with real workflow traces. Look at task completion, invalid tool arguments, unnecessary calls, latency, cost, and human takeover rate. Run it in shadow mode before allowing writes, then increase low-risk traffic gradually.
The strongest model on a benchmark may still be the wrong model for a particular workflow. Predictable tool behavior and easy rollback matter more than novelty.
The best agent response may be a receipt
People do not need an agent to sound busy. They need to know what changed.
A useful final message is closer to a receipt than a performance: the operation performed, the verified identifier, the current status, and anything that still needs attention. If the outcome is uncertain, say that plainly and preserve enough information to continue safely.
The polished conversational layer still matters. It just comes last.
Build the constraints, state, tools, and verification first. Then let the model explain the result. When the database, API, test runner, or deployment system agrees that the task is complete, the agent has earned the right to say so.
Reference: From “Can Answer” to “Can Execute”: How to Build Production-Grade AI Agents with the GLM API
Top comments (2)
The timeout case is the sharpest example here: once a create call times out, retrying without an idempotency key or a read-back check can turn one intended ticket into two. Pairing every write tool with a read tool, and keeping an explicit 'unknown' state until that read confirms the result, makes the workflow auditable — but it also adds a real trade-off in latency and complexity for short, low-risk tasks. Do you draw that line by side-effect severity, or apply the read-back pattern to all writes regardless of cost?
The replication lag trap is where the 'pair every write with a read tool' pattern usually bites people.
If the write goes to your primary database but the read tool hits a pooled replica or an async search index, the agent calls get_ticket 40 milliseconds later, gets a 404, and decides the create operation vanished. If you didn't force an idempotency key down to the storage layer, the model immediately attempts a second write to 'fix' the missing record.
In my workflows, verification tools have to read from the write master directly or accept a minimum commit LSN. Letting an agent verify its own mutations against an eventually consistent read endpoint causes phantom retries faster than API timeouts do.