The database transaction succeeded. The version check passed. The API returned 200 OK. The authorisation was valid. The tool call completed successfully. And the AI agent was still wrong.
That sounds strange until you look at what actually happens when an AI agent starts taking real actions in a production system. An agent doesn't operate on a frozen snapshot of reality. It reads information from several systems, reasons about that information, decides what to do, and then calls one or more tools to make something happen. During that time, the systems around it can continue changing.
A payment can move from pending to failed. Inventory can run out. A customer can cancel an order. A risk score can change. Another agent can update the same business process.
The agent may have made a completely reasonable decision based on what it knew at the time. The problem is that what is known may no longer be true when the action is finally executed.
That isn't always a reasoning problem. Sometimes, it is a consistency problem.
The gap between reading and acting
Consider a simple payment approval workflow. An agent receives a request to approve a high-value payment. It checks several systems and gets something like this:
Payment status: Pending
Customer status: Verified
Risk score: Low
Approval policy: v42
Based on those inputs, the agent decides that the payment can be approved. While the agent is thinking, however, the payment service detects something suspicious and changes the payment status:
Pending → Flagged
A moment later, the agent sends the approval request.
Now, suppose the payment record uses optimistic locking:
if payment.version != expected_version:
raise ConflictError()
payment.status = "approved"
payment.version += 1
The version check passes because nobody changed that particular payment record after the agent read it.
The database is happy.
The API returns success.
But the decision is no longer correct because one of the pieces of information that justified the decision has changed.
This is an important distinction for agentic systems:
A successful write does not necessarily mean that the decision behind the write was still valid.
Traditional applications already deal with concurrency, but agents make the problem more interesting because there is now a reasoning step sitting between reading state and changing state.
An agent is reasoning over a moving target
A useful way to think about an agent is:
Read state → Reason → Take action
The dangerous assumption is that the state observed during the first step remains valid until the final step.
In a distributed system, that assumption is often wrong.
Imagine the following sequence:
Agent Payment Service
| |
|--- Read payment -------------->|
|<-- Pending --------------------|
| |
| Agent reasons |
| |
| Payment becomes
| Flagged
| |
| Decide: Approve |
| |
|--- Approve ------------------->|
|<-- 200 OK ---------------------|
Nothing necessarily failed at the infrastructure level. The database didn't crash. The API didn't timeout. The optimistic lock didn't detect a conflict.
The problem is that the agent made a decision using a state that had become stale.
Once agents are allowed to perform consequential actions, this gap between what the agent observed and what is true when it acts becomes something the architecture has to handle deliberately.
A version check only tells you part of the story
Optimistic locking is still extremely useful. For example:
UPDATE payments
SET status = 'approved',
version = version + 1
WHERE id = 123
AND version = 42;
If another process has already modified that payment, the update fails.
That's good protection.
But imagine that the agent's decision depended on several other services:
Payment Service → payment status
Risk Service → risk score
Customer Service → customer verification
Policy Service → approval policy
Fraud Service → fraud assessment
The agent might have seen:
payment.status = Pending
risk.score = 18
customer.status = Verified
policy.version = 42
fraud.result = Clear
Checking only the payment version doesn't tell us whether the risk score changed, whether the customer was suspended, whether the fraud result was updated, or whether the applicable policy changed.
The write can therefore be completely consistent while the decision itself is stale.
That's where the idea of a read set becomes useful.
Treat the decision inputs as a read set
Instead of recording only the record the agent intends to modify, we can also record the important inputs that influenced the decision.
For example:
{
"payment": {
"id": 123,
"version": 42
},
"risk": {
"customer_id": 781,
"version": 19
},
"customer": {
"id": 781,
"version": 63
},
"policy": {
"id": "high_value_payment",
"version": 7
}
}
Now the system has some understanding of the state on which the decision was based.
Before executing a consequential action, it can determine whether the inputs that matter are still valid.
For example:
def validate_decision(decision):
if payment.version != decision.payment.version:
return False
if risk.version != decision.risk.version:
return False
if customer. version != decision.customer.version:
return False
return True
The interesting part is that not every input needs the same level of freshness.
A payment status may need to be checked immediately before a refund. A policy document may only require the system to know exactly which version was used. A customer's display name probably doesn't need to be revalidated at all.
So the goal isn't necessarily to make every piece of agent context perfectly fresh.
The better question is:
Which inputs must still be valid for this particular action?
That lets you build a more practical consistency model instead of turning every agent operation into an expensive distributed transaction.
Stale context is also an observability problem
There is another reason I like the idea of recording the read set: it makes agent failures much easier to investigate.
Suppose an agent approves a transaction that it shouldn't have approved.
Without decision metadata, the investigation might simply say:
The agent made an incorrect decision.
That's not very useful.
With the versions captured at decision time, you might see:
Decision ID: 8f32
Payment version used: 42
Risk version used: 19
Customer version used: 63
Policy version used: 7
Payment current version: 43
Risk current version: 19
Customer current version: 63
Policy current version: 7
Now there is a much clearer explanation.
The agent made the decision using payment version 42, but the payment had already moved to version 43 before the action was executed.
That is very different from saying, "the model reasoned incorrectly."
This distinction becomes increasingly important as agents become part of real business workflows. When something goes wrong, engineers need to know whether the problem came from the model, stale information, a race condition, a tool failure, or a business rule that wasn't enforced.
Then there is the write-skew problem
Individual version checks also have a limitation that is easy to miss.
Imagine a company has a shared credit limit of $100,000. Current exposure is $50,000, so the system has $50,000 available.
Two agents are processing different accounts at roughly the same time.
Agent A checks the available credit and sees $50,000.
Agent B does the same thing and also sees $50,000.
Both approve a $40,000 transaction.
The important detail is that they are modifying different account records. So each agent's individual optimistic-locking check can succeed.
The resulting exposure is:
Starting exposure $50,000
Agent A approval +$40,000
Agent B approval +$40,000
--------------------------------
Final exposure $130,000
Every individual's write can be valid while the overall system is now violating its credit limit.
This is the classic write-skew problem.
The thing that needs protection isn't just Account A or Account B. It's the shared invariant:
Total exposure <= Credit limit
That means the shared constraint itself has to participate in the consistency mechanism, whether through commit-time validation, coordination around the shared resource, or another mechanism appropriate to the system.
This is one of those cases where simply saying "we have optimistic locking" isn't enough.
Idempotency doesn't solve stale decisions
It's also worth separating this problem from idempotency because they are often discussed together.
Suppose an agent calls:
POST /payments/123/refund
Idempotency-Key: refund-123-abc
The request times out. The agent doesn't know whether the refund actually happened, so it retries.
An idempotency key allows the server to recognise that the retry represents the same logical operation and avoid performing the refund twice.
That's important.
But the idempotency key doesn't tell us whether the original decision to issue the refund is still valid.
The agent could make one perfectly idempotent request based on stale information.
These mechanisms solve different problems:
Optimistic locking
→ Detects conflicting changes to protected records
Idempotency
→ Prevents duplicate execution
Read-set validation
→ Detects stale decision inputs
Invariant validation
→ Protects shared business constraints
A production agent may need several of these mechanisms working together.
The agent should propose the action, not blindly own execution
This leads to an architectural pattern I find useful for consequential agent workflows.
The model can make a decision and propose an action, but a separate runtime layer can determine whether that action is still valid.
Something like:
AI Agent
|
| Decision
v
Decision Record
┌───────────────┐
│ Read Set │
│ Versions │
│ Preconditions │
└───────┬───────┘
|
v
Validator
┌───────────────┐
│ Freshness │
│ Preconditions │
│ Invariants │
└───────┬───────┘
|
┌──────┴──────┐
| |
Invalid Valid
| |
v v
Re-read Execute
Re-plan |
v
External System
The model is responsible for reasoning.
The runtime is responsible for enforcing the rules around execution.
That separation is useful because we don't want a language model to be the final authority on whether an action is safe to execute.
Context isn't truth
This is perhaps the biggest mindset change.
When an agent says:
"The payment is eligible for a refund."
The system shouldn't necessarily interpret that as an absolute fact.
A more accurate interpretation is:
"The agent concluded that the payment was eligible for a refund based on the state it observed."
Those statements are subtly different.
The second one makes the freshness boundary visible.
If the payment status changes before execution, the system can re-read it and either continue, ask the agent to reconsider, or route the action for human approval.
The agent's context becomes evidence with a validity window, rather than permanent truth.
The decision becomes an important unit of the state
Traditional applications tend to focus on records, transactions, and API calls.
Agentic systems introduce another useful concept: the decision itself.
A consequential decision can be thought of as:
Inputs
+
Versions
+
Assumptions
+
Applicable policy
+
Decision
+
Action
If a critical input changes before execution, the decision may no longer be valid.
That doesn't necessarily mean the model made a bad inference.
The world changed.
And that distinction matters when you're debugging autonomous systems.
Where this leaves us
The hardest part of building reliable AI agents may not be getting the model to reason better.
It may be making sure that the world the model reasoned about is still the world in which its action executes.
An agent can produce a sensible plan. The database can accept the write. The API can return 200 OK. Authorisation can be completely correct.
The outcome can still be wrong.
That's why production agent engineering needs to go beyond prompts, memory, tool calling, and model evaluations. Once an agent starts taking real actions, the surrounding system needs mechanisms for freshness, concurrency, idempotency, read-set validation, and business invariants.
The question shouldn't only be:
"Did the agent make the right decision?"
We should also ask:
"
Was that decision still valid when the agent acted on it?"
That is where building reliable AI agents starts to look much more like distributed-systems engineering—and much less like building another chatbot.
Top comments (0)