Mayur Agnani

Posted on Oct 20 Originally published at theaicommit.com

How to Make Your Agent Check the Records It Changed Before It Says Done

#agents #ai #devops #testing

Engineer's view. This is an endpoint that returns 200 after writing the wrong value. The handler logged success, and only a look at the row shows the bug. An agent's "done" is that 200: it comes from what its tools returned. Checking the records it changed is the test it never ran.

TL;DR. An agent decides it is done from what its tools returned, and a wrong write returns success just like a right one. Check the records it actually changed against what the task required, before it closes the task.

Explain Like I'm 5

You ask a friend to put the milk in the fridge. They say "done!" You believe them, because they did walk to the kitchen. Later you find the milk in the cupboard. Your friend was not lying. They opened a door, put the milk behind it, and the door closed fine. Nothing went wrong that they could hear. The only way to know is to open the fridge. So before you say thanks, you look inside.

The Problem

You have shipped this bug before, in an API.

A support tool calls PATCH /orders/A17 to put an order on hold. A mapping bug sends resolved instead of on_hold. The endpoint validates the value, writes it, and returns 200. The client logs success. Your dashboard of error rates stays flat. A week later the customer calls, because nobody ever followed up. The bug was never in the transport. It was in the row, and nothing looked at the row.

An agent that writes to a backend has the same blind spot, at a larger scale. It decides it is done by reading its own transcript. The transcript holds what each tool returned. A write with the wrong value returns "ok" just like a right one.

Microsoft's ThinkingBox team measured how often this happens. Their example is a customer whose $745 appliance sat in a carrier exception for fifteen days. The agent made nine well-formed tool calls and read the refund policy correctly. Then it closed the ticket as resolved. The policy said it had to stay on hold while the exception was open.

That was not a rare case. Across 12 models, they ran 121,680 trials and saw 79,853 failures. Of those failures, 67% ended cleanly. The agent changed state, reported no tool error, and said it was done. A check on the transcript would have passed two thirds of the failures.

Check the records the agent changed, not the messages it wrote. Log every write the agent makes, compare those writes to what the task requires, and refuse to close the task until they match.

The Fix: Check the Agent's Writes Before It Closes the Task

New to how an agent decides it is finished? Start at AI basics → The agent loop.

Why doesn't the agent notice on its own?

Because nothing it can see is wrong. The model reads tool results, not your database. If update_ticket returns {"ok": true}, the model has no signal that the value it sent was the wrong one. Asking it to "double-check" only makes it reread the same transcript.

So the check has to read state. Every write the agent makes is an effect: a table, a row, a field, and its value before and after.

What does the check compare?

A contract per task type. It lists the fields that must end at a given value and the fields the task may touch. Comparing the effects to the contract finds three kinds of failure:

In ThinkingBox's failures, 78% had a wrong field value, 43% had an extra effect, and 25% missed a required one. One run can have more than one.

Where does the check go?

On the tool that ends the task, such as close_ticket. If the check finds a problem, the tool refuses and returns the list of problems as its result. The agent then reads a real signal: which field is wrong, and what it should be. It can fix the field, or hand the ticket to a person.

What This Means for You

When this matters. Any agent that writes to a system of record: tickets, orders, refunds, a CRM, a calendar. It matters most where a wrong write is quiet. A ticket marked resolved sends no alert. A refund issued twice shows up only at month end.

How it affects you. Most of your agent's failures will look like successes in your logs. ThinkingBox's authors read the failures as mostly a problem of tool use and recovery, not reasoning. That is good news, because tool use is the part you control.

There is a second cost. A single run tells you little about the next one. One open-weight model in the study solved 94% of tasks at least once, but only 13% on all 20 tries. Claude Opus 5 solved fewer at least once, yet passed 48% on every try. The code_example.py file shows why. Two simulated agents both pass about 72% of single runs. One fails a little on every task, and passes all 20 runs on none of them. The other is reliable on most tasks and hopeless on a few, and passes all 20 on 38%.

What to do about it. Log every write at the tool layer. Write a contract for each task type that changes records. Put the check on the closing tool. Then score your agent by how many tasks pass every run, not by its average.

Implementing It

The change.

Three roles touch this. Whoever writes the tools records each write. Whoever builds the agent puts the check on the closing tool. Whoever runs the evals scores every task over several clean runs.

Role 1, whoever writes the tools. Route every state-changing tool through one write function. It records the effect before it returns:

def write(self, table, key, field, value):
    row = self.db[table].setdefault(key, {})
    self.effects.append(Effect(table, key, field, row.get(field), value))
    row[field] = value
    return "ok"            # all the model will ever see

Read-only tools skip it. A tool that calls another service records what it asked for, plus the id that service returned. Keep one log per run, never one per process, so two runs never share effects.

Role 2, whoever builds the agent. Write the contract from your policy, not from a golden run. A ticket with an open carrier exception must end on hold:

HOLD = Contract(
    required={("tickets", "T9", "status"): "on_hold"},
    allowed={("tickets", "T9", "note"), ("tickets", "T9", "closed")},
)

The check reads final values for the required fields, and flags any effect outside both sets. Put it on the closing tool. In the OpenAI Agents SDK, a tool input guardrail runs before the tool executes and can reject the call with a message:

from agents import ToolGuardrailFunctionOutput
from agents.decorators import tool, tool_input_guardrail

@tool_input_guardrail
def records_match_task(data):
    problems = contract_for(current_task()).check(db, effect_log.effects)
    if problems:
        return ToolGuardrailFunctionOutput.reject_content(
            "Not closed. Fix these first:\n" + "\n".join(problems))
    return ToolGuardrailFunctionOutput.allow()

@tool(tool_input_guardrails=[records_match_task])
def close_ticket(ticket_id: str) -> str: ...

The rejection message goes back to the model as the tool's output. Without that SDK, do the same check at the top of the tool function. Cap the retries, then escalate to a person.

Role 3, whoever runs the evals. Run each task several times, each from a fresh copy of the data, and count the tasks that pass every run. The fresh copy matters: a run that starts on rows an earlier run changed can pass for the wrong reason. That number is called pass^k:

every = sum(all(run_task(t, fresh_db()) for _ in range(k)) for t in tasks)
print(f"pass^{k}: {every / len(tasks):.0%}")

How you know it worked.

Replay four runs: one right, one with a wrong value, one with an extra write, one with a missing write. A transcript check passes all four. Your gate must close the first and refuse the other three, naming the field each time. In production, count refusals by kind. A rising count of extra effects usually means the agent has a tool it should not have.

When a Completion Check Is the Wrong Tool

When you cannot say what the end state should be. A research agent or a drafting agent has no record to check. Review its output instead, or grade it with a rubric.

When the effect leaves your system. An email, a payment to a third party, or a message in someone else's queue cannot be read back or undone. A check after the fact only tells you what went wrong. Require a person's approval before those calls instead.

When the task types multiply faster than you can write contracts. Write contracts only for the tasks that change money, status, or anything a customer sees. Let the rest through with an effect log you review weekly.

When the agent only reads. A read-only agent has no effects. Its failures are wrong answers, and those belong in an answer eval.

The authors did not measure how much a check like this raises the pass rate, and neither did this session. Measure it on your own tasks before you promise anyone a number.

Before you add a gate, ask:

Can I write down, for this task type, which fields must change and to what?
Does every write go through a layer I can log?
What happens to the customer if the gate refuses and nobody picks it up?

Glossary

transcript — the messages and tool results an agent has seen so far in a run.
effect — one write the agent made: table, row, field, and the value before and after.
contract — for one task type, the fields that must end at a value and those it may touch.
extra effect — a write to a field the task's contract does not allow.
tool input guardrail — an Agents SDK check that runs before a tool and can reject the call.
pass^k — the share of tasks an agent passes on every one of k clean runs.

DEV Community