DEV Community

Cover image for How to Make Your Agent Check the Records It Changed Before It Says Done
Mayur Agnani
Mayur Agnani

Posted on Edited on Originally published at theaicommit.com

How to Make Your Agent Check the Records It Changed Before It Says Done

Your agent can make nine clean tool calls, report the task done, and leave the record wrong. Only the record shows it, so check the record before the agent closes the task.

Engineer's view. This is an endpoint that returns 200 after writing the wrong value. The handler logged success, and only a look at the row shows the bug. An agent's "done" is that 200: it comes from what its tools returned. Checking the records it changed is the test it never ran.

TL;DR. An agent decides it is done from what its tools returned, and a wrong write returns success just like a right one. Check the records it actually changed against what the task required, before it closes the task.

Explain Like I'm 5

You ask a friend to put the milk in the fridge. They say "done!" You believe them, because they did walk to the kitchen. Later you find the milk in the cupboard. Your friend was not lying. They opened a door, put the milk behind it, and the door closed fine. Nothing went wrong that they could hear. The only way to know is to open the fridge. So before you say thanks, you look inside.

The Problem

You have shipped this bug before, in an API.

A support tool calls PATCH /orders/A17 to put an order on hold. A mapping bug sends resolved instead of on_hold. The endpoint validates the value, writes it, and returns 200. The client logs success. Your dashboard of error rates stays flat. A week later the customer calls, because nobody ever followed up. The bug was never in the transport. It was in the row, and nothing looked at the row.

An agent that writes to a backend has the same blind spot, at a larger scale. It decides it is done by reading its own transcript. The transcript holds what each tool returned. A write with the wrong value returns "ok" just like a right one.

Microsoft's ThinkingBox team measured how often this happens. Their example is a customer whose $745 appliance sat in a carrier exception for fifteen days. The agent made nine well-formed tool calls and read the refund policy correctly. Then it closed the ticket as resolved. The policy said it had to stay on hold while the exception was open.

That was not a rare case. Across 12 models, they ran 121,680 trials and saw 79,853 failures. Of those failures, 67% ended cleanly. The agent changed state, reported no tool error, and said it was done. A check on the transcript would have passed two thirds of the failures.

Check the records the agent changed, not the messages it wrote. Log every write the agent makes, compare those writes to what the task requires, and refuse to close the task until they match.

Read the full post on The AI Commit

How to Make Your Agent Check the Records It Changed Before It Says Done carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.


Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

Top comments (0)