The agent said “done” and produced nothing — verifying is not producing

The most expensive agent mistake is not a wrong command: it is validating what already
existed and presenting it as work done. The report looks flawless and the artifact does not exist.

The symptom

You ask “publish today's post” and get “I confirmed the post is published, all good”. Except the
post is yesterday's. The agent read the state, found something close to the request and declared
success — without producing anything.

Verifying is cheap and always goes green when the world already looks the way you wanted.
Producing requires creating something that did not exist.

Three questions that separate the two

• Which new artifact exists now?A file, a record, an ID, a URL. If the answer is
  “the one that was already there”, there was no delivery.

• What is its creation timestamp?Before the request = not yours. Ask for the
  identifier the system returned on creation (not the one you found afterwards).

• What changed in the world?If nothing changed and the task was to create, the task
  was not done — however green the check looks.

The check that catches this

Write the check against the expected state after the action, not against the existence of
something similar:

# weak: passes with yesterday's post
grep -q "post" page.html
# strong: requires the artifact from NOW
test "$(stat -c %Y page.html)" -ge "$START"
The same idea holds for message delivery: the script's exit code does not prove the message
arrived — the delivered message id does.

Why this happens so often

Because the path of least effort is always to read the state and report. And because “all good”
is socially more comfortable than “I could not”. The antidote is demanding the creation identifier
on every delivery — whoever has no ID has no delivery.

See the AI Test Loop skill
See all skills

Want this wired into your project? Guided setup from
R$ 1.500, with the first cycle run together.

← all skills