I made "verified" mean something for AI agent instructions — here's what that actually took

chiragmangaldev3112 — Fri, 18 Sep 2026 09:52:09 +0000

Every AI coding agent I used — Claude Code, Cursor, GitHub Copilot —
failed in the same few ways. It guessed at an ambiguous request instead
of asking. It graded its own work instead of checking independently. It
forgot a project's conventions the moment a new session started.

So I wrote the process down: 37 "playbooks" covering bug fixing, feature
development, code/security review, safe database migrations, incident
response, and more. One instruction set, installed once, read natively by
whichever AI tool you're using (15 supported so far).

That part's a straightforward pitch. What I actually want to write about
is the harder problem underneath it: once you claim something is
"verified," what does that actually have to mean before the word is worth
anything?

The trap: "verified" as a vibe

Early drafts of some of these playbooks said things like "this was tested
and works." That's not verification — it's a vibe wearing verification's
clothes. It has the shape of a checked claim without any of the
substance, and it's exactly the kind of self-assessment I was trying to
get an AI agent to stop doing in the first place. If I'm asking an agent
to never trust its own "looks right," I can't ship a project full of my
own unchecked "looks right"s.

So I went back through every playbook and either found the real evidence
behind every claim, or removed the claim. This is where it got
interesting, because actually running things — not re-reading code,
not reasoning about it — surfaced real bugs I would never have caught
otherwise:

A sparse-page detection script for scanned PDFs was off-by-one on page counting. It assumed "N form-feed characters means N+1 pages," but direct byte-level inspection of a real scanned PDF showed pdftotext emits one trailing form-feed per page, including the last one. The math only looked right until I actually counted bytes in a real file.
A file-extension parser detected format from the full path, not the filename — so a file sitting in a directory like v2.0/README (a dot in the parent directory name, not the file) got misdetected as extension 0/readme and routed to completely the wrong handler.
A security hook meant to scan every file-write for leaked credentials had a matcher that only covered Write and Edit tool calls — MultiEdit calls bypassed it entirely. Nobody wrote that gap in on purpose; it just never got exercised until a re-evaluation pass specifically tried to break it.

None of these were found by review. All of them were found by running
the actual thing against a real case and checking the actual output.

Independent review catches what self-review can't — even for an AI

The MultiEdit gap above is the one I want to dwell on, because of how
it was found. I had a separate pass — same underlying model, zero memory
of why the original hook was built the way it was — re-check the finished
work cold. It didn't know the design history, didn't know what the
original author was trying to protect against. It just tested the actual
behavior against the actual documentation and found they didn't match.

This is the same reason code review works on human teams: the person who
wrote something is the worst-positioned person to spot what's wrong with
it, because they already have a model in their head of what it's
supposed to do, and that model quietly overrides what's actually in
front of them. It turns out this holds for an AI agent reviewing its own
prior work too — a fresh pass with no attachment to the original
reasoning catches things a self-review pass structurally can't, not
because it's smarter, but because it isn't carrying the same blind spot.

Signing matters more once you think about what the content becomes

The playbooks aren't just documentation — once installed, they become
literal instructions an AI agent with shell access follows. That's a
different threat model than a typical doc site. If the backend serving
this content were ever compromised, a swapped file isn't a broken link,
it's an instruction an agent might actually execute.

So every release is signed with Ed25519 (ssh-keygen -Y sign/verify, not
OpenSSL — stock macOS ships LibreSSL, which can't verify Ed25519
signatures at all, confirmed by trying it directly rather than assuming),
and the installer re-hashes every fetched file against a signed manifest
before writing anything. The backend can withhold a release, but it can't
successfully substitute one, because it never holds the private key.

Where this leaves things

37 playbooks, MIT-licensed, free, no account needed:
https://github.com/chiragmangaldev3112/agent-playbooks

I don't think "verified" is a box to check once. It's a standing bar
that gets tested every time someone looks at the thing again with fresh
eyes — which is exactly what a second, independent pass is for, and
exactly what found every bug described above.

Happy to go deeper on any part of this — the signing setup, the
per-tool artifact generation, or the specific bugs — in the comments.

DEV Community: chiragmangaldev3112

I made "verified" mean something for AI agent instructions — here's what that actually took

The trap: "verified" as a vibe

Independent review catches what self-review can't — even for an AI

Signing matters more once you think about what the content becomes

Where this leaves things