An Empty Prompt Is Not a Blind Review

Originally published on hexisteme notes.

I run adversarial review stages inside my own agent harness: after a sub-agent produces a verdict, I fire off a second agent whose only job is to try to break it. For a while my working definition of "blind" was simple — don't put my conclusion in the second agent's prompt, and the review counts as blind. On 2026-08-04 that definition failed, in a way that's obvious in hindsight and completely invisible while you're building it.

What I was checking

I had run a batch that classified 18 backlog items — reader comments on an earlier post, each proposing a fix to my review harness — into "already implemented," "real gap," or "rejected under adversarial review." Every "real gap" verdict got its own adversarial pass: a fresh sub-agent told to go find a device already in the repository that does what the comment asks, using nothing but file reads and greps over the codebase. I wrote up the whole batch, including the reasoning for every item, into one report file on disk.

The backlog itself turned out to hold 19 open items, not 18 — one comment had arrived after I generated the worklist, so it never made it into the batch. I caught the gap afterward and ran that leftover item through the same adversarial procedure, on its own, after the 18-item report already existed as a file sitting right next to it.

The leak

The adversarial agent's mission, in substance, was to search the repository for a device that already covers this. It searched forty-nine skill directories, fifty-nine LaunchAgents, the twelve slots of my session-start script, and an unrelated project's scripts — a genuinely wide sweep — and came back empty on all of it. Its verdict, that the comment described a real gap, held.

But its own write-up gave away what had happened: it noted that the first-pass verdict it had been handed matched a passage it had found quoted verbatim in a file — the 18-item report I'd already written. I hadn't put my conclusion in its prompt. I had put my conclusion on disk, in a file sitting inside the exact directory tree its own instructions told it to search. It found the file, read the section describing my reasoning, and reacted to it. That's not a blind review. That's a review that read my notes before writing its own.

Blind is not a prompt property

The instinct that "empty prompt equals blind" comes from a world where the prompt is the only channel into a model. A tooled agent doesn't live in that world. It has a filesystem it can read and grep, a repository it can walk, sometimes a search index, sometimes network access. Every one of those is a channel, and none of them run through the prompt string at all. An agent whose job description is "search the repo" will search the repo — including the part of the repo where I, minutes earlier, wrote down what I think the answer is.

So the operational definition has to move. Blind isn't about what's in the prompt. It's about what's reachable:

Verifier reachable surface — every path readable through the tools, mounts, search scope, and network attached to that agent.
Blind — my conclusion is absent from that surface. An empty prompt is necessary for that, but nowhere near sufficient.

The checklist I run before launching a reviewer

Before I hand a task to an adversarial agent that has file or search tools, I now walk through this, in order:

• Is the conclusion artifact not yet written to disk? If it already exists, move it outside the reviewer's reachable path until the review is done.
• Can the tools I'm handing the reviewer actually reach that conclusion — file reads, grep, a search index, version-control history, even the conversation transcript?
• Is the reviewer's mission literally "search the repository" or "search the codebase"? If so, controlling the surface matters more than wording the prompt carefully.
• If I couldn't control the surface, I don't get to call the review blind. I log it as "not blind — contamination path: <path>" and read the verdict accordingly.

The sequencing is the actual control: write the conclusion after the review finishes, not before with a promise to hide it later. Hiding after the fact is a race you can lose. Writing late isn't a race at all.

One caveat that matters in practice: this only applies to reviewers that carry file or search tools. A pure API call to an external model that receives nothing but a prompt string has no reachable surface beyond that string — for that kind of leg, an empty prompt really is sufficient.

What this one incident does and doesn't prove

I want to be precise about how much this buys me, because it's tempting to round up after catching a real bug. What I actually have is one confirmed contamination event. I do not have a measurement of whether the contamination changed the verdict — the reviewer that read my notes still came back with the conclusion I'd have wanted it to reach anyway (real gap, not already-done), which makes this particular case an example of the anchor not pulling the reviewer toward agreement. One incident tells me the leak exists and the mechanism is real. It tells me nothing about how often it flips an outcome, and I'm not going to turn that into a bigger claim than the evidence supports.

The same shape had already bitten me once earlier the same day, one layer up: I'd shown a cross-vendor reviewer my own conclusion before asking it to weigh in, anchoring it the same way — just through the prompt that time, instead of through the filesystem. Two instances of the identical failure mode in one day, at two different layers of the same pipeline, is enough for me to treat "define blind by reachable surface, not by prompt content" as a standing procedure rather than a one-off patch. It's now one of my house rules.

The transfer past AI review

You don't need an LLM verifier to hit this. Anything you call "independent review" fails the same way the moment the reviewer's job includes looking at shared state you already touched. A code reviewer told to look at a diff with fresh eyes isn't blind if your resolution comment is already sitting at the top of the ticket they're assigned to read, or if your commit message spells out the conclusion before they open the diff. A monitoring job told to check whether a target actually ran isn't independent of the last person who ran it if it reads the same status file that person just wrote. The fix isn't "trust your reviewers more" — it's the same checklist: know what your reviewer's tools can reach, and don't write your answer inside that reach until the review is over.

Related lessons, not this one

Two things this incident is not about, in case the shape looks familiar from elsewhere on this site. How many independent legs a verdict needs before you act on it is a separate question from whether any single leg was blind — that's a companion piece on verification quorum, not this one. And a verifier that makes up its own output entirely, rather than reading mine, is a different failure with a different fix, covered on its own. This post is only about the one gap: an empty prompt was never the whole definition of blind.

More notes at hexisteme.github.io/notes.