Anyone who has written code and then reviewed it minutes later knows the feeling: the diff looks fine. Of course it does. The person reading it is the same person who just talked themselves into every line of it. The naming made sense because the intent is still warm in memory. The shortcut felt justified because the reason for it is still sitting in working memory, uninspected.
That is the problem the cold-context critic is built to remove.
The core idea
In the harness I built to develop a game, every change passes through a review step performed by a fresh model instance that has no memory of writing the code. It did not plan the change. It did not argue for the approach. It did not feel the small relief of getting something to finally pass. It receives the diff and evaluates it on its merits, the way a stranger would.
The intuition is old and boring, which is part of why I trust it: an author rationalizes; a cold reviewer does not. A model — or a person — that just produced the work carries a bias toward the choices it already made. Strip the memory of making them, and the same mistakes stop being self-evidently correct. The reviewer has nothing to defend. It can only look at what is actually there.
I want to be precise about the claim, because it is easy to oversell. This is a discipline, an architecture choice. I am not going to tell you it caught a specific dramatic production bug, because I am not going to invent an incident to sell an idea. What I can say is the mechanism: cold review catches the class of mistakes an author talks itself into — the plausible-looking shortcut, the assumption never stated out loud, the edge case that "obviously" can't happen. Self-review is structurally bad at exactly those, because the author already accepted them once.
Where it fits
The cold critic is one layer, not the whole story. It sits alongside a few other pieces of the same distrust:
- Per-role model routing. Stronger models run the judgment stages — planning, critique, evaluation — and cheaper ones handle routine execution. The reviewer isn't grading its own homework, and it isn't the same weight class picked for speed.
- A differential oracle. Core game logic is implemented twice and held to shared invariants. When the two implementations disagree, at least one is wrong, and I find out without having to trust either.
- A human approval gate on every irreversible action. Deploys and commits do not happen because a model decided they should. They happen because a person approves them. That gate never moves.
I think about the whole thing on an autonomy ladder, L0 through L4 — from fully hands-on toward more independence. The cold critic is what makes climbing that ladder defensible. More autonomy is only safe if the verification underneath it does not depend on the thing being verified vouching for itself.
The setup, honestly
Concretely: the critic is a separate invocation with a clean context window. It does not inherit the conversation that produced the change. It gets the diff and the standard it is being held to, and it reports back before a human sees the change. That's the shape of it. The value isn't a clever prompt — it's the missing memory. The setup exists to guarantee the reviewer never accumulated a reason to like the code.
That's also why I keep the human gate immovable. The cold critic is a good reader, not an authority. It can flag; it doesn't get to ship.
An honest note
I'm about five to six months into this, career-changing from roughly six years in professional kitchens, no CS degree — I learned through freeCodeCamp and by building. The game is real and published to Google Play, but it is not serving millions, and I'm not running production RAG or fine-tuning pipelines. I'd rather tell you what this is than dress it up.
So the honest framing is this: I don't have the scars yet that teach senior engineers where the bodies are buried. What I can do is build the distrust in as a habit from the start — assume the author is biased, including when the author is me or a model I'm running, and make something with no stake in the answer do the checking.
If you want to see the same idea in code you can actually run, the differential oracle is public: github.com/egnaro9/evals-differential-oracle. It's the same move as the cold critic — distrust your own output — implemented as two independent versions of the logic checked against each other, plus invariant tests over thousands of random boards, with a deliberately buggy implementation included to prove the checks bite. Clone it, run pytest, watch it catch the planted bug.
More at egnaro9.github.io and github.com/egnaro9.
Top comments (5)
The detail that the critic sees the diff and the standard but not the spec is the whole design, and it is a sharper tradeoff than fresh-eyes-catch-what-the-author-rationalized. The amnesia that removes the rationalization also removes the intent. The author talking itself into a shortcut and the task's actual requirement arrive through the same door, the conversation that produced the change, so closing that door to kill the first closes it on the second. That is not an implementation gap, it is structural: the frame is the rationalization channel, which is exactly why you cannot have one reviewer that is both immune to rationalization and able to check against intent.
So the cold critic's clean-context strength is bounded to intent-independent properties: internal consistency, implausible shortcuts, edge cases visible in the code. What it cannot catch by construction is a diff that is locally flawless and solves the wrong problem, because it was never told what the right problem was. It will pass correct code for the wrong requirement every time, and it should. That is the price of the reset, not a bug in it. Which makes the missing layer you point at with "one layer, not the whole story" a specific one: a second gate that does see the spec and checks conformance, not quality. The trap is merging them. A reviewer handed both the spec and the code, told to judge both at once, re-imports the rationalization the cold critic existed to exclude, because now it can talk itself into why the code matches the intent. The two gates catch disjoint failure classes, and keeping them disjoint is what keeps either one independent. Intent-blind and rationalization-immune, or intent-aware and quality-blind. The cost shows up the moment you ask one reviewer to be both.
Best articulation of this tradeoff I've read — and you're right about the structural
part: the amnesia that kills rationalization kills intent, because both ride the same
channel. The one place I'd push: that channel is the author's live reasoning, not
"knowledge of intent" as such. Those come apart.
In the harness, intent isn't reconstructed from the conversation that produced the
change — it's a directive written and frozen before any code exists, with explicit
acceptance criteria. So conformance gets checked against a pre-committed artifact, not
the author's post-hoc story of why the diff is fine. Code-vs-frozen-spec isn't the
rationalization channel; code-plus-the-warm-conversation is. Same "intent," totally
different exposure.
Which makes the dichotomy real but escapable: intent-blind-and-rationalization-immune
(the cold critic, on quality) and a separate intent-aware gate that's still
rationalization-immune — because its intent came from an artifact, not a narrative.
The merge you named — one reviewer handed spec + code, "judge both" — is exactly the
failure, for the reason you gave: it re-opens the door. Keeping intent external and
pre-committed is what lets the second gate exist without becoming it.
Genuinely sharp — this is the kind of pushback that sharpens the design.
You are right, and the correction is a real improvement on what I posed. The dichotomy was too strong because I collapsed intent into the channel it usually arrives on. A frozen directive with pre-committed acceptance criteria carries intent without carrying the author's live reasoning, so conformance against it is intent-aware and still rationalization-immune. The failure was code-plus-warm-conversation, and I wrote it as if intent could only come through the warm conversation. It cannot be retrofitted from an artifact the author committed to before the code existed, which is the whole point of freezing it.
The one thing the freeze converts rather than removes: it kills rationalization but not frame error. A pre-committed directive is immune to the author talking itself into why the diff is fine, because the author could not have written the directive to fit a diff that did not exist yet. It is not immune to the author having framed the requirement wrong in the first place. The conformance gate then faithfully certifies conformance to a wrong directive, cleanly, with no rationalization anywhere in the loop. So freezing intent moves the surviving risk from post-hoc story to upstream framing, which is the exact residue from the harness thread, the spec-level judgment that sits above the reset. The frozen spec is what lets the second gate exist without becoming the first. It is not what audits the spec, and a second frame on the directive is still the open piece. Two gates for two failures, and the freeze is what keeps them from collapsing into one.
The frozen spec lets the second gate exist without becoming the first, and it's worth being exact about what the freeze does to the risk: it converts it rather than removing it. Rationalization needs a live author who can re-narrate the target after seeing the diff. Freeze the directive and pre-commit the acceptance set, and you've amputated the live author from the conformance step. But the framing decision already happened upstream, and the freeze faithfully preserves whatever frame was in force at commit time. So the surviving risk doesn't shrink, it relocates to the one place the conformance gate structurally can't look, because that gate's job is to certify against the frozen thing, not to interrogate it.
The connecting move I want to name: the frame error you point at in this thread is the exact residue the instrument in your other comment goes after. Two independent Strategy frames of the same requirement, diffed on where their acceptance sets diverge, is a gate that reads the directive instead of reading against it. Your two comments are the two halves of one structure: this one names the unreached residue, the other names the mechanism that could reach it. A frame-diff would catch "locally perfect, wrong problem" for the same reason my BoardEngine/React oracle catches a wrong implementation: neither side is gold, and divergence is the whole signal.
Caveat so I'm not claiming credit I haven't earned: I have not built that frame gate. The differential mechanism in the harness today runs only at the correctness tier, two implementations checking each other, not at the spec tier. The closest thing I have upstream is NO-PROOF-NO-CLOSE: the proof each arc must satisfy is committed before the item can close, which locks whatever frame Strategy chose into place. But locking it in is not auditing it. That's just the sequential pipeline preserving the frame, not a gate interrogating it. Auditing the directive is still open, and a second independent frame on it is the cleanest shape I've seen proposed for closing that gap.
Locking the frame without auditing it is worth less than a frame-diff gate, and it's still not nothing, which is the part I'd add before it gets dismissed as just sequential-pipeline residue. A frozen directive that turns out to encode the wrong assumption gives you something the un-frozen version never would: a specific artifact to point at when the bug surfaces. Not "the model misunderstood somewhere in a conversation nobody can replay," but "this directive, committed at this timestamp, assumed X." That's provenance on the failure, even without a mechanism that catches it before ship.
Which means NO-PROOF-NO-CLOSE is doing real work on a different axis than frame-diff: it doesn't reduce the rate of wrong-problem bugs, it makes the ones that happen legible after the fact instead of requiring an archaeology dig through chat history. Cheap, already built, and worth keeping even once the frame-diff gate exists, because the diff gate reduces how often you need the provenance and the provenance is what you fall back on for whatever gets through anyway.