Anyone who has written code and then reviewed it minutes later knows the feeling: the diff looks fine. Of course it does. The person reading it is the same person who just talked themselves into every line of it. The naming made sense because the intent is still warm in memory. The shortcut felt justified because the reason for it is still sitting in working memory, uninspected.
That is the problem the cold-context critic is built to remove.
The core idea
In the harness I built to develop a game, every change passes through a review step performed by a fresh model instance that has no memory of writing the code. It did not plan the change. It did not argue for the approach. It did not feel the small relief of getting something to finally pass. It receives the diff and evaluates it on its merits, the way a stranger would.
The intuition is old and boring, which is part of why I trust it: an author rationalizes; a cold reviewer does not. A model — or a person — that just produced the work carries a bias toward the choices it already made. Strip the memory of making them, and the same mistakes stop being self-evidently correct. The reviewer has nothing to defend. It can only look at what is actually there.
I want to be precise about the claim, because it is easy to oversell. This is a discipline, an architecture choice. I am not going to tell you it caught a specific dramatic production bug, because I am not going to invent an incident to sell an idea. What I can say is the mechanism: cold review catches the class of mistakes an author talks itself into — the plausible-looking shortcut, the assumption never stated out loud, the edge case that "obviously" can't happen. Self-review is structurally bad at exactly those, because the author already accepted them once.
Where it fits
The cold critic is one layer, not the whole story. It sits alongside a few other pieces of the same distrust:
- Per-role model routing. Stronger models run the judgment stages — planning, critique, evaluation — and cheaper ones handle routine execution. The reviewer isn't grading its own homework, and it isn't the same weight class picked for speed.
- A differential oracle. Core game logic is implemented twice and held to shared invariants. When the two implementations disagree, at least one is wrong, and I find out without having to trust either.
- A human approval gate on every irreversible action. Deploys and commits do not happen because a model decided they should. They happen because a person approves them. That gate never moves.
I think about the whole thing on an autonomy ladder, L0 through L4 — from fully hands-on toward more independence. The cold critic is what makes climbing that ladder defensible. More autonomy is only safe if the verification underneath it does not depend on the thing being verified vouching for itself.
The setup, honestly
Concretely: the critic is a separate invocation with a clean context window. It does not inherit the conversation that produced the change. It gets the diff and the standard it is being held to, and it reports back before a human sees the change. That's the shape of it. The value isn't a clever prompt — it's the missing memory. The setup exists to guarantee the reviewer never accumulated a reason to like the code.
That's also why I keep the human gate immovable. The cold critic is a good reader, not an authority. It can flag; it doesn't get to ship.
An honest note
I'm about five to six months into this, career-changing from roughly six years in professional kitchens, no CS degree — I learned through freeCodeCamp and by building. The game is real and published to Google Play, but it is not serving millions, and I'm not running production RAG or fine-tuning pipelines. I'd rather tell you what this is than dress it up.
So the honest framing is this: I don't have the scars yet that teach senior engineers where the bodies are buried. What I can do is build the distrust in as a habit from the start — assume the author is biased, including when the author is me or a model I'm running, and make something with no stake in the answer do the checking.
If you want to see the same idea in code you can actually run, the differential oracle is public: github.com/egnaro9/evals-differential-oracle. It's the same move as the cold critic — distrust your own output — implemented as two independent versions of the logic checked against each other, plus invariant tests over thousands of random boards, with a deliberately buggy implementation included to prove the checks bite. Clone it, run pytest, watch it catch the planted bug.
More at egnaro9.github.io and github.com/egnaro9.
Top comments (0)