AI Needs Peer Review Too

The smartest workflow isn't built around one AI. It's built around distributed responsibility.

Every week brings another comparison. GPT versus Claude. Claude versus Gemini. GPT versus the local models. The benchmark moves and the winner moves, and I've stopped believing the winner is the point.

"Which AI model is best?"

I asked it for longer than I'd like to admit, always reaching for whichever model wrote the best code, the best prose, or held the largest context window. Eventually I saw that the useful question has nothing to do with who wins a benchmark.

We don't design infrastructure by asking "what's the best switch?" We ask about the architecture, the failure domains, how changes are validated before they reach production, and whether a stranger could maintain the thing a year from now.

AI deserves the same questions. The one that matters: what workflow produces the most trustworthy result?

Figure 1 is deliberately simplified: four core steps, with human approval and release left out. The full workflow appears later in the article.

The gate I'm tempted to weaken

I run agents under a governance gate: they're set up to stop before staging or merging until I explicitly authorize the run. I've also felt the pull that makes gates like this fail. Every change made for speed is a small negotiation between safety and momentum, and the most dangerous authorization is the blanket one you give yourself to move faster. The gate keeps me honest on the days my attention doesn't.

I've come to think that gate matters more than any model I could put behind it. The agent is the last thing that would catch its own overreach, because it's the actor, and the actor is exactly what needs checking. Making it smarter doesn't help; a smarter agent just moves faster in whatever direction it's already pointed, mistakes included. What catches that is a second thing, standing outside the agent, with no stake in the work it checks.

That's when it clicked. I wasn't looking for a smarter AI.

I was unconsciously recreating peer review.

This isn't really an engineering problem

I came to this through infrastructure, but the more I sat with it, the less it seemed to be about code.

Think about the last time you used AI for something that actually mattered: a contract clause, a financial model, a medical question, a strategy memo, a piece of writing going out under your name. If you're like many people, at some point you quietly pasted the same question into a second model. The first answer wasn't necessarily bad. You wanted another set of eyes before you trusted it. (For medical or legal questions, a second model is still no substitute for a licensed professional.)

That instinct is correct, and it's old. Newsrooms have editors. Science has peer review. Adversarial courts have opposing counsel. Accounting has audits. Commercial aviation puts two pilots in the cockpit, trained to cross-check and challenge each other. None of these fields decided to hire one impossibly brilliant person and trust them unchecked. They decided that independent review beats individual brilliance for anything that matters.

AI hands each of us the raw talent of a very fast, very well-read colleague. What most of us are missing is the org chart around that colleague.

Engineering already turned this into a system

Every discipline has some version of review. Software engineering is the one that automated it. A serious engineering organization doesn't ship anything that matters on one person's say-so. It runs the work through peer review, architecture review, continuous integration, deterministic validation, governance, and a clean separation between who builds a thing and who approves it.

None of that scaffolding exists because engineers distrust each other. It exists because decades of expensive failures taught a lesson every mature field eventually learns the hard way: an organization that distributes responsibility tends to outperform even its most brilliant individual. The process isn't a tax on the genius. It's what makes the genius safe to depend on.

That raises an uncomfortable question. We wrap our best human experts in all of that review precisely because raw talent isn't trustworthy at scale. AI is fast, often more confident than its accuracy warrants, and can invent details without any signal that it has.

So why would it be the first thing we stop reviewing?

The five things every good team already has

Strip away the tools and the strengths of a high-performing team come down to five principles. You can recreate all of them alone.

• Specialization. The person who frames the problem is rarely the best person to nitpick the implementation. Different tasks need different strengths: different AI models, or the same model given a different role in a separate session.
• Independent review. The reviewer must not be the author. By author I mean the context that produced the work (the conversation, its history, its assumptions), not the brand of model. A conversation reviewing its own output is a student grading their own exam.
• Deterministic validation. Some checks shouldn't be opinions at all. A test suite passes or fails. A link checker finds the broken URL. A schema validator rejects the malformed record. Numbers reconcile against the authoritative source or they don't. Wherever a check has a right answer, give it to a machine; judgment calls, like whether an argument holds up, belong to review.
• Governance. The small set of rules that keeps a thing maintainable as it grows: what gets checked, what needs approval, what's allowed to happen automatically.
• Separation of responsibilities. No single actor should be able to propose, approve, and execute the same consequential change. That includes access. A reviewer gets only what it needs to review, typically read access to the work, and release authority stays outside its control. It's the same reason many businesses require two approvals before a large wire goes out.

Not one of these mentions a vendor. That's on purpose. The tools will change three times before this article is old. The principles won't.

Figure 2 compares the same idea from the organization's side: five things a team provides, four of which a solo operator lacks. The five principles above are how to rebuild them alone. The right-hand column is a target, not a measurement.

"But isn't this slow and expensive?"

Yes, and that's the honest tradeoff, so let me name it.

Running a separate review session and a validation gate costs more time and money than firing one prompt and shipping the answer. Do this for everything and you'll drown in ceremony and quit by Thursday.

So match the rigor to the stakes:

• Throwaway or low-stakes: a quick script, a first draft, a "help me think out loud." One model. Ship it. Extra ceremony here is just procrastination.
• Consequential or hard to reverse: anything going out under your name, touching money, touching production, or that someone else will rely on. This is where independent review earns its cost. The one bug, wrong number, or bad clause it catches is cheaper than the cleanup.

The goal is the least governance that still catches the expensive mistakes, and knowing which bucket you're in before you start.

Your AI should look like an org chart

Once you stop shopping for one model and start designing a workflow, the shape falls out naturally. A single-model workflow is just requirements → one AI → result. It's fast and fine for low stakes, but nobody in that picture has the job of disagreeing.

A team-shaped workflow adds the missing roles: requirements → architecture/framing → implementation/draft → independent review → deterministic validation → human approval.

An engineer sees a CI pipeline. A writer sees outline → draft → editor → fact-check → publish. Someone running a small business sees proposal → second opinion → check the numbers → sign. Same skeleton. It's a control plane for work, and code review is only one instance of it.

Independence comes in degrees, and it helps to be honest about which one you have:

• The same conversation reviewing its own output is weak separation. It shares every assumption that produced the work.
• The same model in a fresh, isolated context gives you contextual separation. It sees the result without the reasoning that led to it.
• A different model in a separate context can be stronger, but it isn't guaranteed to be: models trained on similar data can share blind spots.
• Human approval and deterministic validation are additional controls of a different kind. The strongest setups add them on top of model review rather than using them instead of it.

The subtle part about that review step is that a second reviewer earns its keep through disagreement more than agreement. When two independent reviewers reach the same conclusion, your confidence is better grounded, though still not guaranteed. When they clash, they drag a hidden assumption into the light: the one the author never questioned because it was the one making it. That's what a good architecture review does to a confident design. The point was never consensus; it was the productive argument on the way there.

Here's a real example from producing this essay. I wrote a small tool to send it to DEV.to as an unpublished draft, then gave the code to a fresh session of the same model, one that had never seen it being written. That reviewer found that if the create request timed out after the server had already saved the article, a rerun could post a second copy, and none of my existing tests covered that case. The fix is now pinned by a regression test.

Somewhere along the way, this stopped feeling like an AI workflow. I wasn't prompting a chatbot anymore. I was running a small engineering organization in which some of the team members happened to be AI, and once I saw it that way, the architecture almost designed itself: specialists for distinct jobs, something that orchestrates the handoffs between them, independent review with no stake in the work, and validation that doesn't care whose feelings it hurts. You don't have to impose that structure. You start reaching for it as soon as you treat the system around the intelligence as the product.

One honest caveat. You'll see people (me included) assign specific models to specific roles: "this one for architecture, that one for drafting, a local model for the rote pass." Treat those as how someone happens to wire it today. My wiring will be obsolete by the time you read this. The durable part is the shape: keep the reviewer's context separate from the author's, and put a machine on the checks that have a right answer.

Steal this tonight

You don't need a homelab, a harness, or five repositories. Here's the smallest version, and it works for code, writing, analysis, almost anything:

1. Draft in one context. Use whatever model you already use. Nothing changes here.
2. Review in a separate one. Paste only the output into a fresh session, ideally with a different model, and ask it to argue: "find the flaws, the missing cases, the wrong assumptions." Leave out the drafting conversation; the reviewer should see what a reader sees.
3. Add one deterministic check. For code, run the test suite. For writing, run a link checker. For structured data, validate it against its schema. For numbers, reconcile them against the authoritative source. One check with a right answer, every time.

That's the whole idea at its smallest: a draft, an independent critic, and one thing that can't be argued with. Everything else in this article is that pattern with more discipline added as the stakes climb.

It was never about AI

Step back far enough and this stops being a story about AI. It's a story about organizational design. For decades we've accepted peer review, audits, and deterministic testing as simply how serious work gets done, because we learned, expensively, that intelligence scales better when responsibility is distributed: one person builds the thing, and someone else gets to ask "are you sure?"

I wasn't looking for a smarter AI. I was unconsciously recreating peer review.

Five years from now the models in this article will look quaint, and today's arguments about which one is "clearly the best" will read like old arguments about search engines. The organization around the model is what lasts. Quality was never a property of the smartest component; it comes from arranging the pieces so they catch what any one of them would miss alone.

You already trust editors, auditors, second opinions, and second pilots, because good systems are what make good work dependable. Extend the same courtesy to the fast, confident, occasionally very wrong colleague you've started letting into your most important work.

Give it a reviewer.

---

Disclosure: the argument and the experiences in it are mine. AI tools helped with drafting and with several rounds of independent review, and I edited and checked the final text myself. The diagrams were produced with AI assistance.