I built a multi-agent loop where an adversarial Claude reviewer reads your actual codebase before approving plans

Large language models are surprisingly optimistic reviewers.

Ask an LLM to review an implementation plan and it will often approve things that are objectively wrong:

• Non-existent file paths
• Incorrect function signatures
• Missing edge cases
• Broken assumptions about the codebase
• Incomplete testing strategies

The problem is simple: the model is reasoning from its training data and the conversation context, not from your actual repository.

I wanted something different.

I wanted a reviewer whose default assumption is that the plan is wrong, and whose job is to prove it.

So I built agent-plan-review-loop, an open-source multi-agent orchestration system that repeatedly challenges implementation plans until they survive adversarial review.

The Core Idea

Most AI review workflows look like this:

1. Author generates a plan
2. Reviewer checks the plan
3. Reviewer approves the plan

The problem is that both agents often share the same context and reasoning chain.

My approach intentionally breaks that connection.

Every artifact is stored as a markdown file inside the repository:

• Plan
• Review
• Questions
• Decisions
• Diffs

Each agent runs as a completely fresh process using Claude Code CLI.

The reviewer has no access to the author's reasoning.

It only sees:

• The implementation plan
• The actual repository
• Its instructions

This forces the reviewer to evaluate the plan on its own merits rather than continuing the author's thought process.

In practice, this catches a surprising number of mistakes.

The Workflow

The system runs an Author → Reviewer loop until approval.

Task
 ↓
Classifier
 ↓
Author
 ↓
Reviewer
 ↓
CHANGES_REQUESTED?
 ↓
Yes → Author revises
 ↓
No
 ↓
APPROVED
 ↓
Coder implements

The reviewer is intentionally adversarial.

Its primary instruction is:

You are a SKEPTICAL senior REVIEWER.

Find why this plan will FAIL.
Do not praise it.

Default to CHANGES_REQUESTED;
approve only if genuinely sound.

Instead of asking "what's good about this plan?", the reviewer asks:

• Which assumptions are wrong?
• Which files don't actually exist?
• Which edge cases were missed?
• Which APIs are being used incorrectly?
• Which tests are missing?

The result is far more useful feedback than generic AI approval.

Complexity-Aware Model Routing

One challenge with agent systems is cost.

Running the most expensive model for every task quickly becomes impractical.

To solve that, I added a lightweight classification step using Haiku.

Each task is categorized before planning begins:

Tier	Task Type	Author	Reviewer	Max Iterations
T0	Text, Config, CSS	Sonnet	Opus	3
T1	Small Feature	Sonnet	Opus	3
T2	Complex Refactor	Opus	Sonnet	6

This allows the system to reserve expensive reasoning for genuinely difficult work.

Most routine tasks never need a full Opus planning cycle.

Handling Decisions AI Can't Make

One thing I strongly wanted to avoid was AI guessing business requirements.

When the Author encounters a genuine product decision, it stops and asks.

For example:

STATUS: NEEDS_ANSWERS

Q1: Should exports include archived items?
A:

Q2: CSV, XLSX, or both?
A:

The workflow exits.

A human answers the questions.

The process resumes exactly where it left off.

This prevents the system from inventing requirements simply to keep moving forward.

Isolated Implementation

Once a plan is approved, a separate Coder agent performs the implementation.

The implementation never touches the developer's active working tree.

Instead, it creates an isolated Git worktree:

REPO="$PWD" bash code-run.sh TASK-42

The result is:

• Dedicated branch
• Isolated workspace
• Generated diff
• Human review before merge

This makes experimentation much safer than allowing AI to modify a live working directory.

Pluggable Deployment

Deployment is intentionally simple.

Users define their own validation and deployment commands:

GATE_CMD='bash laravel-gate.sh' \
DEPLOY_CMD='bash ship.sh' \
bash deploy-run.sh TASK-42

If validation fails, the merge is automatically rolled back.

The framework doesn't assume anything about your stack.

The Unexpected Discovery

The most important architectural decision turned out to be the simplest one:

Files are a better memory system than conversations.

When the reviewer starts from scratch and reads markdown artifacts instead of inheriting conversation history, it becomes dramatically more critical.

It behaves less like a second opinion from the same person and more like an independent engineer joining the review process for the first time.

That independence is exactly what makes the feedback valuable.

Mobile Workflow

I also added an optional Telegram bot.

It allows:

• Answering review questions
• Providing steering notes
• Managing multiple tickets
• Monitoring progress remotely

This turned out to be surprisingly useful when away from a laptop.

Example Usage

Generate and review a plan:

REPO="$PWD" bash plan-loop.sh \
  TASK-1 \
  "add CSV export to reports page"

Implement an approved plan:

REPO="$PWD" bash code-run.sh TASK-1

Open Source

GitHub:

https://github.com/execute25/agent-plan-review-loop

The project requires:

• Claude Code CLI
• Bash
• Git

I'm particularly interested in feedback from people building AI coding agents, autonomous development workflows, or review systems.

What would you change in this architecture?

Comments

[Max Quimby]

The "reviewer never sees the author's reasoning" decision is the load-bearing one here — same reason a fresh pair of human eyes catches what the author's brain autocompletes past. Shared context is how you get two agents confidently agreeing on a hallucinated file path.

Two things from running similar adversarial loops. First, a reviewer hard-instructed to "default to CHANGES_REQUESTED" skews toward false positives — it'll invent objections to justify its mandate, and you can land in a revise→reject loop that never converges. A termination guard ("if the only objections left are stylistic, approve"; hard round cap) saved us a lot of wasted tokens. Second, single-reviewer variance is real — running three cheap reviewers and taking majority-approve was more stable than one expensive skeptic, and it surfaces when an objection is idiosyncratic vs. real.

The Haiku classifier for routing is a nice touch, and grounding the reviewer in the actual repo is what separates this from generic "looks good to me" review. How do you handle author↔reviewer deadlock — hard cap, escalate to a bigger model, or kick it to a human?

[Sejin Kim]

Yeah, that CHANGES_REQUESTED default was the single hardest line to tune.

The reviewer prompt already has a partial guard: a "CALIBRATE STRICTNESS TO SCOPE" block that reserves [BLOCKING] for actual defects on trivial plans and pushes process-completeness feedback (e.g. "QA checklist not exhaustive for a one-line copy change") down to [nit]. That helps a lot on T0.

But you're right—it doesn't actually break the loop, it just softens the severity. On T2, the reviewer still manages to find new "concerns" round after round. "If only stylistic objections remain, approve" is a much cleaner termination rule. I'm adding it.

On variance, I made a different bet.

The loop supports cross-family decorrelation via REVIEWER_PROVIDER=cursor: Anthropic authors the plan, Cursor (Composer / GPT) reviews it. Same intuition as majority voting, just along a different axis.

Three cheap Sonnets probably outperform against idiosyncratic reviewer noise. Cross-family review is aimed at a different failure mode: both models were trained on similar data, so both miss the same thing.

I haven't actually A/B tested the two approaches yet, so I can't honestly say which matters more in practice. I probably should.

For deadlock, the philosophy is less "cap and stop" and more "cap and hand the human a lever."

When the loop reaches MAX_ITERS without approval, it stays alive in a stalled state and the Telegram bot offers four actions:

• +3 rounds
• +6 rounds
• Accept as-is (writes Manual override — VERDICT: APPROVED into the review file so the audit trail reflects that a human made the final call)
• Drop

Before extending the loop, you can also send a free-text steering note or even a screenshot. The author incorporates that into the next revision, so the human acts as a tiebreaker with additional context rather than just a rubber stamp.

There's also a separate guard for the genuinely stuck case: if the plan file's hash doesn't change after a revision but the verdict is still CHANGES_REQUESTED, the loop exits immediately. No progress plus another rejection means it's actually stuck, so there's no point burning more iterations.

I also deliberately don't auto-escalate to a larger model. "Throw Opus at it" usually papers over the real issue, which is that the plan or the acceptance criteria are underspecified. Most deadlocks end up being specification problems, not intelligence problems.

Out of curiosity, how do you handle a 2–1 split on your side? Does majority approval win immediately, or does the dissenting reviewer get a second pass?

[Alex Shev]

The adversarial reviewer pattern is useful because it changes the review from vibes to evidence. If the reviewer has to inspect real files, function names, and tests before approving a plan, it catches a lot of the optimistic hallucination that normal planning misses.

[Sejin Kim]

"Vibes to evidence" is exactly the right framing — I might steal that phrase.

The worst hallucinations usually aren't the obvious ones. They're the small assumptions that sound perfectly reasonable: referencing a function that doesn't exist yet, assuming a service is injected when it's actually created inline, things like that.

If you're just reviewing whether the plan sounds right, it's easy to miss those. But if you have to grep the codebase first, you quickly find out what's actually there instead of what the model assumed.

That's the difference between reviewing based on vibes and reviewing based on evidence.

[Alex Shev]

Exactly. The dangerous failures are the plausible ones. A reviewer that must grep first changes the job from "does this sound reasonable?" to "which local fact supports this plan?" I would even make the evidence visible in the approval artifact: function names, files checked, and assumptions that remained unverified.