DEV Community

Emery Chen
Emery Chen

Posted on

If the Agent Agrees First, Discard the Review

If the Agent Agrees First, Discard the Review

A first-pass agreement is a failed agent review.
You heard your own premise read back to you.
Block the merge until a scored dissent log exists.

The claim I want you to hold

A coding agent often agrees because agreement is cheap.
Praise plus one tiny nit is not a review.
That pattern should stop the diff, not bless it.

This piece takes a hard line on purpose.
First-pass agreement is not evidence of design correctness.
Only objections you can falsify may move the patch.

You remain the reviewer of record on this diff.
The agent is a draft critic with a quota of doubt.
If it will not doubt, you discard the run.

Cheap runs invite lazy reviews

Free model access makes another chat feel free.
A free server option makes another run feel free.
Cheap runs are exactly where lazy reviews hide.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode matters here only as a host for those two options.
Spend free model access on a dissent pass, not on applause.

Use the free server option as a pinned, disposable review box.
Availability alone is not a quality claim here.
This article invents no token quota, model name, or hardware spec.

Pin the host and model your operator actually turned on.
Those facts change, and this gate does not need them.
If a dashboard shows a number, record that number yourself.

Do not cite a figure you did not read today.
A screenshot from last month is not a current fact.
Read the live terms, then write down only what you saw.

What a dissent pass must contain

You want objections that can still be wrong.
Each objection needs a file, a claim, and a falsifier.
Drop any item that misses one of those parts.

Require these four outputs before you read tone:

  • Write one sentence that restates the design premise.
  • Collect at least three objections that could still fail.
  • Attach one falsifier command or assertion to each objection.
  • Stop the run when the agent only agrees with you.

If the agent cannot attack the design, stop the thread.
Do not beg the same chat to try harder now.
Open a clean run with the same frozen contract.

Freeze the contract before you spend a token

You write the contract, and the agent does not edit it.
Store that contract beside the diff, under version control.
Paste the diff and the contract, not the answer key.

# dissent-contract.yaml
# Proposal you can run. Not a measured benchmark.
review_id: "REV-014"
premise: "Replace sync file reads with a bounded async pool."
forbidden:
  - "Do not praise the design."
  - "Do not restate the premise as an objection."
  - "Do not invent files you did not receive."
required_objections: 3
fields:
  - id
  - file
  - claim
  - falsifier
  - severity  # low | med | high
stop_if:
  - "objection_count < 3"
  - "any_claim_missing_file"
  - "any_falsifier_is_vague"
Enter fullscreen mode Exit fullscreen mode

The forbidden list is the point of the exercise.
You are paying for friction, not for a friendlier summary.
A review that cannot violate your idea is not a review.

Gate the log before you trust the prose

This script never calls a model at all.
It only accepts or rejects a dissent log on stdin.
Run the gate locally before you discuss any objection.

#!/usr/bin/env python3
"""Local dissent gate. A proposal until you execute it."""
import json
import sys

REQUIRED = {"id", "file", "claim", "falsifier", "severity"}
SEVERITY = {"low", "med", "high"}

def main() -> None:
    doc = json.load(sys.stdin)
    items = doc.get("objections")
    errors = []
    if not isinstance(items, list) or len(items) < 3:
        errors.append("need at least three objections")
        items = items if isinstance(items, list) else []
    premise = str(doc.get("premise", "")).strip().lower()
    seen = set()
    for item in items:
        if not isinstance(item, dict) or not REQUIRED.issubset(item):
            errors.append("missing required field")
            continue
        if item["severity"] not in SEVERITY:
            errors.append("bad severity")
        claim = str(item["claim"]).strip()
        falsifier = str(item["falsifier"]).strip()
        if len(claim) < 12 or len(falsifier) < 12:
            errors.append("claim or falsifier too thin")
        if claim.lower() == premise:
            errors.append("objection restates the premise")
        key = (item.get("file"), claim)
        if key in seen:
            errors.append("duplicate objection")
        seen.add(key)
    if errors:
        print("REJECT: " + "; ".join(errors))
        sys.exit(1)
    print("ACCEPT: dissent log has structure")

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Feed the gate a JSON log and read the exit code.

python3 dissent_gate.py < dissent-log.json
echo "gate_exit=$?"
Enter fullscreen mode Exit fullscreen mode

Exit zero means the log has the required shape.
That exit code does not mean any objection is true.
You still open each named file and try the falsifier.

Keep a failing fixture next to the gate

You should prove the gate rejects a mirror before you trust it.
Save a bad log that only restates the premise.
Expect a non-zero exit, and fail the setup if it passes.

{
  "premise": "Replace sync file reads with a bounded async pool.",
  "objections": [
    {
      "id": "O1",
      "file": "src/pool.py",
      "claim": "Replace sync file reads with a bounded async pool.",
      "falsifier": "No check supplied for this restatement.",
      "severity": "low"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

That fixture should die on count and on the mirrored claim.
A good log needs three distinct attacks, each with a real check.
Do not weaken the gate so a friendly log can sneak through.

Score every row yourself

Use this table after the gate accepts the log.
You fill the last column, and the agent does not.
A fluent sentence with no file is still a fail.

Check Pass looks like Fail looks like
File exists Path is in the diff Path was invented
Claim is specific Names a behavior Vague style worry
Falsifier runs Command or assertion Consider edge cases
Not a mirror Attacks the premise Repeats your idea
Severity fits High means loss or auth High means formatting

A low severity with a real falsifier can still block you.
A high severity with no file cannot block anyone.
Severity is only a hint you may override in writing.

Walk one diff without fake metrics

Consider a small change around local file reads.
The patch wraps sync reads in a bounded async pool.
A weak review says the pool looks cleaner, then stops.

Discard that review, because it agreed with the premise.
A usable objection says the pool hides backpressure from callers.
The falsifier is a test with more waiters than slots.

A second objection says cancellation leaks an open handle.
The falsifier is a timeout that must close the file.
If you cannot build that check, the objection stays open.

A third objection says a retry double-reads a partial file.
The falsifier is a crash between the read and the checksum.
If the diff has no checksum, the objection still stands.

These lines show a shape, not a measured lab result.
This article did not time them or score a live model.
Your repository has to supply the real file paths.

How you post the result

Do not paste the agent praise into the pull request.
Post the three objections, the falsifiers, and your own scores.
A teammate should see your verdict, not the model's mood.

Use a short comment shape, and keep it boring.

  1. State the premise in one frozen sentence at the top.
  2. List each objection id beside the file you opened.
  3. Mark each falsifier as reproduced, blocked, or dropped.
  4. End with merge, revise, or discard, and skip extra adjectives.

If every row is dropped, you still lack a real proof.
You have an empty critic, not a quiet approval.
Silence from the model is not a quiet yes.

Put the run in quarantine

Run the dissent pass on the free server option.
Keep that host pinned to a digest you already checked.
Do not review code on a host you cannot name.

Treat the server as a quarantine for the review transcript.
Do not let that host apply the patch to your tree.
Copy the dissent log out, then end the review run.

If the host digest is missing, spend no token yet.
An unpinned host is a separate failure from a soft review.
Fix the pin first, then ask the model for dissent.

Who should not use this gate

Skip this workflow when a human already owns the review.
A second person beats a forced list of objections.
Use the agent only when the human queue is the bottleneck.

Also skip the gate in the four cases below.

  • You need a formal security audit, and this gate is not one.
  • The change is a comment-only typo, so dissent is waste.
  • You cannot read the named files, so you cannot score them.
  • The diff requires secrets, so do not paste those secrets.

Skip the gate if policy forbids sending source to a hosted model.
A free server option does not erase your data rules.
Read the current terms before the first token leaves your laptop.

Limits you should keep in the README

The local gate checks structure, not the truth of claims.
A model can still emit three fluent and false objections.
Your scoring table is the only truth step, and it is manual.

Free access can change or vanish without any promise here.
Do not make a release depend on a permanent free tier.
Keep the contract and the gate in git either way.

The length check is a crude proxy for thin text.
A short and sharp claim can still be a valid objection.
Allow a written override, and record who granted that override.

This article does not report a live product benchmark.
Treat the YAML and the Python as a proposal you execute.
Save your own exit codes before you teach the gate.

Hold the line

Do not merge on first-pass agreement from a coding agent.
Demand a dissent log with files, claims, and falsifiers.
Score that log yourself, then decide with the diff open.

If you need a disposable host for this dissent pass, try MonkeyCode.
Its free model access and free server option fit this quarantine use.
Keep the gate in your repo and the verdict in your hands.

Top comments (0)