DEV Community

Quinn Zhu
Quinn Zhu

Posted on

Score Every Hunk Before Your First Agent PR

You should not open your first agent pull request blind. Score every changed hunk before you request human review. A fluent model reply is not evidence of safety.

You joined this repo as a junior during hour one. You still lack a map of risky paths. An agent can still draft a wide patch quickly.

This hour is for a gate, not for a demo. You will build a packet you can defend. You will refuse the pull request when a score fails.

Hold the boundary first

Keep the agent run off your real pull request branch. Bring back a diff, a test note, and a revert line. Leave secrets, tokens, and production configs out of the prompt.

Do not allow edits under CI workflows during this pass. Do not allow lockfile edits or migration files either. Those paths need a senior review, not a first hour guess.

Name the tool without inflating it

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode enters here as one place to run that first pass. The operator supplied two availability claims for this draft. Those claims are free model access and a free server option.

This draft does not name any specific model. This draft does not state a token quota. It also does not state hardware, duration, or uptime.

Check the project docs on the day you run the tool. If the docs disagree with this article, trust the docs. A review habit should not pretend to be a product spec.

Use the free server so your laptop clone stays untouched. Use free model access so a bounded retry stays cheap. Still copy the patch home and score it locally.

Shape the prompt before the call

Write the prompt as four short constraints, not a story. Name the ticket line, the allowed files, the forbidden paths, and the test command. Refuse any reply that skips one of those four.

Ticket: reject empty names in parseName.
Allowed: src/parseName.js src/parseName.test.js
Forbidden: .github migrations package-lock.json .env
Test: npm test -- src/parseName.test.js
Output: a patch only. Do not commit.
Enter fullscreen mode Exit fullscreen mode

Keep that prompt in the scratch notes beside the packet. If you change the prompt, change the notes too. A later reviewer should see the exact ask.

Add a local score script

Save the checker below before you call any model. This script is a proposal, not an executed benchmark. Adapt the blocked path list to your repository.

#!/usr/bin/env bash
# proposal: unexecuted junior gate for an agent diff
set -euo pipefail

BASE="${1:-origin/main}"
MAX_FILES="${MAX_FILES:-8}"
MAX_LINES="${MAX_LINES:-200}"

git rev-parse --is-inside-work-tree >/dev/null
FILES="$(git diff --name-only "${BASE}...HEAD" || true)"
COUNT="$(printf '%s\n' "$FILES" | sed '/^$/d' | wc -l | tr -d ' ')"
STAT="$(git diff --numstat "${BASE}...HEAD" || true)"
ADDED="$(printf '%s\n' "$STAT" | awk '{s+=$1} END {print s+0}')"
DELETED="$(printf '%s\n' "$STAT" | awk '{s+=$2} END {print s+0}')"
LINES=$((ADDED + DELETED))

echo "files=${COUNT} lines=${LINES}"

fail=0
if [ "$COUNT" -gt "$MAX_FILES" ]; then
  echo "FAIL too many files for hour one"
  fail=1
fi
if [ "$LINES" -gt "$MAX_LINES" ]; then
  echo "FAIL diff too large for hour one"
  fail=1
fi

BLOCK='(^|/)(\.github/workflows/|migrations/|.*lock$|\.env|credentials?)'
if printf '%s\n' "$FILES" | grep -E "$BLOCK"; then
  echo "FAIL blocked path present in diff"
  fail=1
fi

if [ ! -f REVIEW_PACKET.md ]; then
  echo "FAIL missing REVIEW_PACKET.md"
  fail=1
else
  grep -q '^Revert: ' REVIEW_PACKET.md || { echo "FAIL missing Revert line"; fail=1; }
  grep -q '^Test: ' REVIEW_PACKET.md || { echo "FAIL missing Test line"; fail=1; }
  grep -q '^Unknown hunks: 0$' REVIEW_PACKET.md || { echo "FAIL unknown hunks remain"; fail=1; }
fi

exit "$fail"
Enter fullscreen mode Exit fullscreen mode

Run the checker from the scratch branch after you pull the patch. Pass the base ref your team actually uses. Do not open a pull request when the exit code is nonzero.

chmod +x scripts/score-agent-diff.sh
MAX_FILES=8 MAX_LINES=200 ./scripts/score-agent-diff.sh origin/main
git diff --stat origin/main...HEAD
Enter fullscreen mode Exit fullscreen mode

Read the stat output even when the script passes. A rename can still hide a large rewrite. Your own eyes still have to close the gate.

Write the packet yourself

Create REVIEW_PACKET.md in the scratch branch before review. You write every line in that packet yourself. Do not accept an agent-written packet as your own note.

Ticket: ISSUE-123
Test: npm test -- workspace/foo.test.js
Revert: git revert <merge-sha>
Unknown hunks: 0
Risk: parser change only, no schema edit
Enter fullscreen mode Exit fullscreen mode

The test line must name a command you actually ran. If you have not run it, write Test not run and stop. A missing run is a failed gate, not a minor note.

The revert line must be a command you understand today. If revert would need a data backfill, stop the pull request. Hour one is the wrong time for that class of change.

Follow these eight steps

  1. Copy the ticket acceptance line into your prompt notes.
  2. Create a scratch branch that is not your pull request branch.
  3. Start the agent on the free server option, without secrets.
  4. Limit the prompt to files the ticket actually needs.
  5. Pull the patch back and run the score script locally.
  6. Run the test command and record the real result.
  7. Set unknown hunks to zero only after you explain each hunk.
  8. Open the pull request only when the script exits zero.

Stop the run if the agent edits a file you did not name. Stop again if the summary mentions a file absent from the diff. Those mismatches mean the packet is not ready.

Use this decision table

Do not bargain with a failed row during hour one. Split the work now or discard the scratch branch. Ask a senior when the row is outside your map.

Signal Score Your move
Eight files or fewer Pass Continue to the test line
More than 200 changed lines Fail Split the patch or discard it
Blocked path in the diff Fail Reset the scratch branch
Test command not run by you Fail Run it, or do not open the PR
Unknown hunk count above zero Fail Read again or drop that hunk
Revert line you cannot explain Fail Write a real revert or stop
New dependency or workflow edit Fail Hand the ticket to a senior

A passing size score is not a passing meaning score. You still need to read each hunk in order. Opaque code stays a fail until you can teach it.

Walk one sample hunk

Imagine the ticket says the parser should reject an empty name. The agent changes parseName and also reformats the README file. You score the parser hunk as related to the ticket.

You score the README hunk as unknown, then you drop it. That parser hunk matches the ticket, so it can stay. You still run the named test before you open any pull request.

- if (!name) return null;
+ if (!name || !name.trim()) throw new Error("empty name");
Enter fullscreen mode Exit fullscreen mode

Treat this diff as an illustration, not as a result from your repo. Replace the function names with the files your ticket names. Keep the same pass or fail questions on every hunk.

Read hunks in a fixed order

Open the diff one file at a time. Ask what user-visible behavior changed for this ticket. If the answer needs a guess, mark that hunk unknown.

Watch for formatter noise spread across unrelated files. Watch for a new helper that the ticket never requested. Watch for a comment that hides a behavior change.

Compare the agent summary with the name-only diff output. Every file in the summary should appear in that list. Every file in the list should appear in your notes.

Keep the free options in their lane

The free server is only for the noisy first pass. Keep private fixtures and credentials off that machine. Copy a patch back, not a live environment.

Free model access can make a second prompt cheap. Change the constraint before you retry the same ask. A retry without a new limit is usually noise.

This article states no token total and no expiry date. Those facts were not supplied as verified numbers. Read the current docs before you plan a long session.

If your team bans external coding agents, stop here. Do not treat a free tier as a policy workaround. Ask your lead before you send the first prompt.

Know the checker limits

The block regex is a starter, not a complete risk map. Add your real danger paths before you trust a pass. A green script can still miss a risky file name.

Numstat sums ignore binary files and some renames. Run git diff stat when those numbers look too small. Trust the strange stat over the cheerful summary.

A free server may lack your private test fixtures. A pass on that server can still fail in CI. Prefer the pinned CI command when you can run it.

Skip this flow in these cases

Skip the flow when the ticket needs production data. Skip it when you cannot read the language yet. Skip it when the change is a release or a migration.

Skip it when you need a guaranteed quota or support contract. This draft does not establish either of those promises. Use an approved internal tool when the ticket is urgent.

Skip it when the diff is already on a shared branch. Hour one should not repair a public mistake with another agent pass. Revert first, then review the result with a human.

Leave with a defendable pull request

Your first pull request should stay small enough to explain aloud. The score script is the defense you can show. The model only drafted the patch you scored.

If you use MonkeyCode for that first pass, read the current notes. Confirm the free-access terms on the day you actually run it. Then run the local gate before you ask anyone for review.

Top comments (0)