Scoring Review Surface in a Ninety-Minute Spike

A mid-size payments team merged an AI-written retry helper into a brownfield service on a Thursday afternoon. The demo looked complete, the unit tests were green, and the author had already moved toward the next ticket. By Monday the review thread held fourteen comments about timeout budgets, log redaction, and a swallowed cancel. Generation had been cheap that afternoon, yet the review surface had quietly become the actual delivery cost.

Cheap generation often resembles buying fabric by the bolt while still paying a tailor by the hour. The cloth arrives quickly, folded, and apparently finished, but every unmatched seam still belongs to a human reviewer. Teams that count merged lines as progress then discover that review comments, not tokens, set the calendar. A time-boxed spike can make that trade visible before the next helper lands on the default branch.

The protocol below is a proposed ninety-minute spike, not a completed trial with published scores. It holds one hypothesis, one frozen task, and one kill rule that a team can execute without debating taste. The hypothesis is simple and falsifiable within the window: an assistant-authored patch for a bounded change will exit with a review-surface score no worse than a human patch started from the same card. If the score misses the threshold, the team kills the workflow for that class of change rather than bargaining with the demo.

A free model session on a throwaway server keeps billing setup from leaking into the evidence. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is relevant here only as an evaluation host that offers free model access and a free server option, so the clock can start on the task instead of on account provisioning. The product is not the subject of the spike, and no model names, quotas, or hardware claims are required for the method to stand.

The engineer begins by writing the hypothesis into a log that will outlive the server. The card must name the change, the files that may move, and the kill number before any assistant prompt is typed. A concrete starting log looks like the following block, which should be committed to a throwaway branch so later arguments cannot rewrite the bet.

# spike-log-2026-09-03.txt
hypothesis: assistant patch for bounded retry helper has review_surface <= 8
task: add retry with jitter around existing HTTP client in pkg/upstream
allowed_paths: pkg/upstream/*.go, pkg/upstream/*_test.go
forbidden: new dependencies, config format changes, public API rename
kill_if: score > 8 OR tests missing OR diff touches files outside allowed_paths
clock: 90 minutes, no extension after first failing check

A disposable workspace then isolates the experiment from the laptop that still holds production credentials. The commands below assume a fresh Unix shell on a free server and a shallow clone of an internal service. They are labeled as a proposed sequence, and a team should substitute its own repository URL and module path before running anything against real code.

# proposed evaluation host, not a production jump box
mkdir -p /tmp/review-spike && cd /tmp/review-spike
git clone --depth 1 [e-mail]/payments-upstream.git repo
cd repo
git switch -c spike/review-surface-retry

# freeze the pre-spike snapshot for later scoring
git rev-parse HEAD > /tmp/spike-base.sha
find pkg/upstream -type f | sort > /tmp/allowed-paths.txt

The frozen task should be ugly enough to generate review comments if the assistant cuts corners, yet small enough to finish inside the window. One workable card asks for retry with jitter around an existing HTTP client, a test that fails on exhausted attempts, and logs that never print secrets. The engineer pastes that card once, refuses follow-up prompt shopping, and spends the remaining minutes on tests and the scorer rather than on conversational cleanup.

After the assistant returns a patch, the engineer applies it only on the spike branch and refuses extra files that wander outside the allow list. The next commands capture the diff in a form the scorer can read without a browser. Keeping the artifact on disk matters, because memory of a green demo is a poor substitute for the patch that reviewers will actually see.

git add pkg/upstream
git diff --cached -- pkg/upstream > /tmp/spike.patch
git diff --cached --stat > /tmp/spike.stat
git diff --cached --name-only > /tmp/spike.files

# proposed guard: kill early if the patch escaped the card
if grep -vxF -f /tmp/allowed-paths.txt /tmp/spike.files; then
  echo "KILL: path escape" >> /tmp/spike-log-2026-09-03.txt
fi

Review surface is treated as a countable object rather than a feeling about code quality. The proposed Python scorer below walks a unified diff and adds points for missing tests, bare exception handlers, TODO markers, secret-like literals, and files that never appear in the allow list. Teams may change the weights, but they should change them before the clock starts so the spike cannot negotiate with its own rubric.

#!/usr/bin/env python3
"""Proposed review-surface scorer for a time-boxed AI patch spike."""
from pathlib import Path
import re
import sys

WEIGHTS = {
    "file": 1,
    "hunk": 1,
    "todo": 2,
    "bare_except": 3,
    "no_test_file": 4,
    "secret_literal": 5,
    "path_escape": 8,
}

SECRET = re.compile(r"(api[_-]?key|password|secret)\s*[:=]\s*['\"][^'\"]+", re.I)
BARE = re.compile(r"except\s*:\s*($|#)|catch\s*\(\s*Exception", re.I)
TODO = re.compile(r"\b(TODO|FIXME|HACK)\b")

def score(patch: str, allowed: set[str]) -> dict:
    files, hunks, findings = set(), 0, []
    current = None
    for line in patch.splitlines():
        if line.startswith("+++ b/"):
            current = line[6:]
            files.add(current)
            if current not in allowed:
                findings.append(("path_escape", current))
        elif line.startswith("@@"):
            hunks += 1
        elif current and line.startswith("+"):
            body = line[1:]
            if TODO.search(body):
                findings.append(("todo", current))
            if BARE.search(body):
                findings.append(("bare_except", current))
            if SECRET.search(body):
                findings.append(("secret_literal", current))
    test_hit = any("test" in Path(f).name.lower() for f in files)
    if files and not test_hit:
        findings.append(("no_test_file", ",".join(sorted(files))))
    total = WEIGHTS["file"] * len(files) + WEIGHTS["hunk"] * hunks
    for kind, _ in findings:
        total += WEIGHTS[kind]
    return {"total": total, "files": sorted(files), "hunks": hunks, "findings": findings}

if __name__ == "__main__":
    patch = Path(sys.argv[1]).read_text()
    allowed = set(Path(sys.argv[2]).read_text().splitlines())
    result = score(patch, allowed)
    print(result)
    Path("/tmp/spike-score.txt").write_text(str(result["total"]))

Running the scorer should take seconds, which preserves the remainder of the ninety minutes for a human skim that the script cannot replace. The engineer records the numeric total beside the hypothesis and then walks the findings list once, adding any issue the weights missed, such as a retry loop that ignores cancellation. That extra pass is still part of the spike, because a scorer that never sees control-flow debt will bless a patch that reviewers will reject.

python3 /tmp/score_review_surface.py /tmp/spike.patch /tmp/allowed-paths.txt
SCORE=$(cat /tmp/spike-score.txt)
echo "score=${SCORE}" >> /tmp/spike-log-2026-09-03.txt

# proposed kill rule, written before the assistant ran
if [ "$SCORE" -gt 8 ]; then
  echo "KILL: review surface exceeded threshold" >> /tmp/spike-log-2026-09-03.txt
  git merge --abort 2>/dev/null || true
  git switch -
  git branch -D spike/review-surface-retry
else
  echo "SHIP-CANDIDATE: keep branch for ordinary review" >> /tmp/spike-log-2026-09-03.txt
fi

Ship in this protocol never means skip review. It means the workflow earned another ordinary pull request, not that the assistant became an owner of the module. Kill means the team stops using that assistant path for similar cards until a later spike with a tighter task or a different host beats the same number. The log is the only souvenir worth copying off the free server, because the branch itself was always disposable evidence.

The method has sharp limits that marketing copy would rather omit. The weights are local policy, not a universal quality function, and they will miss design errors that never appear as literals in a diff. A ninety-minute window cannot speak to load, failure injection, or multi-service contracts, and a free server may vanish before anyone reruns the patch. Teams that treat the score as a security audit, a licensing review, or a substitute for an on-call owner are using the wrong instrument.

Engineers in the middle of an incident should not start this spike, and neither should groups that lack authority to kill a workflow after a bad number. The protocol also fails for open-ended architecture work, where the interesting debt lives in diagrams rather than in hunk counts. In those cases the cheaper honesty is to refuse the assistant until the card is small enough that a kill rule can be written in one paragraph.

Readers who already isolate evaluation work on a throwaway host can run the same scoring loop with MonkeyCode's free model access and free server option, then keep or kill the workflow from the log rather than from a demo.