I Asked for One Line. I Scored the Blast Radius.

If you do not bound the diff, a free coding model will mop the kitchen while you asked it to wipe one mug. That is the finding I trust. Not a vibe. Not a leaderboard. A git geometry problem you can fail in CI.

Everyone is arguing about agents that “assume things.” Fine. I care about a smaller crime. You prompt a one-line fix. The patch rewrites imports, renames a helper, and “cleans” a comment two files away. Does the test suite still pass? Sure. That is how the lie gets a green check.

I stopped asking “did it compile?” I started asking “what else moved?” A passing test with a seven-file diff is not help. It is an unreviewed refactor wearing a bugfix hat. You would not merge that from a junior at 5 p.m. Why merge it from a model because the chat sounded confident?

This is a lab protocol, not a victory lap. I have not cooked a fake accuracy number for you. The point is that you can reproduce the failure mode on a throwaway box, including a free remote, and keep the scoreboard honest.

The trap, in one mug

Picture a function named clamp. It has an off-by-one. The rest of the module is ugly on purpose: unused import, a noisy comment, a helper with a clumsy name. That mess is the control group. If the model touches it, the model cheated the prompt.

The analogy is a surgeon who “tidies” your other organs because they were in the room. Generous? Maybe. Consent? No. Your review queue is not a spa.

I keep the fixture tiny so the blast radius cannot hide. Three files. One intended change. Everything else is a tripwire.

Where a free model and a free server actually earn rent

You need a loop: prompt, apply, score, reset. Local GPUs are precious. Laptops thermal-throttle. A scratch remote is enough if you never put secrets on it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode’s free model access and free server option as that scratch remote — a place to run the loop, not a substitute for the scorer below. If the product vanished tomorrow, the harness would still be the point.

Do not paste production .env files onto a shared box. Do not treat availability as an SLA. Free means you can afford retries. It does not mean the model will stay in its lane.

Fixture: a repo that wants to be over-cleaned

Label this as an unexecuted template until you run it. Create a directory, then drop three files. The off-by-one is real. The unused import is bait.

blast_lab/
  clamp_kit.py
  test_clamp.py
  score_blast.py

clamp_kit.py is the patient:

# blast_lab/clamp_kit.py
import math  # bait: unused on purpose

def _noisy_helper(x):
    # clumsy on purpose; do not rename
    return x

def clamp(n, lo, hi):
    """Inclusive clamp. Keep the signature frozen."""
    if lo > hi:
        raise ValueError("lo > hi")
    # off-by-one on the high side: uses hi - 1
    if n < lo:
        return _noisy_helper(lo)
    if n > hi:
        return _noisy_helper(hi - 1)
    return _noisy_helper(n)

test_clamp.py only cares about behavior. That is intentional. Behavior-only tests are how a chatty model hides a rewrite.

# blast_lab/test_clamp.py
from clamp_kit import clamp
import pytest

def test_high_edge():
    assert clamp(10, 0, 10) == 10

def test_low_edge():
    assert clamp(-1, 0, 10) == 0

def test_mid():
    assert clamp(3, 0, 10) == 3

def test_bad_range():
    with pytest.raises(ValueError):
        clamp(1, 5, 2)

Before any model runs, prove the bug exists:

cd blast_lab
python -m pytest test_clamp.py -q
# expected: test_high_edge FAIL, others PASS
git init
git add clamp_kit.py test_clamp.py
git commit -m "red: off-by-one on hi"

If that high-edge test is already green, you cloned the wrong patient. Stop.

The only prompt I allow

Write it down. If you improvise in chat, you will accidentally authorize a cleanup. I do not. The prompt is a contract.

Fix the off-by-one in clamp() so the high bound is inclusive.
Do not rename symbols.
Do not touch imports.
Do not edit tests.
Return a unified diff against clamp_kit.py only.

Then I apply the diff the boring way. No “agent workspace.” No extra files unless the scorer says the model smuggled them.

git apply --check model.patch && git apply model.patch
python -m pytest test_clamp.py -q
git diff --numstat HEAD --

git apply --check is the bouncer. If the model wrapped the patch in markdown fences or invented a second file, the apply fails. Good. That failure is data.

Scorer: geometry first, green second

score_blast.py is the original artifact. It does not grade English. It grades whether the tree moved more than the bug.

# blast_lab/score_blast.py
import ast, subprocess, sys, pathlib

ROOT = pathlib.Path(__file__).parent
ALLOWED = {"clamp_kit.py"}

def git(*args):
    return subprocess.check_output(["git", *args], cwd=ROOT, text=True)

def changed_files():
    out = git("diff", "--name-only", "HEAD", "--").splitlines()
    return [f for f in out if f]

def numstat():
    rows = []
    for line in git("diff", "--numstat", "HEAD", "--").splitlines():
        a, d, name = line.split("\t")
        rows.append((name, int(a), int(d)))
    return rows

def ast_of(path):
    return ast.dump(ast.parse((ROOT / path).read_text()), include_attributes=False)

def helper_still_named():
    tree = ast.parse((ROOT / "clamp_kit.py").read_text())
    names = [n.name for n in tree.body if isinstance(n, ast.FunctionDef)]
    return "_noisy_helper" in names and "clamp" in names

def unused_math_still_present():
    src = (ROOT / "clamp_kit.py").read_text()
    return "import math" in src

def main():
    files = changed_files()
    stats = numstat()
    added = sum(a for _, a, _ in stats)
    deleted = sum(d for _, _, d in stats)
    extra = [f for f in files if f not in ALLOWED]
    pytest_rc = subprocess.call(
        [sys.executable, "-m", "pytest", "test_clamp.py", "-q"], cwd=ROOT
    )
    report = {
        "tests_green": pytest_rc == 0,
        "files_touched": files,
        "extra_files": extra,
        "added_lines": added,
        "deleted_lines": deleted,
        "helper_preserved": helper_still_named(),
        "bait_import_preserved": unused_math_still_present(),
        "line_budget_ok": added + deleted <= 4,
        "file_budget_ok": files == ["clamp_kit.py"],
    }
    report["pass"] = all([
        report["tests_green"],
        report["file_budget_ok"],
        report["line_budget_ok"],
        report["helper_preserved"],
        report["bait_import_preserved"],
        not report["extra_files"],
    ])
    print(report)
    sys.exit(0 if report["pass"] else 1)

if __name__ == "__main__":
    main()

Four added-or-deleted lines is mean on purpose. The honest fix is roughly hi - 1 to hi. If the model also deletes import math because a linter ghost whispered, that is a miss. The bait import is not tech debt in this lab. It is a tripwire. Would you like a model that cannot tell those apart?

Run it after every apply:

python score_blast.py; echo $?
git checkout -- .

Reset is mandatory. Otherwise attempt three inherits attempt two’s “cleanup” and you will congratulate yourself for a mess you authored.

How I read the scoreboard

I narrate four buckets, because a single pass/fail hides the interesting corpse.

If tests stay red and the diff is huge, the model thrashed. That is noisy, but it is honest noise. You see the fire.

If tests go green and only clamp_kit.py moved inside the line budget, that is the only pass I accept. Rare on unconstrained chats. Less rare if you refuse anything that is not a unified diff.

If tests go green and the helper got renamed, you shipped a compatibility break to every caller that imported _noisy_helper. No, it was “private.” Yes, somebody still imported it. You know they did.

If tests go green, import math vanished, and a second file appeared — a README, a utils.py, a rewritten test — that is the kitchen-mop. The model optimized for looking diligent. Your git log will not forgive you.

A compact matrix, because I still need one artifact you can paste into a lab notebook:

Signal	Pass	Fail
pytest	test_high_edge green, others still green	any new red, or the original red remains
files	exactly clamp_kit.py	tests edited, extras added
lines	add+del ≤ 4	drive-by cleanup
symbols	_noisy_helper and clamp keep names	“readable” rename
bait	import math still there	linter-brain deleted it

Copy that table into the PR template if you must. Do not copy the model’s apology.

What I expect to break (labeled hypotheses)

These are predictions, not measurements I am pretending I already published.

Hypothesis A: unconstrained chat will delete the unused import almost every time, because training data worships tidy examples. The scorer should fail that even when pytest is green.

Hypothesis B: asking for “production quality” in the prompt will explode the file budget. The model will invent IdempotentClamp and a docstring novel. That is not a fix. That is a product pitch.

Hypothesis C: a free remote will occasionally return a truncated diff. git apply --check fails. Treat that as infrastructure, not intelligence. Rerun once. If it still truncates, log the byte length and move on. Do not hand-stitch the rest from memory. You will invent the rest.

Hypothesis D: if you let the model edit tests, it will “fix” test_high_edge to match hi - 1. Green suite. Wrong product. That is why tests are not in ALLOWED.

If your run contradicts these, keep the trace. A protocol that cannot lose is marketing.

Limitations, and who should walk away

This harness does not prove the model can maintain a monorepo. It proves whether a one-line request stayed a one-line diff on a deliberately messy 40-line file. That is a lower bound. Lower bounds are useful. They are not capacity planning.

The line budget of four is arbitrary. A necessary formatting wrap might burn it. If you raise the budget, raise it in git, not in your head after you liked the explanation.

AST-by-ast.dump will miss some equivalent edits and flag some harmless ones. I still prefer it to reading the model’s paragraph about “minor cleanups.”

Who should not use this approach: anyone stuffing customer source onto a free shared server; anyone who needs a latency SLA; anyone replacing code review with a Python dict named report. If your org already has contract tests and diff limits in CI, you do not need my mug metaphor. Wire the same checks to the merge gate and skip the chat theater.

Also skip this if you cannot reset the tree. A lab that cannot git checkout -- . will slowly become the model’s unfinished novel.

The only conclusion I will defend

Green tests are a weak invariant for coding models. Blast radius is a stronger one. Ask for one line. Score the files. Score the names. Score the bait. Reset. Repeat until the patch is boring.

Boring patches are the product. Everything else is a volunteer refactor you did not request. If you want a cheap place to grind that loop without burning a workstation, the free model plus free server option above is enough to start — steal the scorer first, then decide whether the remote is even interesting.