Last spring a payments team asked an agent to clean a decade-old checkout directory, then merged after a green unit suite. The agent renamed helpers, inlined tax tables, and reordered private functions that silently encoded rounding order. Production invoices drifted by a few cents on bundled SKUs, and finance noticed only after weekly reconciliation. The suite had never asserted those rounding paths, so the rewrite looked successful until money moved.
That failure mode is becoming common as generated patches get cheaper than careful human review time. The interesting question is not whether an agent can produce a prettier module overnight for reviewers. The useful question is which files it may touch when existing tests do not own the behavior. This walkthrough uses a composite checkout example rather than a claimed personal production incident from one company.
Cheap rewrites enlarge hidden contracts
When generation is inexpensive, the default move is rewriting a whole directory because the prompt is easier than a surgical extract. Each extra file in the patch is an untested coupling that can change without a failing assertion. A messy repository already hides order-dependent I/O, process-wide caches, and implicit column names in shared helpers. Expanding the diff multiplies those hidden contracts instead of isolating one module behind a stable surface.
A more boring sequence works better on brownfield Python services that still take real traffic:
- Build a blast-radius freeze list from import edges and test ownership.
- Record the I/O seam of one candidate module before any production code moves.
- Extract that module behind a facade, leaving every other caller import unchanged.
- Allow an agent to draft only the facade and seam tests, never the frozen internals.
The rest of this article is that sequence, including a local script, a review table, and rejection commands you can paste into a pull request template.
Artifact: freeze untested hubs from the import graph
The script below is a labeled, unexecuted example you can adapt, not a production-grade linter. It walks Python files, records shallow import edges, and flags modules that no test file appears to import. Those untested modules become the freeze list, and an agent must not edit them until a recorded seam exists.
#!/usr/bin/env python3
"""blast_radius.py — labeled example, not a production linter."""
from __future__ import annotations
import ast
import sys
from pathlib import Path
SKIP_DIRS = {".git", ".venv", "venv", "__pycache__", "node_modules"}
def iter_py(root: Path):
for path in root.rglob("*.py"):
if SKIP_DIRS.intersection(path.parts):
continue
yield path
def module_name(root: Path, path: Path) -> str:
rel = path.relative_to(root).with_suffix("")
return ".".join(rel.parts)
def parse_imports(path: Path) -> set[str]:
try:
tree = ast.parse(path.read_text(encoding="utf-8"))
except (SyntaxError, UnicodeDecodeError):
return set()
found: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
found.add(alias.name.split(".")[0])
elif isinstance(node, ast.ImportFrom) and node.module:
found.add(node.module.split(".")[0])
return found
def classify(root: Path):
src_files = []
test_files = []
for path in iter_py(root):
parent = path.parts[-2] if len(path.parts) > 1 else ""
if path.name.startswith("test_") or parent in {"tests", "test"}:
test_files.append(path)
else:
src_files.append(path)
return src_files, test_files
def inbound_count(stem: str, top: str, src_mods: dict[str, Path]) -> int:
count = 0
for other, other_path in src_mods.items():
if other_path.stem == stem:
continue
imported = parse_imports(other_path)
if stem in imported or top in imported:
count += 1
return count
def main(root: Path) -> None:
src_files, test_files = classify(root)
src_mods = {module_name(root, p): p for p in src_files}
imported_by_tests: set[str] = set()
for test_path in test_files:
imported_by_tests.update(parse_imports(test_path))
freeze = []
candidates = []
for mod, path in sorted(src_mods.items()):
top = mod.split(".")[0]
owned = top in imported_by_tests or mod in imported_by_tests
inbound = inbound_count(path.stem, top, src_mods)
row = (mod, inbound, str(path.relative_to(root)))
if not owned:
freeze.append(row)
else:
candidates.append(row)
print("# FREEZE (no test import owns this module)")
for mod, inbound, rel in freeze:
print(f"freeze\t{inbound}\t{mod}\t{rel}")
print("# EXTRACT CANDIDATES (tests import them; prefer low inbound)")
for mod, inbound, rel in sorted(candidates, key=lambda r: r[1]):
print(f"extract\t{inbound}\t{mod}\t{rel}")
if __name__ == "__main__":
main(Path(sys.argv[1] if len(sys.argv) > 1 else ".").resolve())
Run it from the repository root and keep both lists beside the diff:
python3 blast_radius.py .
python3 blast_radius.py . | awk '$1=="freeze" {print $4}' | sort > freeze.txt
python3 blast_radius.py . | awk '$1=="extract"' | sort -k2,2n | head
Interpretation should stay mechanical so review does not turn into a style debate. High inbound plus no test ownership means the module is a silent hub and belongs on the freeze list. Low inbound plus test ownership is the extract candidate for this change. Everything on freeze.txt must remain byte-identical in the first pull request, including whitespace.
How to read one checkout-shaped result
Suppose the labeled run prints rows like the following composite output:
freeze 11 checkout.tax checkout/tax.py
freeze 7 checkout.pricing checkout/pricing.py
extract 2 checkout.cart checkout/cart.py
extract 1 checkout.quote_view checkout/quote_view.py
checkout.tax is the worst place to start, even though it looks like the “messy” file everyone wants cleaned. checkout.quote_view is the better first extract because tests already import it and few modules depend on it. The agent still does not rewrite quote_view.py; it only drafts a facade that delegates to the frozen file.
Decision table: what the agent may draft
Treat the table as a review contract rather than a prompt suggestion that reviewers can waive later.
| Surface | Agent may draft? | Human must verify | Freeze rule |
|---|---|---|---|
| Untested helper with inbound greater than three | No | Do not open the file | Byte freeze |
| Public function that a test already imports | Facade only | Signature and return shape | No body rewrite |
| New wrapper module beside the frozen file | Yes | I/O fixtures still match | Callers stay put |
| Fixture recorder for HTTP or SQL | Sketch only | Secrets redacted, clocks pinned | Record once |
| Test that reimplements business math | No | Prefer recorded outputs | No invented oracle |
| Formatting-only diff on a freeze path | No | Reject the patch | Zero noise |
If a generated diff touches a freeze path, the patch is rejected before anyone debates naming. That rule is the entire point of building the list before the model runs.
Record the I/O seam before any extract
A facade is useless if you cannot replay the old module’s inputs and outputs after the move. The next labeled example records calls through a thin wrapper, then compares later extracts against the tape. Do not let a model invent expected decimals, tax rates, or timestamps.
# seam_recorder.py — labeled example for a checkout quote helper
from __future__ import annotations
import hashlib
import json
from pathlib import Path
from typing import Any, Callable
TAPE = Path("tapes/quote_view.jsonl")
def _digest(payload: Any) -> str:
blob = json.dumps(payload, sort_keys=True, default=str).encode()
return hashlib.sha256(blob).hexdigest()[:16]
def record(fn: Callable, args: tuple, kwargs: dict) -> Any:
result = fn(*args, **kwargs)
TAPE.parent.mkdir(parents=True, exist_ok=True)
row = {
"id": _digest({"args": args, "kwargs": kwargs}),
"args": args,
"kwargs": kwargs,
"result": result,
}
with TAPE.open("a", encoding="utf-8") as handle:
handle.write(json.dumps(row, default=str) + "\n")
return result
def replay_or_fail(fn: Callable, args: tuple, kwargs: dict) -> Any:
key = _digest({"args": args, "kwargs": kwargs})
expected = None
if TAPE.exists():
for line in TAPE.read_text(encoding="utf-8").splitlines():
row = json.loads(line)
if row["id"] == key:
expected = row["result"]
break
actual = fn(*args, **kwargs)
if expected is None:
raise AssertionError(f"missing tape for {key}; record first")
if actual != expected:
raise AssertionError(f"seam drift {key}: {actual!r} != {expected!r}")
return actual
A minimal test then pins the seam without claiming to understand bundled SKU rounding:
# tests/test_quote_seam.py — labeled example
from checkout import quote_view as qv
from seam_recorder import replay_or_fail
def test_quote_matches_recorded_tape():
args = ({"sku": "BUNDLE-40", "qty": 2, "region": "US-CA"},)
replay_or_fail(qv.quote, args, {})
Record production-like fixtures in a staging environment with secrets stripped and clocks pinned. The tape is the oracle for the extract, and the extract must match it exactly. If the tape is missing a path you care about, record that path before asking any model for a patch.
Extract one module behind a facade
After the freeze list and the tape exist, the smallest useful change is a new module that one caller imports instead of the messy file. The old file remains frozen. The facade delegates, and you replace the delegate only when the tape stays green.
# checkout/quote_facade.py — labeled extract
from checkout import quote_view as _legacy
def quote(item: dict) -> dict:
"""Stable surface. Do not reimplement rounding here."""
return _legacy.quote(item)
Then update a single import site, not the entire package graph:
# before
from checkout.quote_view import quote
# after
from checkout.quote_facade import quote
Commands that belong in the pull request description, not in a slide deck:
git diff --name-only | sort > changed.txt
comm -12 changed.txt freeze.txt && echo "REJECT: freeze violated" && exit 1
pytest tests/test_quote_seam.py -q
python3 blast_radius.py . | awk '$1=="freeze" {print $4}' | sort | diff -u freeze.txt -
If comm prints any overlapping path, stop the review. That overlap is a process failure, not a formatting nit. If diff on the freeze list is not empty, the blast radius moved and the change is no longer the change you planned.
A prompt that stays inside the table
Keep the generation request as narrow as the review contract. The labeled prompt below refuses work that would expand the patch:
You may edit checkout/quote_facade.py and tests/test_quote_seam.py only.
Do not modify any path listed in freeze.txt.
Do not invent expected quote amounts; read tapes/quote_view.jsonl.
Return a unified diff. If the freeze list would be touched, refuse.
Anything outside those two files is out of scope, including “drive-by” import cleanup. Cheap generation makes drive-by cleanup feel free; the freeze list makes it obviously expensive.
Where a free model and free server actually help
Drafting a facade, a recorder, and a seam test is repetitive boilerplate that models handle reasonably well. It is also easy to over-generate into a rewrite of checkout/tax.py because that file looks uglier than the facade. A constrained loop on a throwaway server is useful because blast_radius.py remains the gate, not the model’s confidence.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option that can host this loop without standing up a long-lived cluster. The operator-supplied claim is availability of those two things, not a particular model name, quota, hardware profile, or benchmark. Use the server as a scratch workspace: clone a sanitized repository, run the freeze script, ask the model only for facade and test sketches, then discard the machine.
Keep secrets off that server, and treat generated code as untrusted until the seam test and the freeze diff both pass on a machine you control. If the model returns a patch that formats frozen files, drop the patch rather than negotiating exceptions.
Limitations and who should not use this
This workflow is a coupling brake, not a correctness proof for money paths. The import parser misses dynamic imports, importlib loaders, and re-exports through __init__.py barrels. Test ownership based on import names will misclassify files that tests reach only through fixtures, string paths, or coverage of package __main__ modules. JSON tapes do not capture floating clocks, locale, timezone databases, or unordered dict iteration on older runtimes.
Do not use this approach when any of the following are true:
- The module is a cryptographic or payments core that needs a written spec, not a recorded tape.
- You cannot sanitize production-like fixtures before they leave your network boundary.
- The repository is not Python, or the mess lives in schemas and data rather than modules.
- The platform itself is being replaced, so a freeze list would only delay a necessary rewrite.
- Reviewers will rubber-stamp any green agent patch despite a freeze-path violation.
If those constraints apply, stop at the freeze list and write the next change by hand. A list of files you will not touch is still cheaper than a pretty diff that moves money.
A smaller definition of done
A successful first pull request contains one facade, one tape, and zero freeze violations. It does not contain a renamed package, a new framework, or a reformatted tree that no test owns. Cheap generation makes the larger patch tempting because the extra files look related in the prompt. The freeze list makes that expansion obviously out of scope before anyone starts arguing about taste.
If you need a scratch server and free model access for drafting the facade only, MonkeyCode is one place to run that constrained loop. The blast-radius file still decides whether the patch ships.
Top comments (0)