DEV Community

Dakota Huang
Dakota Huang

Posted on

Prove the Smallest Safe Change Is Mechanical: An AST Diff Gate

A rename should never hide a logic change. The smallest safe change is provable, not felt.

Most refactor accidents follow one pattern. A commit mixes a move with a tweak. Reviewers read the diff, get tired, and approve both.

The fix is a gate, not more discipline. Parse old and new files, strip formatting, apply your rename map, compare structure. If the structures match, the commit is mechanical. If they differ, split the commit.

Why line diffs lie

Line diffs are whitespace-sensitive and order-sensitive. A formatter run can triple the diff size and hide one real edit.

Abstract syntax trees ignore all of that. Two files with identical structure after normalization are the same program, modulo names.

So the gate answers one question per symbol: is this change mechanical or behavioral?

The rule: two commits, one change each

Commit A contains only mechanical edits. Commit B contains only behavioral edits. Nothing else.

You can revert B alone and keep the refactor. That is the whole payoff.

Step 1: snapshot the old file

Pull the pre-refactor source out of git. Do not rely on your editor's local history.

git show main:pkg/pipeline.py > /tmp/pipeline.old.py
python -c "import ast,sys; ast.parse(open('/tmp/pipeline.old.py').read())"
Enter fullscreen mode Exit fullscreen mode

The second command fails loudly on a broken snapshot. Cheap check, real value.

Step 2: write the rename map

Keep the mapping explicit and versioned. An empty map is valid and should still pass.

{
  "run_batch": "run_page",
  "batch_size": "page_size"
}
Enter fullscreen mode Exit fullscreen mode

This file becomes the receipt for the refactor. Reviewers can read intent in ten seconds.

Step 3: normalize and compare

The gate below drops imports, drops docstrings, renames symbols, and compares per-symbol dumps.

"""refactor_gate.py: prove a refactor commit is mechanical."""
from __future__ import annotations

import ast
import json
import sys
from pathlib import Path


def _drop_docstring(body: list[ast.stmt]) -> list[ast.stmt]:
    if body and isinstance(body[0], ast.Expr) \
            and isinstance(body[0].value, ast.Constant) \
            and isinstance(body[0].value.value, str):
        rest = body[1:]
        return rest or [ast.Pass()]
    return body


class Normalize(ast.NodeTransformer):
    def __init__(self, renames: dict[str, str]) -> None:
        self.renames = renames

    def visit_FunctionDef(self, node):
        node.body = _drop_docstring(node.body)
        self.generic_visit(node)
        node.name = self.renames.get(node.name, node.name)
        return node

    visit_AsyncFunctionDef = visit_FunctionDef

    def visit_ClassDef(self, node):
        node.body = _drop_docstring(node.body)
        self.generic_visit(node)
        node.name = self.renames.get(node.name, node.name)
        return node

    def visit_arg(self, node):
        node.arg = self.renames.get(node.arg, node.arg)
        return node

    def visit_Name(self, node):
        node.id = self.renames.get(node.id, node.id)
        return node

    def visit_Attribute(self, node):
        self.generic_visit(node)
        node.attr = self.renames.get(node.attr, node.attr)
        return node

    def visit_Import(self, node):
        return None

    def visit_ImportFrom(self, node):
        return None


def surface(source: str, renames: dict[str, str]) -> dict[str, str]:
    tree = Normalize(renames).visit(ast.parse(source))
    ast.fix_missing_locations(tree)
    out: dict[str, str] = {}
    for node in tree.body:
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            out[node.name] = ast.dump(node, include_attributes=False)
    return out


def gate(old_src: str, new_src: str, renames: dict[str, str]):
    old, new = surface(old_src, renames), surface(new_src, renames)
    rows = []
    for name in sorted(set(old) | set(new)):
        if name not in new:
            rows.append((name, "REMOVED"))
        elif name not in old:
            rows.append((name, "ADDED"))
        elif old[name] == new[name]:
            rows.append((name, "MECHANICAL"))
        else:
            rows.append((name, "BEHAVIORAL"))
    return rows


def main(argv: list[str]) -> int:
    old_path, new_path, map_path = (Path(p) for p in argv[1:4])
    renames = json.loads(map_path.read_text()) if map_path.read_text().strip() else {}
    rows = gate(old_path.read_text(), new_path.read_text(), renames)
    for name, status in rows:
        print(f"{status:11s} {name}")
    return 1 if any(status != "MECHANICAL" for _, status in rows) else 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))
Enter fullscreen mode Exit fullscreen mode

Run it against the snapshot and the working file. The exit code is the gate.

python refactor_gate.py /tmp/pipeline.old.py pkg/pipeline.py renames.json
echo "exit=$?"
Enter fullscreen mode Exit fullscreen mode

Report shape for a fixture where run_batch became run_page and clean_row gained a .strip() call:

BEHAVIORAL  clean_row
MECHANICAL  run_page
exit=1
Enter fullscreen mode Exit fullscreen mode

The rename passes. The strip call does not. Split the commit.

Step 4: split, then re-run

Move the behavioral hunk out of commit A. Stash it, or stage hunks selectively.

git add -p pkg/pipeline.py        # stage only the rename hunks
git commit -m "refactor: rename run_batch -> run_page"
git stash push -k                 # park the behavior change
git stash pop                     # reapply it on top
git commit -m "fix: strip cells in clean_row"
Enter fullscreen mode Exit fullscreen mode

Then re-run the gate on the full range. It should report exactly one behavioral symbol.

The decision table

Gate output Reading Action
All MECHANICAL Pure rename or move Commit A, exit 0, done
Any BEHAVIORAL Logic changed Split into A and B, gate A only
REMOVED with no pair Symbol dropped Find callers first, keep a shim
ADDED only New helper Pure extraction: A. New logic: B
BEHAVIORAL on every symbol Snapshot is wrong Re-check Step 1

That last row matters. A bad baseline produces noise, not insight.

Where a model helps, and where it must not

Drafting the rename map is tedious. A model is decent at proposing old_name -> new_name pairs from a diff.

I use MonkeyCode's free model access for that draft step. The gate, not the model, decides whether the mapping is complete.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

When the target repo's dependencies will not install locally, the free server option is a place to run the gate. Availability is operator-supplied; verify current limits before you depend on them.

Keep the model's map in version control. A mapping you cannot review is a mapping you cannot trust.

A two-minute revert rehearsal

Before merging, prove the revert is clean. Revert commit B on a scratch branch.

git switch -c rehearse-revert
git revert --no-edit <sha-of-B>
python refactor_gate.py /tmp/pipeline.old.py pkg/pipeline.py renames.json
Enter fullscreen mode Exit fullscreen mode

If the gate exits 0 after reverting B, the refactor and the fix are separable. That is the property you actually wanted.

Limitations you should know

  • The gate compares structure, not semantics. Equal dumps do not prove equal runtime behavior.
  • Dynamic lookups (getattr, string-keyed dispatch, **kwargs keys) bypass the rename map.
  • Decorators that read __name__ will change behavior on rename, and the gate will not see it.
  • Docstring and comment edits are invisible by design. Add a separate doc check if you care.
  • Generated code and metaclass-heavy files produce noisy trees. Exclude them explicitly.
  • Formatting changes disappear from the report. That is a feature, and it can also hide a formatter bug.

Label the gate as a narrow instrument. It catches mixed commits, not logic errors.

Who should not use this

Skip this workflow if renames are part of a public API contract without a deprecation path. The gate will pass and your users will still break.

Skip it in repos where a single file carries thousands of lines of metaprogramming. The structural dump becomes unreadable before it becomes useful.

Skip it if your team cannot commit mechanical work separately. The gate will report the split and no one will perform it.

The takeaway

Smallest safe change means smallest provable change. Structure comparison makes that provable in seconds.

Run the gate on one file this week. Keep the rename map in the repo so the next refactor starts from evidence, not memory.

Top comments (0)