Start a Messy-Repo Extract With a Package Boundary Freeze

A billing service sat in one Python package for three years, with forty modules sharing helpers through relative imports and a few public functions that plugins actually called. An agent then opened a cleanup pull request that split billing/ into invoices/, tax/, and ledger/ because the folder names looked more professional. Unit tests stayed green, since they imported private helpers the same way the production code did, and the review looked like a tidy architecture win.

Two days later a plugin loader failed in staging because it still imported billing.totals.summarize. The extract had moved the function, deleted the old module, and left no shim. The tests never imported the public path that operators and plugins used. The repo was not missing tests in general; it was missing a freeze of the surfaces that other code was allowed to depend on.

This article treats that failure as a measurement problem, not a taste problem. The workflow below records package boundaries first, then allows only the smallest extract that those records can prove is quiet. Proposals and sample commands are labeled as such; they are a method you can run, not a claim about any particular production system.

Why green tests still ship a broken extract

AI-assisted refactors fail in messy repositories for a boring structural reason. Agents optimize for local readability, while messy packages encode contracts in import paths, __all__ lists, plugin entry points, and call shapes that no unit test names directly. A suite that only exercises internals will endorse a move that production still treats as a public URL.

Current coding-agent reviews often reward a large tree rewrite because the diff looks decisive. That is the same failure mode as shipping work that looks complete while the observable contract never got pinned. If the only evidence is “tests passed,” you have not measured the extract; you have measured the tests you already had.

A boundary freeze is a cheap characterization layer that sits above those tests. It does not prove the new design is better. It only answers whether the extract changed who can call what, and whether any caller still depends on the old path.

Artifact: freeze the public surface, then budget one change

The artifact is a JSON freeze file plus two tests. One test fails if an unexpected public name, export, or import edge appears. The other test fails if you touch more than one module pair in a single extract. Together they force the smallest safe change: move one symbol, keep a shim, and stop.

Proposed layout:

repo/
  billing/                 # the messy package under study
  tools/freeze_boundaries.py
  tests/test_boundary_freeze.py
  tests/test_change_budget.py
  artifacts/boundary_freeze.json

Step 1 — Inventory names and import edges

The following script is a proposed scanner, not a production crawler. It walks one package, records module-level public names, __all__ when present, and import edges that leave the package. Run it against the current tree before anyone moves files.

# tools/freeze_boundaries.py
from __future__ import annotations

import ast
import json
import sys
from pathlib import Path


def iter_py_files(root: Path):
    for path in sorted(root.rglob("*.py")):
        if path.name == "__pycache__":
            continue
        yield path


def module_name(root: Path, path: Path) -> str:
    rel = path.relative_to(root.parent).with_suffix("")
    return ".".join(rel.parts)


def public_names(tree: ast.AST) -> list[str]:
    dunder_all = None
    names: list[str] = []
    for node in tree.body:
        if isinstance(node, ast.Assign):
            for target in node.targets:
                if isinstance(target, ast.Name) and target.id == "__all__":
                    if isinstance(node.value, (ast.List, ast.Tuple)):
                        dunder_all = [
                            elt.value
                            for elt in node.value.elts
                            if isinstance(elt, ast.Constant) and isinstance(elt.value, str)
                        ]
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            if not node.name.startswith("_"):
                names.append(node.name)
        if isinstance(node, ast.Assign):
            for target in node.targets:
                if isinstance(target, ast.Name) and not target.id.startswith("_"):
                    names.append(target.id)
    return sorted(set(dunder_all if dunder_all is not None else names))


def outgoing_imports(tree: ast.AST, self_mod: str, pkg: str) -> list[str]:
    edges: list[str] = []
    for node in ast.walk(tree):
        if isinstance(node, ast.ImportFrom) and node.module:
            target = node.module
            if node.level:
                parent = self_mod.split(".")
                target = ".".join(parent[:-node.level] + ([node.module] if node.module else []))
            if not target.startswith(pkg + ".") and target != pkg:
                edges.append(target)
        elif isinstance(node, ast.Import):
            for alias in node.names:
                if not alias.name.startswith(pkg):
                    edges.append(alias.name.split(".")[0])
    return sorted(set(edges))


def freeze(package_dir: Path) -> dict:
    pkg = package_dir.name
    modules = []
    for path in iter_py_files(package_dir):
        tree = ast.parse(path.read_text(encoding="utf-8"))
        mod = module_name(package_dir, path)
        modules.append(
            {
                "module": mod,
                "public": public_names(tree),
                "imports_out": outgoing_imports(tree, mod, pkg),
            }
        )
    return {"package": pkg, "modules": modules}


def main(argv: list[str]) -> int:
    package_dir = Path(argv[1])
    out = Path(argv[2])
    payload = freeze(package_dir)
    out.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
    print(f"wrote {out} modules={len(payload['modules'])}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Suggested first run:

python tools/freeze_boundaries.py billing artifacts/boundary_freeze.json
git add artifacts/boundary_freeze.json
git commit -m "chore: freeze billing package boundaries before extract"

Commit the freeze on its own. Mixing the inventory with the extract hides which names moved because of the refactor and which names the scanner newly discovered.

Step 2 — Fail the build when the freeze drifts

The characterization test below loads the committed freeze and compares it to a fresh scan. Equality is the point. If a cleanup diff silently drops billing.totals.summarize from the public list, this test is the first red signal, even when unit tests still import a private helper.

# tests/test_boundary_freeze.py
import json
from pathlib import Path

from tools.freeze_boundaries import freeze

FREEZE = Path("artifacts/boundary_freeze.json")
PACKAGE = Path("billing")


def test_public_surface_matches_committed_freeze():
    committed = json.loads(FREEZE.read_text(encoding="utf-8"))
    current = freeze(PACKAGE)
    assert current["package"] == committed["package"]
    assert current["modules"] == committed["modules"], (
        "Package boundary changed. Update the freeze in a dedicated commit "
        "or keep a shim on the old public path."
    )

Step 3 — Enforce a one-pair change budget

Large extracts hide the breaking rename among dozens of file moves. The second test reads git diff --name-only against the freeze commit and allows at most two production modules: the source file and the destination file. Test files and the freeze artifact itself are exempt. This is a process control, not a type system.

# tests/test_change_budget.py
import os
import subprocess
from pathlib import Path

BASE = os.environ.get("BOUNDARY_BASE", "HEAD1")  # set to the freeze commit in CI


def changed_paths() -> list[str]:
    out = subprocess.check_output(
        ["git", "diff", "--name-only", BASE],
        text=True,
    )
    return [line for line in out.splitlines() if line]


def test_extract_touches_at_most_one_module_pair():
    prod = []
    for path in changed_paths():
        p = Path(path)
        if p.parts[:1] == ("tests",):
            continue
        if path.startswith("artifacts/"):
            continue
        if path.startswith("tools/"):
            continue
        if p.suffix == ".py":
            prod.append(path)
    assert len(prod) <= 2, (
        f"Extract exceeded the one-pair budget: {prod}. "
        "Move one symbol, keep a shim, land, then repeat."
    )

In CI, set BOUNDARY_BASE to the freeze commit SHA rather than HEAD1. The sample default is a placeholder so the file stays runnable in a local clone after you substitute the real ref.

Smallest safe change after the freeze is green

Once the freeze is committed and both tests pass on main, an extract is allowed only if it preserves the old public path. The pattern is mechanical:

1. Add the new module with the moved function and no other edits.
2. Keep the old module as a one-line shim that re-exports the same name.
3. Update the freeze in a follow-up commit that records both modules.
4. Delete the shim only after grep, plugins, and the freeze agree that nothing imports it.

Proposed shim, labeled as an example:

# billing/totals.py — kept until callers move
from billing.ledger.summarize import summarize

__all__ = ["summarize"]

That is a smaller diff than a folder taxonomy. It is also easier to revert, because failure still points at one symbol. If an agent returns a fifty-file “domain split,” reject the patch and ask for the shim form. The freeze file tells you whether the agent complied.

Decision table for the extract

Observation after the candidate diff	Freeze test	Budget test	Action
Public names unchanged, one pair of modules touched, shim present	pass	pass	Land the extract
Public name disappeared, no shim	fail	either	Restore the old path before any further moves
Public names unchanged, twelve files moved	pass	fail	Split the patch until one pair remains
New public names appear without a freeze update	fail	either	Record the names in a freeze-only commit
Only tests and docs changed	pass	pass	Out of scope for this protocol

Use the table during review. Do not argue about folder beauty until the first two columns are green.

Where a coding assistant belongs in this protocol

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Generating the first freeze by hand is tedious in a forty-file package, and agents are useful for drafting candidate characterization tests from that JSON. They are not useful as the authority that decides the extract is safe. If you try MonkeyCode here, use its free model access and free server option only to propose extra assertions against boundary_freeze.json, then keep the tests that actually fail on a shim-less move.

Treat those drafts as untrusted text until they run. An assistant that rewrites the freeze to match a large cleanup is doing the opposite of characterization. The committed JSON is the spec; the model is a helper that may add cases you forgot, such as __all__ gaps or re-exports hiding behind import *.

Limitations, and who should skip this

The scanner only sees module-level definitions and static import nodes. It will miss dynamically built module names, runtime getattr plugins, C extensions, and public behavior that exists only as CLI flags or HTTP routes. Teams whose real contract is a network schema need a different freeze, built from response fixtures rather than AST names.

The change-budget test can be bypassed by editing many symbols inside the two allowed files. Pair it with a human grep for deleted def names when a “small” file is actually a god module. The method also assumes a single package root. A monorepo with several messy packages needs one freeze per package, or the budget becomes meaningless.

Skip this protocol if you are deleting a prototype that has no external callers, or if you already have an explicit public API package with compatibility tests. Skip it if the goal is a semantic rewrite rather than a move: characterization of names will not catch a tax-rounding change inside a function that kept the same import path. In that case pin outputs, not boundaries.

The useful result is modest. After a freeze exists, a messy-repo extract becomes a sequence of one-symbol moves with shims, each proven against a committed surface. That is slower than a glamorous tree rewrite, and it is the pace at which plugins, operators, and later reviewers can still explain what changed.