Import First, Refactor Second: Triage a Repo Nobody Can Run

Most refactors die at the import line. The repo runs on one machine, and nobody knows which module breaks first. So the first artifact is not a test. It is a triage table.

I learned this the slow way. A characterization test pointed at a module that cannot import records the breakage, not the behavior. You then "fix" the refactor to satisfy a snapshot of a crash.

This post is a five-step workflow for that situation. Step 1 pins what exists. Step 2 ranks modules by whether they import at all. Step 3 picks a target with a scored table. Step 4 captures one characterization test. Step 5 makes the smallest safe change. All five run on any Python repo.

Why tests come second here

A characterization test asserts on current behavior. Current behavior on a broken import is a traceback.

If you freeze that traceback, every later fix fails the test. You get noise, not safety.

The order is importability, then selection, then snapshot. Never snapshot first.

Step 1: Pin the environment you actually have

Do this before you read a single source file. You are not upgrading anything yet. You are recording ground truth.

python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
pip freeze > requirements.before.txt
python --version > python.before.txt

Commit both files. They are the baseline for every later diff.

If pip install already fails, stop. The refactor is now a dependency archaeology task, which is a different article.

Step 2: Probe every module in a subprocess

Import each module in isolation, with a timeout and a clean PYTHONPATH. Module-level work then appears as a hang or a nonzero exit. Your own process survives.

# triage.py — classify modules by importability. Python 3.9+.
import json, os, subprocess, sys, time
from pathlib import Path

ROOT = Path(sys.argv[1] if len(sys.argv) > 1 else ".").resolve()
SKIP = {".venv", "venv", "build", "dist", "__pycache__", ".git", "node_modules"}
TIMEOUT = 10

PROBE = """
import importlib, json, sys, time
t0 = time.perf_counter()
try:
    importlib.import_module(sys.argv[1])
except BaseException as exc:
    print(json.dumps({"ok": False, "error": type(exc).__name__ + ": " + str(exc)[:200]}))
    raise SystemExit(1)
print(json.dumps({"ok": True, "ms": round((time.perf_counter() - t0) * 1000, 1)}))
"""

def module_name(path: Path) -> str:
    parts = list(path.relative_to(ROOT).with_suffix("").parts)
    if parts and parts[-1] == "__init__":
        parts = parts[:-1]
    return ".".join(parts)

rows = []
for path in sorted(ROOT.rglob("*.py")):
    if SKIP & set(path.parts):
        continue
    name = module_name(path)
    env = {"PYTHONPATH": str(ROOT), "PATH": os.environ.get("PATH", "/usr/bin:/bin")}
    started = time.perf_counter()
    try:
        proc = subprocess.run(
            [sys.executable, "-c", PROBE, name],
            cwd=ROOT, capture_output=True, text=True, timeout=TIMEOUT, env=env,
        )
        status = "ok" if proc.returncode == 0 else "broken"
        tail = (proc.stdout or proc.stderr).strip().splitlines()
        detail = tail[-1] if tail else ""
    except subprocess.TimeoutExpired:
        status, detail = "hangs", f">{TIMEOUT}s at import time"
    rows.append({
        "module": name,
        "status": status,
        "wall_ms": round((time.perf_counter() - started) * 1000, 1),
        "detail": detail[:300],
    })

Path("triage.json").write_text(json.dumps(rows, indent=2))
print(f"{len(rows)} modules -> triage.json")

Run it, then read the counts instead of the file size.

python triage.py .
python -c "import json,collections;r=json.load(open('triage.json'));print(collections.Counter(x['status'] for x in r))"

Expect a line like Counter({'ok': 211, 'broken': 34, 'hangs': 7}). Those three numbers rewrite your plan. The 211 are refactorable. The 34 are triage work. The 7 are landmines.

Note what hangs means. It is module-level network calls, DB connections, or input(). Fix those first, because every test you write will inherit the hang.

Step 3: Rank with a three-column table

Random file order is how messy refactors stall. Score instead.

Signal	Read from	Weight
Blast radius	`grep -rl "import " --include=*.py . \	wc -l`
Import status	triage.json	2
Test cost	your estimate, 1 (pure fn) to 3 (needs DB)	-1

grep -rl "import billing.totals" --include=*.py . | wc -l

Pick the highest scoring module whose status is ok. Refactor only inside the runnable set. Treat everything else as pre-work, and say so in the pull request description.

Step 4: One characterization test, on that module only

Freeze behavior, not implementation. Drive the real entry point. Compare bytes.

# tests/test_characterize_totals.py
import json, pathlib, subprocess, sys
import pytest

GOLDEN = pathlib.Path(__file__).parent / "golden" / "totals_sample.json"

def run(args):
    proc = subprocess.run([sys.executable, "-m", "billing.cli", *args],
                          capture_output=True, text=True)
    return {"code": proc.returncode, "out": proc.stdout, "err": proc.stderr}

def test_totals_are_byte_stable():
    got = run(["totals", "--file", "fixtures/sample.csv"])
    if not GOLDEN.exists():
        GOLDEN.parent.mkdir(parents=True, exist_ok=True)
        GOLDEN.write_text(json.dumps(got, indent=2, sort_keys=True))
        pytest.skip("golden written; review it by hand before committing")
    assert got == json.loads(GOLDEN.read_text())

The skip is the point. A golden file written by a passing run is evidence. A golden file written by a crashing run is a bug with a commit message.

Strip nondeterminism before you freeze anything. Absolute paths, timestamps, and durations will flake. Normalize them inside run() and assert on the normalized form.

Step 5: Smallest safe change, one seam per commit

Three rules. They are boring on purpose.

1. Extract or move one thing per commit.
2. Never change a signature in the same commit as a move.
3. Run the same three commands before every push.

python triage.py . > /tmp/after.json
pytest tests/test_characterize_totals.py -q
git diff --stat

The triage.json diff should show zero status regressions. If a module flipped to broken, you moved something its importers needed.

Where free tooling fits, honestly

The tedious part is not the refactor. It is drafting a characterization test per module and running the probe loop again and again.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. Both are operator-supplied availability claims; treat them as such and check current terms yourself. In this workflow the free model access has exactly one narrow job: turn a triage.json row plus the module source into a test skeleton, then hand-check every assertion. Never accept generated assertions unreviewed. A generated test that asserts nothing is worse than no test, because it passes.

The free server option matters when the loop is long. Probing a few hundred modules in subprocesses is CPU-bound and slow on a laptop you still need. A remote box keeps the loop off your machine. I make no claim about its size or runtime. Measure your own run first with time python triage.py ..

Neither option decides the refactor. The table and the golden file do.

Limitations and who should skip this

The import probe is a heuristic. It misses lazy imports, plugin registries, and anything behind a feature flag.

Modules with heavy global state can report ok and still be unsafe to move. Add a second probe that touches exported attributes if that bites you.

Golden files break on environment-dependent output. If your CLI prints a version string or a temp path, normalize before freezing or the test becomes a flake generator.

Skip this workflow when import itself needs live credentials or a network. Fix fixtures first. A probe that logs in 300 times is an incident, not a baseline.

Skip it too for a monorepo where one probe run takes an hour. Shard by package first, then triage each shard.

The order is the whole trick

Import triage, then ranking, then one golden file, then one seam.

That order keeps you inside the runnable set. Every commit stays comparable to the last. And the pull request ships evidence instead of vibes.

If you run the triage script, tell me which bucket surprised you most: broken, hangs, or ok.

Comments

[Alex Shev]

Importing before refactoring is a useful discipline because it separates environmental uncertainty from code-design judgment. I would capture the first runnable baseline in a small document: runtime version, install command, one command that succeeds, and the first failing path. That baseline makes later refactors measurable instead of leaving everyone to rediscover setup assumptions.