Lock Import Side Effects Before One Registrar Extract

Core conclusion

Messy packages hide real cost at import time. Extracting a helper without a tape breaks callers.

Pin those side effects before any structural edit. Change one registrar only after the tape is green.

A free coding model cannot invent missing tests. It can only move code you already pinned.

The failure mode

Import-time work looks cheap until the extract lands. Distant packages then fail during routine CI runs.

Typical messy __init__.py files perform four extra jobs. They mutate paths, seed env defaults, register hooks, and import extras.

Callers silently depend on that hidden import setup. A broad extract drops one of those jobs.

Tests that only assert return values stay green. Production then misses a previously registered cleanup handler.

Why return-value tests miss this

run_job("x") can pass after a broken extract. The helper never used the atexit hook.

Path mutations also stay invisible to unit tests. A plugin loader then fails in another repository.

Env seeds fail in the same quiet way. Formatters read MESSY_PKG_LOG and get empty strings.

Public __all__ drift breaks star imports later. Downstream from messy_pkg import * then loses names.

An import tape makes those four failures local. You see them before the extract merge.

What this article pins

This workflow freezes four import observables on purpose. It does not freeze any business logic outputs.

1. sys.path entries added during package import.
2. Environment keys written during package import.
3. New messy_pkg* names in sys.modules.
4. atexit.register calls intercepted by a spy.

Those four values form the full characterization tape. You extract one registrar only when they match.

Who this is for

Use this on a Python package with a thick __init__.py. Skip it for a greenfield library with explicit setup functions.

Do not use this as cover for a large rewrite. The extract is one function, not a layer.

Artifact: a side-effect harness

The harness imports the package inside a subprocess. The parent interpreter state then stays completely clean.

The child prints one JSON tape to stdout. Pytest then compares that object to a committed expected dict.

Label the tree below as a proposed fixture. Rename those modules to match your repository.

0. Build the messy package under test

Create this tree only inside a scratch directory.

messy_pkg/
  messy_pkg/
    __init__.py
    worker.py
  tests/
    test_import_tape.py
  characterize_import.py

worker.py holds one real function. __init__.py registers cleanup as a side effect.

# messy_pkg/worker.py
def run_job(name: str) -> str:
    if not name:
        raise ValueError("name required")
    return f"job:{name}"

# messy_pkg/__init__.py
from __future__ import annotations

import atexit
import os
import sys
from pathlib import Path

from messy_pkg.worker import run_job

_ROOT = Path(__file__).resolve().parent.parent
_EXTRA = _ROOT / "vendor_shims"


def _ensure_shim_path() -> None:
    extra = str(_EXTRA)
    if extra not in sys.path:
        sys.path.insert(0, extra)


def _seed_env() -> None:
    os.environ.setdefault("MESSY_PKG_MODE", "compat")
    os.environ.setdefault("MESSY_PKG_LOG", "warn")


def _flush_compat_buffers() -> None:
    # Proposed leftover compatibility flush. Not production code.
    os.environ.pop("MESSY_PKG_FLUSH_ONCE", None)


def register_compat_cleanup() -> None:
    atexit.register(_flush_compat_buffers)


_ensure_shim_path()
_seed_env()
register_compat_cleanup()

__all__ = ["run_job", "register_compat_cleanup"]

That registrar is the only extract target. Leave path and env mutations untouched for now.

1. Snapshot import observables in a child process

# characterize_import.py
from __future__ import annotations

import json
import os
import subprocess
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parent

CHILD = r'''
import atexit
import json
import os
import sys
from pathlib import Path

_register_calls = []
_orig_register = atexit.register

def _spy_register(*args, **kwargs):
    _register_calls.append({"argc": len(args), "kw": sorted(kwargs)})
    return _orig_register(*args, **kwargs)

atexit.register = _spy_register

before_path = list(sys.path)
before_env = dict(os.environ)
before_modules = set(sys.modules)

import messy_pkg  # noqa: F401

after_path = list(sys.path)
after_env = dict(os.environ)
after_modules = set(sys.modules)

path_added = [p for p in after_path if p not in before_path]
env_added = sorted(k for k in after_env.keys() - before_env.keys())
env_changed = sorted(
    k for k in after_env
    if k in before_env and after_env[k] != before_env[k]
)
mods_added = sorted(
    m for m in (after_modules - before_modules)
    if m == "messy_pkg" or m.startswith("messy_pkg.")
)

tape = {
    "atexit_count": len(_register_calls),
    "env_added": env_added,
    "env_changed": env_changed,
    "env_log": after_env.get("MESSY_PKG_LOG"),
    "env_mode": after_env.get("MESSY_PKG_MODE"),
    "modules_added": mods_added,
    "path_added_suffixes": [Path(p).name for p in path_added],
    "public_names": sorted(getattr(messy_pkg, "__all__", [])),
}
print(json.dumps(tape, sort_keys=True))
'''


def capture_tape() -> dict:
    env = os.environ.copy()
    env["PYTHONPATH"] = str(ROOT)
    proc = subprocess.run(
        [sys.executable, "-c", CHILD],
        check=False,
        capture_output=True,
        text=True,
        env=env,
        cwd=str(ROOT),
    )
    if proc.returncode != 0:
        raise RuntimeError(proc.stderr or proc.stdout)
    return json.loads(proc.stdout)


if __name__ == "__main__":
    print(json.dumps(capture_tape(), indent=2, sort_keys=True))

The spy wraps atexit.register before the import. Packages that bind from atexit import register will bypass it.

Treat that bypass as a known limitation. Convert those imports before you trust the count.

2. Freeze the tape as a pytest gate

# tests/test_import_tape.py
from __future__ import annotations

import json
from pathlib import Path

from characterize_import import capture_tape

EXPECTED = {
    "atexit_count": 1,
    "env_added": [],
    "env_changed": [],
    "env_log": "warn",
    "env_mode": "compat",
    "modules_added": ["messy_pkg", "messy_pkg.worker"],
    "path_added_suffixes": ["vendor_shims"],
    "public_names": ["register_compat_cleanup", "run_job"],
}


def test_import_tape_matches_expected():
    tape = capture_tape()
    assert tape == EXPECTED


def test_tape_roundtrip_bytes(tmp_path):
    # Proposed local helper. Do not auto-write goldens in CI.
    tape = capture_tape()
    path = tmp_path / "tape.json"
    path.write_text(json.dumps(tape, indent=2, sort_keys=True) + "\n", encoding="utf-8")
    assert json.loads(path.read_text(encoding="utf-8")) == tape

setdefault does not add keys that already exist. Inherited CI variables can hide the seed.

Scrub known keys in the child environment. Then env_added reflects true import writes.

3. Scrub inherited environment keys

Add this loop inside capture_tape before subprocess.run.

    for key in ("MESSY_PKG_MODE", "MESSY_PKG_LOG", "MESSY_PKG_FLUSH_ONCE"):
        env.pop(key, None)

Re-run the tape after the scrub. env_added should now list both keys.

Update EXPECTED in the same edit. Do not leave a stale golden dict.

EXPECTED = {
    "atexit_count": 1,
    "env_added": ["MESSY_PKG_LOG", "MESSY_PKG_MODE"],
    "env_changed": [],
    "env_log": "warn",
    "env_mode": "compat",
    "modules_added": ["messy_pkg", "messy_pkg.worker"],
    "path_added_suffixes": ["vendor_shims"],
    "public_names": ["register_compat_cleanup", "run_job"],
}

4. Run the tape before any extract

python -m pip install pytest
python characterize_import.py
python -m pytest tests/test_import_tape.py -q

Record the JSON output once. Commit the expected dict beside the test.

Future extracts must keep that object stable. Any extra key is a failed gate.

Smallest safe change

Do not split sys.path logic in this pass. Do not touch the setdefault calls either.

Move only register_compat_cleanup and its flush helper. That is the entire allowed diff.

5. Extract one registrar into compat.py

# messy_pkg/compat.py
from __future__ import annotations

import atexit
import os


def _flush_compat_buffers() -> None:
    os.environ.pop("MESSY_PKG_FLUSH_ONCE", None)


def register_compat_cleanup() -> None:
    atexit.register(_flush_compat_buffers)

Point __init__.py at the new module. Keep the original call order identical.

# messy_pkg/__init__.py  (after extract)
from __future__ import annotations

import os
import sys
from pathlib import Path

from messy_pkg.compat import register_compat_cleanup
from messy_pkg.worker import run_job

_ROOT = Path(__file__).resolve().parent.parent
_EXTRA = _ROOT / "vendor_shims"


def _ensure_shim_path() -> None:
    extra = str(_EXTRA)
    if extra not in sys.path:
        sys.path.insert(0, extra)


def _seed_env() -> None:
    os.environ.setdefault("MESSY_PKG_MODE", "compat")
    os.environ.setdefault("MESSY_PKG_LOG", "warn")


_ensure_shim_path()
_seed_env()
register_compat_cleanup()

__all__ = ["run_job", "register_compat_cleanup"]

6. Re-run the same tape immediately

python characterize_import.py
python -m pytest tests/test_import_tape.py -q

modules_added will now include messy_pkg.compat. That delta is expected.

Update EXPECTED in the same commit as the extract. Do not mix unrelated golden edits.

"modules_added": [
    "messy_pkg",
    "messy_pkg.compat",
    "messy_pkg.worker",
],

If atexit_count, env keys, path suffixes, or __all__ shift, stop. Restore the registrar and inspect the diff.

Decision table

Observation after extract	Action
Only modules_added gained the new module	Accept. Update the golden tape.
atexit_count dropped from 1 to 0	Reject. Registrar never ran.
atexit_count rose above 1	Reject. Double registration.
path_added_suffixes changed	Reject. Wrong extract scope.
env_added or values changed	Reject. Seed order drifted.
__all__ lost a public name	Reject. Public contract broke.
Child import raised an exception	Reject. Cycle or missing module.

Use the table as a hard gate. Do not negotiate extra edits into the same diff.

Commit shape

Keep the registrar extract on one commit only. Keep golden tape updates on that same commit.

Do not mix formatter noise into the extract. Run formatters later in a separate change.

The PR description should list the decision table row you hit. Reviewers then know which tape delta was accepted.

Where a free remote model fits

Local tape comes first. Remote edits come second. Never invert that order.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option. Those two facts matter only after the tape exists.

You can send the golden dict, the harness, and one extract instruction. The remote session proposes the registrar move. Your pytest gate still decides accept or reject.

Do not send an unpinned __init__.py and ask for cleanup. The model will relocate path hacks, env seeds, and hooks together.

The tape exists to forbid that bundle. Keep the instruction limited to one function name.

If a remote session is useful, send the tape plus one function name. Paste the decision table next to that prompt.

Skip invented model names and quota claims here. Treat the server as a workspace, not evidence.

Limitations

The atexit spy cannot see C-level registrations. Native extensions can still hide extra hooks.

The harness ignores threads, sockets, and file opens. Import-time HTTP calls will not appear.

Add a separate interceptor if network setup is your mess. Do not overload this import tape.

sys.path suffix checks remain coarse by design. Two directories with one name collide.

Prefer resolved paths when your tree is stable. Suffix mode is for vendor shim folders.

setdefault behavior depends on the parent environment. Always scrub known keys in the child.

This tape is not a substitute for run_job unit tests. Worker behavior stays explicitly out of scope.

Subprocess cost adds seconds on every run. That cost is acceptable for import characterization.

Do not place this tape on a hot unit-test path. Keep it in a slow, explicit CI job.

Who should not use this

Do not use this on a package with a thin documented setup() entry. You already have an explicit seam.

Do not use this to justify a weekend rewrite. Limit the work to one registrar and one commit.

Do not hand the repo to a coding model first. Commit the golden tape before any remote edit.

Free or paid access does not change that rule. An unpinned extract is still an unpinned extract.

Checklist

1. Capture the import tape inside a child subprocess.
2. Scrub inherited environment keys for the child process.
3. Commit the expected dict beside the pytest file.
4. Extract one registrar and change nothing else.
5. Re-run the tape and apply the decision table.
6. Update modules_added only for the new module.

Stop after step six on purpose. Further extracts need a fresh tape cycle.

Import side effects are the real API of a messy package. Pin them first, then move one function.

The rest of the mess can wait until the next cycle.