The Import Worked in the Agent Shell. pytest Still Loaded Last Week's Wheel.

Have you ever watched a coding agent celebrate a clean import while your test job still traces into last week's code? I have, and on the first night I blamed pytest collection instead of the quieter sys.path problem. The shell and the test runner were both "Python 3", which is the most misleading agreement in this trade. This is a 48-hour field notebook about that mismatch, plus a dump script I will actually rerun.

Hour 0–6: the agent could import the fix

I started from a real failure: a helper raised AttributeError in CI, and I wanted a second machine that was not my laptop. I moved the reproduction onto MonkeyCode because free model access and a free server option kept the dump beside the test.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The agent opened a shell at the repo root and imported the patched symbol with a one-liner. It printed the new attribute immediately, so I told myself the original bug was gone. Did I print module.__file__ before celebrating, or did that successful import already feel like enough proof? That feeling is how you lose an evening to two copies of the same package name.

Here is the command that gave me false confidence. Notice how little it asks of the interpreter, and how much it assumes about the working directory.

python -c "from billing.ledger import post; print(post)"

That command succeeds whenever a billing/ directory exists under the current working directory. It does not prove the installed project, the built wheel, or the CI image will import the same file. Would you ship a release on that evidence alone, without asking which file won? I almost did, and the traceback later named a path I had not edited.

Hour 6–18: two interpreters, one package name

I finally printed the module file in both places, and the paths were not even cousins. The agent shell loaded .../repo/billing/ledger.py from the working tree I had just edited. The pytest process loaded .../site-packages/billing/ledger.py, which still had Tuesday's class and Tuesday's missing attribute. It was the same import name, a different inode, and a completely different conversation with the model.

Why did that split happen so quietly that both sides still said import billing? python -c puts the current directory at sys.path[0], so my local tree won inside the agent shell. Pytest ran after pip install . inside a throwaway venv, then from a working directory that did not contain a billing/ folder. The local tree disappeared from the front of sys.path, so the installed wheel won instead. The agent had never left the source tree, so the local tree always won and the wheel never got a vote.

I was debugging two different copies and calling it one failure. Have you done that while insisting the tests were flaky, because one command was green? I have, and the flake was a path, not a race.

Commands I should have run in the first hour, before any patch and before any prompt:

python -c "import sys, billing; print(sys.executable); print(billing.__file__)"
python -m pytest -q tests/test_ledger.py --tb=short
python -c "import sys; print(sys.executable)"
command -v python
ls -l "$(python -c 'import sys; print(sys.executable)')"

If those executables differ, stop talking about the product code for a minute. You are not looking at one environment with two opinions. You are looking at a coincidence of command names that both happen to print Python 3.

Hour 18–30: the dump script I wish I had on hour one

I got tired of typing the same prints into two shells, so I wrote a tiny dump that any interpreter can run. It is not clever, and it is not a framework you have to adopt. It is a snapshot of the things I kept forgetting: executable, prefixes, cwd, sys.path, and the file that actually satisfied the import.

# dump_import_env.py
from __future__ import annotations

import json
import os
import platform
import sys


def dump(modname: str) -> dict:
    info = {
        "argv": sys.argv,
        "executable": sys.executable,
        "version": sys.version.split()[0],
        "cwd": os.getcwd(),
        "prefix": sys.prefix,
        "base_prefix": sys.base_prefix,
        "in_venv": sys.prefix != getattr(sys, "base_prefix", sys.prefix),
        "pycache_prefix": getattr(sys, "pycache_prefix", None),
        "path0": sys.path[0] if sys.path else None,
        "sys_path": sys.path,
        "platform": platform.platform(),
    }
    try:
        module = __import__(modname)
        info["module_file"] = getattr(module, "__file__", None)
        spec = getattr(module, "__spec__", None)
        info["module_origin"] = getattr(spec, "origin", None) if spec else None
        search = getattr(spec, "submodule_search_locations", None) if spec else None
        info["module_submodule_search"] = list(search) if search else []
    except Exception as exc:
        info["import_error"] = repr(exc)
    return info


if __name__ == "__main__":
    name = sys.argv[1] if len(sys.argv) > 1 else "sys"
    print(json.dumps(dump(name), indent=2))

Run it more than one way before you trust a fix, because launch style changes sys.path[0]:

python dump_import_env.py billing
python -m dump_import_env billing
python -c "import dump_import_env, json, sys; print(sys.path[0])"

Then force the pytest interpreter to speak the same language, using the same file and the same module name:

python -m pytest --version
python dump_import_env.py billing
diff -u <(python dump_import_env.py billing) <(python -m pytest --version >/dev/null; python dump_import_env.py billing)

If you need the answer from inside a test, print a small JSON blob and fail on purpose once:

import json
import sys

import billing


def test_show_where_billing_came_from():
    payload = {
        "executable": sys.executable,
        "file": billing.__file__,
        "path0": sys.path[0],
    }
    print(json.dumps(payload, indent=2), flush=True)
    raise AssertionError("remove after you compare this blob with the agent shell")

The point is not pretty JSON, and it is not a new test style to publish. The point is a diff you can paste into a note without squinting at two tracebacks that both say billing.ledger.

A decision table I now keep next to the script

I needed a cheat sheet more than I needed another explanation of import machinery. This is the table I would tape above the terminal, because launch style is the whole bug.

Launch style	What lands at sys.path[0]	Copy you probably import	Trust it for install tests?
python -c at repo root	empty string (the cwd)	local package tree if it exists	No
python dump_import_env.py	directory of that script	depends whether that directory contains the package	Only after you read __file__
python -m pytest at repo root	cwd	local tree, often shadowing a wheel	Maybe for unit tests
pip install . then pytest from /tmp	/tmp	installed wheel under site-packages	Yes, for packaging
pip install -e .	editable .pth hook	local tree again	Yes for iteration, no for wheel contents

I treat the last two rows as different products now, even when they share a package name. Editable installs answer a developer question: does my editor match pytest on this machine? Wheel installs answer a release question: does the artifact I ship match pytest on a machine without my tree? Mixing those questions is how you get a green agent shell and a red pipeline, then spend a day patching the copy nobody runs.

Hour 30–48: a small test plan, not a vibe

I wrote a four-step plan I will reuse instead of arguing with the model about whose Python is real. It is boring on purpose, which is the feature.

1. Freeze the interpreter. Print sys.executable in the agent shell and in pytest, and refuse to continue if they differ.
2. Freeze the module file. Import the package under test and record __file__ plus spec.origin.
3. Freeze the launch. Reproduce with python -m pytest from a directory that does not contain a shadowing package folder.
4. Only then apply a code change, and rerun the dump. If __file__ still points at site-packages, the tree edit never entered the artifact.

A pytest assertion I now drop into one smoke test, labeled as a proposal you should edit for your own layout:

import sys
from pathlib import Path

import billing


def test_billing_is_the_copy_we_intended():
    origin = Path(billing.__file__).resolve()
    allowed_root = Path(sys.prefix).resolve()
    # Proposal: invert this if you intentionally test the working tree.
    assert allowed_root in origin.parents, (
        f"imported {origin} via {sys.executable}; "
        f"sys.path[0]={sys.path[0]!r}"
    )

That assertion is not a universal rule, and it will be wrong if you mean to test the working tree. If you mean to test the working tree, require the repo root instead of sys.prefix. The value is making the intended copy explicit so an agent cannot "fix" the suite by importing a different tree. Can a model still edit the assertion to match the wrong copy? Yes, which is why the dump JSON lives in the ticket, not only in the test.

What broke, and what I would repeat

What broke was not the model, and it was not a mysterious pytest plugin hiding a second conftest. I asked a shell that stood inside the source tree to certify an installed package. The free server was useful as a clean room, but only after I stopped treating its working directory as production layout. I also learned that importlib.reload() will cheerfully reload the wrong file, which feels like progress and is not.

What I would repeat on the next import ghost, before I let anyone patch production code:

• Keep dump_import_env.py in the repo and run it before the first "please fix this import" prompt.
• Compare JSON, not feelings. If executable or module_file changed, the rest of the conversation is noise.
• Use the second machine as a second cwd, not as an oracle. Drop the package folder from the pytest cwd when you care about wheels.
• Tell the agent the decision table row you are on. "We are on the wheel row" is a better prompt than "tests are still failing."

Would I skip the dump next time because the traceback already shows a path? I might try, and I would be wrong again by lunch. Tracebacks show the file that raised, not the file you thought you edited, and they do not show sys.executable unless you print it.

Limitations, and who should not use this

This workflow will not save you if the bug is a race, a clock, or a network timeout. It also will not prove that a wheel contains non-Python data files; it only tells you which .py satisfied an import. Do not use it as evidence of performance, and do not treat a free remote server as a replica of production hardware, OS packages, or secret material. If your package is a namespace split across several distributions, mod.__file__ can still lie by omission, and you need submodule_search_locations instead.

People who should skip this include anyone shipping extension modules where the .so path matters more than the package __init__. Skip it if you cannot put even a dump script on the machine that runs tests. If you cannot print sys.executable, you are not ready to ask a model to patch the import. Skip it if your policy forbids sending source to a hosted editor, because a convenient second server is still a second server.

I am not claiming the free model access or the free server option last forever, or that they replace a CI image you already trust. They were a convenient second pair of hands for this notebook. If you steal one thing, steal the table and the dump, and make the agent show module_file before it claims the failure is gone.