I Trusted Git for 48 Hours. Python Still Imported the Wheel.

Have you ever stared at a green git status while the running process still behaved like yesterday's code? I spent two days in that loop, and the mismatch started to feel oddly personal. The assistant kept rewriting files under src/, yet the worker still executed the old function body. Why does a patch sitting in the working tree refuse to exist at runtime on the remote box?

What I thought was broken

I assumed the remote interpreter was caching bytecode, because that is the first story we all tell. I deleted every __pycache__ directory until the tree looked sterile, and the traceback still printed stale lines. I restarted the worker, then the shell, then the whole box, and nothing in the logs moved.

Then I blamed the model, because blaming the nearest assistant is the second story we tell. Was it editing a copy of the file that the process never imported at all? I grepped the whole checkout for the function name and found exactly one definition. That should have closed the mystery, but the running worker still refused to show the new log line.

The 48-hour timeline

Hours 0–8: local looked fine

Locally I ran the module as a script, and the new logging line showed up immediately in stdout. I committed that change with a smug little message and pushed it toward the remote environment. On the remote side I pulled the branch, installed the package, and started the same entrypoint again. The new log line never appeared, even though git said the file on disk was current.

git pull --ff-only
pip install -e .
python -m myapp.worker

The editable install printed success, so I trusted the environment without asking which file was loaded. That quiet trust was the first real mistake I made during this whole two-day detour.

Hours 8–24: the model kept saving the wrong layer

I pasted the traceback into a coding assistant that was running against a scratch remote environment. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access and free server option as a throwaway box, not as a production runtime.

The model did what models do when a function looks wrong: it rewrote the function body again. It added a guard clause, then a log line, then a unit test that matched the new signature. The test file in git looked healthier after those edits, but the running worker did not change.

I asked out loud whether we were even editing the file that Python had imported. A source tree alone cannot answer that question, and the model could not invent the missing path. Can you honestly diagnose an import bug like this without printing the loaded filename first?

Hours 24–40: what actually broke

The remote box already had an earlier pip install myapp sitting in site-packages from day one. The later editable install did not fully replace it, because a regular install, an egg-link, and leftover dist-info were mixed. Python's import machinery is not a git client, and it does not care which file you just saved.

The smoking gun was one line:

python -c "import inspect, myapp.worker as w; print(w.__file__); print(inspect.getfile(w.handle))"

Locally that one-liner printed a path under the repository, which matched the file I had just edited. Remotely it printed a path under site-packages, sitting beside a leftover myapp-0.1.0.dist-info directory. The model was patching src/myapp/worker.py while the process imported the wheel I installed before lunch.

Hours 40–48: the boring fix

I uninstalled every copy of the package, then installed it exactly once, then printed __file__ again. After that check finally passed, the model's patches showed up in the worker traceback at last. The code change itself was tiny, and the operational change was the entire story of the outage.

Reproducible artifact: prove which file is running

Here is a minimal layout you can recreate in an empty directory. I am labeling this as a procedure I would repeat, not as a benchmark and not as a claim about production traffic.

shadow-demo/
  pyproject.toml
  src/myapp/__init__.py
  src/myapp/worker.py
  check_import.py

pyproject.toml:

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[project]
name = "myapp"
version = "0.1.0"

[tool.setuptools.packages.find]
where = ["src"]

src/myapp/__init__.py can stay empty. src/myapp/worker.py carries a marker that makes the loaded copy obvious:

MARKER = "repo-copy"

def handle(payload: dict) -> str:
    return f"{MARKER}:{payload.get('id', 'missing')}"

check_import.py is the witness I wish I had run on hour one:

from __future__ import annotations

import inspect
import sys

import myapp
import myapp.worker as worker


def main() -> None:
    print("executable=", sys.executable)
    print("path0=", sys.path[0] or "<empty>")
    print("package_file=", myapp.__file__)
    print("worker_file=", inspect.getfile(worker))
    print("handle_file=", inspect.getfile(worker.handle))
    print("marker=", worker.MARKER)
    print("handle=", worker.handle({"id": "n1"}))


if __name__ == "__main__":
    main()

Reproduce the shadow

Run these in order. Do not skip the regular install, because that copy is the entire trap.

cd shadow-demo
python -m venv .venv
. .venv/bin/activate
python -m pip install -U pip
python -m pip install .
python check_import.py

Now edit MARKER in src/myapp/worker.py to repo-copy-v2 and run the checker again:

python check_import.py
python -c "import myapp.worker as w; print(w.MARKER, w.__file__)"

The second run still prints repo-copy if the interpreter is loading the copied files from site-packages. That is the whole incident in a dozen commands. Git can be clean, the buffer can look right, and the process can still be somewhere else.

Then try the fix people reach for when they are tired:

python -m pip install -e .
python check_import.py
python -m pip show -f myapp
python -c "import sys; print('\n'.join(sys.path))"

If pip show -f still lists copies under site-packages/myapp plus an egg-link, you are in the mixed-install state I hit. Uninstall until pip show myapp fails, then install one way only.

python -m pip uninstall -y myapp
# repeat until pip says skip / not installed
python -m pip install -e .
python check_import.py

Change MARKER again after that loop. The checker should follow the repo file this time. If it does not, stop guessing and print __file__ before you ask any model to rewrite the function.

Extra trap: python -c and an empty sys.path[0]

I also wasted a few hours running one-liners from the repo root and thinking that proved the worker. python -c puts an empty string at sys.path[0], which is not the same as python -m myapp.worker. Compare them on purpose:

python -c "import sys, myapp; print(sys.path[:3]); print(myapp.__file__)"
python -m check_import
python check_import.py

If those three prints disagree, you do not have an application bug yet. You have three different import contexts, and the assistant will happily patch only one of them.

Decision table I now keep in the notes

Observation	Likely layer	Do this next	Do not do this
Git diff has the patch, runtime does not	import path / installed files	print __file__ and pip show -f	rewrite the function again
Tests pass, worker does not	different interpreter or cwd	compare sys.executable and sys.path	add time.sleep
Editable install reported success	leftover regular install	uninstall until missing, reinstall once	stack another pip install -e .
Model keeps touching tests	assertion drift	freeze the test, fix the import first	let the model align the assert
Only remote is stale	server still has a wheel	recreate the venv, then rerun the checker	chmod random files
One-liner disagrees with -m	sys.path[0] / launch style	run the same entrypoint the worker uses	debug from python -c only

Test plan I would actually repeat

1. Create a clean venv on the box that will run the worker, and refuse to reuse a mystery environment.
2. Install the project exactly one way: editable for iteration, or a single wheel for a freeze.
3. Run check_import.py and store the printed paths next to the commit hash.
4. Change MARKER, rerun the checker, and refuse to debug behavior until the marker changes.
5. Start the worker with python -m myapp.worker using the same sys.executable as the checker.
6. If you involve an assistant, paste package_file= and worker_file= before you paste the traceback.

Would I let a model generate the next patch? Yes, after the path print. Before that print, the model is editing a ghost, and you are reviewing theater.

What broke, in plain language

Several ordinary things stacked, and none of them were exotic runtime science.

• A regular pip install . copies files into site-packages, so later edits in src/ are invisible.
• A later editable install can leave both copies visible to pip show.
• Process start order matters: an already imported module in a long-lived worker will not see a new file.
• Deleting __pycache__ does not help if you never imported the file you edited.
• Assistants optimize for making the open buffer look correct. They do not automatically ask which file the interpreter loaded.

Is this the model's fault? Not really. I handed it a traceback and a repo path, and I never handed it inspect.getfile(). That gap is on me, and it is cheap to close.

Limitations, and who should skip this

This workflow assumes a CPython package installed with pip inside a single virtualenv. It will not diagnose namespace packages that live in several directories at once. It will not catch an import hook, a zipimporter, or a frozen binary either.

If you ship with PyInstaller, conda, or a vendored directory, still print __file__, but do not trust this uninstall loop as gospel. Do not use a throwaway remote box as your only copy of production data, and do not paste secrets into a scratch server just because the install is free.

If your team already pins environments with lockfiles and rebuilds an image on every change, you may never hit this. If you iterate by ssh-ing into a box and asking a model to just fix the worker, you will hit it.

What I would repeat next time

I would print the loaded file before I read the traceback, every single time. I would refuse mixed installs, and I would recreate the venv instead of stacking another editable layer. I would keep the checker next to the package, not in my head, and I would give an assistant the path output first.

I would also treat "the patch is in git" as a claim that still needs a runtime witness. Two days is a long time to learn that Python is not git. I would rather spend the first ten minutes asking a blunt question. Which file did you actually import?