48-Hour Field Notes: The Diff Looked Right. The Import Still Hit the Old Wheel.

I keep coming back to one annoying question: why does a job still behave like yesterday when the diff already looks right? The function in the review sits in the working copy, and the comment even quotes the new branch. Then a printed module path can point at an old wheel under site-packages, which means the process never loaded your edit. Have you ever approved a line that the running job was not actually importing at all?

The note I wish I had opened first

These notes are a method I would repeat, not a scoreboard pulled from someone else's cluster. I am not claiming a measured outage, a customer save, or a benchmark you should quote in a review. What I am claiming is narrower, and it is still worth a quiet evening with a terminal. Python will happily import a different file than the one your editor has focused on screen.

If that sentence feels obvious, ask yourself when you last printed the module file path before trusting a green log. I split the forty-eight hours into three short passes, and I write each result before I start the next pass. Pass one stays on the laptop, where an editable install can hide the very bug you think you fixed. Pass two uses a clean virtual environment, ideally on a machine that never saw an editable install. Pass three is only a comparison of the two notes, looking for a path that changed while the diff did not.

What actually breaks

The usual story is not malice, and it is not even a badly written patch in review. You run the module as a package from the repo root, so the current directory leads the import path and the working tree wins. A job runner later starts that same module with another working directory, or it starts a console script from a virtualenv. That console script imports the distribution copied at install time, which can be older than the branch you just reviewed.

A second cousin of that bug is a plain name collision sitting on the import path. A scratch folder called the same name sits ahead of the real project, so the probe imports the scratch copy instead. You then edit the real project and wonder why the string on the job never changes at all. Would your log line tell you about that path, or would it only say that a function returned the old string?

Current CPython still puts the script directory in sys.path[0] when you launch a file, and it puts the current directory there when you use python -m. An installed console script does not owe you either of those favors. I would write that distinction into the note before I blame the patch, the reviewer, or the scheduler.

A probe you can paste and run

The script below is a proposed check, not a capture copied from a private incident file. I would keep it under a notes folder and treat a failing exit code as a reason to stop editing. It takes the import name from the first argument, and it reads an expected root from the environment. That split lets the laptop and the clean host use different checkout paths without editing the script.

import importlib
import os
import sys

def main() -> int:
    name = sys.argv[1]
    expected = os.environ.get("EXPECTED_ROOT", "")
    module = importlib.import_module(name)
    origin = getattr(module, "__file__", None)
    print(f"cwd={os.getcwd()}")
    print(f"argv0={sys.argv[0]}")
    print(f"path0={sys.path[0]!r}")
    print(f"origin={origin}")
    marker = getattr(module, "MARKER", None)
    print(f"marker={marker!r}")
    if not expected:
        print("expected_root=missing")
        return 0
    root = os.path.realpath(expected)
    found = os.path.realpath(origin) if origin else ""
    inside = found == root or found.startswith(root + os.sep)
    print("verdict=inside_expected_root" if inside else "verdict=outside_expected_root")
    return 0 if inside else 2

if __name__ == "__main__":
    raise SystemExit(main())

If I control the library, I would also put a boring marker in the package itself. The string is a proposal for the note, not a version scheme you must adopt.

# mypkg/__init__.py — proposal, unexecuted until you add it
MARKER = "working-tree"

Commands for the laptop

On the laptop I would run the three commands below, after I set the root to the checkout I trust. I want the probe, the installer metadata, and a one-liner import to agree before I touch the patch again. If those three disagree, the diff is not the next place I should spend the afternoon. Do you really want another commit while the running process is still loading yesterday's installed file?

export EXPECTED_ROOT="$PWD"
python notes/import_probe.py mypkg
python -m pip show mypkg
python -c "import mypkg, sys; print(mypkg.__file__)"

Commands for a clean host

On a clean host I would not copy the laptop virtualenv, because that copy is how the bug travels. I would create a new environment, install from the same lock or sdist the job uses, and run the probe again. If that host has only a wheel and no working tree, I stop expecting the path strings to match. I compare origin and marker to the laptop note, and I write down that the question changed.

python -m venv /tmp/probe-venv
/tmp/probe-venv/bin/python -m pip install /path/to/your.whl
/tmp/probe-venv/bin/python notes/import_probe.py mypkg
/tmp/probe-venv/bin/python -m pip show mypkg

Leave EXPECTED_ROOT unset on that second run when the wheel is supposed to live under the new virtualenv. Setting it to the laptop checkout would fail on purpose, and that failure would only prove you asked the wrong question. I want the note to say which question I asked.

How I read a disagreement

I keep the note to six lines, because a longer note turns into a diary and I stop comparing it. The list below is the whole comparison, and I do not add a seventh line just to feel thorough. If you cannot fill a line, write missing, because a blank looks like success when you reread it later. Have you noticed how a tidy blank cell quietly becomes the story you already wanted to tell?

1. Record the command, the working directory, and whether you used a file path or the module switch.
2. Record path zero, origin, and marker from the probe, plus the location line from the installer metadata.
3. If origin sits outside the expected root, stop and fix the environment before you edit any more code.
4. If origin matches but marker does not, you are on the right tree and the wrong revision, so reinstall.
5. If both match and the bug remains, the import shadow was not the cause, and the note should say so.

That fifth line matters more than it looks, because a single-bug hunt will eventually blame the wrong layer. I would rather write not this than invent a story that happens to fit the log I already disliked. A wrong story feels productive in the moment, and it costs you the next morning when the job still fails. So the note is allowed to acquit the import path, and then I have to go look somewhere else.

A table I check before the second pass

I use a short table so I do not talk myself into running the wrong pass next. The table is a reminder, not a policy your team has to adopt on Monday morning. If a row does not fit your setup, skip the row instead of stretching the check. Have you ever turned a handy checklist into a rule that nobody on the team can follow?

What I just saw	What I do next	What I refuse to conclude
Laptop origin is inside the checkout and marker matches	Move on to config, data, or the long-running process	That every other host loads this same file
Clean host origin is under that host's site-packages	Compare marker or version with the laptop note	That the path strings themselves must match
pip show cannot find the name	Treat the import as a path accident until install is explicit	That a green local run used the package I meant
Marker is missing on both sides	Add a marker or compare version metadata before judging	That two empty markers prove the same revision

Where a free model and a free server fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I use MonkeyCode's free model access only to draft the probe and the six-line note from constraints I already wrote. I still read that draft, I still run it locally, and I drop it if the exit codes disagree. A generated script that I have not executed is a proposal, and I label it that way in the note. I do not ask the model to declare the patch correct.

The free server option is the part I actually want for the second pass of this note. I want a machine that does not inherit my editable install, my extra path variable, or a forgotten scratch folder. I am not stating a quota, a hardware shape, a model name, or a promise that the free option remains available. If that server is busy, missing, or outside your policy, a disposable virtual machine is the same experiment.

The product is a convenient place to run the second pass, not the evidence that the import is honest. Would I paste a production traceback into that chat just to save a few minutes of typing? No, because the probe only needs an import name and a root path, and even those can be sensitive. If the import itself reads secrets, I would not run this probe against that module on a shared server.

What I would repeat next time

• I would repeat the two-pass note whenever a review says the code changed and the job still returns the old string.
• I would repeat a marker constant in libraries I control, because path equality is awkward across machines and a string is easy to diff.
• I would repeat the installer metadata check beside the probe, because an old version under site-packages is a faster story than a debugger.
• I would not repeat a laptop-only check and call the evening done, since the editable install is often the thing hiding the wheel.

Who should skip this approach

Skip this approach if the failure is not an import, such as a wrong config file or a stale image. A path probe will look healthy in those cases, and you will waste the second day chasing a shadow that is not there. Skip it when the package is mostly native code and you need an ABI or libc match before you can trust the result. A matching file path does not prove the extension was built for that host, so do not pretend it does.

Also skip it if the code handles secrets or personal data and your rules forbid external servers or external models. Skip it if you need an audited production replica, because a free server is a second opinion and not a release gate. I would also skip the model draft if you are not willing to read every line before you run it. A short script can still do the wrong import, and speed is not a reason to stop looking.

Limits I do not want to hide

This check does not see modules loaded by a process that started before you reinstalled the package. It does not notice a zip import, a namespace package spread across several paths, or a path edit after the first import. It also does not tell you that the tested revision is the revision you meant to ship. You have to put that fact in the marker or in the version metadata yourself, or the note cannot know it.

Restart the worker after you reinstall, or the note will describe a process that already exited in spirit. Watch for a later sys.path insert inside a worker, because the probe only sees the path at the moment you import. I would rather leave those gaps on the page than add another tool that pretends to close them.

If you already have a MonkeyCode account, run this probe once on the free server and keep the six-line note beside the diff. If you do not, run the same commands in any clean virtualenv and keep the note anyway. The useful part is the comparison between the two notes, not the name printed on the host. I would repeat that comparison the next time a green diff and an old string show up in the same hour.