DEV Community

Jordan Huang
Jordan Huang

Posted on

A Successful Import Is Not a Runtime Receipt

Did the agent really install your dependencies today?
I keep seeing the same false comfort in chats.
An import works. Then someone ships. Then CI explodes.

This is not a rant about model quality.
It is a runtime-identity problem instead.
The chat log is not sys.executable and never was.

I will knock down five claims I still hear.
Then you get a receipt you can rerun.
You will see no mystery scores or fake hardware lists.

What this piece refuses to do

I will not quote a latency number I did not measure.
I will not name models I cannot verify here.
I will not treat a transcript as a build log.

The scripts below are a proposed workflow.
Label them that way in your own notes.
Run them on the same host the agent used.

Myth 1: A clean import means the package is pinned

Claim. "It imported httpx. We are fine."

Really? What version loaded into memory?
Which extra was present on disk?
Which interpreter performed the import?

An import only proves one module object exists.
It does not prove a pin or a content hash.
It also does not prove tomorrow's host will match.

Ask one blunt question after every import.
Which file recorded that wheel on disk?
If the answer is the chat, you have theater.

Corrected model. An import is a smoke test.
A lockfile is the actual install contract.
Keep those two jobs in separate boxes.

Myth 2: The host Python is your project Python

Claim. "The loaner box already has Python. Skip the venv."

System Python is only a bootstrap tool.
It is not your project's runtime profile.
sys.prefix and sys.base_prefix tell the truth.

If those two strings match, you are not in a venv.
The agent may still print a friendly "installed."
That print is social talk rather than isolation.

Run this on the machine the agent used:

python3 -c "import sys; print(sys.executable); print(sys.prefix); print(sys.base_prefix)"
command -v python3
type python
Enter fullscreen mode Exit fullscreen mode

Do those lines agree with each other?
If they disagree, the agent guessed the binary.
Guessing the binary is not a packaging step.

Corrected model. The project owns the interpreter.
The host only loans a bootstrap binary.
Create a venv before any install command runs.

Myth 3: "Already satisfied" means the right version

Claim. "pip said already satisfied. Stop nagging."

That phrase only means a name was found.
It may still be the wrong version.
It may be a user-site install instead.
It may be leftover cache from another job.

Name match is not the same as version match.
Version match is not the same as extra match.
Extra match is still not a lockfile.

Check the two sources that actually exist:

python3 -m pip show httpx
python3 -m pip freeze | grep -E '^httpx=='
Enter fullscreen mode Exit fullscreen mode

If show drifts from your lock, you drifted.
If freeze is empty, you queried the wrong Python.
Wrong Python is the usual failure, not "pip is broken."

Corrected model. Satisfied is only a cache message.
It is not a reproducibility statement at all.
Diff pip freeze against the lock every session.

Myth 4: The chat remembered the virtualenv

Claim. "We activated .venv two prompts ago."

Did we? In which process, exactly?
Shell state dies with the process that created it.
A new command often gets a brand new shell.

VIRTUAL_ENV is an environment variable.
It is not model memory in the transcript.
The transcript cannot export it for you.

printf '%s\n' "${VIRTUAL_ENV:-unset}"
.venv/bin/python -c "import sys; print(sys.executable); print(sys.prefix != sys.base_prefix)"
Enter fullscreen mode Exit fullscreen mode

If the first line prints unset, activation died.
If the second binary is missing, no venv exists.
If the boolean is false, you are on host Python again.

Corrected model. Activation is strictly per process.
Re-export it in every command block you run.
Or call .venv/bin/python by absolute path.

Absolute path beats folklore every time.
Folklore is how agents "forget" the venv.
Do you still trust a sentence that says "activated"?

Myth 5: Cheap rehearsal means you can skip the receipt

Claim. "It is only a loaner host. Why record it?"

Because loaner hosts are where drift hides.
A free model can still emit unpinned pip install.
A free server can still pick another default Python.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I reach for MonkeyCode when a free model and a free server belong in the same rehearsal loop. That pairing is handy for trying the receipt without burning a laptop. It does not pin wheels for you, and it does not turn a chat into CI.

Corrected model. Cost and identity are different questions.
Cheap rehearsal still needs a written receipt.
Free does not mean the runtime goes unrecorded.

The artifact: write a runtime receipt

Here is a small proposed checker.
I am not publishing timings for it.
Copy it. Run it. Keep the JSON.

#!/usr/bin/env python3
"""Write runtime-receipt.json for the current interpreter.

Proposed workflow. Run it yourself before you trust it.
"""
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
from pathlib import Path
from shutil import which


def sha256_if_exists(path: Path) -> str | None:
    if not path.is_file():
        return None
    digest = hashlib.sha256()
    digest.update(path.read_bytes())
    return digest.hexdigest()


def pip_freeze_lines() -> list[str]:
    completed = subprocess.run(
        [sys.executable, "-m", "pip", "freeze"],
        check=False,
        capture_output=True,
        text=True,
    )
    if completed.returncode != 0:
        return [f"ERROR:{completed.returncode}"]
    return [line for line in completed.stdout.splitlines() if line.strip()]


def main() -> int:
    root = Path.cwd()
    receipt = {
        "cwd": str(root.resolve()),
        "sys_executable": sys.executable,
        "sys_version": sys.version.replace("\n", " "),
        "sys_prefix": sys.prefix,
        "base_prefix": sys.base_prefix,
        "in_venv": sys.prefix != sys.base_prefix,
        "which_python3": which("python3"),
        "which_python": which("python"),
        "VIRTUAL_ENV": os.environ.get("VIRTUAL_ENV"),
        "PATH_head": os.environ.get("PATH", "").split(os.pathsep)[:8],
        "requirements.txt": sha256_if_exists(root / "requirements.txt"),
        "requirements.lock": sha256_if_exists(root / "requirements.lock"),
        "pyproject.toml": sha256_if_exists(root / "pyproject.toml"),
        "pip_freeze": pip_freeze_lines(),
    }
    out = root / "runtime-receipt.json"
    out.write_text(json.dumps(receipt, indent=2) + "\n", encoding="utf-8")
    print(out)
    if not receipt["in_venv"]:
        print("warning: not in a virtualenv", file=sys.stderr)
        return 2
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

Need a tiny test beside it?
Yes. Assert identity, not vibes.

# test_runtime_receipt.py
from __future__ import annotations

import json
import sys
from pathlib import Path


def test_receipt_file_exists():
    assert Path("runtime-receipt.json").is_file()


def test_receipt_matches_running_interpreter():
    payload = json.loads(Path("runtime-receipt.json").read_text(encoding="utf-8"))
    assert payload["sys_executable"] == sys.executable
    assert payload["in_venv"] == (sys.prefix != sys.base_prefix)


def test_pin_file_was_recorded():
    payload = json.loads(Path("runtime-receipt.json").read_text(encoding="utf-8"))
    has_req = payload["requirements.txt"] is not None
    has_lock = payload["requirements.lock"] is not None
    has_pyproject = payload["pyproject.toml"] is not None
    assert has_req or has_lock or has_pyproject
Enter fullscreen mode Exit fullscreen mode

What the JSON keys actually mean

  • sys_executable: the binary that actually ran.
  • in_venv: prefix inequality, not a vibe check.
  • which_python3: whatever PATH says right now.
  • VIRTUAL_ENV: empty means activation did not survive.
  • pip_freeze: the live view of site-packages.
  • file hashes: did the pin files change under you?

If sys_executable and which_python3 disagree, stop.
You are debugging the wrong interpreter now.
Everything after that disagreement is noise.

Proposed commands

Do not paste this blindly into production.

python3 -m venv .venv
.venv/bin/python -m pip install -U pip
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python runtime_receipt.py
.venv/bin/python -m pytest test_runtime_receipt.py -q
Enter fullscreen mode Exit fullscreen mode

What should you commit after that?
Commit runtime_receipt.py and the tests.
Commit the lockfile you actually used.
Treat runtime-receipt.json as session evidence.
Redact paths if they leak a username.

Decision table

Use this before you trust an agent install.

Situation Write a receipt? Free model + free server okay? Ship from that host?
Exploring a script Yes Yes, as rehearsal No
Pinning dependencies Yes Yes, if you copy freeze out No
App tests after pins Yes Only if the receipt matches CI No
Production release Yes, inside your CI Not as the release runner No
Secrets in the tree Yes, on a machine you own Avoid shared loaner hosts No

Read the last column again, slowly.
Rehearsal is not a release channel.
That is the whole table.

Who should not use this

Skip this workflow if you need a full SBOM.
Skip it if you cannot run Python at all.
Skip it if the host must never see your lockfile.
Skip it if legal wants an attested builder.

Also skip the shared free server when secrets sit in .env.
A receipt will not unsay a leaked token.
Keep secrets off loaner disks on purpose.

Limitations I will not hide

pip freeze is incomplete for some extras.
It does not replace uv lock or Poetry.
It does not understand Conda environments.
It does not prove your app tests passed.
It can record absolute paths. Redact them.

The venv check uses prefix inequality.
That is the usual CPython signal today.
It is not a law of every embedding.

I also do not claim the receipt is tamper proof.
It is a snapshot you can diff later.
Diffs beat vibes. That is enough for rehearsal.

FAQ, compressed

  • Did the import pass? Then you have smoke, not a pin.
  • Did pip look happy? Then you have a cache message.
  • Did the chat mention a venv? Then you have a story.
  • Do you have JSON for sys.executable? Then we can talk about installs.

Stop asking "did it import?" as the finish line.
Start asking "which interpreter, which freeze, which file?"
Three answers. One JSON. Then the debate is real.

The agent can help type the commands.
You still own the receipt after that.
If those two roles blur, you are back in myth land.

If you rehearse on a free server, commit the receipt.
That is the only ask.

Top comments (0)