I Kept Patching Local pytest. The Agent Had Run on Another Host

The chat returned a clean pytest summary, and the path even matched my repo name. Why would I question a relative path that looked exactly like the tree on my laptop? I still opened local files, edited assertions, and reran tests that had never been the failing process. Two days later a hostname print proved the agent had executed Python on a different machine.

These field notes reconstruct the debugging workflow I now use; the command blocks are a labeled reproduction rather than a production incident report. I needed a method that still works after the chat window makes remote stdout feel local. The rest of this piece is that method, including the part where my clever theories failed in public.

Hour 0–8: I treated the chat like a local terminal

Have you ever watched an agent emit pytest output and assumed it shared your interpreter? I did that for most of the first evening, because the module names were mine. The failure even cited tests/test_billing.py:88, a file that definitely exists on disk beside my editor. So I patched that line, ran pytest myself, and watched a very local green bar appear.

What I tried during those first hours looked completely reasonable, at least when I viewed each step in isolation:

1. I reran pytest tests/test_billing.py -q in my own terminal until the assertion I had just edited went green.
2. I asked the agent to run the same file again, and I compared the traceback line numbers by eye.
3. I hashed tests/test_billing.py locally, then asked the agent to hash the path it had reported.
4. I grepped both sides for BILLING_MODE, because the assertion mentioned a flag I thought was environment-only.

The hashes matched after a sync, which made me more confident even while it made me more wrong. Matching file bytes do not prove matching interpreters, matching environment variables, or even matching machines underneath. I already needed a host identity at that point, and I still did not ask for one. Have you noticed how chat UIs make remote stdout feel like it happened under your thumbs?

Hour 8–24: two Pythons, one familiar path

Overnight I convinced myself the flake was pytest-xdist, then cached .pyc files, then a dirty PYTHONPATH. None of those theories survived a boring print of sys.executable from the same process that ran pytest. My laptop used a virtualenv under .venv/bin/python, and the agent printed a different prefix entirely. The test file was the same, but the runtime was not even pretending to be local.

Here is the command I should have run once, before I edited a single failing assertion:

python3 -c "import os, sys, socket, platform; print('host=', socket.gethostname()); print('exe=', sys.executable); print('cwd=', os.getcwd()); print('py=', platform.python_version()); print('pid=', os.getpid())"

Example output on my laptop looked like the block below, captured during the labeled reproduction:

host= local-dev
exe= /home/dev/proj/.venv/bin/python
cwd= /home/dev/proj
py= 3.12.7
pid= 41822

Example output from the agent's run, pasted back into the chat, did not share the hostname or the executable prefix. After that paste, I finally stopped treating relative paths as any kind of proof of locality. A path like ./.pytest_cache can exist on two machines at once, and both will look equally sincere in Markdown.

What actually broke

The breakage was not a mysterious pytest plugin; it was a workflow split I had invited on purpose. I was iterating with MonkeyCode because free model access and a free server option let the agent run commands off my laptop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That combination helps when you want real command execution without making your laptop the only available host. It also creates a split-brain if chat transcripts never carry a host fingerprint beside the test summary. I kept applying fixes to the machine under my keyboard, while the failing process lived on the free server workspace.

These symptoms fooled me, so I am writing them down before I can pretend they were subtle:

• Relative paths in the tracebacks matched my repo layout character for character, which felt like evidence.
• The test names were identical, which only meant I had synced the tree before the run.
• The failing assertion mentioned an env flag I had exported locally and never confirmed remotely.
• The chat paraphrased a green or red summary without a session header I could pin to a host.

Did the agent lie to me in the transcript? Not really, and that is the annoying part. I asked for pytest, received pytest, and then assigned every line of output to the wrong computer. The useful question was never "is billing broken." The useful question was "which interpreter just spoke."

Hour 24–36: a fingerprint the chat cannot shrug off

I wanted something a future agent run could print first, before it was allowed to emit any test report. The script below is the artifact for this write-up, and you can run it with the interpreter you are about to test. It prints identity, not verdicts, and it refuses to dump token values.

#!/usr/bin/env python3
"""host_fingerprint.py — print a stable, secret-free identity for this process."""

from __future__ import annotations

import hashlib
import os
import platform
import socket
import sys
from pathlib import Path


def _hash_exe(path: str) -> str:
    digest = hashlib.sha256()
    try:
        with open(path, "rb") as handle:
            for chunk in iter(lambda: handle.read(65536), b""):
                digest.update(chunk)
        return digest.hexdigest()[:12]
    except OSError as exc:
        return f"unreadable:{exc.__class__.__name__}"


def fingerprint() -> dict[str, str]:
    exe = sys.executable
    cwd = Path.cwd().resolve()
    env_names = sorted(
        key
        for key in os.environ
        if key.startswith(("PY", "VIRTUAL", "PATH"))
        and "TOKEN" not in key
        and "SECRET" not in key
        and "KEY" not in key
    )
    return {
        "hostname": socket.gethostname(),
        "fqdn": socket.getfqdn(),
        "pid": str(os.getpid()),
        "cwd": str(cwd),
        "python_version": platform.python_version(),
        "implementation": platform.python_implementation(),
        "executable": exe,
        "executable_hash": _hash_exe(exe),
        "prefix": sys.prefix,
        "base_prefix": sys.base_prefix,
        "venv": str(sys.prefix != sys.base_prefix),
        "platform": platform.platform(),
        "uid": str(getattr(os, "getuid", lambda: "nt")()),
        "env_names": ",".join(env_names[:20]),
    }


def main() -> None:
    data = fingerprint()
    width = max(len(k) for k in data)
    print("=== host fingerprint (not a test result) ===")
    for key, value in data.items():
        print(f"{key:<{width}}  {value}")


if __name__ == "__main__":
    main()

I also drop a tiny pytest hook so a green bar cannot appear without the fingerprint. Save this as tests/conftest.py if you do not already own that file, or merge the hook into the existing one.

# tests/conftest.py
import json
from pathlib import Path

import pytest

from host_fingerprint import fingerprint


@pytest.hookimpl(tryfirst=True)
def pytest_sessionstart(session):
    payload = fingerprint()
    out = Path(session.config.rootpath) / ".host_fingerprint.json"
    out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
    print("\n[host-fingerprint]")
    for key in ("hostname", "executable", "cwd", "python_version", "venv"):
        print(f"  {key}={payload[key]}")
    print("[/host-fingerprint]\n")

Run the commands below on whichever host you currently believe is the active one:

python3 host_fingerprint.py
python3 -m pytest tests/test_host_contract.py -q -s
cat .host_fingerprint.json

Run both commands from the repository root, because the hook writes .host_fingerprint.json beside the checkout. If the chat cannot paste those five keys, I now refuse to debug the assertion. That rule sounds petty until you remember the way I burned the entire previous evening.

A decision table I now keep above the fold

Observation	Do not assume	Check next
Traceback path matches your repo	The process is local	hostname and sys.executable
File hashes match after a sync	Runtimes match	python_version, prefix, venv flag
pytest is green on your laptop	The agent's failure is stale	Diff .host_fingerprint.json from both runs
Chat says the suite ran	Tests executed on a known host	Fingerprint printed before the summary
An env flag "is set" in your shell	The test process inherited it	Print selected os.environ keys from pytest
Coverage or JSON reports appeared	You are reading the producing host	Hash the report against that session's fingerprint

I now keep this small table above the fold in the README for agent-run tests. The table is intentionally boring, because clever theories are how I lost the first day. Boring checks like hostname and executable would have ended this incident before hour eight arrived. Would you have kept editing assertions after two mismatched executable prefixes?

Hour 36–48: what I would repeat, and what I would not

Would I still use a free remote execution host for agent commands after this kind of mess? Yes, I would, when the job is to run a test on a machine I can throw away. I would not use it as an invisible substitute for my laptop without a fingerprint in the transcript.

The free model access is handy for drafting the test plan, and the free server option is handy for actually executing that plan. Mixing those two without labeling the host is how I invented a 48-hour bug that was never in the assertion. Have you drawn that line between drafting and executing, or do the two still collapse in one chat?

These are the habits I would repeat on the next project, and I am not planning to debate them:

• I will print a host fingerprint as the first output of any agent-run test command.
• I will store .host_fingerprint.json beside the reports, then diff that file before opening HTML coverage.
• I will keep local-pytest.log and remote-pytest.log as separate files, and I will refuse to concatenate them.
• After I sync the tree, I will verify executable_hash instead of stopping at the test file hash.
• I will ask the agent to quote cwd, hostname, and sys.prefix in the same code fence as pytest.

These are the habits I will not repeat, even when the chat window sounds completely certain:

• I will not debug an assertion only because the traceback path looked familiar on my disk.
• I will not export secrets into a remote run just to make two environments match faster.
• I will not trust a coverage folder that was never hashed against the session fingerprint file.
• I will not let the chat paraphrase pytest output when I still need the raw session header.

A minimal reproduction you can run without billing tests

This is a labeled reproduction you can run locally, not a claim about production traffic. Create tests/test_host_contract.py, then run that file on each host you think you are using.

# tests/test_host_contract.py
import json
from pathlib import Path


def test_fingerprint_file_exists_and_names_a_host():
    path = Path(".host_fingerprint.json")
    assert path.is_file(), "pytest_sessionstart should have written the fingerprint"
    data = json.loads(path.read_text(encoding="utf-8"))
    assert data.get("hostname"), "hostname missing"
    assert data.get("executable"), "executable missing"
    assert data.get("cwd"), "cwd missing"

If this test fails, the session hook never ran on the interpreter you actually launched. If it passes on both machines with different JSON bodies, you finally have the bug I spent two days refusing to name. Do not merge those JSON files into one pretty summary, because the whole point is the mismatch.

Limitations, and who should skip this

This fingerprint does not prove two hosts are safe, equivalent, or allowed to see your code. It only proves the two runs are not the same process, which is a much smaller claim. Identical container images can share hostnames if you never set them, and then the table above goes quiet. Hashing sys.executable also fails on some stub launchers, especially on Windows, where the file you open is not the real runtime.

Do not treat the JSON file as an audit log, because it is only a debugging aid. I strip token-like names with a crude filter, and that filter is not a security boundary. Machine names and home-directory paths can still leak if you paste the file into a public issue. If you handle regulated data, a free remote execution host may be the wrong place to send the tree, fingerprint or not.

Who should skip this approach:

• Skip this if your agent already runs only inside a single local checkout that you control.
• Skip this if your team needs reviewed isolation, data-handling agreements, and a real CI identity.
• Skip this if you hoped a hostname print would replace pinning dependencies and Python versions.
• Skip this if you are about to paste fingerprints that include home directories into a public issue.

The method also will not help if you never read the fingerprint and just keep arguing with the assertion. I speak from the kind of stubbornness that argues with line 88 while two interpreters wait politely. A fingerprint is only useful when you let it veto the next edit.

I am not going to pretend a remote agent host is always faster, cheaper, or smarter than a local venv. I only needed the fingerprint so the remote host would stop impersonating my laptop in chat. If you already bounce work between a laptop and a remote agent host, steal the fingerprint script. The rest of this write-up is optional color, and I honestly wish my evening had been optional too.