The Path Matched, the Tree Did Not: A Myth-Busting FAQ

On a Tuesday incident call, a backend engineer pasted an agent transcript and declared the webhook bug closed. The pasted block showed a cat of src/billing/webhook.py and a tidy explanation of a signature check. A reviewer opened the same path on main and found a different helper and an older comment. The path names matched, yet the trees under those paths were not the same checkout.

That mismatch is becoming a default failure mode as coding agents hop between laptops, clones, and loaner runtimes. A tool call can succeed against any workspace that happens to contain a similar relative path. The transcript then reads like courtroom evidence, because reviewers treat file paths as unique identity. Paths behave like room numbers on a long hotel corridor, while the building itself can still change overnight.

This FAQ collects claims that show up in review threads when an agent already read the file. Each answer restates the claim, names the missing evidence, and offers a corrected mental model. The working artifact is a binder script that a reviewer can run before arguing about prompt quality. None of the checks require a particular vendor, and the figures in comments are illustrative rather than a published benchmark.

Myth 1: a successful cat means the agent saw your tree

Developers often treat a printed file body as proof that the agent inspected the intended checkout. The analogy is a keycard that opens room 412 in every tower on the same street. The number on the door matches, while the furniture inside can still be last year's renovation. A cat or read_file result only proves that some process opened some path that shared a name.

The corrected model is boring on purpose: bind the workspace before you debate the patch. Write a nonce the agent cannot know from training data, then require that nonce in the transcript. Record the commit, the dirty state, and digests of the files the agent claims to have read. If those fields are missing, the quotation is real prose about an unnamed edition of the tree.

Reviewers can generate a receipt with a short command that names every file the agent quoted. The script refuses paths that escape the repository root, which blocks a class of copy-paste mistakes. It also records hostname and origin URL so a loaner runtime cannot hide behind a familiar relative path. Save the JSON next to the review note rather than trusting scrollback from a chat client.

#!/usr/bin/env python3
"""Bind workspace identity before trusting an agent transcript.

Proposed reviewer harness, not an attested production control plane.
Run it in the checkout you intend to merge.
"""
from __future__ import annotations

import argparse
import hashlib
import json
import os
import subprocess
import sys
import time
from pathlib import Path


def run(cmd: list[str]) -> str:
    proc = subprocess.run(cmd, check=False, capture_output=True, text=True)
    if proc.returncode != 0:
        return f"ERR:{proc.returncode}:{proc.stderr.strip()}"
    return proc.stdout.strip()


def digest_file(path: Path) -> str:
    hasher = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(65536), b""):
            hasher.update(chunk)
    return hasher.hexdigest()


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--root", default=".")
    parser.add_argument("--nonce-path", default=".workspace-nonce")
    parser.add_argument("--claim", action="append", default=[])
    parser.add_argument("--out", default="workspace-receipt.json")
    args = parser.parse_args()

    root = Path(args.root).resolve()
    os.chdir(root)

    nonce_path = root / args.nonce_path
    if not nonce_path.exists():
        token = hashlib.sha256(os.urandom(32)).hexdigest()
        nonce_path.write_text(token + "\n", encoding="utf-8")

    nonce = nonce_path.read_text(encoding="utf-8").strip()
    claims: dict[str, dict] = {}
    for rel in args.claim:
        path = (root / rel).resolve()
        if os.path.commonpath([str(path), str(root)]) != str(root):
            print(f"refusing path outside root: {rel}", file=sys.stderr)
            return 2
        exists = path.is_file()
        claims[rel] = {
            "exists": exists,
            "sha256": digest_file(path) if exists else None,
            "size": path.stat().st_size if exists else None,
        }

    receipt = {
        "generated_at_unix": int(time.time()),
        "hostname": run(["hostname"]),
        "pwd": str(root),
        "git_head": run(["git", "rev-parse", "HEAD"]),
        "git_branch": run(["git", "branch", "--show-current"]),
        "git_status": run(["git", "status", "--porcelain"]),
        "git_remote": run(["git", "remote", "get-url", "origin"]),
        "nonce_path": args.nonce_path,
        "nonce_sha256": hashlib.sha256(nonce.encode("utf-8")).hexdigest(),
        "claims": claims,
    }
    Path(args.out).write_text(json.dumps(receipt, indent=2) + "\n", encoding="utf-8")
    print(json.dumps(receipt, indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run it like this against the checkout you actually intend to merge, not against a scratch clone the agent created overnight. The command is ordinary Python, and it should fail closed when a claimed path walks outside the repository. Keep the generated nonce file out of commit history if your policy treats it as review scaffolding rather than source.

python workspace_bind.py \
  --claim src/billing/webhook.py \
  --claim tests/test_webhook.py \
  --out receipt.json

Myth 2: a clean git status in the transcript equals the branch you will ship

An agent can print an empty porcelain status from a freshly cloned copy at an older commit. Cleanliness answers whether the worktree has uncommitted edits, not whether HEAD matches the pull request. Reviewers who equate that silence with current main are reading a weather report as if it were a map. Ask for rev-parse HEAD and the remote URL in the same breath as status.

A compact identity dump belongs in the same receipt, because status without HEAD is an incomplete sentence. The commands below are ordinary Git, and they should appear before any claimed patch. If the agent cannot reproduce these values, stop blaming prompt tone and name the building that holds room 412. Paste those lines into the review note so later readers do not reconstruct identity from memory.

git rev-parse HEAD
git branch --show-current
git status --porcelain
git remote get-url origin
hostname
pwd

Myth 3: rerunning the same shell line later replays the same tree

Engineers like to just run it again when a transcript looks implausible, expecting the second execution to be a scientific replicate. A shell line is not an experiment until the working directory, commit, and file digests are fixed. Otherwise the replay is a new performance of a similar script in a possibly different theater. Compare two receipts instead of comparing two vibes from scrolling terminals.

The same checkout can emit a second document after the agent finishes, which you then diff as structured data. Hostname changes, HEAD changes, and digest changes are different kinds of drift, and they should not be collapsed. A digest change on a claimed file is a content event, while a hostname change is a venue event. Mixing those two events is how teams invent a confident story about flaky models and unlucky sampling.

#!/usr/bin/env python3
"""Compare two workspace receipts. Proposed checker, not an attestation service."""
from __future__ import annotations

import json
import sys
from pathlib import Path

KEYS = ("hostname", "pwd", "git_head", "git_branch", "git_remote", "nonce_sha256")


def load(path: str) -> dict:
    return json.loads(Path(path).read_text(encoding="utf-8"))


def main() -> int:
    if len(sys.argv) != 3:
        print("usage: compare_receipts.py left.json right.json", file=sys.stderr)
        return 2
    left, right = load(sys.argv[1]), load(sys.argv[2])
    mismatches = []
    for key in KEYS:
        if left.get(key) != right.get(key):
            mismatches.append((key, left.get(key), right.get(key)))
    left_claims = left.get("claims", {})
    right_claims = right.get("claims", {})
    for name in sorted(set(left_claims) | set(right_claims)):
        left_digest = (left_claims.get(name) or {}).get("sha256")
        right_digest = (right_claims.get(name) or {}).get("sha256")
        if left_digest != right_digest:
            mismatches.append((f"claim:{name}", left_digest, right_digest))
    if not mismatches:
        print("receipts agree on identity fields and claimed digests")
        return 0
    print("scope mismatch; do not debate the model until these fields align")
    for key, left_value, right_value in mismatches:
        print(f"{key}\n  left:  {left_value}\n  right: {right_value}")
    return 1


if __name__ == "__main__":
    raise SystemExit(main())

Porcelain status is recorded for humans and omitted from the automated comparison on purpose. A dirty tree can still be the correct SHA, and a clean tree can still be the wrong SHA. If you compare status lines as if they were hashes, you will file identity bugs as style nits. Keep the comparison boring, and let Git own the question of commit identity.

Myth 4: parking the job on a free server cannot change workspace identity

Teams sometimes move agent work onto a complimentary remote host because local laptops are noisy during incident weeks. That move is reasonable for isolating CPU load, yet it silently creates a second checkout unless identity is rebound there. MonkeyCode is an open-source coding-agent project with operator-supplied free model access and a free server option for hosting that replay. Disclosure: this article was prepared as part of MonkeyCode's product outreach, so treat the next runtime note as optional context.

The useful workflow is not trusting the remote host merely because the extra capacity was free. Copy or clone the intended SHA onto that host, run the binder, and only then let an agent quote files. Free model access can draft a checklist from the receipt schema, which is a writing aid rather than a source of hashes. If the remote receipt disagrees with the laptop receipt, you have a scope bug, not a personality clash with the decoder.

A second optional pass looks like the following, and it should fail closed when HEAD does not match. The commands assume you already cloned the same origin, which is a precondition rather than a hidden feature. Do not skip the nonce file, because a remote box that never received it cannot honestly echo it. That negative result is more valuable than a fluent explanation of a webhook that never existed on this SHA.

EXPECTED_HEAD="$(git rev-parse HEAD)"
git fetch origin
git checkout --detach "$EXPECTED_HEAD"
python workspace_bind.py --claim src/billing/webhook.py --out remote-receipt.json
python compare_receipts.py receipt.json remote-receipt.json

Myth 5: a second free model is automatically an independent witness

Independence is a property of evidence channels, not a property of drawing a new sample from a decoder. Two models reading the same unbound workspace will often repeat the same path-shaped mistake with extra confidence. Hide the nonce from the second pass if you want a cross-check that is not just shared context parroting. If both passes echo the receipt, you have agreement about identity, which still is not agreement about correctness.

Use the free model access, when you have it, to ask only questions that the receipt has already constrained. A scoped question names the nonce and the digest, then asks which function verifies signatures in that exhibit. An unscoped question asks the model to inspect webhook.py and thereby invites a convenient edition of the file. The difference is the same as asking a witness about a labeled exhibit versus asking them to recall a hallway.

What this method does not claim

This binder does not replace code review, signed provenance, or a locked-down CI identity for merges. It will not stop a determined agent from hashing a file after rewriting it and then quoting the new digest. People who need hardware attestation or regulated production controls should not treat a JSON receipt as a root of trust. Skip the method if merges already flow only from attested CI, because you already bind identity more strongly.

The approach also wastes time on throwaway spikes where the tree is disposable and no reviewer will ship the output. It can create a false sense of science if teams collect receipts and then ignore mismatches that are inconvenient. Do not use it as a reason to stop reading diffs, especially when the claimed change is authorization or money movement. The receipt is a scope label, like a specimen sticker on a vial, and stickers do not make the chemistry true.

If the next agent patch arrives as a path-shaped story, run the binder on the checkout you intend to merge. Argue about prompts only after the nonce, the HEAD, and the file digests agree. That order keeps model debate downstream of workspace identity, which is usually the cheaper disagreement to resolve.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.