After Each Tool Span, Hash the Working Tree

A coding agent prints Task complete at 02:14.
The trace shows one successful apply_patch span.
The unit tests still fail on the next command.

The log looks honest. The tree is not.
The span recorded a tool name and a short result string.
It never recorded the bytes that landed on disk.

This gap is now common in agent debugging.
Model quality debates miss it.
The missing join sits between the tool span and the working tree.

The failure is a join, not a vibe

Public threads still argue whether models already outcode most developers.
That debate does not help a failing patch land.
You need a join key from each tool span to the files it touched.

A green span only means the wrapper returned.
It does not mean the file changed.
It does not mean the change matches the argument digest.

Treat three records as one debug unit:

• the tool span id and name
• a stable hash of the tool arguments
• a hash of selected paths after the call

If any piece is missing, stop reading the dashboard.
Start reading the join file.

A scene you can reproduce

Create a tiny repo. One module. One test. One agent tool.
The tool claims it patched tax.py.
The test still expects the old rounding path.

workspace/
  tax.py
  test_tax.py
  traces/joins.jsonl

tax.py before the run:

def round_cents(amount: float) -> int:
    return int(amount * 100)

test_tax.py:

from tax import round_cents


def test_round_cents_bankers() -> None:
    assert round_cents(1.225) == 122

The agent wrapper returns {"ok": true, "path": "tax.py"}.
Pytest still fails.
Without a tree hash, you debug the model.
With a tree hash, you debug the write.

Artifact: span-to-tree join records

Label the next block as a local helper, not a production SDK.
It is a proposal you can run in a scratch repo.
It writes one JSONL line after every tool call.

# join_trace.py — example helper, run locally
from __future__ import annotations

import hashlib
import json
import time
from pathlib import Path
from typing import Callable, Iterable

JOIN_PATH = Path("traces/joins.jsonl")
WATCH = ("tax.py", "test_tax.py")


def sha256_bytes(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()


def hash_args(payload: object) -> str:
    blob = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    return sha256_bytes(blob.encode("utf-8"))


def hash_tree(root: Path, rel_paths: Iterable[str]) -> dict[str, str]:
    out: dict[str, str] = {}
    for rel in rel_paths:
        path = root / rel
        if not path.is_file():
            out[rel] = "MISSING"
            continue
        out[rel] = sha256_bytes(path.read_bytes())
    return out


def append_join(record: dict) -> None:
    JOIN_PATH.parent.mkdir(parents=True, exist_ok=True)
    with JOIN_PATH.open("a", encoding="utf-8") as handle:
        handle.write(json.dumps(record, sort_keys=True) + "\n")


def call_with_join(
    root: Path,
    span_id: str,
    tool_name: str,
    args: dict,
    tool: Callable[[dict], dict],
) -> dict:
    before = hash_tree(root, WATCH)
    started = time.time_ns()
    result = tool(args)
    ended = time.time_ns()
    after = hash_tree(root, WATCH)
    changed = sorted(path for path in WATCH if before.get(path) != after.get(path))
    append_join(
        {
            "span_id": span_id,
            "tool_name": tool_name,
            "args_sha256": hash_args(args),
            "result_sha256": hash_args(result),
            "before": before,
            "after": after,
            "changed_paths": changed,
            "started_ns": started,
            "ended_ns": ended,
        }
    )
    return result

The helper does not need a vendor SDK.
It needs a root path, a watch list, and one append.
Keep the watch list small. Hashing the whole monorepo is waste.

A fake tool that lies without writing

This example is labeled and unexecuted in this article.
It shows a wrapper that returns success and writes nothing.

# example_only.py — labeled example, not a live agent
from pathlib import Path

from join_trace import call_with_join


def apply_patch(args: dict) -> dict:
    # Looks complete. Does not touch tax.py.
    return {"ok": True, "path": args["path"], "bytes": 214}


if __name__ == "__main__":
    root = Path(".")
    call_with_join(
        root,
        span_id="span-01",
        tool_name="apply_patch",
        args={"path": "tax.py", "patch": "use bankers rounding"},
        tool=apply_patch,
    )

Run it, then read the JSONL line.
changed_paths will be empty.
after["tax.py"] will match before["tax.py"].
The span can still say ok.

That is the bug class.
The model may have planned a patch.
The tree never moved.

The debug loop

Do not start with prompt edits.
Start with a four-step join check.

1. Load the latest join line for the failing span.
2. Confirm args_sha256 matches the prompt’s tool payload.
3. Confirm changed_paths is not empty when a write was claimed.
4. Confirm after hashes match a second hash taken by hand.

# check_join.py — example checker
import json
from pathlib import Path

from join_trace import WATCH, hash_tree

line = Path("traces/joins.jsonl").read_text(encoding="utf-8").splitlines()[-1]
rec = json.loads(line)
live = hash_tree(Path("."), WATCH)

mismatches = [p for p in WATCH if rec["after"].get(p) != live.get(p)]
claimed_write = rec["tool_name"] in {"apply_patch", "write_file"}
no_change = rec["changed_paths"] == []

print("span_id", rec["span_id"])
print("claimed_write", claimed_write)
print("changed_paths", rec["changed_paths"])
print("after_vs_live", mismatches)
print("lie_score", int(claimed_write and no_change))

Interpret the checker with a small table.
Do not treat the table as a benchmark.
It is a decision aid for one repo.

claimed write	changed_paths	after vs live	next action
yes	empty	match	wrapper never wrote; fix the tool
yes	nonempty	mismatch	later process mutated the tree
yes	nonempty	match	tree moved; debug the patch content
no	nonempty	match	unexpected mutation; find the extra writer
no	empty	match	no fs side effect; look at HTTP or memory

Most “the model is dumb” nights fall in row one.
The span was green.
The file digest never changed.

Commands for a clean replay

Keep the commands boring. Boring commands survive on-call.

git status --short
python -m pytest test_tax.py -q
python example_only.py
python check_join.py
sha256sum tax.py test_tax.py

If git status is dirty and changed_paths is empty, your tracer is blind.
If both are clean and tests fail, the patch never targeted the assertion.
If after and sha256sum disagree, something wrote after the span closed.

Record that third writer as its own span.
Formatters, codegen, and test fixtures belong in the join file.
Hidden writers make agent diffs look random.

Where a free remote loop helps

Local joins catch silent writes.
Remote joins catch “works on my laptop” trees.
Same JSONL schema. Different disk.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.
Those two facts are the only product claims used here.
No model names, quotas, or hardware details are assumed.

A useful remote setup is narrow.
Copy the watch list, the helper, and the failing test.
Run the same tool wrapper against the same files.
Compare args_sha256 and after hashes across hosts.

If the hashes match and tests still disagree, the environment is the bug.
If the hashes differ, the write path is the bug.
Either result is cheaper than another prompt rewrite.

Use the remote box as a second witness, not as a demo.
Keep secrets out of the watch list.
Keep the join file next to the test output.

What this method refuses to claim

It does not score models.
It does not prove a patch is correct.
It only proves whether the tree moved with the span.

Hash collisions are not the practical risk.
The practical risk is a bad watch list.
Ignore node_modules and build output.
Ignore caches and compiled artifacts.
Watch source and tests only.

Timestamps are not hashes.
Clock skew already mis-orders spans in other writeups.
A content hash still works when clocks drift.
Use nanoseconds only as a duration hint.

Do not store raw file contents in JSONL by default.
Hashes are enough for the join.
Contents belong in git, not in the trace stream.

Limits

This loop fails on generated binaries.
It fails when tools write outside the watch list.
It fails when two tools share one file in one tick.

It also fails for pure network agents.
If the side effect is a ticket or a mail, hash the API body instead.
The same join idea applies. The code above does not.

Path normalization can lie on mixed OS replay.
Store relative POSIX paths in the join file.
Resolve them with pathlib on each host.

Large trees make naive hashing slow.
Cap the watch list.
If you need more files, hash a manifest, not the repo.

Who should not use this

Skip this if you already emit file digests on every OpenTelemetry span.
Skip this if the agent must not read the workspace.
Skip this if the tree holds production secrets.

Skip this for one-shot chat with no tools.
There is no working tree to join.
A message log is enough.

Also skip it when legal logging rules forbid file fingerprints.
A hash of a confidential file is still a fingerprint.
Get a policy review first.

A short checklist before the next agent run

• Watch source and tests only.
• Hash arguments, results, and selected paths.
• Fail the run when a write tool changes nothing.
• Re-hash by hand after the span closes.
• Compare local and remote join lines.

The agent can still be wrong after a clean join.
That is a patch-content problem.
It is no longer a missing-write problem.

Separate those two bugs.
The dashboard will look less finished.
The tree will finally show up in the trace.

If you need a second machine for the same JSONL join, MonkeyCode’s free server option can host the watcher. Keep the disclosure in mind, and keep the hashes in git.