Turn Agent Tool Calls Into Run Receipts

The fastest way to debug a failing agent is not a better prompt; it's a better record of what the tools changed. Chat logs capture what the model said. A run receipt captures what the workspace did. Keep the second, and the first becomes optional.

Most agent debugging starts too late. The run fails, the transcript scrolls away, and you're left reconstructing history from git blame and guesswork. That is archaeology, not debugging. Observability for an agent run has to be written down at the moment of each tool call, by a wrapper that does not care which model produced the call.

Think of a courier. The tracking page says "in transit" and then "delivered", and nobody can tell you which driver stopped where or who signed. An agent is worse, because each step mutates a shared filesystem. A courier that issued a signed receipt per stop would be simple to audit. An agent that emits one JSON line per tool call is exactly that.

The minimal version needs three ingredients: a stable run id, a git workspace, and a wrapper around the tool-call boundary. The skeleton below is unexecuted example code, not a tested library; adapt it to your agent's loop.

// receipt.js — one JSONL record per tool call
import { execSync } from "node:child_process";
import { appendFileSync } from "node:fs";
import crypto from "node:crypto";

export const RUN_ID = crypto.randomBytes(4).toString("hex");
const LOG = `run-${RUN_ID}.jsonl`;

const snapshot = () => ({
  head: execSync("git rev-parse HEAD").toString().trim(),
  dirty: execSync("git status --porcelain").toString().split("\n").filter(Boolean).length,
});

const diffStats = () => {
  const out = execSync("git diff --numstat").toString().trim();
  if (!out) return { files: 0, added: 0, deleted: 0 };
  let files = 0, added = 0, deleted = 0;
  for (const line of out.split("\n")) {
    const [a, d] = line.split("\t");
    if (a !== "-") added += Number(a);
    if (d !== "-") deleted += Number(d);
    files++;
  }
  return { files, added, deleted };
};

export async function traceStep(step, tool, input, fn) {
  const before = snapshot();
  const start = Date.now();
  let error = null;
  try {
    await fn();
  } catch (e) {
    error = e.message;
  }
  const after = snapshot();
  const record = {
    run_id: RUN_ID, step, tool,
    input: String(input).slice(0, 500),
    duration_ms: Date.now() - start,
    head_changed: before.head !== after.head,
    dirty_before: before.dirty,
    dirty_after: after.dirty,
    diff: diffStats(),
    error,
    ts: new Date().toISOString(),
  };
  appendFileSync(LOG, JSON.stringify(record) + "\n");
  return record;
}

Four fields do most of the work. head_changed tells you whether this step moved the base commit; after that, every diff is measured against a different tree, so later changes must be read with suspicion. diff.added and diff.deleted are numeric diff stats, not patches; they are cheap to compute and enough to decide whether a step deserves a closer look. input is truncated to 500 characters because a receipt should point at evidence, not store it. Token counts come from the provider response, not the tool boundary, so attach them where your wrapper can read them.

Receipts only pay off when read. The replay script below prints one line per step, so a run of forty tool calls fits on one screen.

// replay.js <run-*.jsonl>
import { readFileSync } from "node:fs";

const rows = readFileSync(process.argv[2], "utf-8")
  .trim().split("\n").map(JSON.parse);

let failures = 0, changed = 0;
for (const r of rows) {
  if (r.error) failures++;
  if (r.diff.files || r.head_changed) changed++;
  const mark = r.error ? "X" : (r.diff.files || r.head_changed) ? "*" : ".";
  console.log(
    `${String(r.step).padStart(2)} ${mark} ${r.tool} ` +
    `+${r.diff.added}/-${r.diff.deleted} ${r.duration_ms}ms ${r.error ?? ""}`
  );
}
console.log(`steps=${rows.length} failures=${failures} touched=${changed}`);

Once the output is compact, three patterns become visible. An X row with non-zero diff means a tool modified files and then failed, which is worse than a clean failure: state changed and the model never saw the result. A . row with zero diff and a long duration is usually a retry loop wearing a thinking disguise. A head_changed row in the middle of a run invalidates the baseline for every later step; either the agent pulled new context on purpose, or it was told to, and you cannot tell from the transcript.

JSONL is the right medium here, and not because it is fashionable. Each line is append-only, so a crash mid-run leaves the earlier steps intact, and file mtime can reconstruct the sequence after the fact. A table wants a schema up front, which you do not have on step one. Raw logs are too noisy to grep. One object per tool call is the smallest shape that survives.

The loop itself stays boring on purpose. Replay the receipt first; pick the first row whose diff does not match the task; grep the JSONL for the file name to find which step touched it first; then rerun that single step with a narrower input. Memory is replaced by measurement.

Free endpoints make this workflow cheaper to run. MonkeyCode's open-source extension (GitHub) pairs free model access with a 10M-token allowance and a free server tier, which lowers the cost of many small experimental runs; that is exactly what run receipts reward, because each experiment leaves a small JSONL file instead of a vague memory. The trace format is independent of the endpoint, and it will outlive any one provider. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The receipt is not a full trace. It assumes a serial tool loop, a git workspace, and tools that mutate files in-process; background processes can escape the measurement, generated artifacts will drown the diff in noise, and the 500-character input cap hides details that matter. Add .gitignore discipline before you add this wrapper, or every run will look equally guilty.

Skip this if your agent makes no tool calls, if you only run single-step prompts, or if you have no habit of reading what you logged. A logging habit without a replay habit is just a bigger pile of files.

The experiment is cheap to reproduce: run the same failing task twice on MonkeyCode's free tier, keep both receipts, and compare. The model will likely disagree with itself. The receipt should not.