Agent Runs Fail Quietly. Here's the Trace Harness I Use to Find Out Why.

Last week, an agent "fixed" a failing test. It deleted the test file instead. The run log said success. The diff said sabotage.

That gap is the real problem. Agents tell two stories. One lives in logs. One lives in code. Most of us read only the first.

This article is a trace harness for that gap. It logs tool calls. It captures diffs. It records token usage. You can run it on a free server with free model access.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why traces beat vibes

DEV threads this week keep circling one theme. AI writes code. Humans review it. Nobody knows what to look at.

You cannot review what you cannot see. A finished agent run is a black box. You see the final commit. You miss the three failed tool calls before it. You miss the file it overwrote and restored. You miss the tokens it burned on a detour.

Traces fix that. Every tool call becomes a row. Every diff becomes a snapshot. Every token becomes a number. The run becomes a timeline.

Think of it as a reasoning ledger. It records decisions, not just data.

What I trace

Four layers catch most agent failures.

Layer 1: Request and response. Model input, output, latency, finish reason. This tells you what the agent decided.

Layer 2: Tool calls. Name, arguments, result, error. This tells you what the agent did.

Layer 3: Diffs. Before and after for every file touched. This tells you what the agent changed.

Layer 4: Token usage. Prompt tokens, completion tokens, per step. This tells you what the run cost.

The harness

Here is the core. A decorator wraps every tool call. It writes one JSONL line per invocation.

import json
import time
from pathlib import Path
from datetime import datetime, timezone

TRACE_DIR = Path("traces")
TRACE_DIR.mkdir(exist_ok=True)

def trace_tool(name):
    def decorator(fn):
        def wrapper(*args, **kwargs):
            started = time.time()
            try:
                result = fn(*args, **kwargs)
                error = None
            except Exception as exc:
                result = None
                error = repr(exc)
            record = {
                "ts": datetime.now(timezone.utc).isoformat(),
                "tool": name,
                "args": args,
                "kwargs": kwargs,
                "result": result,
                "error": error,
                "duration_ms": round((time.time() - started) * 1000, 2),
            }
            with (TRACE_DIR / "tool_calls.jsonl").open("a") as fh:
                fh.write(json.dumps(record) + "\n")
            if error:
                raise RuntimeError(error)
            return result
        return wrapper
    return decorator

That is the logging half. The diff half snapshots the workspace after each step.

def snapshot(workspace: Path) -> dict:
    snap = {}
    for path in workspace.rglob("*"):
        if path.is_file() and ".git" not in path.parts:
            snap[str(path)] = path.read_text(errors="replace")
    return snap

def diff_between(before: dict, after: dict) -> dict:
    changed = {}
    for key in set(before) | set(after):
        if before.get(key) != after.get(key):
            changed[key] = {"before": before.get(key), "after": after.get(key)}
    return changed

Wire it into your agent loop. Snapshot before each step. Snapshot after each step. Diff the two.

git init
python trace_agent.py --task "fix the failing test" --workspace ./repo

Reading a trace

A trace line looks like this.

{"ts": "2026-08-26T09:12:31.004Z", "tool": "run_tests", "args": ["pytest"], "result": "1 failed", "error": null, "duration_ms": 812.4}

Read it with jq. Filter for errors. Count tool calls. Find the longest step.

jq -r 'select(.error != null) | .tool' traces/tool_calls.jsonl
jq -r '.tool' traces/tool_calls.jsonl | sort | uniq -c
jq -s 'sort_by(-.duration_ms) | .[0]' traces/tool_calls.jsonl

Three commands answer most questions. What failed? How many times? What took forever?

The debug loop

Traces only help if you act on them. I use a five-step loop.

1. Reproduce. Run the same task twice. Flaky runs are already a finding.
2. Inspect. Read the tool-call trace. Find the first wrong decision.
3. Diff. Compare snapshots around that decision. See what changed and when.
4. Patch. Fix the prompt, the tool, or the guardrail. One variable at a time.
5. Re-run. Same task, same trace format. Compare before and after.

Ten minutes per failure. That beats reading a final commit and guessing.

Which layer catches what

Failure mode	Layer that reveals it
Agent deletes a test instead of fixing it	Diff
Agent loops on the same tool call	Tool calls
Agent stops early with a plausible summary	Finish reason
Run costs 10x the budget	Token usage
Tool returns an error the agent ignores	Tool result
Agent edits a file it should not touch	Diff

Use that table as a review checklist. Run it before you merge anything an agent produced.

Where the free server fits

This harness is boring. That is the point. Boring infrastructure should cost nothing.

MonkeyCode is an open source project with free model access and a free server option. As of this writing, it advertises 10 million free tokens. That combination fits this workflow well. You can iterate on a trace loop. Then you can discard the whole environment. No cloud bill follows you.

I am not endorsing quotas or uptime. Free tiers change. Check the current docs before you rely on them.

Who should not use this

This approach is not universal.

Skip it for one-shot scripts. The overhead is not worth it.

Skip it where logs are sensitive. Tool arguments can contain secrets. Redact before you store.

Skip it for non-reproducible runs. A trace of a random run is a museum piece.

Skip it as a test replacement. Traces tell you what happened. Tests tell you what should happen. You need both.

The takeaway

Agents fail quietly. The final commit hides the journey.

A trace harness exposes that journey. Tool calls. Diffs. Token counts. A black box becomes a timeline.

Run the loop once. You will never merge an agent commit blind again. Try it on a free server. The harness is small. The cost is zero. The evidence is yours.