Shrink the Trace, Not the Bug: A Prefix Minimizer for Failing Agent Runs

Your CI job fails on Tuesday. The same agent run passed on Monday. No prompt change, no model bump, no new tool.

The trace holds 61 spans. A full replay takes four minutes. Two spans actually matter.

That gap is the debugging tax on agent work. This post closes it with one artifact: a prefix minimizer for failing traces.

The trace is readable. The failure is not.

Most teams log something. Fewer log enough to compare. Fewest log a verdict for a truncated run.

• Logs show text. You need digests of tool arguments and results.
• Spans show order. You need a stable sequence number per span.
• Replay shows drift. You need a post-condition, not a transcript diff.

A post-condition is boring, and that is the point. Example: /app/config.yaml hashes to sha256:9f2c.... Either it matches or the run failed.

Record spans so a machine can compare them

One JSONL line per span. Keep the line flat and hash the payloads.

{"seq": 14, "kind": "tool", "name": "write_file", "args_digest": "sha256:6d1f0c2a9b3e4a71", "result_digest": "sha256:0b8e5f3c1d2a7c90", "status": "ok", "duration_ms": 42}

Four rules pay off later:

1. seq is monotonic and never reused.
2. Digests use canonical JSON, so key order cannot lie.
3. status records failures, not only successes.
4. Payload bodies live in storage, keyed by their digest.

If your trace cannot answer "what did span 14 send and receive", stop here. Fix logging first.

Why full replay is the wrong default

Replaying 61 spans costs model calls, tool latency, and wall clock. Each extra span is another chance to flake. Long traces also hide the small causal set that actually breaks.

A minimizer gives you a shorter question. A ten-span prefix is something a human can read in a review.

Step 1: find the first diverging span

Compare the recorded trace against a replayed one. Report where they part ways.

import hashlib
import json


def digest(obj) -> str:
    blob = json.dumps(obj, sort_keys=True, separators=(",", ":")).encode()
    return "sha256:" + hashlib.sha256(blob).hexdigest()[:16]


def load_trace(path):
    with open(path) as f:
        spans = [json.loads(line) for line in f if line.strip()]
    return sorted(spans, key=lambda s: s["seq"])


def first_divergence(recorded, replayed):
    for a, b in zip(recorded, replayed):
        if a["args_digest"] != b["args_digest"]:
            return a["seq"], "args"
        if a["result_digest"] != b["result_digest"]:
            return a["seq"], "result"
    if len(recorded) != len(replayed):
        return min(len(recorded), len(replayed)), "length"
    return None, None

This tells you where the run changed. It does not tell you what the minimum trigger is.

Step 2: replay with a verdict, not a diff

Your harness takes a prefix and returns pass or fail. Wrap it in repeated trials, because agent runs wobble.

class CassetteHarness:
    """Replay tool calls from a recorded tape. Never touch production."""

    def __init__(self, tape, workspace, expected_hash):
        self.tape = tape
        self.workspace = workspace
        self.expected_hash = expected_hash

    def run(self, spans):
        self.workspace.reset()
        for span in spans:
            if span["kind"] == "tool":
                self.workspace.apply(span["name"], self.tape[span["seq"]])
        return "fail" if self.workspace.hash() != self.expected_hash else "pass"


def verdict(spans, harness, trials=3):
    fails = 0
    for _ in range(trials):
        try:
            if harness.run(spans) == "fail":
                fails += 1
        except TimeoutError:
            continue
    if fails == 0:
        return "pass"
    if fails == trials:
        return "fail"
    return "flaky"

Treat flaky as a signal, not an error. A flaky prefix usually means a race or a live external dependency.

Step 3: shrink the prefix with verification

Binary search assumes the failure is monotone: adding spans cannot unbreak it. Verify that assumption, then fall back to a linear scan.

def linear_scan(spans, harness, trials=3):
    for i in range(1, len(spans) + 1):
        if verdict(spans[:i], harness, trials) == "fail":
            return i
    raise SystemExit("no failing prefix found")


def shrink(spans, harness, trials=3):
    if verdict(spans, harness, trials) != "fail":
        raise SystemExit("not reproducible: collect more traces")
    lo, hi = 0, len(spans)
    while lo < hi:
        mid = (lo + hi) // 2
        if verdict(spans[:mid], harness, trials) == "fail":
            hi = mid
        else:
            lo = mid + 1
    if lo > 0 and verdict(spans[: lo - 1], harness, trials) == "fail":
        lo = linear_scan(spans, harness, trials)
    return spans[:lo]

Run it from the command line and write the minimal trace next to the original.

python -m trace_shrink traces/run-2291.jsonl --trials 3 --out min-2291.jsonl

Example output below is illustrative, not a benchmark:

loaded 61 spans
full trace      fail 3/3
prefix 30       fail 3/3
prefix 15       fail 3/3
prefix 7        pass 0/3
prefix 11       fail 3/3
prefix 9        pass 0/3
prefix 10       fail 3/3
minimal prefix: 10 spans (seq 1..10)
wrote min-2291.jsonl

Ten spans is a reviewable artifact. Attach it to the issue and stop arguing about the other 51.

The loop I reuse

1. Pull the failing trace and its post-condition hash.
2. Replay the full trace three times. Confirm it fails.
3. Run the minimizer until the prefix stops shrinking.
4. Read the last two spans of the minimal prefix by hand.
5. Convert the cause into a regression test with a fixed cassette.

Step five is the one teams skip. Without it, the same bug returns next month wearing a new seq.

Decision table

Symptom	First move	Why
Wrong final output, normal step count	Compare result digests per span	The bad value is usually local
Replay passes locally, fails in CI	Repeat trials, then diff environment	Flaky input, not flaky logic
Failure only on long runs	Shrink the prefix	Cuts cost and noise together
No failing prefix at all	Revisit logging and post-conditions	You cannot minimize what you cannot detect

Where a free tier fits, honestly

Minimization is a volume problem. Binary search on 61 spans means several replays, times three trials. That is a lot of cheap calls and a small amount of compute.

MonkeyCode's free model access and free server option fit that shape. The repeated trials are the cost driver. Hosting the replay worker and the trace store next to each other removes a network hop from every probe. The operator reports a free allowance of ten million tokens; confirm the current limits and terms in the project docs before you queue a batch, since quotas change.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

If you want to try the loop without a billing conversation, the free model access is the cheapest way to start. Keep the harness offline and the side effects stubbed.

Limitations and who should skip this

• Non-monotone failures. An early span can mask a later bug. The verifier catches some cases, not all.
• Irreversible side effects. Never replay deploy, payments, or email sends against real systems. Cassette them or refuse to replay.
• Stateful tools. A database cursor is not a pure function. Snapshot it or the verdict lies.
• Causally coupled spans. The minimal prefix may exclude a span that made the bug possible. Read the prefix, do not worship it.
• No post-condition. If success is "looks fine", this method has nothing to measure.

Skip it if your traces lack digests, or if your tool calls cannot be stubbed. Fix observability first. A minimizer on bad data just produces a shorter wrong answer.

The real win is not the tool. It is the shift from reading transcripts to asserting outcomes. Traces become evidence, not logs.