How I Debug a Misbehaving AI Coding Agent: My 4-Step Playbook

TL;DR

AI coding agents don't fail like normal software — they fail confidently, and the bug is usually three turns upstream from where the damage shows up. After months of running an autonomous Claude Code setup, I settled on a 4-step debug playbook: reproduce minimally → find the divergence turn in the transcript → diff tool reality vs model assumption → fix at the right layer. This post walks through the playbook with a real war story where my agent read a JSON error response and decided it was a success. 🐛

The Problem

When a regular program breaks, you get a stack trace. It points at a line. You fix the line.

When an AI agent breaks, you get... a finished task. A cheerful summary. Maybe even green tests. And then two days later you discover it "refactored" a config loader that never existed, or built a feature on top of an API call that has been returning 403 the whole time.

I run a fully autonomous implementation system built on Claude Code (mid-2026 builds, running on Node.js 22.x). It plans work, edits code, runs tests, and reports back — largely unattended. Which means when something goes wrong, there's no human in the loop who saw it happen. All I have is the aftermath and a transcript.

Early on, my "debugging" looked like this:

1. Notice weird output
2. Re-run the whole task and stare at it
3. Add an ALL-CAPS rule to the prompt like NEVER DO THE BAD THING
4. Watch it do the bad thing again next Tuesday

That's not debugging. That's superstition. ⚠️

The turning point was realizing that agent failures are almost never where the symptom is. The symptom is in the final output; the cause is a specific earlier turn where the model's internal picture of the world stopped matching reality. Everything after that turn is the model being perfectly logical about wrong facts.

So the job isn't "why is the output bad?" It's "at which exact turn did the model's beliefs and reality split?" That reframe gave me a repeatable process.

How I Solved It: The 4-Step Playbook

flowchart TD
    A[Symptom: bad output] --> B[Step 1: Reproduce with a minimal prompt]
    B --> C[Step 2: Walk the transcript, find the divergence turn]
    C --> D[Step 3: Diff tool reality vs model assumption]
    D --> E{Which layer failed?}
    E -->|Ambiguous instructions| F[Fix the prompt/docs]
    E -->|Model skipped verification| G[Add a deterministic check]
    E -->|Tool output was misleading| H[Fix the tool contract]
    F --> I[Re-run minimal repro to confirm]
    G --> I
    H --> I

Step 1: Reproduce with a minimal prompt

Resist the urge to re-run the full task. A 40-turn session is a haystack. Instead, extract the smallest prompt that still triggers the misbehavior.

When my agent mangled a database migration, the full task was "implement the new billing fields end to end." The minimal repro turned out to be just:

Read migrations/ and tell me the current schema version.

That single instruction reproduced the bug: the agent reported a schema version that didn't exist. Forty turns collapsed into one. Now I had something I could iterate on in seconds instead of minutes, and I knew the failure had nothing to do with billing logic at all.

Rule of thumb: if your repro takes more than 3 turns, keep cutting. The bug survives minimization far more often than you'd expect, because the bug is usually in perception (reading files, parsing tool output), not in the complicated reasoning you were worried about.

Step 2: Walk the transcript and find the divergence turn

Claude Code keeps full session transcripts (JSONL on disk — check your ~/.claude/projects/ directory). This is the flight recorder. Most people never open it. Open it.

I read transcripts with one question in mind: what did the model claim right after each tool call? You're looking for the first turn where the model's summary of a tool result doesn't match the raw result sitting right above it.

A trick that saves a lot of scrolling — pull out just the tool results and the assistant text that follows each one:

jq -r 'select(.type == "tool_result" or .type == "assistant")
       | .content // .text' session.jsonl | less

Read it top to bottom like a diff between two narrators: the tools tell one story, the model tells another. The first place the stories disagree is your divergence turn. Everything downstream is fruit of the poisoned tree — don't waste time analyzing it.

Step 3: Diff what the tool returned vs what the model assumed

Here's my favorite war story, because it's so dumb and so instructive.

My agent needed to register a webhook with an internal service. The service replied:

{
  "ok": false,
  "error": "duplicate_endpoint",
  "message": "Endpoint already registered",
  "id": null
}

HTTP status: 200. Because of course the service returns errors with a 200. 🙃

The agent's very next message: "Webhook registered successfully. Moving on to the notification handler." It then spent six turns building a notification handler around a webhook id of null, including writing if (webhookId) guards that silently skipped the entire code path. Green tests, by the way — the guards made sure nothing ran, and nothing that doesn't run can fail.

At the divergence turn, the diff was brutally clear:

• Tool reality: "ok": false, "id": null
• Model assumption: registration succeeded, an ID exists

Why did it happen? The model pattern-matched on 200 + a JSON body + the word "registered" in the message field. Skim-reading. The same failure mode as a tired human at 2am, honestly.

This step is where you classify the failure. In my experience nearly every agent bug is one of three kinds:

1. Ambiguity bug — the instructions genuinely supported two readings, and the model picked the other one
2. Verification bug — the info was available, the model just didn't check it (my webhook story)
3. Tool contract bug — the tool's output actively invites misreading (also my webhook story — a 200-with-error API is a trap for humans and models alike)

Step 4: Fix at the right layer

This is the step everyone gets wrong, including past me. The instinct is to always fix with prompt rules: "ALWAYS check the ok field!" But each failure class has a correct layer:

Ambiguity bugs → fix the instructions. Rewrite the ambiguous sentence in your CLAUDE.md or task prompt. One precise sentence beats five warning paragraphs. If you find yourself writing a rule in caps, the underlying sentence is probably still ambiguous.

Verification bugs → add a deterministic check, not a prompt rule. Prompt rules are probabilistic; the model follows them usually. For anything where "usually" isn't good enough, put the check outside the model. I added a tiny response validator the agent must run after registration calls:

# validate-response.sh — fail loudly so the agent can't skim past it
ok=$(jq -r '.ok' response.json)
if [ "$ok" != "true" ]; then
  echo "REGISTRATION FAILED: $(jq -r '.error' response.json)" >&2
  exit 1
fi

A non-zero exit code is impossible to misread. The model can ignore a sentence in a JSON blob; it cannot ignore a failed command.

Tool contract bugs → fix the tool. If a tool returns errors as 200s, or dumps 4,000 lines when 10 matter, no amount of prompting fixes that permanently. I wrapped the offending service client so errors became actual errors. The bug never came back — for the agent or for me.

Then re-run your Step 1 minimal repro to confirm the fix. Because the repro is one turn, this takes seconds. If you skipped minimization, you're now re-running 40-turn sessions to test a one-line fix, and you'll stop verifying out of sheer boredom. Minimization is what makes the whole loop sustainable.

Lessons Learned

1. The symptom is downstream; debug upstream. The final bad output is almost never where the bug lives. Find the divergence turn and ignore everything after it.

2. Agents fail at perception more than reasoning. I expected to debug bad logic. What I actually debug, over and over, is bad reading: skimmed tool output, misparsed errors, assumed file contents. Optimize your debugging (and your tools) for perception failures.

3. Prompt rules are for ambiguity; exit codes are for safety. If a failure genuinely must not happen again, the fix is deterministic — a validator, a hook, a wrapper. Adding a prompt rule where you needed a hard check is the #1 way to meet the same bug twice.

4. A 200-with-error API will eventually fool your agent, guaranteed. Anything that fools a skimming human fools a model. Fixing tool contracts is the highest-leverage, least-glamorous agent debugging there is.

5. Transcripts are your stack trace — build the habit of reading them. The answer is almost always sitting in plain text in a JSONL file. Ten minutes of reading beats an hour of re-running and guessing.

What's Next

The playbook is manual today, and steps 2–3 are begging for automation. I'm experimenting with a "transcript skeptic": a second, cheaper agent pass that walks a finished session and flags turns where the assistant's claim doesn't match the preceding tool result. Early results are promising — it caught a misread test failure last week before I did. If it stabilizes, that's a future post.

Wrap-up

Agent debugging felt like voodoo until I stopped treating the model as the unit of failure and started treating turns as the unit of failure. Reproduce small, find the divergence, diff belief against reality, fix the right layer. It's just debugging — the stack trace is a transcript now.

If you're running Claude Code or any autonomous agent: next time it does something baffling, don't re-prompt. Open the transcript and find the turn. 💡

If this was useful, follow me here on Dev.to — I write regularly about building and running autonomous coding agents in production: the wins, the faceplants, and the playbooks that come out of them. And if you haven't yet, grab Claude Code and let an agent surprise you. Then debug it. 🚀

What's the most confidently wrong thing your agent has ever done? Tell me in the comments — I'm collecting war stories.