Why your AI coding agent keeps breaking prod (and the 3-layer fix)

Todd Doane — Sun, 18 Oct 2026 12:08:23 +0000

Claude Code, Cursor and Codex are excellent at single steps. Production work is not a single step. Here is the arithmetic of why agents fall apart on real tasks, and the structure we use to run a small company's platform with agents without babysitting them.

The symptom: brilliant in the demo, flaky on Tuesday

You ask the agent to deploy a fix. It edits the right file, writes a plausible commit message, runs the deploy, and then something small goes wrong: a CRLF line ending breaks a shell script on the server, an environment variable is read from the wrong file, a webhook key rotates on redeploy. The agent notices the failure, "fixes" it by retrying or resetting state, and reports success. Next week, in a fresh session, it does exactly the same thing.

None of this is the model being dumb. It is the shape of the work.

The arithmetic: errors compound

Suppose your agent gets each step right 90% of the time, which is generous for a step like "parse this API response and write the right rows". Chain steps together and the probabilities multiply:

Steps the model executes	Success rate
3 decisions, scripts do the rest	0.9³ = 73%
5 steps	0.9⁵ = 59%
10 steps	0.9¹⁰ = 35%

Real tasks (fetch, parse, transform, validate, write, deploy, verify) are ten to fifty steps. If the model executes every one of them by improvising code or commands in the moment, the success rate of the whole job collapses, and you end up watching it.

The fix is not a better prompt. It is moving the steps that have a right answer out of the model entirely.

The 3-layer fix: Directive, Orchestration, Execution

We call it D.O.E. Each layer does one kind of work, and the boundaries between them are the point.

Layer	Nature	Lives in	Changed by
Directive	Intent, in prose: goal, inputs, tools, outputs, edge cases, how to verify	`directives/*.md`	Humans write; agents append learnings
Orchestration	Judgment: which directive applies, which inputs, did it work	The agent; `CLAUDE.md` is its job description	You, rarely
Execution	Determinism: fetch, parse, transform, deploy	`executions/*.py`	Agents write; tests guard

A directive never contains code. A script never contains policy ("ask the user if…"). The agent never contains either: it reads one and runs the other. Now the model makes three decisions instead of fifty executions, and decisions are the part a model is actually good at. A script that parses a response correctly once parses it correctly forever.

What a directive looks like

# Directive: Deploy the API

## Goal
Ship the current branch to the API server and prove it's live.

## Execution tools
executions/deploy_service.py --target api   # normalizes CRLF/BOM, uploads, restarts

## Edge cases & learnings
- Shell scripts committed from Windows had CRLF endings and
  failed with "bad interpreter". deploy_service.py now normalizes them.
- The webhook key rotates on redeploy; the script refreshes it.

## Verification
GET /health returns 200 and the commit hash matches HEAD.

The Edge cases & learnings section is where the return on investment lives. Every gotcha the agent meets is written down where the next session reads it before acting, instead of being rediscovered at 2 a.m.

The part that makes it get better: self-annealing

Structure alone stops the bleeding. What makes month three dramatically better than week one is a rule about failure: recovery is not a fix. When something breaks, the agent runs a fixed loop, in this order:

Diagnose the root cause: keep asking "why" until you reach something that, once fixed, prevents recurrence.
Fix it in code, config or infrastructure.
Add the test that would have caught it.
Add a standard to the directive (and a lint, if the pattern is mechanically detectable).
Then recover: reprocess the stuck items, restart the service.

Most agent sessions do step 5 first and stop. "Let's just reprocess the stuck items" and "I'll add a cron to retry failures" feel like progress and leave the cause in place. Making recovery the last step means every failure leaves behind a test, a rule and sometimes a lint, so whole classes of mistakes stop being possible.

A real example from building our own store

While building the checkout for our store, a secrets linter found a live Stripe key hard-coded in an old helper script in our own repo. Removing that one key would have been recovery. The fix was making lint_secrets.py run before every commit and adding a line to CLAUDE.md requiring it. That kind of mistake can no longer reach git in that repo, whoever (or whatever) is typing.

How to start today, for free

Put a CLAUDE.md (or .cursorrules / AGENTS.md) at your repo root that states the three layers and the self-annealing loop.
Pick the one job your agent fails at most often. Write a directive for it with the six sections above, and point it at a script.
Move the deterministic parts of that job into the script. Give the script a test.
Tell the agent: "Read CLAUDE.md, then run the deploy directive." Watch what it appends to Edge cases & learnings.

We publish a free starter, Agent Ops Kit Lite on GitHub (MIT), with the CLAUDE.md, the operating-principles directive, the directive template and the secrets linter. It is enough to try the approach on one job this afternoon.

Originally published at calyx-ai.net. Disclosure: I sell the full Agent Ops Kit ($29) — the extra directives, tested execution scripts and multi-agent templates. The free lite kit above stands on its own. This article was drafted with AI assistance and reviewed against the framework we run in production.

DEV Community: Todd Doane