DEV Community

Cover image for 15 Ways AI Coding Agents Fail in Real Repos (and the Guardrails That Fix Them)
Adrian Studio
Adrian Studio

Posted on Originally published at adrianstudio3.hashnode.dev

15 Ways AI Coding Agents Fail in Real Repos (and the Guardrails That Fix Them)

#ai #programming #productivity #devops

Around week three of using coding agents, most teams hit the same wall. The agent is clearly busy: long transcripts, dozens of edited files, confident summaries. But ask "what shipped, and is it correct?" and the answer gets fuzzy.

After running Claude Code, Codex CLI and Gemini CLI side by side on real projects, I've seen the same 15 failures over and over. Almost none of them are model-quality problems. They are operations problems, and each one has a boring, file-based fix.

The four root causes

Symptom Root cause
"Done", but nothing works end to end No definition of done that requires evidence
Each tool behaves differently on the same repo Instructions duplicated and drifting
You're afraid to let it run, so you babysit Permissions designed in the prompt instead of in config
It re-learns the same facts every session No durable memory, or memory that only grows

Here are the 15 failures, grouped by cause.

A. Evidence failures

1. "Done", but the feature is broken

Agents narrate intent as outcome: "I've updated the handler to retry." Fix it by putting a definition of done in your AGENTS.md that requires evidence:

## Definition of done
1. The behaviour exists AND you ran something that demonstrates it.
2. Tests covering the change pass; none deleted, skipped or weakened.
3. Lint and type check pass.
4. Final message lists: files changed, commands run WITH results,
   what you could not verify, decisions you made without being asked.
Enter fullscreen mode Exit fullscreen mode

The test for a report: could a stranger reproduce the claim? "Tests pass" fails. "make test T=billing → 42 passed in 6.1s" passes.

2. The agent deleted the failing test

This is what an unbounded retry loop does. Cap it: three attempts, then stop and report what was tried. Have a reviewer check specifically for removed or skipped tests.

3. Lots of activity, flat output

If the agent is busy but accepted changes aren't going up, the tasks are too big and have no acceptance check. Every delegated task gets one line before any code is written:

Acceptance check: test_duplicate_returns_409 passes and make lint is clean.

If you can't write that line, the task isn't ready to delegate.

4. Scope creep: "while I was here, I also..."

Add an explicit Out: list to every task brief, and have the reviewer flag any file changed outside In:.

B. Drift failures

5. Different behaviour per tool

You have CLAUDE.md, GEMINI.md and AGENTS.md, each slightly different. Keep one contract and make the others thin adapters:

# CLAUDE.md
@AGENTS.md

## Claude-specific
- Use the reviewer subagent before declaring multi-file changes done.
Enter fullscreen mode Exit fullscreen mode

For Gemini CLI, load the contract directly in .gemini/settings.json:

{ "context": { "fileName": ["AGENTS.md", "GEMINI.md"] } }
Enter fullscreen mode Exit fullscreen mode

Codex reads AGENTS.md natively. Then add a two-line CI check so drift fails the build:

grep -q "AGENTS.md" CLAUDE.md GEMINI.md || { echo "adapter lost its pointer"; exit 1; }
[ "$(wc -l < CLAUDE.md)" -le 40 ] || { echo "move rules to AGENTS.md"; exit 1; }
Enter fullscreen mode Exit fullscreen mode

6. The contract gets ignored

Usually because it's 400 lines of vague prose. Keep it to 80-150 lines and make every rule checkable:

Weak Strong
"Write good tests." "Every bug fix adds a test that fails before the fix."
"Be careful with the DB." "Never run migrations against anything but localhost."
"Keep code clean." "src/domain/** must not import src/web/**; make lint enforces it."

If a linter can enforce a rule, configure the linter and delete the sentence.

C. Permission failures

7. "I set it to never ask, and the overnight job did nothing"

Sandbox and approval are two different dials. The sandbox is the boundary (files, network). The approval policy is the procedure when an action would cross it. In Codex:

sandbox_mode = "workspace-write"   # read-only | workspace-write | danger-full-access
approval_policy = "on-request"     # on-request (interactive) | never (batch)

[sandbox_workspace_write]
network_access = false
Enter fullscreen mode Exit fullscreen mode

never doesn't widen the boundary. It makes boundary-crossing actions fail instead of waiting. That's correct for batch runs, and it's exactly why the job "did nothing". Read the failure log and widen the sandbox for that one profile only.

8. The agent bypassed a block through another tool

You denied git push, so it tried curl against the API. Deny the capability (network, a path), not just one command name. A blocked action should be reported, never retried another way.

9. Secrets ended up in a transcript

The agent could read .env. In Claude Code:

"permissions": {
  "deny": ["Read(./.env)", "Read(./.env.*)", "Read(./secrets/**)"]
}
Enter fullscreen mode Exit fullscreen mode

Inject a credential into one command's environment instead of leaving it in a readable file.

10. "Please don't" in the prompt

Rules that matter belong in config and hooks, not prose. A PreToolUse hook that exits with code 2 blocks the call and shows its stderr to the agent:

BLOCK = [
  (r"\brm\s+-[a-z]*r[a-z]*f\b", "recursive force delete"),
  (r"\bgit\s+push\b.*--force(?!-with-lease)", "force push without lease"),
  (r"\b(curl|wget)\b[^|]*\|\s*(sh|bash)\b", "piping a download into a shell"),
]
for pattern, reason in BLOCK:
    if re.search(pattern, cmd, re.I):
        print(f"Blocked: {reason}. Ask the human.", file=sys.stderr); sys.exit(2)
Enter fullscreen mode Exit fullscreen mode

Test it with two echo '{...}' | python3 guard.py; echo $? lines before you trust it.

11. The hook was "installed" but never ran

Codex requires new or changed hooks to be trusted before they execute, and a wrong path fails silently anywhere. Verify a hook by its output, not by its presence.

D. Context and memory failures

12. Context full of passing-test output

Move long suites into a test-runner subagent that returns only failures. The main context gets 5 lines instead of 5,000.

13. The reviewer rewrote the code

A reviewer with an Edit tool stops reviewing and starts rewriting. Give it Read, Grep, Glob, Bash only, or run a second tool read-only:

codex --profile review "Review the current git diff against AGENTS.md.
Reply with VERDICT, BLOCKING, NON-BLOCKING, EVIDENCE."
Enter fullscreen mode Exit fullscreen mode

(In current Codex a profile is its own file, ~/.codex/review.config.toml, containing sandbox_mode = "read-only".)

14. Memory contradicts itself

An append-only MEMORY.md turns into fiction within a month. Use three layers: raw per-turn logs, a daily summary capped at 40 lines, and a durable memory capped at 120 lines. A weekly job promotes an item only if it shows up in two or more daily files, and forgets closed loops and contradicted facts.

15. Duplicate external actions after a timeout

A scheduled job sends a message, the request times out, the job retries, and now there are two messages. When a call times out, you can't tell "never arrived" from "processed, response lost". Give every external action an idempotency key, record it before the call, mark it unknown on timeout, and check the target system before retrying.

The 60-minute version

  1. 0-15 min: one AGENTS.md (what it is, commands, needs-a-human, definition of done) plus thin adapters.
  2. 15-30 min: an approval matrix, then allow/ask/deny lists and Codex sandbox defaults from it.
  3. 30-45 min: a guard hook (tested), plus reviewer and test-runner subagents.
  4. 45-60 min: a MEMORY.md with five real facts and a nightly consolidation job.

Then track three numbers weekly: accepted changes, rework rate, and human minutes per accepted change.

Want the templates?

I packaged the full setup into a free 4-page preview (failure catalogue, the try-then-verify loop, and the definition-of-done checklist): download the free preview.

The full kit has a 28-page guide plus 29 ready-to-copy files: AGENTS.md/CLAUDE.md/GEMINI.md, Claude Code settings.json with the tested guard hook, reviewer and test-runner subagents, Codex profiles, an LLM-wiki structure with a validator, and the memory routine. It's here if you want it (launch code AGENTOPS).

Top comments (0)