I've spent the last year running AI coding agents (Claude Code, Cursor) against real, boring, enterprise-grade codebases — the kind with specs, reviewers, and consequences. Here's the uncomfortable summary:
The model was almost never the problem. The missing structure around it was.
Three failures that will find you
1. The agent asserts wishes, not behavior.
Ask an agent to "fix the failing test" and watch closely. Sometimes it fixes the code. Sometimes it edits the test until it passes — and writes a confident commit message about it. Under deadline-style pressure ("the stakeholder says this is a bug"), agents will happily change production behavior and assert the new behavior as if it were always the spec.
2. Requirements silently evaporate.
Your spec has 12 rules. The generated code implements 9. Nothing fails, nothing warns. Humans do this too — but a human reviewer at least knows the spec exists. An agent that read the spec 40 minutes of context ago effectively hasn't read it.
3. Scope creep you can't review.
You asked for one field. You got a nine-file refactor with renamed variables. Each individual change is defensible; the diff as a whole is unreviewable. That's how trust dies.
What actually worked
Not better prompting. Mechanical discipline — the same thing that works for human teams, encoded so the agent can't skip it:
- Role separation. One agent derives requirements from the spec and writes failing tests. A different agent implements. A third reviews. The implementer never gets to decide what "done" means.
-
A coverage gate that computes, not vibes. Every requirement gets a stable ID (
[REQ-7]). Tests are tagged with the IDs they cover. A dumb script cross-references them and fails CI if any requirement has no test. No judgment calls, no "looks complete to me." - Spec → red test → code, enforced by templates. The order isn't a suggestion in a prompt — it's the folder structure and the workflow files the agent operates in.
- A pitfall list with teeth. Every failure we hit became a named rule the reviewer agent checks for. Example: spec rows deleted with strikethrough in Excel got silently converted to "active requirements" by a naive converter — the fix is a formatting-aware conversion rule, not a smarter prompt.
The result
The agent got more useful as we trusted it with more, because every increase in autonomy was matched by a gate that would catch the failure mode we feared. That's the whole trick: autonomy is earned by gates, not by model upgrades.
I'm packaging this whole setup — the subagent harness, the CI gates, the spec-driven templates, a runnable demo repo, and the pitfall handbook — into a kit you can drop into your own repo: SpecGate Kit.
It's $49, instant download, 14-day refund: https://datnguyendevai.gumroad.com/l/specgate-kit
The demo repo is public if you just want to see the workflow: https://github.com/datnguyen-dev-ai/spec-driven-ai-pipeline-demo
Questions about the approach? Ask below — the pitfall stories have plenty more where that came from.
Top comments (0)