12 Rules for Building AI Agents That Survive Production

Szymon Paluch — Fri, 17 Jul 2026 12:30:14 +0000

I sat the Claude Certified Architect exam expecting questions about model parameters, context limits, and API flags. I got something else. The exam barely tests trivia. It tests judgment: given a broken agent and four plausible fixes, which one actually addresses the root cause?

The interesting part was how few ideas the whole thing rests on. The same handful of rules kept deciding the "right" answer, and they are the same rules that decide whether an agent holds up once real users touch it. Below are the twelve I kept running into, plus the four traps that look like solutions and are not.

This is my own study material, derived from publicly available exam guidance. It reflects how I build, not an official Anthropic position.

The twelve rules

Enforce determinism in code, not in prompts
If a rule has to fire every single time, it is not a job for a prompt. A prompt is a suggestion the model usually follows. "Usually" is not a guarantee. When you need a guarantee, put it in a hook, a gate, or an allowlist. Code enforces. Prose requests.
Pick the cheapest fix that hits the root cause
Before you build a subsystem, try the levers that cost minutes: a sharper tool description, an explicit acceptance criterion, a config change. Most "we need to build X" moments dissolve once you test the cheap fix first. Reach for the classifier only after the one-line change fails.
Bad tool selection? Start with the descriptions
When an agent keeps picking the wrong tool, the description is almost always the culprit, not the model. Tool descriptions are the primary signal the model uses to choose. Rewrite them to say exactly when to use the tool and when not to, before you go anywhere near few-shot examples.
Over-engineering is almost always the wrong answer
Narrowing scope and improving the prompt beat a new subsystem far more often than engineers expect. Every subsystem you add is one more thing to debug, monitor, and keep in sync. Complexity is a cost you pay forever, not once.
A bigger context window does not fix attention
Stuffing more into the window does not make the model pay better attention to what matters. It often does the opposite. Decomposition and structure fix attention: smaller tasks, cleaner inputs, explicit boundaries. "Just use the model with the larger context" is a non-answer.
Model confidence and tone are not signals
A confident-sounding answer is not a correct one. Self-assessed confidence is poorly calibrated, and on the hard cases the model is often most sure precisely when it is wrong. Never gate a decision on how certain the model claims to be. Verify with something external.
An independent instance beats self-review
A model grading its own work is biased toward its own work. A fresh instance, with no memory of having produced the output, catches far more errors. If you want a real review step, spin up a separate reviewer, do not ask the author to check itself.
Least privilege for tools
Give each agent role only the tools it needs, roughly four to five. This is not only a security point. More tools measurably degrade selection quality: the model gets worse at choosing when the menu is longer. Fewer, sharper tools beat a giant toolbox.
Structure beats free text, everywhere
Structured errors, structured outputs, structured handoffs between agents, all with explicit provenance. Free-text prose between components is where information quietly rots. When agent A hands to agent B, pass a schema, not a paragraph.
Do not suppress errors, and do not panic
Handle transient failures locally: retry the flaky network call, back off, move on. For everything else, escalate with context so the coordinator can decide. The two failure modes to avoid are swallowing the error silently and blowing up the whole run over something recoverable.
Match the API to your latency requirement
The Batch API costs about half as much but can take up to 24 hours. That is great for an overnight job and useless for anything a human is waiting on. Pick synchronous calls for human-blocking tasks and batch for the rest. Cost and latency are a trade you make on purpose.
A retry fixes format, not missing data
Retrying a call can correct a malformed structure. It cannot conjure information that was never in the source. If the data is not there, running it again just wastes tokens and time. Know which problem you have before you hit retry.

The four traps
These are the answers that feel smart in the moment and cost you later. Every one of them showed up as a tempting-but-wrong option.

"Just write in the prompt that it is mandatory." Shows up exactly when you need determinism, which is the one thing a prompt cannot give you. See rule 1.
"Add few-shot examples." Genuinely useful for ambiguous judgment calls. Useless for hard rules, where it creates the illusion of a guarantee without the guarantee.
"Build a classifier." A heavy solution reached for before anyone tested the cheap levers. Sometimes correct, usually premature. See rule 2.
"Make one tool that does everything." Merging tools feels like simplification. It makes selection worse, because now the model has to guess intent inside a single overloaded tool. See rule 8.
It collapses into three ideas
Strip away the specifics and the twelve rules are really three:

What must always work goes in code, not in a request. Determinism is an architecture decision, not a wording decision.
Match the complexity of the fix to the size of the problem. Cheap lever first, subsystem last.
Pass structure, not prose. Between components, schemas beat sentences.
None of this is exotic. It is the difference between an agent that demos well and one that is still running in three months. The exam rewards it because production rewards it.

Szymon Paluch. I work with teams on agentic system strategy, taking Claude Code and agents from demo to production.

12 rules for building AI agents | Szymon Paluch

The full cheat-sheet from prepping for the Claude Certified Architect exam: 12 rules, 7 recurring traps, the specifics of all 5 domains, and a facts table. The rules that decide whether an agent survives production.

szymonpaluch.com

DEV Community: Szymon Paluch

12 Rules for Building AI Agents That Survive Production

12 rules for building AI agents | Szymon Paluch