TL;DR: Production agents work because their autonomy is contained, not because it is unlimited. Put agentic decisions inside a predictable workflow, cap every loop, restrict tools by consequence, checkpoint state, validate with code where possible, and ask humans to approve only high-impact actions. If safety lives only in the prompt—or success is measured by a polished demo—you do not have a production-ready agent yet.
Table of Contents
- The Autonomy Demo and the Production Gap
- A Working Definition, Because the Word Is Mush
- Pattern 1: The Bounded Loop
- Pattern 2: Workflow Skeleton, Agentic Muscles
- Pattern 3: Tool Authorization Tiers
- Pattern 4: Checkpoint and Resume
- Pattern 5: The Critic Loop (Used Sparingly)
- Pattern 6: Human Gates That Don't Destroy the Value
- Anti-Patterns: The Demo-Ware Hall of Fame
- A Reference Shape for a Production Agent
- Common Mistakes
- Best Practices
- Key Takeaways
- FAQ
- Conclusion
The Autonomy Demo and the Production Gap
The agent demo is a genre now, and it follows conventions as rigid as a sonnet: give the model some tools, show it planning aloud, watch it chain six calls to book the meeting or fix the bug or reconcile the invoices, and end before anything goes wrong. The demo's power comes precisely from what it omits — the run where step three returned an empty result and the agent hallucinated a substitute, the run that looped for forty minutes, the run that "helpfully" emailed a customer.
I've now shipped and reviewed enough agentic systems to hold a firm, mildly contrarian position: the difference between agent demo-ware and agent production-ware is that production systems spend most of their design budget constraining the autonomy the demo celebrates. Not eliminating it — constraining it, on purpose, at specific joints. The patterns below are those joints: where working teams put the structure, the budgets, the authorization, and the humans, and why each placement earns its complexity.
A Working Definition, Because the Word Is Mush
"Agent" currently means anything from a prompt with one tool to a fleet of self-delegating processes, which makes architectural conversation nearly impossible. The definition that makes the patterns legible: an agent is a system where the model chooses the next action — which tool, which arguments, whether to continue — rather than executing a fixed pipeline. That choice-making is the source of both the value (it handles situations you didn't enumerate) and the risk (it handles them in ways you didn't enumerate).
The design question is therefore never "agent or not" but where the choosing happens and what contains it. Every pattern below is an answer to the containment half.
Pattern 1: The Bounded Loop
The primitive at the heart of every agent — observe, decide, act, repeat — ships to production only when wrapped in explicit budgets on every axis on which it can run away:
class LoopBudget:
max_steps: int = 12 # actions per task, hard stop
max_tokens: int = 150_000 # spend ceiling per task
max_wall_clock: int = 300 # seconds; agents must not outlive user patience
max_tool_errors: int = 3 # consecutive failures before surrender
max_repeats: int = 2 # identical tool+args = probable loop
# On any breach: stop, checkpoint state, surface partial work with
# an honest status — never silently truncate, never silently retry forever.
Two of these deserve special defense because teams routinely omit them. Repeat detection (max_repeats) catches the signature agent pathology — retrying an identical failing action with identical arguments, forever, at your expense; a fingerprint of the last few tool calls costs nothing to compare and kills the pathology dead. And surrender protocol matters more than the ceilings themselves: an agent that hits a budget must report — what it tried, where it stopped, what remains — because a partial answer with an honest boundary is a feature, while a timeout with no explanation is a support ticket. The budget breach isn't an error path; it's a first-class outcome that deserves UX.
Calibration note from production: most useful task classes converge at surprisingly low step counts (5–15). If your agent regularly needs 40 steps, you don't have an autonomy problem — you have a decomposition problem, which is the next pattern's business.
Pattern 2: Workflow Skeleton, Agentic Muscles
The single highest-value structural decision in the genre. Fully open-ended agents — one loop, all tools, "figure it out" — maximize both flexibility and variance; fixed pipelines minimize both. The production sweet spot is a deterministic workflow skeleton whose individual stages are agentic:
The skeleton contributes what models are bad at: guaranteed ordering, stage-scoped tool access, deterministic gates between stages, and a legible progress model ("it's in the analyze stage" beats "it's thinking"). The agentic stages contribute what pipelines are bad at: handling the infinite variety inside a step — which searches to run, how to interpret a messy result, what the fix should be.
Teams consistently arrive at this shape from both directions: open-ended agents get skeletons bolted on after the variance bites; rigid pipelines get stages agentified after the edge cases bite. Starting at the sweet spot skips both bites. The diagnostic for where stages belong: anywhere you can write a deterministic check ("do we have enough evidence to proceed?"), you've found a gate; the stretches between gates are where the model chooses.
Pattern 3: Tool Authorization Tiers
An agent's real capability surface is its tool list — and the cardinal production sin is handing one undifferentiated toolbox to a probabilistic chooser. Tools must be tiered by consequence, with the tiers enforced outside the model:
- Tier 0 — Observe: read, search, fetch, list. Freely available; worst case is wasted budget.
- Tier 1 — Reversible acts: draft, stage, comment, create-in-sandbox. Available within the loop; mistakes are undo-able.
- Tier 2 — Consequential acts: send, merge, deploy-to-staging, modify-shared-state. Require a deterministic precondition check (validation rules, state assertions) before execution — the model requests, the policy layer decides.
- Tier 3 — Irreversible or high-blast-radius: payments, deletions, production deploys, external communications at scale. Require a human gate (Pattern 6) or are simply absent from the agent's world.
Two enforcement details separate real implementations from theater. First, tiering lives in the tool executor, not the prompt — "please be careful with the send tool" is a wish; an executor that refuses Tier 2 calls failing preconditions is a control. Second, arguments get validated against the requester's authority, not the agent's — an agent acting for user X can only touch resources X could touch; the agent inherits scoped credentials per task rather than wielding a god-token. This is ordinary least-privilege engineering, and it's remarkable how often the excitement of agents causes teams to forget they already knew it. Prompt injection makes this non-optional: any agent that reads external content (web pages, emails, documents) will eventually read instructions aimed at it, and the tier system is what makes that a curiosity instead of an incident.
Pattern 4: Checkpoint and Resume
Demo agents live for one glorious uninterrupted run. Production tasks span minutes to hours, meet flaky tools, restart with deploys, and get interrupted by the humans they serve. The pattern: externalize the agent's task state — plan, completed steps, tool results, pending intent — into a durable store at every step boundary, making the loop itself stateless and resumable.
The payoffs compound beyond crash recovery. Resumability converts model-provider hiccups from task failures into pauses. Checkpoints give you audit — the exact reconstruction of what the agent knew when it acted, which is what your postmortem (and possibly your compliance team) will want. Human gates (Pattern 6) stop being awkward blocking calls and become checkpoint states awaiting input — the agent parks, the human answers hours later, the task resumes with context intact. And checkpoint diffs are the best debugging artifact agentic systems produce: "between step 6 and 7, the plan changed from X to Y" localizes misbehavior that raw transcripts bury.
Durable-execution frameworks (Temporal-class) fit agents naturally here, and the fit is no accident: an agent is a workflow whose next step is chosen at runtime. Teams already operating such infrastructure should run agent loops on it rather than reinventing checkpoint machinery in application code.
Pattern 5: The Critic Loop (Used Sparingly)
Generate-then-critique — a second model pass reviewing the first's output against a rubric before it proceeds — measurably improves quality on tasks with checkable properties: does the code compile and pass tests, does the summary cite only present facts, does the plan touch only permitted systems. The pattern earns its place in the catalog with two sharp caveats that demos omit.
First, critics work where checking is easier than doing. Code review by critic works because execution and tests provide ground truth to critique against. Open-ended judgment tasks ("is this analysis insightful?") get critic theater — a second model confidently blessing the first's correlated errors. Spend critic budget where verifiable properties exist; use deterministic validators instead wherever they're possible at all (cheaper, and actually reliable).
Second, cap the loop at one round. Generate-critique-revise converges in one iteration on most real tasks; further rounds produce oscillation (the revision un-fixes what the previous round fixed) at linear cost. The multi-round self-refinement of demos is mostly token combustion. One generation, one critique, one revision, then a gate — deterministic or human — is the production shape.
Pattern 6: Human Gates That Don't Destroy the Value
Every consequential agent needs human approval somewhere, and naive placement destroys the economics — an agent that interrupts for confirmation eight times per task is a slower, wordier form on top of which you've added inference costs. The design discipline is treating approvals as a scarce UX budget:
- Gate at consequence boundaries, not step boundaries. One approval of the complete proposed action set ("send these 3 emails, file these 2 tickets — approve?") beats five sequential micro-approvals, both for throughput and for reviewer attention quality.
- Make the approval artifact rich and diff-shaped. Humans approve well when shown what will change — recipients, amounts, before/after states — and rubber-stamp when shown a wall of agent reasoning. Design the approval view like a code review, not a transcript.
- Tier the gating by trust earned. New agent (or new task class): gate everything in Tier 2+. As measured performance accumulates, widen the auto-approve envelope — low-risk action types first, monitored by sampling audits rather than universal review. This graduated autonomy is how teams get from "human approves everything" to genuine leverage without a leap of faith.
- Never let the gate become a bottleneck silently. Queued approvals age; tasks parked on humans need SLAs, reminders, and escalation, or the agent system's throughput quietly becomes one distracted reviewer's attention span.
Anti-Patterns: The Demo-Ware Hall of Fame
Recurring shapes that reliably predict production pain: the self-delegating swarm (agents spawning agents — multiplies every failure mode in this article by the fan-out, delivers coordination overhead in exchange, and is almost never what a business task needs; a workflow skeleton with parallel stages captures the parallelism without the anarchy); the god-context loop (append every observation to one ever-growing context until the model drowns in its own history — checkpointed state with summarized memory exists precisely to prevent this); prompt-enforced safety (any sentence shaped like "the model is instructed not to..." presented as a control); and the demo metric ("it completed the task in our tests" with no denominator — production agents need completion rates, intervention rates, and cost-per-completed-task, measured on real traffic, or you're navigating by anecdote).
A Reference Shape for a Production Agent
The patterns assembled, as they'd appear in a real system — a support-operations agent that investigates and resolves account issues:
Workflow skeleton with four stages (triage → investigate → propose → execute), each stage an agentic loop under a LoopBudget, tools tiered per stage: investigation gets Tier 0 only; proposal builds a Tier 1 draft; execution holds scoped Tier 2 credentials for the specific account, with refunds and closures behind a Tier 3 human gate presented as a diff. State checkpoints at every step boundary into a durable store; the human gate is a parked checkpoint with a 4-hour SLA. One critic pass validates the proposal against account state before the gate. Telemetry emits completion rate, intervention rate, spend per resolution, and step-count distribution — the four numbers that tell you whether the thing is earning its complexity.
None of this is exotic; every piece is a known engineering material. What's distinctive about strong agent architecture is the judgment about where model autonomy belongs and where it must be fenced — which is also precisely what AI-focused design interviews now probe. Engineers building toward architect-level ownership of these systems can pressure-test that judgment against the skill matrix and scenario assessments in an AI Architect track — the gap between "can wire up an agent loop" and "can decide where the gates go" is exactly the gap those assessments are built to expose.
Common Mistakes
- Budgets as afterthoughts. Loops ship with no step ceiling, no repeat detection, no surrender protocol — then meet their first pathological input on a weekend.
- One toolbox, no tiers. The model that drafts replies can also send them; the first injection or hallucination finds this immediately.
- Autonomy where a pipeline belongs. If the steps are known and fixed, an agent adds variance and cost to a solved problem; agentify the stages that need judgment, not the sequence that doesn't.
- Blocking human gates. Synchronous approval calls make humans a latency component; parked checkpoints make them a workflow stage.
- Critic loops on unverifiable tasks. Two correlated models agreeing is not verification.
- Shipping on anecdotal success. Without completion/intervention/cost rates on real traffic, you cannot distinguish an agent that works from an agent that has worked.
Best Practices
- Wrap every loop in explicit multi-axis budgets with repeat detection and a designed surrender path.
- Structure tasks as deterministic skeletons with agentic stages, gated by checkable conditions.
- Tier tools by consequence; enforce tiers and argument scoping in the executor with per-task least-privilege credentials.
- Checkpoint state at step boundaries; make resume, audit, and parked human gates fall out of the same mechanism.
- Apply critics only where verification beats generation in difficulty, and cap at one round.
- Spend the approval-UX budget at consequence boundaries with diff-shaped artifacts, widening autonomy as measured trust accumulates.
- Instrument completion rate, intervention rate, and cost per completed task from day one.
Key Takeaways
- Production agent architecture is the discipline of constraining autonomy at specific joints — budgets, skeletons, tiers, checkpoints, gates — not maximizing it.
- The workflow-skeleton-with-agentic-stages shape is the genre's sweet spot; both extremes migrate toward it after production pain.
- Tool tiering enforced in the executor (never the prompt) is the load-bearing safety mechanism, and prompt injection makes it mandatory.
- Externalized, checkpointed state converts crashes, interruptions, and human approvals from failure modes into workflow states.
- An agent without completion-rate telemetry is a demo with a deployment pipeline.
FAQ
When does a task justify an agent over a fixed pipeline at all?
When the path varies genuinely per instance — investigation, diagnosis, multi-source synthesis — such that enumerating branches is infeasible. If a senior engineer could flowchart the task completely, build the flowchart; it will beat the agent on every production metric.
How do multi-agent architectures fit these patterns?
Mostly as stages: specialized agents as stages in one skeleton (a research stage, a drafting stage) inherit all the containment machinery cleanly. Peer-to-peer agent negotiation, by contrast, multiplies unverifiable interactions and is rarely justified by business need — treat it as a research posture, not a production default.
What model tier should the loop's choosing run on?
The choosing (planning, tool selection) benefits from the strongest model more than the doing does — errors there compound through every subsequent step. Common production economics: flagship model for plan/decide steps, small models for extraction and summarization inside stages, per the routing logic of standard cost design.
How do I test an agent before real traffic?
Replay-based evaluation: record real task inputs and tool-call transcripts, then run the agent against simulated tool responses (including the recorded failures — empty results, timeouts, malformed data). Agents are systems whose hard cases live in tool-response space, so that's the space your test harness must control.
Do these patterns apply to coding agents specifically?
Directly — coding agents are the pattern set's best case, because verification is cheap (compile, test, lint = deterministic gates; the sandbox = Tier 1 by construction; the PR = a diff-shaped human gate that already existed). That's much of why coding is the domain where agents genuinely work today.
Conclusion
Every era of software gets a technology whose demos outrun its deployments, and agents are this era's champion of the gap. The gap closes the way it always closes — not by the technology becoming magic, but by engineers building the boring exoskeleton that lets a powerful, unreliable core do work: budgets around the loop, structure around the choosing, authority tiers around the tools, durable state under the whole thing, and humans stationed exactly where their judgment is irreplaceable and nowhere else.
Constraint, it turns out, is what autonomy ships in. The teams winning with agents right now aren't the ones who trusted the model most — they're the ones who fenced it best, measured it honestly, and widened the fences only as the numbers earned it. Build like that, and the demo's promise stops being a genre convention and starts being a completion rate.
Top comments (2)
The checkpoint-and-resume pattern is underrated. Agents need to be treated more like durable workflows than simple LLM loops. Great practical breakdown.
Really liked the point about “constraining autonomy.” In production, guardrails are often more important than making the agent smarter. The tool-tiering approach is especially practical.