Most AI agent demos die on contact with production. Not because the model is weak, but because the integration was an afterthought: no bounded scope, no enforced permissions, no durable state, no evaluation beyond "looks good to me."
This post covers five integration patterns that separate agents surviving in production from agents stuck in demo purgatory. Each pattern is technology-agnostic — it applies whether you build on LangGraph, CrewAI, AutoGen, or plain API calls.
I expanded these patterns into a full implementation guide with checklists here: AI Agents in Enterprise Workflows: A Practical Integration Guide. What follows is the condensed, opinionated version.
Pattern 1: One bounded workflow with a measurable finish line
Never start with "let's add an agent to the department." Start with one workflow where completion is unambiguous.
Consider an illustrative invoice-dispute assistant: it reads an invoice, gathers the relevant policy, and proposes a resolution. Finance keeps approval authority for any credit. The finish line is measurable — dispute resolved or escalated — and the boundary is explicit: the assistant gathers evidence and recommends; it never issues refunds or edits bank details.
Two rules from this pattern:
- Map the process before adding the model. Record how a request arrives, which systems staff consult, who approves changes, and what counts as done. Establish baselines: handling time, correction rate, unresolved cases — including the time reviewers spend checking the agent's output.
- Use deterministic code where the next action is already known. A tax calculation or invoice lookup doesn't need an open-ended planner. Add the model where interpretation matters: extracting the customer's complaint, or deciding which policy document is relevant.
If the bounded workflow doesn't deliver value, a bigger agent won't either.
Pattern 2: Narrow tool contracts, enforced outside the model
The most common production failure I see: tools with god-mode access and a system prompt that says "be careful."
Expose tools like "read invoice by authorized account and invoice ID" — never unrestricted database access. Validate fields, check the caller's identity, and apply account-level authorization in the service layer. A system prompt is guidance; it is not a security boundary.
Three non-negotiables:
- Separate read tools from write tools. A proposed credit must carry invoice ID, amount, currency, reason, and supporting evidence — and the execution service independently checks limits and current invoice state before committing.
- Re-read critical values before committing. An old recommendation must never approve a changed invoice.
- Treat retrieved content as untrusted. Emails, attachments, and documents can contain instructions that conflict with the user's task. Restrict tool destinations, keep credentials out of model-visible context, and include adversarial documents in your evaluation set.
Pattern 3: Durable state, idempotency, and explicit approvals
Conversations are not databases. Store workflow state outside the model: request ID, evidence references, proposal version, approval status, execution result. After a worker restart, the system resumes from that record — it never asks the model to reconstruct "what probably happened."
For anything retried, use an idempotency key and persist the action result against it, so a network timeout can't create two credits. If the result is unknown, reconcile with the business system before retrying. A model's statement that a tool "probably failed" is not evidence.
Approvals deserve the same explicitness. Model the states — evidence collected, approval pending, approved, execution uncertain — and define the allowed next step and retained evidence for each. Persisted states prevent an interrupted session from bypassing approval or repeating a business action.
Pattern 4: Evaluate with difficult cases, not happy paths
An evaluation set of ten easy cases proves nothing. Build yours from the cases staff actually dread: the missing invoice, the already-paid invoice, the mismatched account, the policy exception, the request to change payment details. Mask personal data where appropriate, and keep the expected outcome and the reason alongside each case.
Score more than the final sentence:
- Did the agent fetch the correct record?
- Did it cite evidence that actually supports the amount?
- Did it stop at the approval boundary?
- What happened during tool outages, malformed results, or malicious instructions in retrieved text?
Set acceptance criteria by consequence. A wrong summary is repairable in review; an unauthorized credit is a release blocker. Keep a held-out set that prompt authors never tune against, and rerun it whenever the model, tool schema, or retrieval configuration changes.
Pattern 5: Roll out in recommendation-only mode, operate like software
Begin with the agent preparing drafts while staff run the existing process. Compare outcomes and record why reviewers edit or reject drafts — that log reveals missing context and unclear policies before actions become automatic.
Then route a small class of low-risk requests through the assisted flow, with a visible takeover path and preserved evidence on handoff. Cap tool calls, elapsed time, and spend per request; exhausting a limit must produce an explicit handoff, never an invented result.
Monitor backlog, completion rate, reviewer effort, and correction rate together. Faster drafts are not a productivity gain if review takes longer. And keep the previous workflow available during rollout, with a clear control to disable execution tools while retaining read-only assistance.
Once live, operate the agent like the changing software system it is: version prompts, tools, policies, and model settings together. Track cost per accepted resolution and the share of cases needing manual recovery — far more useful than raw token counts.
When one agent isn't enough
These patterns cover the single-agent case. When you genuinely need multiple agents — say, separate researcher, executor, and reviewer roles — the coordination design matters more than the model choice. A separate guide compares the three practical options: supervisor orchestration, shared state, and handoff patterns, with task contracts and recovery guidance for each.
Rule of thumb: add a second agent only for a specific coordination need, never because the first agent felt underpowered. Most "we need multi-agent" problems are actually "we need narrower tools and better evaluation" problems.
The checklist
- One workflow, measurable finish line, baseline metrics first.
- Narrow tool contracts; authorization enforced in code, not prompts.
- Durable state, idempotency keys, explicit approval states.
- Evaluation from difficult and adversarial cases; acceptance criteria by consequence.
- Recommendation-only rollout, visible fallback, versioned operation.
Get these five right and your agent has a fighting chance in production. Skip them and you have a demo with a deployment date.
This article was written with AI assistance and reviewed by the author. Examples are illustrative. The extended implementation guide with figures and checklists is published here.
Top comments (0)