I reviewed a system last quarter that was spending a five-figure monthly sum on inference to do work a state machine would have done for the cost of the compute.
The process was: receive a webhook, validate the payload against a schema, look up two records, apply a pricing rule, write a result, emit an event. Seven steps. Same seven steps, ten thousand times a day, in the same order, every time.
It had been built as an agent with tool access, because that was the architecture the team had been reading about.
The four tiers
The most expensive mistake in automation is using a sophisticated technique where a simple one would do. Language models are seductive precisely because they work on almost anything — which makes it easy to deploy one where twelve lines of conditional logic would have been faster, cheaper, deterministic, and testable.
Work down this list. Stop at the first tier that solves your step.
Tier 1 — deterministic rules and workflow orchestration. If the decision can be expressed as conditions over structured data, express it that way. Microseconds, zero marginal cost, identical output every time, unit testable. Most steps in most processes belong here, and this tier is chronically under-used because it is unglamorous.
Tier 2 — classical ML. High-volume bounded prediction where you have labelled history: fraud scoring, churn, demand forecasting, classification into a fixed taxonomy. Cheap at inference, explainable enough for audit, far more predictable than an LLM on the same task.
Tier 3 — LLM as a bounded function. Extraction, summarisation, classification of unstructured text, normalising messy input into a schema. Called with a specific input and a validated output shape. No autonomy. This is where most of the genuine 2026 gains live, and it is the tier teams skip past on their way to agents.
Tier 4 — agentic workflows. A model that plans, calls tools, and iterates toward a goal. Genuinely powerful for open-ended work. Also the hardest to test, the most expensive per execution, and the one that fails most surprisingly.
A well-built pipeline uses all four. Invoice processing might use Tier 3 to extract line items from a PDF, Tier 1 to match against a purchase order, Tier 2 to score anomaly risk on a mismatch, and Tier 4 only for the small residue where something needs investigating across the ERP, an email thread and a supplier portal.
The decision rule
If you can draw the process as a flowchart without a large number of branches, do not use an agent. If drawing the flowchart is itself the hard part, an agent may be right.
That is the whole rule, and it holds up better than any more elaborate framework I have tried.
Agents win when the path is unknown at design time. Investigating why a shipment is late — check the carrier API, then the warehouse system, then the order notes, then maybe email the supplier, with each answer determining the next question — has no fixed flowchart. Encoding every branch is a losing effort.
Agents lose when the path is known and volume is high. You are paying latency, per-token cost and non-determinism to rediscover a sequence you already know.
The failure mode that should worry you
An agent that fails loudly is fine. You retry, you alert, you handle it.
An agent that confidently completes a task incorrectly is the dangerous case, and it is the one teams under-plan for. It called the wrong tool with plausible arguments, got a plausible-looking result, and moved on. Nothing errored. Your monitoring is green.
The mitigations are architectural, not prompt-level:
Constrain the tool surface. An agent should have access to the minimum set of tools for its task, not your whole API. Every additional tool is a way for it to be confidently wrong in a new direction.
Make destructive operations require confirmation. Reads can be free. Writes that move money, send communications, or change customer state should return a proposed action for approval rather than executing — at least until measured accuracy justifies otherwise.
Validate outputs at the boundary. Schema validation on everything leaving the agent. Not just structural validity — semantic constraints too. If the amount field can only be positive and below a ceiling, enforce that in code, not in the prompt.
Log the full reasoning trace. When something goes wrong in three weeks, you need to reconstruct what the agent was thinking. Store the tool calls, the intermediate outputs, and the decision points.
Bound the loop. Maximum iterations, maximum wall clock, maximum spend per execution. An agent in a retry loop against a flaky API will happily burn your budget.
A pattern that works
For most real processes, the right shape is a deterministic workflow with an agent as one bounded step:
workflow ProcessClaim {
step validate -> Tier 1 (schema + business rules)
step extract -> Tier 3 (LLM, bounded, schema-validated)
step scoreRisk -> Tier 2 (classical model)
step route:
if risk < 0.2 -> autoApprove (Tier 1)
if risk < 0.7 -> humanReview (task queue)
else -> investigate (Tier 4, bounded agent)
step record -> Tier 1
}
The agent handles the genuinely open-ended slice. Everything else is deterministic, testable, and cheap. You get the capability where it is needed and predictability everywhere else.
This also gives you a migration path in the right direction: as you learn what the agent actually does in the investigate step, recurring patterns get promoted into deterministic rules, and the agent's share shrinks over time. That is the correct direction of travel. A system where the agent's scope grows is usually a system where nobody is analysing what it does.
Full guide — process discovery, exception architecture, human-in-the-loop patterns, ROI framing and a 12-week rollout: Business Process Automation in 2026.
TechCirkle: agentic workflow development | LLM integration
More from TechCirkle
- Custom AI agent development
- AI development services
- Generative AI development services
- Enterprise AI development services
- Top AI automation tools for businesses
- Custom software development services
- Digital transformation services
Frequently Asked Questions
When is an AI agent the right choice?
When the path through the process is unknown at design time — a variable number of steps in an order determined by what you discover. Investigating a discrepancy across several systems fits. Running the same seven steps ten thousand times a day does not.
What is wrong with using an agent for a deterministic process?
You pay latency, per-token cost and non-determinism to rediscover a sequence you already know. It is also far harder to test: a state machine has enumerable paths, while an agent has a distribution of behaviours you can only sample.
How do I stop an agent from being confidently wrong?
Constrain the tool surface to the minimum needed, require confirmation for destructive operations, validate every output against a schema including semantic constraints, log full reasoning traces, and bound iterations, wall clock and spend per execution. Prompt instructions are not a control.
What is the difference between Tier 3 and Tier 4?
Tier 3 calls a model as a bounded function — specific input, validated output shape, no autonomy. Tier 4 gives the model agency to plan and call tools iteratively. Most value attributed to agents is actually available at Tier 3, at a fraction of the cost and with far better testability.
Should the agent's scope grow over time?
Usually the opposite. As you observe what the agent does in practice, recurring patterns should be promoted into deterministic rules, shrinking its share. A growing agent scope typically means nobody is analysing its behaviour.
How do I test an agentic step?
Build a fixture set from real historical cases including the difficult ones, run it on every change, and track outcome accuracy rather than output similarity. Add adversarial cases as you find them in production. Accept that you are measuring a distribution, not asserting equality.
Can I mix tiers in one process?
You should. A typical pipeline uses Tier 3 for extraction, Tier 1 for matching and routing, Tier 2 for scoring, and Tier 4 only for the residue that genuinely has no fixed shape. Designing that layering well is most of the engineering value.
Top comments (0)