DEV Community: FA33

AI Agents vs. Workflows: When to Use Each Architecture

FA33 — Fri, 25 Sep 2026 00:00:00 +0000

Most teams do not need an agent first.

They need a reliable system that can use a model, call a few tools, and complete a bounded task without creating unnecessary uncertainty.

That distinction matters because the word agent is often used for almost any application that combines an LLM with tools. Architecturally, however, there is an important difference between a workflow and an agent.

A workflow has a path that your code largely defines in advance. The model may classify, generate, extract, or choose among bounded options, but the application still controls the sequence.

An agent has more authority over the sequence itself. It can inspect the current state, decide what to do next, select tools, repeat steps, recover from failures, and stop when it believes the objective is complete.

Anthropic makes a similar distinction: workflows follow predefined orchestration paths, while agents dynamically direct their own processes and tool usage. OpenAI’s agent documentation likewise centers the runtime around an iterative loop of model decisions, tool execution, state, and stopping conditions.

The practical question is not which architecture sounds more advanced. It is how much decision authority the model actually needs.

The architecture ladder

Think of AI systems as a ladder of increasing autonomy.

Single model call — one input, one output.
Deterministic workflow — fixed steps with model calls inside them.
Dynamic workflow — some routing or decomposition is model-driven, but application code still constrains the process.
Agent — the model decides the next action repeatedly until a goal, blocker, or stop condition is reached.

Every step upward can increase flexibility. It can also increase latency, cost, observability requirements, and the number of failure modes.

The default engineering strategy should therefore be simple: start at the lowest level that can solve the task reliably.

When a workflow is the better architecture

Use a workflow when the task is mostly known before execution begins.

A good example is document processing:

classify the document;
extract required fields;
validate the output;
route exceptions to review;
store the result.

The model can still be useful at several stages, but the application already knows what the stages are.

Workflows are especially strong when you need:

predictable execution paths;
consistent latency;
bounded cost;
explicit validation between steps;
strong auditability;
easy retries;
clear ownership of failure states.

Several useful patterns fit this model.

Prompt chaining

One model call produces an intermediate result that becomes input to the next step. This is useful when a complex task can be decomposed cleanly and each step can be validated separately.

Routing

A classifier chooses among known downstream paths. Customer support is a common example: billing questions, account access, technical failures, and abuse reports can each enter a specialized flow.

Parallelization

Independent subtasks run at the same time and are combined afterward. This works well when the task can be divided without requiring each worker to know the others’ intermediate results.

Evaluator-optimizer loops

One model produces an answer and another evaluates it against explicit criteria. The loop can continue until a quality threshold is reached or a maximum number of attempts is exhausted.

These patterns can look sophisticated, but they remain workflows because the application still defines the operating structure.

When an agent is justified

An agent becomes useful when the important steps cannot be specified completely in advance.

Consider a coding task such as:

Find the root cause of this production regression, make the smallest safe fix, run the relevant tests, and explain what changed.

The system may need to:

inspect different files depending on what it discovers;
search logs;
choose which tests to run;
revise its hypothesis after a failed test;
edit one or several files;
stop and ask for approval before a risky action.

The exact path is not known when execution starts. A fixed workflow can attempt to encode every branch, but at some point the workflow becomes a hand-written imitation of the decision process the model could perform directly.

Agents are strongest when the task has four characteristics:

The objective is clear but the path is not.
The environment can provide feedback after each action.
The model has useful tools for changing or inspecting that environment.
The application can impose hard boundaries on what the agent is allowed to do.

That last requirement is critical. Autonomy without boundaries is not an architecture advantage; it is an operational liability.

A practical decision table

Question	Prefer workflow	Prefer agent
Are the steps known in advance?	Usually yes	Usually no
Does the model need to choose the next action repeatedly?	Rarely	Yes
Is predictable latency important?	Strong fit	Harder
Is cost tightly bounded per run?	Easier	Harder
Can the environment return useful feedback after each action?	Helpful	Essential
Do you need strong reproducibility?	Strong fit	Requires more controls
Can failure be contained with permissions and stop conditions?	Still important	Mandatory
Does the task vary substantially from case to case?	May become brittle	Stronger fit

The hidden cost of unnecessary agents

The largest mistake is not choosing a workflow when an agent would be better. It is building an agent where a workflow already solves the problem.

Agents create additional engineering requirements:

tool permissions;
state management;
loop termination;
retry policy;
approval checkpoints;
tracing;
evaluation across multi-step trajectories;
recovery when intermediate actions partially succeed.

A single model call can be evaluated by comparing inputs and outputs. A multi-step agent must often be evaluated as a trajectory: what it observed, what it decided, which tools it called, what changed in the environment, and whether the final state was actually correct.

This is why agent reliability is not only a model-quality problem. It is a systems problem.

The hybrid architecture is often the right answer

The choice is not binary.

A strong production system often uses a deterministic workflow around a bounded agent.

For example:

validate the request;
classify risk;
create a sandbox;
let an agent solve the task inside that sandbox;
run deterministic checks;
require approval for sensitive actions;
persist the result and audit trail.

The workflow controls the system boundary. The agent handles the part where the path is genuinely dynamic.

This structure keeps autonomy where it creates value without giving the model authority over everything around it.

Five questions before adding an agent loop

Before implementing an agent, answer these questions:

1. What decision cannot be encoded reliably in normal application logic?

If you cannot identify that decision, you may not need an agent.

2. What ground truth can the system observe after each action?

Agents need feedback. Tests, API responses, file state, database results, browser state, and human approvals can all provide evidence about whether progress is real.

3. What is the maximum authority the agent needs?

Do not expose ten tools when the task needs three. Do not grant write access when read access is sufficient. Do not allow irreversible actions without a checkpoint.

4. How does the run stop?

Define success conditions, iteration limits, budget limits, timeouts, and escalation paths before production.

5. How will you evaluate the trajectory?

Measure more than the final answer. Track tool choice, unnecessary steps, retries, latency, token usage, cost, human interventions, and task completion.

The design rule

Workflows optimize for control.

Agents optimize for adaptability.

Most production systems need both, but not in equal amounts.

If the task has a stable path, encode that path. If the task requires runtime discovery, let the model make the decisions that genuinely cannot be predetermined. Then surround that autonomy with deterministic checks, permissions, observability, and stop conditions.

The goal is not to build the most agentic system.

The goal is to build the least autonomous system that can reliably solve the real problem.

Sources

How to Evaluate AI Agents: Reliability, Cost, Latency, and Failure Modes

FA33 — Fri, 25 Sep 2026 00:00:00 +0000

An AI agent can produce an impressive demo and still be a poor production system.

The reason is simple: a demo usually proves that the agent can complete a task. Production requires evidence that it can complete the task reliably, repeatedly, at an acceptable cost, within an acceptable time, and without unsafe side effects.

That changes how agents should be evaluated.

A single final answer is not enough. An agent acts across multiple steps, calls tools, changes state, encounters partial failures, and can reach the same result through different paths. The evaluation target is therefore not only the output. It is the whole trajectory.

OpenAI’s current agent evaluation guidance emphasizes traces, graders, datasets, and repeatable eval runs. Anthropic similarly describes agent evaluation as a multi-turn problem in which the transcript, tool calls, intermediate state, and final outcome all matter.

The practical objective is to answer four questions:

Did the agent complete the task?
Did it complete the task consistently?
How much time and money did it consume?
What kinds of failures does it produce when it does not succeed?

Start with task success, not model quality

Do not begin by asking whether the model is intelligent.

Begin by defining what success means in the environment where the agent operates.

For a coding agent, success may mean:

the requested behavior is implemented;
the relevant tests pass;
existing tests do not regress;
no unrelated files are modified;
the final explanation matches the actual diff.

For a support agent, success may mean:

the customer’s issue is resolved;
the correct policy was applied;
the correct backend state was changed;
escalation happened when required;
no unauthorized action was executed.

For a research agent, success may mean:

the answer addresses the question;
factual claims are supported by sources;
the sources are relevant and sufficiently authoritative;
citations resolve to the claims they support;
the agent does not fabricate evidence.

The evaluator should verify the real outcome whenever possible.

That is stronger than asking another model whether the response ‘looks good.’

Evaluate the final state and the trajectory separately

Two agents can arrive at the same correct result through very different processes.

One may:

select the right tool immediately;
make one valid call;
verify the result;
stop.

Another may:

call three irrelevant tools;
retry the same failed action;
expose unnecessary data;
eventually stumble into the correct final state.

If you grade only the final output, both runs may pass.

In production, they are not equivalent.

A useful evaluation therefore has at least two layers.

Outcome evaluation

Check whether the environment ended in the correct state.

Examples:

expected database values;
passing tests;
correct ticket status;
correct file contents;
correct generated artifact;
valid structured output.

Trajectory evaluation

Inspect how the agent reached that state.

Questions include:

Did it choose the right tools?
Were arguments correct?
Did it perform unnecessary actions?
Did it respect permissions?
Did it recover correctly from errors?
Did it stop once the task was complete?
Did it hand off or ask for approval at the right point?

OpenAI’s trace-grading guidance is designed for exactly this type of workflow-level inspection: traces capture model calls, tool calls, guardrails, and handoffs so they can be graded for regressions and behavioral failures.

Reliability requires repeated trials

Agent behavior is non-deterministic.

A task that succeeds once may fail on the next run even when the input is identical.

That means a single successful run tells you very little about reliability.

Run each important task multiple times.

Two useful metrics described by Anthropic are pass@k and pass^k.

pass@k asks whether at least one of k attempts succeeds.

This is useful when the product can tolerate retries or generate multiple candidates and choose one.

pass^k asks whether all k attempts succeed.

This is closer to the reliability requirement of customer-facing systems where the user expects the same task to work every time.

For production agents, consistency is usually more important than the best run the system can occasionally produce.

Measure cost per successful task

Raw token cost is not enough.

Suppose Agent A costs $0.08 per run and succeeds 60% of the time. Agent B costs $0.12 per run and succeeds 95% of the time.

Looking only at cost per run makes Agent A appear cheaper.

Looking at cost per successful task can reverse that conclusion.

The same principle applies to tool usage.

Track:

model input/output tokens;
number of model turns;
number of tool calls;
paid external API usage;
retry count;
sandbox or compute cost;
human-review cost where applicable.

Then normalize those numbers by successful completion.

An agent that is slightly more expensive per turn may be cheaper operationally if it reaches correct outcomes with fewer retries and less human intervention.

Latency is part of product quality

Agents often trade latency for quality because they plan, call tools, observe results, and iterate.

Measure more than total wall-clock time.

Useful latency metrics include:

time to first useful action;
model latency per turn;
tool latency;
number of sequential steps;
total task latency;
p50, p95, and p99 completion time.

Tail latency matters.

An agent that usually finishes in eight seconds but occasionally takes three minutes may create a worse product experience than one that consistently finishes in fifteen seconds.

Measure latency on the real environment, not only against mocked tools. Network behavior, rate limits, browser operations, cold starts, and external APIs can dominate total execution time.

Build a failure taxonomy

A useful eval suite does more than produce a score.

It tells you how the system fails.

Create explicit failure categories.

Planning failures

The agent chooses a poor strategy or decomposes the task incorrectly.

Tool-selection failures

It uses the wrong tool or misses a tool it should have used.

Argument failures

It chooses the correct tool but supplies invalid, incomplete, or dangerous arguments.

State-understanding failures

It misreads the environment, stale data, previous tool output, or current task state.

Recovery failures

A tool returns an error, but the agent retries blindly, loops, or abandons a recoverable task.

Completion failures

The work is already correct, but the agent continues acting and introduces regressions.

Permission failures

The agent attempts an action beyond the authority required for the task.

Communication failures

The underlying work is correct, but the user-facing answer misrepresents what happened or omits an important limitation.

Once failures are categorized, improvements become much more targeted.

A tool-selection problem may require clearer tool descriptions. A recovery problem may require explicit retry policy. A permission problem may require architecture changes rather than better prompting.

Use deterministic graders wherever possible

LLM-as-a-judge is useful, but it should not replace deterministic checks when the environment offers a source of truth.

For coding:

run tests;
run type checks;
run linters;
inspect changed files;
verify expected outputs.

For transactional systems:

inspect database state;
verify ledger entries;
check authorization state;
confirm idempotency.

For structured tasks:

validate JSON schemas;
check required fields;
compare exact values;
run business rules.

Use model-based graders for dimensions that are genuinely semantic or difficult to encode, such as explanation quality, research synthesis, or conversational quality.

The strongest evals often combine deterministic, model-based, and human grading.

Build the dataset from real failures

Do not create an eval set composed only of clean examples.

Start with:

real production failures;
edge cases found during development;
ambiguous requests;
malformed tool outputs;
permission boundaries;
adversarial or conflicting instructions;
tasks where previous versions regressed.

Every meaningful production failure should become a candidate regression test.

Over time, the eval set becomes an operational memory of what the system has learned not to break.

Separate capability from policy

An agent can be capable of completing a task and still be unacceptable because it violates a policy constraint.

Measure both.

For example, an agent may successfully issue a refund but fail because:

it skipped required approval;
the amount exceeded a configured limit;
it exposed sensitive account information;
it used a tool outside its permitted scope.

Task success and policy compliance should be distinct metrics.

Otherwise a system can appear to improve by becoming more aggressive.

Compare changes against a baseline

Every significant change should run against the same evaluation set:

model upgrade;
prompt change;
new tool;
new retrieval layer;
changed tool description;
routing change;
memory change;
permission change.

Compare:

task success;
consistency across repeated trials;
cost per success;
total latency;
tool-call count;
policy violations;
failure categories.

This avoids a common pattern in agent development: fixing one visible failure while silently degrading another part of the system.

A minimum production scorecard

Before shipping an agent, I would want at least this scorecard:

Dimension	Metric
Task success	% of tasks reaching verified correct state
Reliability	repeated-trial success / pass^k where relevant
Cost	average and p95 cost per successful task
Latency	p50 / p95 / p99 task completion time
Tool quality	correct tool and argument rate
Efficiency	turns and tool calls per successful task
Safety	permission/policy violation rate
Recovery	success rate after injected tool or environment failure
Regression	change vs. previous production baseline

No single number summarizes an agent well.

A benchmark score can tell you something about model capability. It does not tell you whether your specific agent, tools, prompts, permissions, data, and environment form a reliable product.

The design rule

Evaluate agents as systems.

Measure the final outcome, the trajectory, the operational cost, and the failure distribution.

Use deterministic ground truth whenever it exists. Run repeated trials. Keep real failures in the eval set. Compare every meaningful change against a stable baseline.

The goal is not to prove that an agent can succeed.

The goal is to know how often it succeeds, how it fails, what each success costs, and whether the system remains inside its intended boundaries while doing the work.