Most teams do not need an agent first.
They need a reliable system that can use a model, call a few tools, and complete a bounded task without creating unnecessary uncertainty.
That distinction matters because the word agent is often used for almost any application that combines an LLM with tools. Architecturally, however, there is an important difference between a workflow and an agent.
A workflow has a path that your code largely defines in advance. The model may classify, generate, extract, or choose among bounded options, but the application still controls the sequence.
An agent has more authority over the sequence itself. It can inspect the current state, decide what to do next, select tools, repeat steps, recover from failures, and stop when it believes the objective is complete.
Anthropic makes a similar distinction: workflows follow predefined orchestration paths, while agents dynamically direct their own processes and tool usage. OpenAI’s agent documentation likewise centers the runtime around an iterative loop of model decisions, tool execution, state, and stopping conditions.
The practical question is not which architecture sounds more advanced. It is how much decision authority the model actually needs.
The architecture ladder
Think of AI systems as a ladder of increasing autonomy.
- Single model call — one input, one output.
- Deterministic workflow — fixed steps with model calls inside them.
- Dynamic workflow — some routing or decomposition is model-driven, but application code still constrains the process.
- Agent — the model decides the next action repeatedly until a goal, blocker, or stop condition is reached.
Every step upward can increase flexibility. It can also increase latency, cost, observability requirements, and the number of failure modes.
The default engineering strategy should therefore be simple: start at the lowest level that can solve the task reliably.
When a workflow is the better architecture
Use a workflow when the task is mostly known before execution begins.
A good example is document processing:
- classify the document;
- extract required fields;
- validate the output;
- route exceptions to review;
- store the result.
The model can still be useful at several stages, but the application already knows what the stages are.
Workflows are especially strong when you need:
- predictable execution paths;
- consistent latency;
- bounded cost;
- explicit validation between steps;
- strong auditability;
- easy retries;
- clear ownership of failure states.
Several useful patterns fit this model.
Prompt chaining
One model call produces an intermediate result that becomes input to the next step. This is useful when a complex task can be decomposed cleanly and each step can be validated separately.
Routing
A classifier chooses among known downstream paths. Customer support is a common example: billing questions, account access, technical failures, and abuse reports can each enter a specialized flow.
Parallelization
Independent subtasks run at the same time and are combined afterward. This works well when the task can be divided without requiring each worker to know the others’ intermediate results.
Evaluator-optimizer loops
One model produces an answer and another evaluates it against explicit criteria. The loop can continue until a quality threshold is reached or a maximum number of attempts is exhausted.
These patterns can look sophisticated, but they remain workflows because the application still defines the operating structure.
When an agent is justified
An agent becomes useful when the important steps cannot be specified completely in advance.
Consider a coding task such as:
Find the root cause of this production regression, make the smallest safe fix, run the relevant tests, and explain what changed.
The system may need to:
- inspect different files depending on what it discovers;
- search logs;
- choose which tests to run;
- revise its hypothesis after a failed test;
- edit one or several files;
- stop and ask for approval before a risky action.
The exact path is not known when execution starts. A fixed workflow can attempt to encode every branch, but at some point the workflow becomes a hand-written imitation of the decision process the model could perform directly.
Agents are strongest when the task has four characteristics:
- The objective is clear but the path is not.
- The environment can provide feedback after each action.
- The model has useful tools for changing or inspecting that environment.
- The application can impose hard boundaries on what the agent is allowed to do.
That last requirement is critical. Autonomy without boundaries is not an architecture advantage; it is an operational liability.
A practical decision table
| Question | Prefer workflow | Prefer agent |
|---|---|---|
| Are the steps known in advance? | Usually yes | Usually no |
| Does the model need to choose the next action repeatedly? | Rarely | Yes |
| Is predictable latency important? | Strong fit | Harder |
| Is cost tightly bounded per run? | Easier | Harder |
| Can the environment return useful feedback after each action? | Helpful | Essential |
| Do you need strong reproducibility? | Strong fit | Requires more controls |
| Can failure be contained with permissions and stop conditions? | Still important | Mandatory |
| Does the task vary substantially from case to case? | May become brittle | Stronger fit |
The hidden cost of unnecessary agents
The largest mistake is not choosing a workflow when an agent would be better. It is building an agent where a workflow already solves the problem.
Agents create additional engineering requirements:
- tool permissions;
- state management;
- loop termination;
- retry policy;
- approval checkpoints;
- tracing;
- evaluation across multi-step trajectories;
- recovery when intermediate actions partially succeed.
A single model call can be evaluated by comparing inputs and outputs. A multi-step agent must often be evaluated as a trajectory: what it observed, what it decided, which tools it called, what changed in the environment, and whether the final state was actually correct.
This is why agent reliability is not only a model-quality problem. It is a systems problem.
The hybrid architecture is often the right answer
The choice is not binary.
A strong production system often uses a deterministic workflow around a bounded agent.
For example:
- validate the request;
- classify risk;
- create a sandbox;
- let an agent solve the task inside that sandbox;
- run deterministic checks;
- require approval for sensitive actions;
- persist the result and audit trail.
The workflow controls the system boundary. The agent handles the part where the path is genuinely dynamic.
This structure keeps autonomy where it creates value without giving the model authority over everything around it.
Five questions before adding an agent loop
Before implementing an agent, answer these questions:
1. What decision cannot be encoded reliably in normal application logic?
If you cannot identify that decision, you may not need an agent.
2. What ground truth can the system observe after each action?
Agents need feedback. Tests, API responses, file state, database results, browser state, and human approvals can all provide evidence about whether progress is real.
3. What is the maximum authority the agent needs?
Do not expose ten tools when the task needs three. Do not grant write access when read access is sufficient. Do not allow irreversible actions without a checkpoint.
4. How does the run stop?
Define success conditions, iteration limits, budget limits, timeouts, and escalation paths before production.
5. How will you evaluate the trajectory?
Measure more than the final answer. Track tool choice, unnecessary steps, retries, latency, token usage, cost, human interventions, and task completion.
The design rule
Workflows optimize for control.
Agents optimize for adaptability.
Most production systems need both, but not in equal amounts.
If the task has a stable path, encode that path. If the task requires runtime discovery, let the model make the decisions that genuinely cannot be predetermined. Then surround that autonomy with deterministic checks, permissions, observability, and stop conditions.
The goal is not to build the most agentic system.
The goal is to build the least autonomous system that can reliably solve the real problem.
Top comments (0)