Day 11/90: Does Your AI App Need an Agent—or Just a Workflow?

In Day 10, I explored a problem in AI tool calling: an LLM can generate valid JSON and still choose the wrong action.

But there’s a more fundamental question:

Should the model be making that decision at all?

For Day 11 of my AI engineering journey, I’m exploring when to use a workflow, when to introduce an agent, and what that decision means for production systems.

1. A tool call doesn't automatically make an agent

Imagine a customer asks:

My order arrived damaged. Can I get a refund?

The application needs to:

1. Identify the order.
2. Retrieve its details.
3. Validate the refund policy.
4. Decide whether the request qualifies.
5. Explain the outcome to the customer.

If the application follows a predefined sequence and uses an LLM only to extract information or draft a response, it's primarily a workflow.

An agent gives the model greater control over execution: selecting tools, deciding what information to investigate next, and determining whether it has enough evidence to proceed.

The key distinction isn't whether the system uses tools.

It's who controls the next step.

2. Workflow vs. agent: Where does control live?

Workflow

The application defines the execution path. The model handles specific tasks within that path.

order = get_order(order_id)
policy = get_refund_policy()

eligible = check_eligibility(order, policy)

response = generate_response(
    order, eligible
)

The sequence is predictable, easier to test, and generally simpler to debug.

Agent

The model chooses its next action based on the task and the results of previous actions.

while not done:
    action = agent.decide(context)
    result = execute_tool(action)
    context.append(result)
    done = agent.should_stop(context)

This is a simplified illustration, not a complete production implementation. A real system also needs tool validation, authorization, error handling, and execution limits.

The advantage is flexibility. The trade-off is a larger space of possible execution paths.

3. When should you use each?

A workflow is often the better choice when:

• The business process is well-defined.
• The required steps are known in advance.
• Decisions can be expressed as explicit rules.
• Predictability and straightforward testing matter most.

An agent becomes more useful when:

• The next step depends on information discovered along the way.
• The task requires selecting between multiple tools or information sources.
• The investigation cannot be fully specified in advance.
• Dynamic planning provides a measurable benefit.

For example, processing a standard refund may fit a workflow. Investigating a complicated support issue across order history, shipping records, and troubleshooting documentation may benefit from an agent.

Even then, a hybrid architecture can be the better design: let the agent investigate while deterministic application code enforces business rules and permissions.

4. More autonomy means more things to evaluate

A successful final answer doesn't prove that an agent behaved correctly.

For a production system, I'd evaluate:

• Task success: Did it actually solve the user's problem?
• Tool selection: Did it choose appropriate actions?
• Evidence quality: Did it gather enough reliable information?
• Efficiency: Did it repeat calls or perform unnecessary steps?
• Termination: Did it stop when the task was complete?
• Safety: Did backend controls prevent unauthorized actions?
• Performance: What were the latency and cost per successful task?

One particularly important distinction: a model can decide that a refund should be issued, but it should not be the authority that grants itself permission to issue one.

Authorization, eligibility rules, and transaction limits belong in enforceable application logic.

5. The experiment I'd run

Rather than assuming an agent is better, I'd compare two implementations on the same support-request dataset.

Metric	What I'd measure
Task success	Percentage of requests completed correctly
Tool efficiency	Average tool calls per request
Latency	Median and p95 completion time
Cost	Average cost per successful task
Reliability	Failure rate across repeated runs
Safety	Unauthorized actions blocked by backend controls

I'd also include ambiguous requests that require clarification, because that's where dynamic decision-making might provide a meaningful advantage.

These are proposed evaluation criteria, not measured results.

The goal isn't to prove that agents are better. It's to determine whether their additional flexibility justifies their operational complexity.

6. What the research says

Two useful references helped shape this distinction:

• Anthropic — Building Effective Agents: Practical guidance on workflows, agents, and choosing the simplest architecture that meets the requirements.
• ReAct: Synergizing Reasoning and Acting in Language Models: Research on combining reasoning and actions, allowing observations from tool use to inform subsequent decisions.

They approach the subject differently, but both are useful for understanding how model-driven decisions can interact with application execution.

My takeaway

I'm starting to see agent design less as a framework choice and more as a question of where autonomy actually adds value.

A workflow gives you a predefined path. An agent gives the model more control over the path. Neither is automatically the better architecture.

My rule of thumb:

Start with a workflow. Introduce an agent when dynamic decisions measurably improve task completion. Keep authorization and business rules enforceable outside the model.

Autonomy is an architectural trade-off, not a feature checkbox.

Your turn

When building AI applications, what makes you decide that a workflow is no longer sufficient and an agent is worth the added complexity?