There's a notion that an AI agent is basically an LLM enhanced with RAG, tools, MCPs, and instructions. In fact, there are several configurations we could call an AI agent, and they are not the same in terms of predictability and security.
When we talk about building production-grade AI, the real question is about determining what is actually safe, reliable, and useful to deploy in a business environment.
The Baseline: Traditional Software
For context, our software architectures looked like this for decades:
- Driven entirely by predefined business logic and rules.
- Algorithms solve problems in a strictly predictable way.
- Maximum security (hopefully, and compared to what's coming next).
The real world is complex, though, and not everything can be processed. Which is where AI comes into play: unstructured data, estimates, human specific metrics (tone, mood, sentiment).
LLMs as Components
The simplest AI usage is calling an LLM for a specific task, like
- Summarizing texts
- Extracting data from messy sources
- Classifying, labeling, and tagging
Here, the LLM is treated as an external black box to the system, turning fuzzy data into usable data. It makes no decisions. We manage to solve the tasks that are "impossible for algorithms," while retaining full predictability and security.
This is safe, but you probably won't call this approach agentic. And what if we need dynamic decision-making?
AI Agent as a Workflow
If we need fuzzy decision-making in our process, we can use LLM for it. In this approach, our system is a predefined graph, where different LLM nodes handle specialized reasoning steps with a strict set of possible outcomes.
Here is an example inspired by Dify's research agent template:
- A Planning LLM breaks the user's query into different research topics.
- A microservice executes web searches and data crawling for each topic.
- A Summary LLM turns search results into usable data.
- A Judge LLM evaluates the gathered data. If it's insufficient, it loops back to the Planner to produce more research angles.
- If the Judge decides we have what we need, the data is routed to a Report LLM which produces the report.
Flexibility increases because the system dynamically loops and adapts its plan based on the environment. Security and predictability naturally remain bounded by the strict control you have over the predefined paths and tools.
For many businesses, providing an agent this level of structured freedom is plenty.
Autonomous AI Agents
At the far edge of the spectrum, we drop the graph entirely. There is no predefined plan.
Instead, you provide the LLM with tools, instructions, and data, and the model plans and decides every step at runtime. This approach offers maximum flexibility in how a task is solved.
The way the agent will solve the task becomes unpredictable. Security concerns rise as a wide variety of tools are available to the agent.
We use techniques like guardrails, rigorous testing. We can still enforce control by deriving the permissions for the tools from the initial user request, or at least by defining an identity for the agent and restricting it from critical access.
The Hidden Menace
Unfortunately, the hype around autonomous agents leads to a dangerous architecture, a variant of the Autonomous AI Agent we described: the God-Agent. This happens when an organization builds an agent with:
- Full access to company systems
- Full access to data stores
- Expectations that it can be treated like a 'digital employee'
With a God-Agent, there is absolutely no predictability and security is thrown right out the window. Most will agree that this is a production anti-pattern. The blast radius is huge, and risks are business critical.
You have probably seen the news about companies losing entire databases or critical documents, or leaking data -- often, this is the work of a god-agent. Microsoft is dispatching AI security specialists to its AI buying customers in an effort to promote better practices and avoid such security breaches.
God agent is the outer edge of the agentic spectrum. What lies beyond it?
The Hard Boundary
When defining the boundaries of what your AI agent can do, it's critical to remember what we should not give to an AI agent.
An AI agent:
- Bears no responsibility
- Cannot be held accountable
- Has no liability
The hard boundary in production is this: if a task requires legal, professional, or financial responsibility, it stays with the human. (For now?)
Finding Your Sweet Spot
So, where do you draw the line for production?
Would you prefer strictly defined workflows, or are you willing to give an LLM free roam over the way to solve your tasks?
Let's debate in the comments.
Top comments (0)