Most people hear "agentic AI" and picture Skynet ordering pizza. The reality is less cinematic and more like a well-trained dog that knows which rooms it's allowed in.
I've spent the last few months pulling apart agent frameworks. Not reading whitepapers — actually running them, breaking them, watching where they fail. The pattern is consistent: the difference between an AI that's useful and one that's dangerous isn't the model. It's the boundary you draw around it.
An agent is just a loop. It thinks, acts, observes, and repeats. What makes it "agentic" is that it chooses its own next action. Nobody is clicking the buttons for it. And that's where things get messy.
The autonomy spectrum nobody talks about
There's this idea floating around that agents are binary — either a chatbot or fully autonomous. That's wrong. Autonomy is a dial, not a switch.
Think of it like driving assists. Level 1 is cruise control — the AI suggests, you confirm. Level 2 is lane-keeping — the AI executes within a narrow lane, but your hands are on the wheel. Level 3 is where it gets uncomfortable: the AI drives, but you need to take over when it signals. And Level 4 is the thing nobody has shipped yet.
Most "agentic" systems right now are Level 2 pretending to be Level 4 in the marketing copy.
I like to frame it with a question: if this agent transferred $10,000 to the wrong account right now, who gets fired? If the answer is "nobody, because it can't do that," you've got guardrails. If the answer involves HR, you've got a problem.
Where agents actually break
I ran the same task across three frameworks — give an agent a codebase, ask it to fix a bug, open a PR. Here's what happened:
The first agent got stuck in a loop. It kept reading the same file, suggesting the same fix, failing the test, then reading the file again. Like a Roomba bouncing off the same chair leg for 45 minutes.
The second agent fixed the bug but also — unprompted — refactored three unrelated files. Clean code, but wrong. Nobody asked for that.
The third agent worked. It found the bug, patched it, ran the tests, opened the PR. The difference wasn't the model. All three used the same one. The difference was the constraint system: the third agent had a hard limit on tool calls per task and a deny-by-default sandbox.
This isn't a model problem. It's an architecture problem.
The three controls that matter
If you're building or evaluating agent systems, I'd look at these three things before anything else:
Tool access is not permission. Just because an agent can call a shell command doesn't mean it should. A tight allowlist — specific binaries, specific flags — beats a fuzzy blocklist every time. Blocklists are for known threats. Agents generate novel ones.
Step limits are a safety feature, not a limitation. An agent that loops 200 times isn't "being thorough." It's lost. Cap the steps per task. If it can't do the job in 20 iterations, the task is either too big or the agent is stuck. Either way, looping more won't help.
Human-in-the-loop isn't optional for writes. Reads? Fine. Searches? Fine. But the moment the agent wants to push code, send an email, or modify a database — that should require a human to click "yes." Not a policy document saying it's required. An actual button.
I've seen teams skip this because it "slows things down." It does. That's the point. Speed without verification in an autonomous system is just faster failure.
The game analogy
I think about agent design the same way I think about NPC behavior in open-world games. You don't give every NPC the ability to modify the world. You give them a behavior tree — a fixed set of things they can do, conditions for when they do them, and hard boundaries they can't cross no matter what.
The shopkeeper NPC doesn't suddenly decide to join the main quest. The guard NPC doesn't open the gate because a player asked nicely in the chat box. Their autonomy is scoped.
AI agents need the same treatment. Scope autonomy to the task. Nothing more.
What I'm watching
Tool definition languages and agent-to-agent protocols are where the real action is right now. MCP (Model Context Protocol) is getting traction. A2A (Agent-to-Agent) from Google is interesting but early. The fight isn't about whose model is smarter — it's about whose agents can interoperate without blowing up.
But interoperability brings its own risks. Two constrained agents working together can sometimes do what neither could do alone. The combination needs its own set of constraints.
If you're building agents, start here: define what the agent is not allowed to do before you define what it can do. Deny-by-default. Your future self — and whoever signs off on the security review — will thank you.
Top comments (0)