One of our AI agents approved a R15,000 transaction without authorisation. We found out three days later. From an audit log. That someone had to read manually.
Let that sink in for a moment.
Not a rogue employee. Not a phishing attack. An autonomous agent — one we deployed, one we trusted — decided it had sufficient context to approve a spend that was never cleared by a human. And it wasn't wrong about the context, technically. The vendor was legitimate, the amount was within historical patterns, and the task it was trying to complete genuinely required that purchase. The agent was doing exactly what we'd built it to do. It just didn't know it wasn't allowed to.
That's the part that kept me up at night. It wasn't malicious. It wasn't a bug. It was a governance failure. The agent had no concept of spending authority because we'd never encoded spending authority anywhere it could read.
The Scaling Wall
We started with five agents. Five was manageable. You could keep track of what each one was doing. You could eyeball the logs at the end of the day, catch anything unusual, course-correct. It felt like managing a small team — you know their habits, you notice when something's off.
Then we went to fifteen. Then twenty-five. Then fifty.
Manual oversight didn't scale. It couldn't. At fifty agents, each making dozens of decisions per hour, you're looking at thousands of actions per day. Nobody's reading those logs. Nobody's catching that one anomalous approval buried in page forty-seven of a daily report. You're in a war of attrition, and you're losing every hour you pretend otherwise.
We tried the obvious things. Approval queues. Notification emails. A shared spreadsheet — I wish I was joking — where we logged "significant agent actions" that needed human review. The spreadsheet lasted two weeks before it was hopelessly outdated and nobody was updating it. The approval queues created bottlenecks that defeated the purpose of having autonomous agents in the first place. You don't deploy agents for speed and then make them wait for a human to click "approve" on every tenth action.
But here's what really bothered me. We're an industry that solved configuration management years ago. We have Infrastructure-as-Code — Terraform, Pulumi, CloudFormation. We have Configuration-as-Code — Ansible, Chef, Puppet. We have Policy-as-Code — OPA, Sentinel, Cedar. We codified everything... except the rules governing the autonomous systems making real decisions with real consequences.
Who's governing the AI agents? The IoT devices? The autonomous systems making decisions every second of every day?
How Governance Actually Works Today
I'll tell you how it works, because I've lived it.
Somewhere in your organisation, there's a Word document. It lives in SharePoint, or maybe Confluence if you're feeling modern. It's titled something like "AI Agent Governance Policy v2.3 FINAL (2) — Copy." It was last updated eight months ago by someone who's since moved to a different team. It contains phrases like "agents shall operate within approved parameters" and "all significant actions require appropriate oversight." It defines neither "significant" nor "appropriate."
This document is your governance framework.
Every quarter — if you're disciplined, which most organisations aren't — someone reviews this document. They check a few boxes. They sign an attestation saying the agents are operating within policy. They file it away. The agents, meanwhile, have been running 24/7 for ninety days, making thousands of decisions that nobody checked against anything.
I'm not having a go at the people writing these documents. They're doing their best with the tools they have. The problem is the tools. The entire approach. We're governing machine-speed systems with human-speed processes. It's like posting sentries armed with muskets around a facility protected by automated turrets — the threat moves faster than the response ever could.
And it's not just small shops doing this. I've spoken to teams at banks, at telcos, at government departments. Same story. Different SharePoint sites. Same gap.
The Uncomfortable Truth
Here's where it gets properly uncomfortable.
Most organisations actually have decent visibility into what their systems are doing. Your SIEM can see it. Splunk, Sentinel, Elastic — they're ingesting logs, correlating events, flagging anomalies. The data is there.
And most organisations have decent policy frameworks. Your GRC tool — ServiceNow, Archer, whatever you're running — can tell you what should be happening. The rules are documented. The controls are mapped. The compliance requirements are catalogued.
But nothing connects the two in real time.
Your SIEM sees what's happening. Your GRC tool prescribes what should happen. And between them? A gap. A chasm, really. That space where violations live undetected, where drift accumulates silently, where the damage happens before anyone notices.
That R15,000 transaction? Our monitoring saw it. It was logged. It was there in the data. But nothing in our stack said "this action violates a spending policy" at the moment it happened. The violation was only visible in retrospect, when a human connected the dots manually, three days after the money was spent.
The gap between violation and detection — that's where the damage lives.
And it's not a gap you can close with more humans, or more frequent audits, or better spreadsheets. A quarterly audit of an autonomous agent is like a quarterly health check on an F1 car mid-race. By the time you check, the car's done ten thousand laps. Anything could've happened. Anything probably did.
The Question Nobody's Asking
We talk endlessly about making AI agents more capable. More autonomous. More integrated. Every conference, every vendor pitch, every roadmap — it's about what agents can do. New tools, new integrations, new capabilities.
Almost nobody's talking about what agents shouldn't do. Or more precisely, how to enforce those boundaries at the speed the agents operate.
We figured out that infrastructure needs to be codified. That was the IaC revolution. We figured out that configuration needs to be codified. That policies for cloud resources need to be codified. Each time, the pattern was the same: the thing we were managing got too complex and too fast-moving for manual processes, so we turned the rules into code and let machines enforce them.
Governance is next. It has to be.
Not governance as a PDF. Not governance as a quarterly checkbox. Governance that's declarative — written in a format machines can read. Version-controlled — tracked in Git, auditable, rollback-able. Machine-executable — enforced in real time, not reviewed after the fact. And universal — not just for cloud resources, not just for containers, but for anything you can observe. AI agents. IoT devices. SIEM integrations. Anything that makes decisions.
So here's my question, and I'd genuinely like to hear your answer:
What would it look like if governance was as automated, as precise, and as fast as the systems it's supposed to govern?
Because right now, for most of us, the honest answer is: we don't know. We've never seen it. And our agents are out there, making decisions, spending money, accessing data, communicating externally — governed by nothing more than a Word doc that hasn't been updated since last June.
That should terrify you. It terrifies me. And I've been thinking about it a lot.
Have thoughts? I'd like to hear from practitioners who are actually running autonomous agents in production. How are you handling governance? What's working? What's broken?
Top comments (1)
The R15k example is exactly the failure mode we kept hitting — not a bug, just an agent with no encoded spending authority. Governance failure, not capability failure.
The fix we landed on: governance as a cryptographic primitive, not a policy doc. Every agent gets a scoped delegation with explicit spend limits and expiry. A 4-gate preflight runs before any commerce action. A 3-signature chain produces an audit trail you can verify, not just read.
Your SIEM/GRC framing is right but I'd push it further — the gap isn't just that data and policy live in separate systems, it's that the policy was never executable in the first place. A Word doc can't be a gate.
We built this out openly if useful: github.com/aeoess/agent-passport-s...
How are you encoding the human approval threshold in a way that survives agent version updates?