DEV Community

Cover image for We gave software the ability to act, then forgot to supervise it
Mr Recruiter
Mr Recruiter

Posted on

We gave software the ability to act, then forgot to supervise it

Something changed about AI at work this year and I don't think everyone clocked it.

For a while, AI meant a chatbot. You asked, it answered, you decided what to do with the answer. The worst it could do on its own was make something up and hope you pasted it into a doc without checking. Bounded. A person was always the one who actually did anything.

That's over. The new thing is agents, and an agent doesn't wait to be asked. It acts. It reads the inbox, decides something, sends a reply, opens the CRM, pulls a list, updates a record, moves to the next task, and loops, fast, with nobody watching any single step.

The distance between those two things is enormous. Most org charts haven't noticed it exists.

Here's the way I've started describing an agent to people who don't live in security. It's a new hire. One who never sleeps, never takes a weekend, got onboarded in about thirty seconds with zero paperwork, and was handed the keys to several systems because scoping the access properly was annoying and everyone was busy. You would never let a human employee in the building on those terms. We do it for agents constantly, and we call it a productivity win.

The reason this slips past companies is that the tools built to catch problems assume a human is on the other end.

An agent doesn't trip a multi-factor prompt. It logged in once and just kept going. It doesn't leave the kind of trail your monitoring was designed to read, because that trail assumes a person, at a time, from a place. And it doesn't stop at 6pm. It's the quiet one still working at 2am, using access nobody's looked at since the day it was set up.

So the whole model of "someone did a thing, let's see who and when" doesn't fit. The actor is software, running non-stop, spending credentials it was given and never once questioned.

And the autonomy is the actual risk, not some side effect of it.

Think about it this way. A chatbot given a bad instruction produces a bad sentence. You read it, roll your eyes, ignore it. No harm. An agent given a bad instruction produces a bad action, and the action already happened by the time anyone could have objected. That's the whole difference, and it's a big one.

There's data catching up to this now. Forrester named AI agent threats the top security risk for chief information security officers heading into 2026. And a 2026 insider-risk report from Ponemon found something that stuck with me: 92 percent of organisations say generative AI has changed how their people access and share data, while only 13 percent have any formal policy for it. Agents widen exactly that gap, and they widen it in the worst possible direction, by pulling the human out of the loop entirely.

The frustrating and slightly reassuring truth is that the fix isn't some new category of magic. It's the boring discipline we already know, applied to something that moves faster and never rests.

Give every agent its own identity instead of letting it borrow a person's login, so you can actually tell what did what. Give it the narrowest slice of access its job needs and nothing spare. Put a human checkpoint in front of the things you can't take back, the sends, the payments, the deletions, so autonomy stops short of the irreversible. And watch what the agent does, not just whether it logged in, because with agents the login was never the interesting part.

None of that is clever. It's least privilege and monitoring, advice so old it's almost embarrassing to repeat. Agents just raised the price of ignoring it, because the distance between a mistake and its consequences shrank to about a second.

This is a lot of where my time goes at Arclogiq now, and it usually starts with a question that makes the room go quiet. For every agent you've turned on, if someone fed it a bad instruction tomorrow, what's the worst thing it's currently allowed to do? People rarely know. The answer is almost always worse than they'd guessed, and narrowing it down is most of the job.

We spent years learning to secure the humans. The humans aren't the ones we handed the keys to this year.

Top comments (0)