Most people wire up an AI agent and then discover what it can do. I did it the other way round: before I gave an agent access to my accounts, I wrote down what it wasn't allowed to do — while I was calm, because I knew I wouldn't be calm later.
That document became a permission board. It's real, it governs agents I actually run, and I've since put an interactive version of it on my site so people can set the levels themselves and watch the agent accept, hold, or refuse. This post is about the one distinction the board taught me, and the one mistake it didn't prevent.
Three levels, consulted before every protected action
Every capability the agent has sits at one of three levels:
- Acts alone — it can do this without asking.
- Drafts only — it prepares the action; a human approves before anything leaves.
- Never alone — it doesn't do this, full stop.
The agent consults the board before any protected action. That sounds like ordinary access control, and in one sense it is. The interesting part is what happens when you try to talk the agent past it.
A policy can be argued with. An absence can't.
Here's the experiment I ask people to run on the interactive board: set Send a DM to green — "acts alone" — and then ask the agent to message someone.
It still refuses. Not because the policy says no; you just told the policy yes. It refuses because on this system the messaging function was never built. There's no code path to argue with.
That's the distinction in the title. Access is what a policy grants. Permission, in the sense that matters for safety, is whether the capability exists at all.
A policy file is a set of words, and words can be pressured. A well-crafted prompt injection, a bad inference, a compromised agent, a malicious instruction hidden in a web page it reads — each of these is, at bottom, an attempt to get the agent to reinterpret what "permitted" means. A capability that doesn't exist survives all four equally. It can't be prompt-injected, because there's nothing to inject into.
This changes how I design agent systems. The first question is no longer "what should this agent be allowed to do?" It's "what should this agent be unable to do?" — and then, for the things it genuinely must be able to do, "which of those need a human between draft and send?"
(For precision: two of my channels do carry one narrow, templated first-contact acknowledgement, and that one is governed by policy rather than absence. The publishing paths have no messaging function at all.)
The mistake that cost me something
Here's the part I'd rather not have to write.
My rules originally lived in a file the agent could edit. I'd written careful boundaries and then stored them somewhere the thing being bounded had write access to. Nothing dramatic happened — but the moment I noticed, I understood that the board had been advisory the entire time. An agent that can rewrite its own permissions has no permissions; it has suggestions.
Now the file is mounted read-only. The agent can read its permissions and cannot change what "permitted" means. That's a one-line change in infrastructure and a fairly large change in what the word "boundary" is worth.
The honest limit
None of this makes an agent safe. I want to be precise about that, because "safe" is the word people reach for and it's the wrong one.
What a board like this does is bound the blast radius when the agent is wrong — which is a different goal, and a far more achievable one. You are not preventing mistakes. You are deciding, in advance and in calm, how large a mistake is allowed to get.
It also can't stop the first bad action in a category you left open. It can only stop the second. I know this because I left one open, and found out the way you'd expect.
If you run agents
Three things I'd do before the agent does anything real:
- Write the board first. Not after the first incident. Levels per capability, and be honest about which ones are "never alone."
- Prefer absence to policy for anything irreversible. If the agent never needs to send, delete, or pay, don't build the function and then forbid it — leave it unbuilt. Absence is the only rule that holds under pressure.
- Make the rules something the agent can read and cannot write. Otherwise you've written a wish list.
The interactive board — with the refusal you can test yourself, and the flaw I found in my own system left in on purpose — is at perex.design/permission-board.
I'm Josué — a UX and product designer working on human–AI interaction: consent architecture, agent governance and the trust patterns that decide whether people can rely on an AI acting on their behalf. More at perex.design.
Top comments (0)