A Policy Kernel, Not a Prompt

#ai #opensource #security #softwareengineering

Originally published on humbertovillanueva.dev.

An agent that runs all night with access to my email, my files and my repositories has to answer one question before anything else: what exactly can it touch? If the answer is a paragraph in the prompt, the honest reply is "whatever it decides." A model can be talked out of an instruction. It cannot be talked past a function that never calls the tool.

That is the whole idea behind the policy kernel in Garu, the open-source runtime I built for always-on agents. These are the decisions that shaped it.

The rules live in the agent's file, and the kernel reads them

An agent is one YAML file. Next to its model, its tools and its schedule sits a list of rules. Each rule names a tool, optionally a condition on the arguments, and one of three actions: allow, ask, or block. This is a trimmed version of the rules for Bea, the agent that sorts my inbox every weekday morning:

policy:
  - tool: gmail.search_threads      # reading: always fine
    action: allow
  - tool: gmail.label_thread        # filing: one click undoes it
    action: allow
  - tool: gmail.create_draft        # a reply to an address nobody reads
    when: { to: { matches: "no-?reply|notifications?", ignoreCase: true } }
    action: block
  - tool: gmail.create_draft        # words in my name: I see each one first
    action: ask
  - tool: "*"
    action: block

Every tool call the model wants to make goes to the kernel first. The kernel reads the list top to bottom, and the first rule that matches decides. The model never sees this list. It only sees the result: the tool ran, the tool is waiting for a person, or the tool was refused.

Nothing matching means ask

The default matters more than any single rule. If no rule matches a call, the kernel does not allow it quietly and it does not refuse it quietly. It asks. A card appears in the inbox and on my phone, and the run waits. If nobody answers within thirty minutes, the answer is no.

That last part took me a while to accept. An unattended agent that gets a yes by default is not unattended; it is unsupervised. Silence has to be a deny, even when the thing it was asking for was probably fine.

Hide what can never be allowed

If a rule blocks a tool for every possible argument, the model should not know the tool exists. Garu removes it from the list the model is offered. Rook, the agent that watches my repository through GitHub's hosted server, is offered three of that server's forty-six tools. It cannot waste a turn trying to open a pull request, and there is nothing to inject a request into.

Engineering rule: a rule the model cannot see is a rule it cannot argue with. Hide the tool, not just the permission.

Arguments are the real boundary

Tool names are too coarse. "Write a file" is harmless in the agent's own folder and dangerous everywhere else. So rules match on arguments too: a write is allowed when the path matches briefs/ followed by today's date, a fetch is allowed for four named hosts, a draft is blocked when the recipient looks like an automated sender. Tomay, the agent that writes my morning brief, has exactly one write rule and never pauses, because the one thing it may write is the one thing it does.

The kernel also says why. Each rule can carry a reason, and the reason is what the inbox card shows and what the model reads when a call is refused. "Blocked: a reply to an automated sender goes nowhere" teaches more than a bare error.

A sandbox for the rest

Policy decides whether a tool runs. It does not decide what a tool server can reach while it runs. For that, each tool server can run in its own container: no network, a read-only root, only the agent's workspace mounted, destroyed after the run. The policy and the sandbox answer different questions, and an agent I would leave alone overnight needs both.

What none of this stops

The kernel stops a blocked tool. It does not stop a model from choosing a bad argument for a tool it is allowed to use. Text in an email or a web page can still talk the model into something, and if that something is inside an allowed rule, it happens. The defense is the shape of the rules: allow reads, allow writes to the agent's own folder, and make everything that leaves the machine ask. Narrow allows are the design, not a workaround.

What I would build first

Put the decision in code between the model and the tools, never in the prompt.
Make "nothing matched" mean ask, and make an unanswered ask a no.
Hide every tool the rules could never allow.
Match on arguments, and attach a reason to every rule.
Record every decision, so a 3 a.m. action can be replayed in the morning.

The result

The test I keep applying is simple: would I let this run while I sleep? With the rules in code, the record on disk and a cost cap on every run, the answer became yes. That is the only feature that matters.

Garu is an independent open-source project under the Apache-2.0 license, unrelated to my employer. Everything in this note comes from its public repository: github.com/humbertovillanueva/garu.

DEV Community