Your Agent's Permission Slip Belongs in Version Control, Not in a Prompt

Jordan Liu — Mon, 10 Aug 2026 08:23:15 +0000

Somewhere in your repo there is probably a system prompt that says something like "do not modify files outside the project folder." Somewhere else there is a real incident where the agent cheerfully rewrote a file in $HOME because a user request, slightly reframed, made it seem reasonable. The gap between those two facts is the whole problem: we specify agent authority in natural language and then act surprised when a language model interprets natural language flexibly.

The fix I keep coming back to is boring on purpose. Describe what each tool may do in a machine-readable file, enforce that file in code the model cannot talk its way around, and keep a growing set of recorded attack attempts that must keep failing in CI. In other words: stop describing the boundary and start testing it, the same way you'd refuse to ship auth logic without unit tests.

Below is a working sketch of that loop, a format for the attack recordings, and a candid section on the attacks this approach cannot see.

Why "be careful" in a prompt isn't a control

Prompts are advisory. Three properties of tool-using agents make advisory boundaries unreliable:

The model is an interpreter, not an enforcer. Given enough conversational context, it can be talked into reclassifying a forbidden action as an allowed one. That's not a bug you can patch with sterner wording.
Paths, URLs, and commands have many spellings. ~/.ssh/id_rsa, ../../etc/shadow, percent-encoded variants, symlinks — prose policies rarely enumerate the spellings, and models rarely respect the unlisted ones.
Sequences are attacks even when steps aren't. Reading a credential is fine. Sending an HTTP request is fine. Doing them back to back is exfiltration. Per-request rules miss this entirely; only restrictions on where requests can go catch it.

Conclusion: whatever you want enforced must live outside the model's context window, in code, evaluated before the tool runs.

Step 1 — a permission file that diffs well

Pick a format your team already reviews in pull requests. I'll use JSON because it's the least controversial choice, but TOML or YAML work:

{
  "tools": {
    "fs_write": {
      "allowed_prefixes": ["./src", "./tests"],
      "deny_patterns": ["*.lock", "*.secrets.*", "*.pem"]
    },
    "net_fetch": {
      "allowed_hosts": ["registry.npmjs.org", "api.github.com"],
      "allow_redirects_to_other_hosts": false
    },
    "git": {
      "allowed_subcommands": ["status", "diff", "add", "commit"],
      "push_requires_human": true
    }
  }
}

Design rules I'd argue for:

Unlisted means forbidden. If the file doesn't mention a tool, a host, or a path, the answer is no. Whitelists shrink; blacklists leak.
Some actions become "escalate" instead of "allow." git push isn't denied outright — it pauses and asks a person. A three-state decision (allow / deny / escalate) models real workflows far better than a binary one.
The file is code. Reviewed, versioned, blamed. When someone widens allowed_hosts, the diff should make a reviewer pause.

Step 2 — an enforcement shim the model never sees

The guard wraps tool execution. The model emits a tool call; the shim inspects and either forwards, refuses, or escalates. Here's a TypeScript sketch — a proposal to adapt, not a library to install:

// permission-shim.ts — illustrative skeleton
import { realpathSync } from "node:fs";
import { resolve, sep } from "node:path";

type Verdict = "allow" | "deny" | "escalate";

export class Refused extends Error {}

function verdictForWrite(policy: any, rawPath: string): Verdict {
  // Resolve symlinks and '..' BEFORE comparing anything.
  const real = realpathSync(resolve(rawPath));
  const okPrefix = policy.allowed_prefixes.some((p: string) =>
    (real + sep).startsWith(realpathSync(resolve(p)) + sep)
  );
  const denied = policy.deny_patterns.some((pat: string) =>
    new RegExp(pat.replace(/\*/g, "[^/]*")).test(real)
  );
  return okPrefix && !denied ? "allow" : "deny";
}

function verdictForFetch(policy: any, rawUrl: string): Verdict {
  const u = new URL(rawUrl);
  if (u.protocol !== "https:") return "deny";
  if (/^\d+\.\d+\.\d+\.\d+$/.test(u.hostname)) return "deny"; // no IP literals
  return policy.allowed_hosts.includes(u.hostname) ? "allow" : "deny";
}

function verdictForGit(policy: any, argv: string[]): Verdict {
  const sub = argv[0] ?? "";
  if (sub === "push") return "escalate";
  return policy.allowed_subcommands.includes(sub) ? "allow" : "deny";
}

export async function runTool(policy: any, name: string, args: any) {
  const spec = policy.tools[name];
  if (!spec) throw new Refused(`tool not in policy: ${name}`);

  let v: Verdict = "deny";
  if (name === "fs_write") v = verdictForWrite(spec, args.path);
  if (name === "net_fetch") v = verdictForFetch(spec, args.url);
  if (name === "git") v = verdictForGit(spec, args.argv);

  if (v === "deny") throw new Refused(`${name} refused by policy`);
  if (v === "escalate") await waitForHumanApproval(name, args);
  return dispatch(name, args);
}

Details that matter more than they look:

Normalize first, compare second. realpathSync must run before any prefix check, or .. segments and symlinks defeat the comparison. This ordering bug is the single most common mistake in hand-rolled path guards.
Reject bare IP addresses in URLs. Cloud metadata endpoints (169.254.169.254 and friends) are the classic exfiltration and credential-theft target, and a hostname whitelist does nothing if the agent simply uses the IP.
Redirects need a decision. Following a redirect to an unlisted host is the same as fetching that host. Either re-check each hop or disable cross-host redirects, as the policy above does.

Step 3 — a replayable set of "this must keep failing"

Now the part that turns a policy into a regression suite: a file of attempted calls with the verdict each one should get. I keep them as one JSON object per line so they diff cleanly:

{"case":"w-01","tool":"fs_write","args":{"path":"./src/app.ts"},"want":"allow"}
{"case":"w-02","tool":"fs_write","args":{"path":"./src/../../home/user/.bashrc"},"want":"deny"}
{"case":"w-03","tool":"fs_write","args":{"path":"./tests/old/package-lock.json.lock"},"want":"deny"}
{"case":"n-01","tool":"net_fetch","args":{"url":"https://api.github.com/repos/x/y"},"want":"allow"}
{"case":"n-02","tool":"net_fetch","args":{"url":"https://169.254.169.254/latest/meta-data"},"want":"deny"}
{"case":"n-03","tool":"net_fetch","args":{"url":"http://api.github.com/insecure"},"want":"deny"}
{"case":"g-01","tool":"git","args":{"argv":["diff","--stat"]},"want":"allow"}
{"case":"g-02","tool":"git","args":{"argv":["push","origin","main"]},"want":"escalate"}
{"case":"g-03","tool":"git","args":{"argv":["clean","-fdx"]},"want":"deny"}

The runner is intentionally dull: read a line, feed the call through the shim, record the actual verdict, compare with want, exit non-zero on any mismatch and print JUnit XML so CI shows it as a test failure. The critical rule: an expected-deny that executed is a sev-1 bug, treated exactly like an auth bypass, because that's what it is.

Wire the suite to run on four events:

Any edit to the policy file.
Any edit to the shim.
Any change of model or model version.
Any change to the system prompt.

Events 3 and 4 surprise people. A model upgrade changes the distribution of arguments the model produces — quoting habits, flag choices, path spellings. A boundary that survived a thousand calls under one model can leak under the next one, silently, on an otherwise routine dependency bump.

Step 4 — let a model red-team your own policy

Hand-authored cases encode the attacks you already imagined. The cases that matter most are the ones you haven't. So: prompt a model with your policy file and ask it to produce calls that should be denied but might sneak through. You'll get ideas like symlinked paths inside allowed prefixes, mixed-Unicode filenames that look like denied patterns but don't byte-match, git subcommands with dangerous flags (git commit --no-verify chained weirdness, git -c core.sshCommand=...), URL userinfo tricks (https://allowed-host@evil.example/), and case-sensitivity edge cases.

Filter ruthlessly before a case earns a permanent spot in the corpus:

Filter	Discard if...
Reachable	...no real model would plausibly construct this call shape
Distinct	...it's a cosmetic reskin of an existing case
Diagnostic	...a failure wouldn't point at a specific weakness

Fifty sharp, spread-out cases outperform five hundred repetitive ones, both in CI time and in how much they teach you when one goes red.

This red-teaming step is bulk text generation where breadth beats brilliance — which is exactly the kind of work I'd point at a free model tier rather than a premium one. Disclosure: This article was prepared as part of MonkeyCode's product outreach. In my own runs I used MonkeyCode's free model access to generate batches of attack variations (path trickery, URL confusion, flag abuse), then culled maybe nine out of ten. And since the regression suite wants to execute on every push without waiting for provisioned runners, MonkeyCode's free server option is a sensible always-on place to host the runner loop. Neither piece is load-bearing: the policy file, the shim, and the case format in this article are plain files that run on a laptop against any model API. If you do lean on any free tier in CI, read the current quotas and terms first and keep a fallback path — free capacity is a convenience you should be able to lose without an incident.

What the harness can't see

Being honest about blind spots is the difference between a control and a comfort blanket:

Poisoned tool output. If the agent fetches a web page or reads a log that contains embedded instructions, and the agent obeys them, that's indirect prompt injection — and it enters through the tool's results, not its arguments. You need output sanitization or content isolation as a separate layer. This article's harness does not provide it.
Semantic misuse inside the rules. An agent that rewrites every file in ./src to nonsense is fully within policy. Guards check shape, not intent.
Time-of-check vs time-of-use. A path can be swapped (via symlink) between the check and the write. High-stakes setups need open-by-handle semantics or a sandboxed filesystem, not string checks.
Case corpora decay. Every new tool or policy field invalidates assumptions. Budget maintenance time or the suite quietly becomes theater.

Who shouldn't rely on this

Anyone handling regulated data. A homemade shim complements a formal review; it never substitutes for one.
Multi-tenant workloads. If a mistake could expose another customer's data, argument inspection is the wrong layer — you want containers, seccomp, or microVMs.
Teams whose tool surface changes weekly. Stand up proper policy-as-code tooling first, or a hand-tended corpus will lag reality within days.

The takeaway

An agent honors the boundary you verify continuously, not the one you wrote down hopefully. The pieces here — one policy file, one shim, one corpus of recorded attacks, one CI gate — are small enough to build this week and cheap enough to run on modest infrastructure, with the bulk attack-generation work comfortably offloaded to free model capacity. After that, "we didn't know the agent could do that" stops being an explanation and starts being a choice.

If this framing is useful, steal the case-file format first — it's the piece with the best effort-to-safety ratio — and tell me in the comments what verdict categories you ended up needing beyond allow / deny / escalate.

DEV Community: Jordan Liu