- Book: AI That Acts
- The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
Almost every agent starts the same way. You give it a service account so it
can reach the systems it needs, you write tools against that account, and it
works.
What you have built is a component that can do anything any user can do,
driven by a language model, reading text from support tickets and web pages
and uploaded documents. The gap between "the agent can do everything" and "the
agent did something it should not" is one persuasive paragraph in an untrusted
input.
The fix is not a better prompt. It is that the agent should never have held
that authority in the first place.
The rule
An agent's capabilities are a subset of the capabilities of the user it is
acting for. Never a superset, never a different set.
If Ana cannot refund another customer's order in the UI, the agent acting for
Ana cannot either, not because the prompt says so, but because the tool has no
credential that would allow it.
Capabilities as data
export type Capability =
| { kind: "orders.read"; userId: string }
| { kind: "orders.refund"; userId: string; maxCents: number }
| { kind: "email.send"; fromAddress: string; toDomains: readonly string[] }
| { kind: "docs.search"; docSets: readonly string[] };
export type Caps = readonly Capability[];
export function capsFor(user: User, session: Session): Caps {
const caps: Capability[] = [
{ kind: "orders.read", userId: user.id },
{ kind: "docs.search", docSets: user.entitledDocSets },
];
if (user.roles.includes("support")) {
caps.push({ kind: "orders.refund", userId: user.id, maxCents: 10_000 });
caps.push({ kind: "email.send",
fromAddress: `${user.id}@support.acme.com`,
toDomains: ["acme.com"] });
}
if (session.elevated) {
// step-up auth widens the ceiling, and only for this session
caps.push({ kind: "orders.refund", userId: user.id, maxCents: 100_000 });
}
return caps;
}
Two things are already better than a role string. The limits are values,
so maxCents can differ per user, per plan, per session, and the capability
carries the userId, so a tool cannot act on behalf of someone else even if
the model asks it to.
Deriving capabilities once, at the start of a run, also means the set cannot
drift mid-conversation as the model talks itself into something.
Tools check capabilities, not requests
function requires<K extends Capability["kind"]>(
caps: Caps, kind: K,
): Extract<Capability, { kind: K }> {
const c = caps.find((x) => x.kind === kind);
if (!c) throw new NotPermitted(kind);
return c as Extract<Capability, { kind: K }>;
}
const refundOrder = tool({
name: "refund_order",
schema: z.object({ orderId: z.string(), amountCents: z.number().int().positive() }),
async run({ orderId, amountCents }, ctx) {
const cap = requires(ctx.caps, "orders.refund");
if (amountCents > cap.maxCents) {
throw new NotPermitted(
`refund ${amountCents} exceeds this session's limit of ${cap.maxCents}`);
}
const order = await db.order.findUnique({ where: { id: orderId } });
if (!order) throw new NotFound("order");
if (order.userId !== cap.userId) throw new NotPermitted("not this user's order");
return refunds.create(order.id, amountCents, { actor: cap.userId });
},
});
The ownership check uses cap.userId, not an id from the model's arguments.
That distinction is the whole control: arguments are attacker-influenced,
capabilities are not.
actor: cap.userId on the write means your audit log records the human, not
the service account, which is what makes an incident investigable afterwards.
Filter the tool list to what is permitted
A tool the model cannot use should not be in its context at all.
export function toolsFor(caps: Caps) {
return ALL_TOOLS.filter((t) => t.requires.every(
(k) => caps.some((c) => c.kind === k)));
}
const res = await client.messages.create({
model: MODEL,
tools: toolDefs(toolsFor(ctx.caps)),
messages,
});
Three benefits. The model cannot attempt what it cannot see, so you get fewer
refusals to explain. The context is smaller and cheaper. And an attempted call
to an unlisted tool becomes a strong signal — a legitimate model does not
invent tool names, so log it loudly.
Keep the server-side check anyway. Filtering is an ergonomic improvement, not
a security boundary.
Credentials scoped to the request, never ambient
The strongest version pushes scoping below your code, so a bug in the checks
above still cannot cross a boundary.
export async function withUserDb<T>(
cap: { userId: string; tenantId: string },
fn: (db: Client) => Promise<T>,
): Promise<T> {
const client = await pool.connect();
try {
await client.query("BEGIN");
await client.query("SET LOCAL app.user_id = $1", [cap.userId]);
await client.query("SET LOCAL app.tenant_id = $1", [cap.tenantId]);
const out = await fn(client);
await client.query("COMMIT");
return out;
} catch (e) {
await client.query("ROLLBACK"); throw e;
} finally {
client.release();
}
}
With row-level security reading those settings, a tool that forgets its
ownership check still cannot read another user's rows. SET LOCAL inside the
transaction is essential — a session-level SET on a pooled connection leaks
to the next borrower.
The same idea applies to outbound APIs: mint a short-lived token carrying the
user's scopes rather than reusing a long-lived service key.
Sub-agents inherit, never widen
If your agent spawns another agent, pass the same capabilities down:
export function spawn(parent: Ctx, task: string, subset: Caps) {
const invalid = subset.filter((c) => !parent.caps.includes(c));
if (invalid.length) throw new Error("sub-agent cannot exceed parent caps");
return runAgent(task, { ...parent, caps: subset });
}
The check makes privilege escalation through delegation impossible to write.
Narrowing is fine and often wise — a research sub-agent gets docs.search and
nothing else.
Say why, in terms the model can use
catch (err) {
if (err instanceof NotPermitted) {
return errorResult(block.id,
`Not permitted: ${err.message}. Do not retry. Tell the user this ` +
`requires approval from someone with higher access.`);
}
throw err;
}
"Do not retry" prevents the loop. Telling the model what the user should do
next turns a dead end into a useful reply, rather than an agent that
apologises three times and gives up.
Be careful what the message reveals. "Booking bk_123 is not available" is
better than "that booking belongs to another user" — the second confirms the
record exists, which is an information disclosure through an error string.
Audit what was attempted, not just what happened
logger.info("agent_capability_check", {
runId: ctx.runId,
actorUserId: ctx.userId,
tool: block.name,
granted: ok,
reason: ok ? undefined : err.message,
sourcesInContext: ctx.retrievedDocIds,
});
granted: false events are the ones to alert on. A support agent whose run
attempted refund_order for 500,000 cents did not misfire on its own, something
in that context asked for it, and sourcesInContext tells you which document
was in the window when it did.
Before production
Six checks. Capabilities derived from the authenticated user at run start.
Tools enforcing capabilities rather than trusting arguments. Ownership checked
against the capability, not the model's input. Tool list filtered to what is
permitted. Credentials scoped per request, not ambient. Denied attempts
audited with the context that produced them.
None of it depends on the model behaving. That is the point — every other
defence assumes a model that can be persuaded, and this one assumes it already
was.
If this was useful
AI That Acts covers giving an agent
real authority safely — capability design, ownership checks, scoped
credentials, and error messages that neither leak nor cause loops.
Prompt-injection defence in depth is book five. The series is at
xgabriel.com/ai-in-typescript.
Top comments (0)