DEV Community

Rain S
Rain S

Posted on

Why AI Writes Need Risk Tiers: The R0-R5 Tool Risk Model

The previous piece, "Runtime over Prompt", argued that the security boundary belongs on the execution path — immediately before a tool call can produce a side effect.

This one goes a level deeper. Once the boundary sits on the execution path, the runtime has to answer a question it can't dodge: should this tool be allowed to run at all?

And the operations enterprises actually hesitate over are rarely reads. Querying a customer is fine. Updating one? Emailing one? Deleting one?

1. Reads and writes are not symmetric

When an AI gets a read wrong, it "saw it wrong" — you fix it and move on. The blast radius is small.

When an AI gets a write wrong, it "broke it" — a wrong order amount, an overwritten customer record, an email sent to the wrong person. Much of it is irreversible or expensive to undo.

So there's a gap between "let it look" and "let it act", and the gap is really this: should every operation be governed by the same policy?

Obviously not. Querying can be automatic. Changing a contract needs a human. Deleting data shouldn't be allowed at all.

That's what tool risk tiering is for: give every AI operation a risk level, then let the level decide the execution policy — instead of a blanket "all automatic" or "all human-reviewed".

2. R0-R5: a tiering you can actually ship

Six levels, low to high, each mapping to one execution policy:

Tier Meaning Example Execution policy
R0 Informational Explanations, summary stats Automatic
R1 Read List customers Automatic
R2 Low-risk write Update a note Policy decides (governance-configurable)
R3 Business-sensitive write Create a follow-up task, change an order Human confirmation
R4 High-impact action Operations needing dual approval Dual approval
R5 Irreversible / external action Delete data, send email Blocked

Three design decisions worth calling out:

  • Reads automatic, writes confirmed, high risk blocked. Low-risk work stays automatic; high-risk work gets a human; irreversible work is stopped outright.
  • Explicit declaration wins; otherwise derive from semantics. If a tool declares its tier, that tier is used. If it doesn't, writes default to R3 (confirmation) and reads to R1 (automatic). An undeclared tool is not waved through — it is conservatively held.
  • Policy can override. The governance layer can override a single tool's enablement and confirmation requirements at runtime, with immediate effect.

The point of the tiering: it turns "do we dare let the AI do this" from a feeling into a rule. Nobody has to judge tool by tool; they set policy by tier.

3. How the tiering is enforced at runtime

A tiering is paper until the runtime enforces it. The chain:

Tool declares its tier (R0-R5) → permission check (row-level scope: own / org) → policy gate (R5 blocked / R3-R4 confirmed / R0-R2 automatic) → human confirmation (executes only on approval) → write executes → side effect recorded (revocable) + audit chained

Using the open-source KeelBase implementation as the example, the chain is verifiable.

Repository: https://github.com/rain6fish/KeelBase

  • Declaration: every AI tool carries a riskLevel, and the MCP endpoint surfaces it in the tool declaration (_meta.keelbase) — visible to external clients before they call anything.
  • Gate: R5 is blocked outright (the response states plainly that the operation was blocked by security policy); R3/R4 return a confirmation marker and execute only after approval.
  • Audit: every execution lands on the audit hash chain; denied calls are recorded too — an authorization denial writes isError=true plus the reason list. Tamper-evident and traceable.
  • Revoke: side effects created by AI are recorded and can be revoked, including cross-system compensation.

The protocol conformance suite pins these semantics. Output from Server-NestJS/scripts/verify-protocol-conformance.mjs:

─ Tool risk tiers (protocol §4) ─
  ✓ RISK_STRATEGY table matches the vectors
  ✓ R1 (read) → auto / no confirmation
  ✓ R3 (business-sensitive write) → confirmation
  ✓ R4 (high-impact) → human_approval
  ✓ R5 (irreversible/external) → block
  ✓ Derivation: undeclared write tool → R3 confirmation
  ✓ Derivation: undeclared read tool → R1 auto

═══ Conformance: 34/34 passed (0s) ═══
Enter fullscreen mode Exit fullscreen mode

Tampering, tier escalation, and confirmation bypass all get rejected here.

Closing

AI writing data isn't the problem. Tiering it, confirming it, and being able to trace it is what makes it shippable. Turn "do we dare let the AI act" into a rule you can execute, and it can move from assistant to doing real work.

If you're building agents, three questions worth asking about your own system:

  1. Does your tool inventory contain writes with no execution policy at all — callable, so they run?
  2. For a tool that declares no risk level, does it default to allowed, or to conservatively confirmed?
  3. When a high-risk operation is stopped, did the model say "I can't do that", or did the execution layer actually refuse?

The third is the one that matters. If the answer is "the model refused", the boundary is still in the prompt, not on the execution path.

If you find a way to bypass a tiering rule or a gate, open an issue — I'm more interested in the failure cases than the successes.

Top comments (0)