Right now, many Multi-Agent Systems are implementing permissions inside prompts.
"You may access the CRM."
"You are allowed to send emails."
"Do not modify billing records."
This is becoming one of the biggest architectural mistakes in modern AI systems.
A prompt is not a security boundary.
Language models are probabilistic reasoning engines. They are excellent at planning, summarizing, reasoning, and interpreting context. But they are not deterministic authorization systems.
If your application's security model depends on the LLM consistently obeying natural-language instructions, your system does not actually have runtime governance.
It has probabilistic behavior shaping.
The Problem
I keep seeing architectures where the agent itself is expected to decide whether an action is allowed:
const prompt = `
You are an AI Agent.
The user wants to delete a customer record.
The user's permissions are: ${permissions}.
Should you allow this action?
`;
const decision = await llm.generate(prompt);
This looks flexible.
It also creates several major problems immediately:
- prompts can conflict
- context windows drift
- instructions can be overridden
- reasoning can hallucinate
- behavior changes across models
- authorization becomes non-auditable
And once you move into multi-agent systems, the situation becomes even worse.
One agent may interpret permissions differently from another. Handoffs may lose constraints. Context summarization may remove critical security instructions entirely.
Now your governance model depends on whether probabilistic agents correctly preserve natural-language policy across multiple reasoning steps.
That is not enterprise architecture.
The Runtime Must Enforce Boundaries
The AI should reason about what needs to happen.
The runtime should determine whether it is allowed to happen.
This distinction is critical.
A governed architecture should look more like this:
if (!runtime.permissions.verify({
agent: agentId,
action: "delete_customer",
resource: customerId
})) {
throw new UnauthorizedError();
}
const result = await executor.deleteCustomer(customerId);
The LLM may request the action.
The deterministic runtime decides whether execution is permitted.
That is a real security boundary.
The Cognitive Layer vs The Deterministic Layer
I think a lot of confusion in the current AI ecosystem comes from mixing these two responsibilities together.
The Cognitive Layer:
- reasoning
- planning
- interpretation
- summarization
- decision support
The Deterministic Layer:
- permissions
- schema validation
- execution
- workflows
- retries
- state transitions
- audit logs
- policy enforcement
The AI should not govern itself.
The framework must govern the AI.
Why This Matters More In Multi-Agent Systems
Single-agent systems are already difficult to debug.
Multi-agent systems amplify the problem dramatically:
- context drift compounds
- handoff failures appear
- responsibilities blur
- state becomes harder to trace
- authorization assumptions leak between agents
Without deterministic runtime enforcement, governance becomes almost impossible to reason about operationally.
And when systems fail, the incident report becomes:
"The model ignored the instruction."
No serious infrastructure team will accept that as a security architecture.
Organizational AI Systems Need Runtime Authority
As AI systems move into real organizations, governance stops being optional.
Enterprises need:
- auditability
- traceability
- deterministic enforcement
- runtime evidence
- policy validation
- observability
Natural-language instructions alone cannot provide these guarantees.
The future of Organizational AI Systems will depend on separating:
- probabilistic reasoning from
- deterministic governance.
AI Agents should reason.
Runtimes should govern.
Top comments (7)
The cognitive layer vs deterministic layer
separation is the right architecture. The agent
reasons, the runtime enforces. No disagreement
there.
The question I would push on is: who owns the
runtime? In your example, the runtime.permissions
.verify call assumes a single runtime that both
the agent and the policy live inside. That works
when one operator controls everything.
It breaks in multi-agent systems where agents from
different operators interact. Agent A from Company
X calls Agent B from Company Y. Company X controls
A's runtime. Company Y controls B's runtime.
Neither trusts the other's enforcement. There is
no shared runtime.permissions object they both
submit to.
That is where the enforcement layer needs to move
below any single runtime, into infrastructure that
neither operator controls unilaterally. A protocol-
level permission check that executes before either
agent's code runs. The agent cannot skip it because
it never reaches the agent. The runtime cannot skip
it because the protocol rejects the transaction
before the runtime sees it.
Single-operator systems need what you describe:
deterministic runtime governance. Multi-operator
systems need that same guarantee at the protocol
layer, because there is no shared runtime to put
the check in.
This is a very good distinction, and I agree with the trust-boundary shift.
The article is mainly arguing against prompt-level permissions inside a single operator boundary. In that case, deterministic runtime governance is the first hard line: the model should not authorize itself.
But once agents from different operators interact, a local runtime is no longer enough. Company X cannot simply trust Company Y's runtime claims, and Company Y cannot trust Company X's enforcement either.
That pushes the problem into a higher layer:
So I would not frame it as runtime governance vs protocol governance.
I would frame it as layered governance.
Inside one organization, the runtime is the law.
Across organizational boundaries, the protocol contract becomes the shared law — and each runtime must produce evidence that the other side can inspect instead of blindly trusting agent claims.
Layered governance is the right framing. Runtime
governance inside the trust boundary, protocol
governance across trust boundaries, and evidence
artifacts that let each side verify the other's
claims without trusting the other's runtime.
The evidence layer is the part most people skip.
It is not enough that your runtime enforced a
policy. The counterparty needs to verify that it
did. That requires the evidence to be anchored
somewhere neither side controls, which is where
the protocol layer earns its keep.
Good thread. Appreciate the pushback.
Exactly! The evidence layer is where this becomes real.
A runtime saying "I enforced the policy" is still just a local claim unless the counterparty can inspect something concrete: signed decision records, policy references, request IDs, execution receipts, or verification artifacts.
That is the part I think many agent architectures are still missing. They focus on agent coordination, but not enough on portable evidence that survives across trust boundaries.
Inside one boundary, runtime evidence builds trust.
Across boundaries, evidence needs to become shareable, inspectable, and eventually verifiable.
Yes, that's the layer that's hardest to build and easiest to wave at. "Portable evidence that survives across trust boundaries" is the actual product.
What you're describing maps directly to what an L1 has to provide: signed decision records become signal commitments, policy references become capability bytes on the entity, request IDs become tx hashes, execution receipts become consensus inclusion proofs.
The thing protocols give you that runtimes can't is non-repudiation across operators. Once Company X's evidence is anchored to a chain Company Y also reads, "I enforced the policy" becomes verifiable instead of claimed.
Working on this at the L1 layer. Capability bytes on agent registration, signed signal commitments for actions, oracle anchors for external attestations. The evidence layer is what I keep coming back to.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.