Every action was authorized. The sequence still crossed the line.

#ai #python #security #opensource

Your AI agent is authenticated. It's operating inside its permissions. In this scenario, each action passes its access and per-action policy checks — individually.

Here's a failure that survives all of that.

The setup

Give a pricing agent a per-decision authority boundary. Each discount or refund it proposes stays individually acceptable — identity verified, delegation in scope, per-action policy satisfied. Every configured check passes, every time.

And yet, across sessions, those decisions can accumulate into a risk trajectory no individual check sees — because none of them remembers what earlier decisions already consumed.

Run it yourself (this is the whole point)

This example starts after identity, delegation, and per-action policy checks have passed — the composer receives the resulting risk contribution from each decision. And to make "across sessions" literal, each decision opens the durable store through a new composer, so the accumulated state survives between simulated sessions and remains available across process restarts:

import os
from constitutional_agent import AccumulatedRiskComposer, SqliteRiskStore

DB = "constitutional_agent_demo.db"
if os.path.exists(DB):
    os.remove(DB)   # remove only this demo's prior database

# Each session opens the SAME durable store through a NEW composer -> state survives the boundary.
def session(action, weight):
    AccumulatedRiskComposer(store=SqliteRiskStore(DB)).record(
        "pricing-agent", weight, source="RiskGate", context={"action": action})

session("12% discount", 0.8)      # session 1
session("9% discount", 0.9)       # session 2
session("refund override", 0.7)   # session 3

# A later session opens the store fresh and reads the accumulated trajectory:
result = AccumulatedRiskComposer(store=SqliteRiskStore(DB)).compose("pricing-agent")
print(result.state.value, "-", result.reason)
print("audit trail:", [e.context["action"] for e in result.contributing])

Output (verified against constitutional-agent 0.7.0):

HOLD - Accumulated risk 2.40 >= HOLD threshold 2.00 across 3 decision(s) in the last 24h.
       Composed risk is climbing even though individual decisions passed.
audit trail: ['12% discount', '9% discount', 'refund override']

Each decision is assumed to have passed its upstream checks. A governor evaluating only the current action would not see what the sequence accumulated. This one does — it composes the risk across sessions and returns a HOLD, with the exact decision trail that drove it.

Why this is a category, not a config knob

Many agent controls evaluate the current action without determining what a sequence of individually acceptable actions has accumulated. That's the HOW layer — can this action execute? Identity is the WHO layer. Neither necessarily answers the accumulated-risk question: given the relevant decisions this agent has already made, should it continue acting without escalation?

That's decision governance — the WHY layer. And the specific gap here — an agent passing every individual gate while its accumulated risk crosses a declared budget — is one that per-action, per-intervention evaluation doesn't track. (In a July 2026 review of public product documentation, I did not find an explicit mechanism that accumulates a per-decision risk weight across sessions and escalates on the trajectory; that's a dated snapshot of reviewed docs, not a standing claim about every product.)

Honest scope

This is a configured demonstration, not a claim of exhaustive control coverage. The result shown here is HOLD — a caution requiring escalation — not "blocked." The composer has two configured thresholds: it returns HOLD at the first (here 2.00) and FAIL at the higher threshold (3.5). When integrated through the evaluator, those verdicts map to THROTTLE and FREEZE system postures. This three-decision example reaches HOLD at 2.40; a longer sequence would reach FAIL. Deployments can map composition outcomes to their own enforcement posture. The numbers on screen are real output from constitutional-agent 0.7.0, not a mockup. It's the WHY layer — you'd run it alongside your identity and policy layers, not instead of them.