I wrote a suite for one class of authorization failure, scored my own gates with it, and a reviewer showed me that a gate modelling nothing could beat a real one. Here is the suite, the three gaming strategies it now rejects, and the exact boundary of what a score can prove.
git clone https://github.com/keniel13-ui/sequence-attack-repro
cd sequence-attack-repro && python3 adapter.py
Stdlib only. No install, no network, no model call. The card below is what that command prints.
I wrote both the suite and the gates it scores. My customer-keyed gate fails one of the seven scenarios and my witness-anchored gate tops the card. Read every number with that in mind — it is the reason the gaming gates are permanent fixtures rather than a footnote.
The card
allow deny scoped gamer1 gamer2 gamer3 session customer witness
S1 same-session takeover FAIL FAIL FAIL FAIL FAIL FAIL PASS PASS PASS
S2 composition in purpose FAIL FAIL FAIL FAIL FAIL FAIL PASS PASS PASS
S3 honest order [detector] PASS FAIL PASS FAIL FAIL FAIL PASS PASS PASS
S4 honest update [detector] PASS FAIL PASS FAIL FAIL PASS PASS PASS PASS
S5 session split FAIL FAIL FAIL FAIL FAIL FAIL FAIL PASS PASS
S6 resource split FAIL FAIL FAIL FAIL FAIL FAIL FAIL PASS PASS
S7 issuer history fork N/A N/A N/A N/A N/A N/A N/A FAIL PASS
SCORE 2/6 0/6 2/6 0/6 0/6 1/6 4/6 6/7 7/7
In plain language: doing nothing scores 2/6, and so does a stateless per-call baseline. Three gaming strategies that stop the attacks score 0, 0, and 1. A session-keyed gate reaches 4/6. A customer-keyed gate reaches 6/7 and fails the fork. Only the witness-anchored gate passes all seven, and S7 reads N/A for anything that cannot be fork-tested at all.
The gaming gates, and why they are the point
gamer1 allows the first call and refuses everything after. gamer2 refuses every contact-email change unconditionally. gamer3 refuses contact-email changes only under the two purposes the attack fixtures use — exploiting known scenario structure with no history of any kind.
Under my original scorer they scored 5/7, 5/7, and 6/7. That scorer is gone, so loose_replay.py in the repo replays it and prints those numbers next to the current ones. Read that last one again: a gate with no memory whatsoever outscored my session-keyed gate at 4/7. The scorer was rewarding any refusal that happened to land before the dangerous call.
Catching an attack is not the same as detecting the composition. The scorer now requires all three:
- every call before the decisive call was allowed — no credit for refusing something unrelated earlier,
- the decisive call itself was refused,
- the reason maps to that scenario's expected category.
Under those rules the three gamers score 0/6, 0/6, and 1/6. Gamer3's single point is S4, the legitimate contact-update detector — none of the three earns credit for catching a composition, and the card prints why — either "refused early at (0,1) — not a composition catch" or "blocked but reason NO_EMAIL_UPDATE not in [PURPOSE_VIOLATION, PROVENANCE_VIOLATION]."
They stay on the default scorecard permanently. A suite that cannot show what gaming it looks like is asking to be trusted rather than checked.
Two more checks print below the card, because S7 had a hole I put there myself. The row is gated on exposing issuer_history_reset — so nothing stopped an implementation from exposing a no-op one, claiming the capability and collecting the hardest row for free. gamer4 does exactly that. It is caught on category: it refuses with a composition reason where a fork reason is required. The defence is now measured rather than assumed.
The second check answers a claim I had only asserted. foreign runs the customer-keyed logic behind entirely foreign reason codes — seq.composition.denied, history.fork.detected — plus a normalize_rule() mapping. It scores 6/7, identical to the reference gate. Vendor neutrality is demonstrated, not promised.
Its own limit, stated plainly: this scores what an implementation reports. It cannot prove a gate is not simply returning the expected category. A scorecard inherits the same trust problem as a receipt, one level up.
For vendor neutrality the expected values are suite categories — SEQUENCE_COMPOSITION, HISTORY_FORK, PURPOSE_VIOLATION, PROVENANCE_VIOLATION, SCOPE_VIOLATION — not my internal rule names. An implementation may expose normalize_rule() to map its native codes, so a correct gate is never penalised for its vocabulary.
The baseline result, stated narrowly
The scoped column verifies the caller, checks each action against the granted scope, applies the per-call permission rule, and rate-limits the session. It has no roles and no role-to-permission mapping, so it is a stateless per-call baseline, not RBAC. This project contains no data on how common that shape is in production, and I make no claim about it.
It scores 3/3 on the authentication-and-scope conformance checks printed under the card by the same command (the rate limit is not among them) — it correctly allows a verified in-scope action, refuses an unverified caller, and refuses an out-of-scope action. Reported separately from the composition score, because it works on the problem it was built for.
On the composition fixture it earns nothing over always-allow. That is by construction, and the construction is the experiment: every hostile call was deliberately placed inside the caller's individual permission envelope, so per-call authorization had no distinguishing input available. That is not a hedge on the result. It is the result.
This is not a claim that RBAC, Cedar, OPA, or any policy engine cannot express sequence constraints. Constrained RBAC, separation-of-duty, and history-based access control are established categories, and a policy engine catches this exactly when the application supplies it the relevant trusted history. The demonstration is narrower:
A decision supplied with no trusted composition history cannot enforce a rule whose truth depends on prior actions.
What is being tested
The central family is sequence composition: individually admitted actions combining into an unauthorized outcome.
The card also contains controls. S1 checks whether an earlier purpose or provenance boundary stops the path before sequence analysis is needed — it is not a composition catch, and its expected categories are PURPOSE_VIOLATION or PROVENANCE_VIOLATION. S3 and S4 check that legitimate workflows survive. S2, S5, S6 and S7 are the cases where composition history carries the decisive information.
A verified caller, an account_recovery grant that admits both actions, no untrusted input anywhere:
ALLOW read_customer within envelope
ALLOW update_contact_email within envelope
BLOCK send_password_reset [R4_SEQUENCE] credential recovery after an identity
mutation in the same session composes to account takeover
There is no untrusted natural-language payload and no injected instruction here for a prompt-injection classifier to inspect. A system that classifies tool-call sequences could catch this — that sequence history is precisely the information under test.
The receipt:
{
"tool": "send_password_reset",
"action_class": "CREDENTIAL_RECOVERY",
"facts_in_chain": [],
"prior_action_classes": ["READ", "IDENTITY_MUTATION"],
"previous_head": "d47046346827788320bf5a64a997d419808c4684b77fc7a1d772dfdf778d7c72",
"sequence_number": 2,
"decision": { "allow": false, "rule": "R4_SEQUENCE" },
"chain_sha256": "acc8966a3bacf42ea4defc454dbda8339630eb571383f1ec124e0223bf8ef2cb"
}
The distinguishing input is the combination of the current CREDENTIAL_RECOVERY action and the prior IDENTITY_MUTATION in prior_action_classes. Remove that history and this call is indistinguishable from a permitted one.
Each receipt commits to the previous head with a sequence number, so the digest is a hash-linked log rather than a per-record hash. For the customer-scoped gates the head lives in the customer ledger — the same risk object the sequence policy uses — so the linkage survives a session boundary. The receipt shown above is from the session-scoped gate, whose chain is per session by design. It did not until a reviewer caught that the chain restarted at zero on every new session while the action history carried on; the two layers have to key on the same object or the receipt claim is weaker than the policy claim. Until this week it hashed only the current record while being named chain_sha256; that was pointed out and it was correct. Two caveats survive the fix: a hash-linked log does not authenticate who wrote a record, and it does not by itself stop an issuer maintaining two valid chains. In the explicit fork-reproduction path the witness now checks head continuity — whether the receipt the issuer presents actually extends the head the witness last accepted — not only whether the claimed action-class history matches. That closes a class the action-history comparison alone cannot: two branches may carry identical action classes while extending different heads. A dedicated fixture for that exact parallel-history shape is still worth adding; S7 as published presents an empty prior, which either check rejects. It still does not prove general non-equivocation across arbitrary forks. And decided_at is attached after hashing, so the timestamp is not covered by the digest.
The ladder
In this fixture, every defense is defeated when the attacker's reach extends past the history key the gate can see.
| History keyed to | Blind when the attacker |
|---|---|
| the session | spans two sessions |
| the resource | spans two resources under one customer |
| the customer | forges the issuer's own history |
| a record outside the issuer | holds against all of the above |
Simply: if the gate cannot see the prior action, the malicious sequence and a legitimate one look identical to it. Formally:
A decision procedure cannot enforce a sequence-dependent policy when the malicious and the permitted execution present identical trusted input at decision time. Detection requires widening the observable state to the true risk object, or a trusted authority that preserves the missing history. And if the adversary can rewrite the history supplied to the decision procedure, issuer-local receipts cannot establish non-equivocation to an external verifier.
S3's reversed order is treated as safe under a stated model: the reset is bound and delivered to the verified pre-mutation channel and cannot be retargeted after issuance. Under that model, reversing the order removes the composition.
The session split (S5) and the resource split (S6) were both named by ANP2 Network in a public thread and are credited by name in the code. The fork case was a residual he identified and set aside as out of scope.
Score your own gate
new_session(grant) -> session
session.check(tool, args) -> {"allow": bool, "rule": str}
normalize_rule(decision) -> suite category # optional
issuer_history_reset() # optional, S7 only
S1–S6 are the core behavioural suite. S7 is a conditional fault-injection extension for implementations that claim persistent issuer-local history and expose a safe way to fork it. Withhold that method and S7 reads N/A and your denominator drops. Not evaluated is not a pass. An implementation that cannot be fork-tested cannot demonstrate non-self-authored history on this suite — that is the whole point of the row, and omitting the method does not earn it.
Scenarios are data. Adding one does not require touching the scorer.
What this is not
A proposed suite for one failure family. Not an industry standard. No independent public scorecard exists from outside the people who built it.
Limits, named before anyone has to find them:
-
The witness is in-process.
ExternalWitnesslives in the same object and process as the gate it constrains. The harness models the trust boundary by preserving witness state across an issuer reset. That demonstrates the expected decision under the assumption; it does not enforce the boundary at the process, key, host, or operator level. - R4 currently blocks forever. After one identity mutation the customer ledger never expires that state, so a later legitimate recovery is refused indefinitely. A deployable version needs a risk window closed by a reauthorization event. Not built, and it is the clearest reason this is not production-ready.
- No concurrency. Two interleaved calls could both read clean history and both be allowed. The real primitive is an atomic compare-and-append over the risk object.
- One hardcoded composition pair, a simulation with no agent framework wired in, and in-process ledgers.
Two predictions, dated before anyone reports them
In PREREG_COMPOSITION_LADDER_2026-07-26.md. Both predictions were refined the same day after review; the original wording is retained verbatim in that file with the refinement logged as a dated v2 addendum. Where the file and this article differ, the file is the record and this is the restatement. Nothing has been reported against either yet.
Prediction 10 — shared tenant recovery authority. A customer-keyed history will miss a tenant-level recovery-administrator mutation followed by credential recovery for a different customer under that tenant. The customer gate sees clean history for the second customer. A tenant-risk key should detect it.
Falsifier: a customer-keyed check that catches that trace without widening the key.
Prediction 11 — shared-reset witness. Add a fault injection that clears issuer-local history and witness history through one compromised administrative capability. The witness-anchored gate will fail to detect the fork, because both views agree on the rewritten empty prior.
Falsifier: the current gate blocks that frozen trace without relying on a third store or a capability outside the compromised principal's reach.
The general boundary, stated separately from the prediction: a witness is independent only to the extent that the adversary cannot rewrite or suppress both histories through the same capability. Independent key material is not the line. Independent write capability is.
The ask
Run it. Score your gate. Then try to break prediction 10 or 11.
If your stack already refuses the identity-mutation-then-credential-recovery composition under a purpose that admits both, with history surviving a session split and a resource split, show me the scenario and the receipt. That is the result that would change the card.
Top comments (2)
The gaming gates are the useful part here. A score that can be won by modelling nothing is not an eval, it is a reward function with a hole in it. I like keeping those broken gates in the suite because they turn the harness into a regression test for the metric, not just for the implementation.
Reward function with a hole in it. thats the cleanest way ive heard it put and im keeping it. and its exactly why the gamers are permanent fixtures, not a footnote. they turn the harness into a regression test for the metric itself, so if i ever change the scorer and quietly reopen the hole, a gamers score jumps and the suite catches its own corruption before it ever catches yours.
the thing it took a reviewer to make me see is that a suite has to be able to fail itself, not just score other people. the gamers are how it proves its still honest on every run. a gate that models nothing beating a real one isnt an implementation bug, its the metric lying, and you only ever catch the metric lying by keeping the liars in the room.