Two security stories from July 2026 make the same point about AI agents.
Hugging Face disclosed that an autonomous agent spent 4.5 days moving thr...
For further actions, you may consider blocking this person and/or reporting abuse
The transition table makes failures enumerable, which is a big deal, but it only bounds per-step legality. The expensive failure mode with agents that have real write access is trajectory-level: a batch of N actions, each legal under the state machine, fired at machine speed inside a minute. Every canTransition call returns true. The incident is the aggregate.
I think the enforced boundary needs history as an input: canTransition(current, next, history). That lets the router apply rate and diversity predicates in the same place it already applies state legality: writes per window, distinct external targets per window, maybe credential-specific cooling periods after sensitive classes of action. A velocity budget is just data, like the transition table, so it stays outside the model's reach.
This also extends the "verify against the owning system" point. Some owning systems have abuse detectors you cannot query. They score the shape of the trajectory rather than the validity of each call. A burst of valid writes from a low-history account can look like automation abuse even if each call would pass isolated review.
Where would you put that history window: per-agent, per-credential, or both?
The comment's closing question is the useful one, and the abuse-detector detail points at a sharper version: some owning systems score the shape of the trajectory, not the validity of each call, so a legitimate burst from a low-history account can look like automation abuse even when every call would pass isolated review. That flips the problem: verification against the owning system is not just about gating before the call, it is about matching the owning system's own notion of a suspicious shape. The velocity budget we put on our side has to be at least as conservative as the score the other side applies, otherwise the agent passes our gate and gets throttled by theirs anyway. Which is a stronger reason to track history per-credential: the remote abuse detector almost certainly does.
Per-credential is where I would land too, though "at least as conservative as theirs" is the piece I cannot make operational. Their threshold is not published, it drifts, and hiding it is part of what makes it work. There is no number to aim at. You can never confirm your budget is conservative enough. The only thing you ever get back is the moment it wasn't.
Which reframes throttles. A 429, a soft degrade, an unexplained latency penalty, a shadow limit that quietly drops your throughput: that is the only sample anyone gets of the remote's scoring function. Read that way, the velocity budget stops being a constant guessed once in a config file and becomes an estimator fit against observed refusals. Per-credential history then carries two jobs at once. It is the state the remote is scoring, and it is the sample you fit against.
The trap is that retry-with-backoff eats exactly that signal. Attempt two succeeds, the call returns 200, and nothing upstream ever learns a limit was touched. Throttle responses belong in the same evidence line mads_hansen wanted denied transitions in, recorded as events rather than absorbed at the transport layer. Then the budget is re-derivable from the log instead of asserted by the component doing the throttling.
Against my own position: per-credential is a lower bound on the unit being judged. Address, network, client fingerprint, timing regularity all give them ways to join credentials you are keeping apart. The finest grain you control is not necessarily the grain they score on.
Agreed on the estimator framing. A budget fixed in config is a guess. A budget fitted to observed refusals is a model of a system you cannot inspect. The retry-with-backoff detail is what makes it concrete: when attempt two returns 200, the transport layer absorbs the signal and the budget never learns a limit was touched. Treating throttle responses as first-class events, in the same evidence line as denied transitions, is what lets the estimator actually fit. The self-correction is right too. Per-credential is the finest grain you control, not necessarily the grain the remote scores on.
The grain mismatch in that last line does more damage than it looks like it does. If the remote scores per account and the fit is per credential, refusals arrive as a function of load that was never observed, because a sibling credential's burst can push the account over while your own series looks quiet. The estimator still converges. It fits a curve to your traffic and charges your budget for a refusal another key caused, so it reads as well calibrated right up until two credentials are busy at the same time. That is an identifiability problem, so more observations do not repair it. The repair is putting the joint series in as an input, every credential sharing an account with timestamps, so the regression runs against the quantity the remote actually scored. Which needs the sharing structure to be known, and that is a fact on their side of the boundary you mostly have to guess at.
The other half is where the instrumentation sits. Backoff usually lives inside a vendor SDK, below the call site. When it does, the 429 never crosses into your process, and no amount of policy about first-class throttle events will record it, since the event was resolved before anything you wrote could see it. Capturing has to happen under the retry, at the transport, where the response lands before the library decides what to do with it. Otherwise the evidence line is curated by code you did not write, filtered by whatever selection rule its author picked for a use case that was not this one.
The identifiability point sharpens the estimator argument past where it started. The unit you fit has to match the unit the remote scores, and you cannot observe their unit directly, so the joint series is a proxy that depends on knowing the sharing structure, which is a fact you mostly have to guess at. That makes the practical question one of discovery: how do you learn the account structure before the estimator converges on a false calibration? Do you treat the mismatch as a detect and-alert condition, or do you design the budget around the assumption that the scoring unit is unknowable?
@odejobiabiolasamuel The reversibility framing is the one that survived contact with the data. The original intuition was 'anything that validates its own output validates nothing' but that is a tautology without a cost test attached. Reversibility turns it into a decision rule: can I undo this in bounded time if the judgment was wrong? If yes, approve. If no, escalate.
The probes analogue works because a probe that reads its own output as input produces the same result regardless of whether it checked anything. The cost of the check collapses to zero information gain. That is why reversibility is the right approve-gate — it is the cheapest test that actually measures whether the gate did anything at all.
Thanks for the read.
Reversibility as the gate test works because it measures something. A self-check can always answer yes. A bounded-time undo test has a real failure state. The information-gain framing makes it precise: a probe that reads its own output returns the same result whether or not it checked, so the check adds nothing. Reversibility adds a cost that can actually be wrong, which is what turns it into a decision rule instead of a tautology.
Odejobi,
"Bounded-time undo test has a real failure state" — exactly. The reversibility framing works because it gives the gate something it can actually check: not "is this action good" but "can this action be taken back within the window where taking it back still matters." The time bound is what makes it operational rather than philosophical.
The window is the operational piece. "Can I undo this" only becomes a test once you add the time bound, because most actions are reversible for a while and irreversible after.
That's the part that quietly moves the work. Once reversibility has a deadline, the deadline itself needs monitoring too — otherwise reversible is just irreversible that hasn't expired yet. The gate ends up watching the clock as much as the action.
The generate-verify-approve-act loop is the right skeleton. The part I keep coming back to is where verification lives relative to the model. Your GitLost example makes it precise: when the verification step is inside the reasoning context, any text can override any other text. The guardrail dissolves because it was never separate from the thing it was guarding against.
The FSM-as-enforcement pattern is the strongest version of this. A prompt is soft, and soft workflows break the moment the model encounters an edge case it was not trained on. A state machine that owns the only path to side effects is the hard version. I have seen the same distinction in a different context: probes that validate their own output are unreliable; probes that are validated by a separate structural check hold up.
One thing the article could sharpen: the approve step is where most teams actually differ. Some gate everything; some gate nothing above a cost threshold. The practical question is not whether to approve but what the cost of a false approve looks like. If the action is reversible, skip the human gate. If it is not, the gate pays for itself the first time it fires.
The probes analogue is the clearest statement of the core idea I have seen: anything that validates its own output validates nothing. Reversibility as the approve-gate test is the right, it turns the cost question into a decision rule.
The “FSM must be the only path to side effects” caveat is the part teams should test continuously, not just document. I would add a transition receipt containing the prior state/version, exact payload digest, actor, policy version, approval digest, and resulting state/version. Then make the side effect conditional on a compare-and-swap of that prior version. That closes the gap where two workers both hold valid approvals for stale state. Useful negative tests are replaying an approval with changed arguments, racing two legal transitions, revoking the grant after approval but before execution, and attempting the provider call without a transition receipt. The evidence should include denied attempts too; otherwise a clean state history can hide repeated bypass attempts.
The compare-and-swap detail closes a race that the article's approval-gate pattern leaves
open: two workers can both hold valid approvals for the same logical state, and either one can act on it after the other has moved the state forward. A transition receipt that binds the side effect to a specific prior version turns that race into a rejected transition instead of a double effect. The negative-test list is the more valuable half of the comment, though, replaying an approval with changed arguments and revoking a grant between approval and execution are exactly the cases that look fine in a happy-path demo and break in production. Making the denial path first-class evidence is what turns a clean state history into a useful one, because a clean history without denials is just an untested one.
State machines are the part I wish more agent demos showed. They force the system to name which transition it is taking and who can approve the risky ones. Without that, least privilege turns into a vibes policy sitting next to an overpowered tool token.
The transition table is where approval policy stops being prose and becomes data, it names the transition and the approver in the same place, so least privilege has a structure to attach to.
Moving verification outside the model's reasoning is exactly right, and the approval gate is the part most teams skip. Our boundary in Opportunity Skill is strict about this. Discovery and message triage may run on a schedule, but human outreach is explicitly excluded from recurring execution. Every contact action has to pass human confirmation before anything is sent, no exceptions. It is a deliberately boring state machine. The agent drafts the proposal, presents it, and waits. Trust in agent systems comes from which actions are structurally unable to complete without a human.
Excluding a whole action class from recurring execution is a stronger boundary than gating each call, because it removes the case where the schedule itself creates the default. The agent drafts, presents, and waits, and there is no path where sending happens because nothing stopped it. "Deliberately boring" is the honest description of the pattern, and the trust property follows from the actions that cannot complete without a human.