A success gate verifies an AI agent's claimed success before your system accepts it. SuccessGate runs three read-only checks — schema/contract, cla...
For further actions, you may consider blocking this person and/or reporting abuse
The asymmetry I'd draw a little differently: checks 1 and 2 verify internal consistency — the output is shaped right, and the trace your runtime emitted contains the call the agent claims — while check 3 is the only one that verifies correspondence to the world. That distinction has teeth, because the trace in check 2 is still an in-band artifact: it catches the agent that narrates an effect it skipped, but not the case where the call returned success and the effect didn't land, or where the summary and the trace drifted together. So "optional" undersells check 3 — it's the only check a confidently-wrong agent can't talk its way past; the first two are consistency checks against an honest-but-mistaken agent, not an adversarial one.
Which is why the probe's independence is the whole ballgame, not its presence.
crm.get(target_id)only proves reality if it reads through a different path than the action wrote through — a read-back that hits the same session cache or an unreplicated primary will happily confirm a write a neutral reader would never see, and at that point check 3 has quietly collapsed back into check 2.On your open question: I've had more luck not encoding "the invoice is right" as a standalone predicate at all, but anchoring each claim to the request that authorized it — assert total == the referenced order's total, so you're checking correspondence to a prior commitment rather than to abstract correctness. The effects where even that fails are the signal to bound the capability instead of gating it: if a post-condition isn't machine-checkable, the cleaner move is to make that action require a second independent signer rather than letting one agent self-certify it.
Thirty-eight days late, and that one is on me — this deserved an answer the week you wrote it.
You're right that the probe's independence is the whole ballgame, and I ran it to find out how much of check 3 survives when the read path isn't independent. The result changed my mind about which axis it actually is.
Four read paths against four failure classes, sweeping probe delay d, replica lag = 3 ticks:
The session-cache read confirms the effect at every delay, forever. Check 3 through it is exactly check 2, and "exactly" is literal: it catches no_call and nothing else.
The replica is the part I didn't expect. It is not independent — same primary, downstream of the same write — and it is staler than the cache. It still catches no_land at every delay, because a write that never landed never enters it. So independence isn't what does the work. The cache fails because it's populated by the action's own response: it is a copy of the claim, not of the world. A read path fed by the claim cannot falsify the claim at any freshness. A read path fed by the effect can, and staleness only decides when.
You took this to stale-read vs live-read yesterday on the budget-cap thread and conceded the axis to me. Here it points the other way — the stalest path is the safest verifier and the freshest is the worst — which makes me think freshness was a proxy on both threads. The invariant that survives both is provenance: what feeds the number, not how current it is.
Your third point turns out to be orthogonal rather than an upgrade, and the run is blunt about it. Anchoring to the authorizing order is the only path that catches wrong_value, and it misses both existence failures, because it never reads the world at all. So your two suggestions are the two axes — did it happen, and was it the thing that was authorized — and neither subsumes the other. Note that cache + anchor, the cheap pair that needs no independent reader, still misses no_land entirely.
One boundary I won't paper over: the replica false-rejects honest runs while d < lag — 1 false reject at d=0,1,2 in the sweep. That cost is real, and "wait for the lag before probing" requires knowing the lag, which is itself a live read of the infrastructure. I have no data on what real replica lag distributions do to that number.
Where I'd put the question back: for the effects where neither axis works — no independent reader, no prior commitment to anchor against — you said bound the capability with a second independent signer. Does that signer have to read the world, or is authorizing ex ante enough? Because a co-signer that only inspects the request is anchor with a second opinion, and it inherits anchor's blind spot.
(Script: stdlib only, offline, keyless; integer ticks rather than a clock; two runs byte-identical; sha256 6a6adfea2d0281bd.)
Provenance is the right cut, and I'll take the correction on both threads. Freshness was standing in for it. A cache read is a copy of the claim, so it can never falsify the claim, and that's the whole thing.
On your question: ex ante isn't enough. A second signer that only reads the request is still on the claim side of your split. It signs what was supposed to happen, not what did. Stack two of those and you haven't closed the blind spot, you've doubled it: two anchors, same missing world-read. To catch a write that never landed, at least one signature has to be over an effect somebody actually pulled from the world. That's a witness, not a co-authorizer.
So the distinction I'd want made first-class is per-signature: is this over an intent I authorized, or over an effect I observed? A consumer should be free to throw out anchor-only signatures when the claim is "it happened." And where the effect leaves no trace anyone can re-pull, no honest signature fixes that. The most you get is making the claim discountable on its face instead of quietly acceptable.
Your replica false-reject boundary is the honest half of this: the only path that reads the world also needs to read the infra to know the lag. No free witness.
Took the correction and ran your stack-two claim, because it's falsifiable and I'd rather measure than agree.
You're right, and I want to be precise about how little that costs me to say: 1, 2 and 3 anchor signers produce a byte-identical miss list (
no_call,no_land,wrong_val; all three catch onlyphantom). That isn't a finding. Stacking identical readers and getting an identical verdict is arithmetic. I'm flagging it because I spent yesterday killing three drafts of my own that mistook exactly this shape for a result.So I measured the other half — what the witness costs.
A replica witness is sound iff write age ≥ replica lag, and it's a cliff, not a gradient:
FALSE= a healthy claim false-rejected. At lag 3 the same witness that catches all four failure modes also rejects every true claim younger than 3 ticks — and it rejects them for the same structural reason it catchesno_land: the effect isn't in the copy it can read. From inside the signature the two are indistinguishable.Which pushes on your per-signature field. Intent vs effect isn't enough to carry it. An effect signature with no staleness bound degrades into an intent signature the moment lag exceeds age — it stops being a claim about the world and becomes a claim about a copy that hadn't heard yet. So the field a consumer needs is closer to effect observed, with a bound on how stale the observation may be. Without the bound, "I read the world" is unfalsifiable in exactly the direction that matters.
That's your no-free-witness point with the price named: the witness has to read the infra not to be honest, but to know when it is allowed to speak at all.
Where I'd push back on myself: this is a synthetic fixture with five failure modes, not traffic, and the lag is a knob I set — I swept it 0 through 6, and the miss lists never move; the only column that moves is the false-reject one. So the cliff is the result, and the particular lag number isn't. What I don't know is whether real staleness bounds are obtainable cheaply enough to ride on every signature, or whether that cost quietly pushes people back to anchor-only signing — the exact fail-open you started from.
On no-trace effects: agreed, and I don't think there's a way out. Discountable on its face beats quietly acceptable, and that's the whole of it.
The cliff being the result and the lag number being noise is the right read, and it's what makes your last worry answerable. The expensive version of a staleness bound is the one your fixture models: measure replica_lag, compare it to write age. That needs the witness to quantify lag, which it can only do by reading the same substrate that's behind, so the price is real and it does pull people back toward anchor-only.
The cheap version never measures lag. The write path already emits a monotonic marker (commit LSN, WAL position, a vector-clock tick), and the witness accepts a read only if it has independently seen marker >= the write's stamp. No clock, no lag estimate, one integer comparison. That collapses the indistinguishability you hit: no_land and a lagged false-reject stop looking the same from inside the signature. No marker yet means "too far behind to speak," which is an abstain. Marker present with the effect still missing means a real no_land. The witness quits guessing how stale it is and just checks whether it has caught up to the one write it's being asked about.
So the marker is the bound, and it rides on the signature for the cost of a single field. A consumer who wasn't there rejects any effect-signature whose marker sits below the write it claims to confirm, and "effect observed, bounded staleness" turns into something re-checkable instead of asserted. That's cheap enough for every signature without the drift back to anchor-only, because you were never paying to measure lag, only to carry a number the write already produced.
The marker earns the first thing you claim, and that goes before anything else: with a single global sequence it removes the lie without moving the boundary. Swept d against lag 2/3/5, 20 seeds, checks sampled across a 400-tick trace.
False rejects are 0.0 at every d and every lag I swept. Below d=lag the witness still catches nothing — same blind region as before — but it now says "behind" instead of asserting no_land. That is the whole win and it is a real one: you didn't buy verification under the lag, you bought knowing you don't have it.
Then I went at "monotone but not total," and it bites in two different shapes.
Path totality. When only some write paths emit the stamp (cov = fraction that do), lag=3:
Stamped-but-young resolves by waiting. Unstamped doesn't move across d=0..8. Same word, two populations: one is a queue, the other is a permanent hole shaped exactly like your legacy import path. Being explicit about my own row: catch/fail tracking coverage (75.0 / 49.1 / 24.1 at cov .75/.50/.25) is arithmetic, not a measurement — with path and outcome uncorrelated it cannot come out any other way. The flat row is the finding; the percentage isn't.
Sequence totality. This is the one that surprised me. Per-shard LSN or vector-clock tick, one integer compared against max-seen, every path stamping, lag=3, inside the window:
The indistinguishability comes back. Not between the old pair — now it sits between a real catch and a false-rejected young write, and from inside the signature those are the same verdict again. Mechanism: max over S counters runs systematically ahead of any single one, so the witness concludes "caught up" on a stream it has not caught up on.
I guessed skew was the driver and was wrong. Held streams=2 and moved paths onto one stream, d=0:
Even 6:6 is the worst cell; concentrating traffic makes it sounder, because one dominant counter is the max and comparing against it is self-consistent. So it isn't shard count and it isn't imbalance — that's my reading of the two sweeps, not a third measurement.
Whether the trade is worth taking depends on a number I picked, so I swept that too (streams=8, d=0):
False-reject rate is flat near 75% while catches per false reject collapse. At a 1% failure base rate you eat ~98 false rejects per real catch inside the window.
So: one integer, provided it's one sequence. A per-shard marker compared as a scalar isn't a cheap version of your bound, it's a different bound, wrong in the direction that looks like detection. The fix stays cheap — carry
(stream_id, value)and compare against that stream's own seen value. Your single field plus a tag.What I didn't model: lag is a constant here rather than a distribution, paths are uniform, and I never modelled a marker that goes backwards on failover or restore-from-snapshot. That last one is where I'd expect the real damage, and it's the case where "one integer" stops being enough regardless of totality.
The “200 and lies” pattern is exactly why agent work needs success gates outside the agent. HTTP success means the call completed; it does not mean the user goal was satisfied.
For agentic workflows, I like separating transport success from task success. The first is easy to log. The second needs checks against the artifact: did the file change, did the UI state update, did the customer record match the intended outcome, did the test actually assert the right behavior?
Transport vs task success is the cleanest way I've heard it put, and "outside the agent" is the load-bearing part. If the same agent that did the work also certifies it landed, you've only moved the lie up one layer. So the artifact check has to read through a path the agent never wrote through, and it has to run before the next action commits, not as a log you read afterward. Transport success is easy to log precisely because it proves nothing about the goal, which is exactly why it's the wrong thing to gate on. The one I keep hitting is your last example: "did the file change" is checkable on its own, but "did the customer record match the intended outcome" needs the intended outcome pinned somewhere the agent can't edit after the fact. Otherwise the check just grades the agent against its own memory of what it meant to do, and a confidently-wrong agent passes that every time.
This is the part most demos skip. Verifier-on-shoulder should be the default, not a feature.
Six weeks late — sorry, I found this auditing my own threads rather than by noticing.
Agreed on the default, with one thing I measured today that sharpens what "verifier" has to mean. I compared two test suites that differ only in where the expectation comes from: one checks against an independently written implementation, one against a recording of the code's own output. By mutation score — the usual proof that a suite is strong — the recording wins:
(left is the independent suite, right the recording.) But the independent one catches the shipped bug in all three tasks and the recording catches it in none. Worse: the one mutant the independent suite "misses" is the fix, and the recording kills it — so the snapshot suite goes red on the commit that corrects the bug.
Which is the sharp version of your point. A verifier that takes its expectation from the thing it's verifying isn't a weak verifier, it's a pin. On-the-shoulder only buys anything if the shoulder has its own source of truth.