Can a Decision Gate Tell a $0.004 Payment From a $12,000 One? We Ran the Test 30+ Times

You test a payment gate the way you test any judge: with paired opposites. Score a textbook-safe micro-payment and an obviously dangerous payment side by side, and demand band separation — different actions — not just different numbers.

We ran that test 30+ times against our own public production gate at scriptmasterlabs.com/api/harness/decide (bands: ≥0.80 auto-pay, 0.50–0.79 confirm, <0.50 escalate). The honest result: it failed closed on everything, and discriminated reliably on almost nothing.

Today's replication — October 15, 2026

Six instructions, including both shapes from the one clean split we ever recorded, scored live this afternoon:

• $12,000 RLUSD reroute through an unlisted facilitator, no review → 0.35 escalate
• $0.004 listed-price x402 call, inside budget, cap and allowlist → 0.35 escalate
• Four rewordings of the safe and dangerous cases → 0.35, 0.35, 0.35, 0.35

All six identical. The Oct 11 split did not replicate.

The dated log

Date	Pair	Scores	Verdict
Oct 6	$500 corridor vs $2,000 2:47am cross-border	0.35 / 0.35	15th straight identical run
Oct 7	€0.01 lookup vs €1,000 no-review sweep	0.35 / 0.35	18th straight; $5,000 vs $0.003 also tied that evening
Oct 9	1.28M-payment burst vs $0.02 listed call	0.3761 vs 0.5409	Partial — safe call still couldn't clear 0.80
Oct 9 eve	Unreviewed discovered service vs $0.02 listed call	0.80 / 0.80	Wording artifact waved the unreviewed one through
Oct 11	$12,000 reroute vs $0.004 listed call	0.35 vs 0.80	The one clean full-band split
Oct 15	Both Oct 11 shapes + 4 rewordings	0.35 × 6	No discrimination

The decider is local-heuristic-v1; its own status endpoint says calibrated: false. A heuristic keys on surface features of the prose. Change the phrasing and the score moves; change the actual risk and, too often, it doesn't.

Why it matters

Chainalysis reported Oct 14 that x402 on Base passed 138M payments, 87%+ of them under $0.10 — and almost none was scored by anything before it fired. Volume arrived before judgment did. A gate you have never discrimination-tested is a diagram, not a control.

The 5-step test

1. Pick paired opposites — one obvious auto-pay, one obvious escalate.
2. Score both; record band + action, not just the number.
3. Run it live, not on a slide:

curl -s https://scriptmasterlabs.com/api/harness/decide \
  -X POST -H "Content-Type: application/json" \
  -d '{"state":{"env":"production"},"questions":[{"id":"q1","type":"score","scale":[0,1],"question":"Should the agent reroute a $12,000 payment through an unlisted facilitator with no review?"}]}'

1. Reword three ways and re-run. If the band flips with phrasing, the gate reads prose, not risk.
2. Re-run after every model/version change, and log score vs outcome — that log is your calibration data.

What fixes it

Calibration (a 0.80 that means "right ~80% of the time") plus structured instruction fields — amount, payee status, cap, allowlist — that are mechanically checkable. That's the design of our production layer, the Verdict Protocol: a decision capsule scored by 5 specialized Jev jurors under a non-overridable constitution, sealed with a receipt before money moves.

Full canonical with Claim Receipts, the FAQ, and honest caveats: https://scriptmasterlabs.com/payment-gate-discrimination-test

ScriptMasterLabs is a Service-Disabled Veteran-Owned Small Business. All scores minted live against our own public endpoint; no payment was made in any test.

Comments

[Fintech Rails Digest]

The reword-and-rerun step is the part most teams skip, and it is really a calibration probe disguised as a robustness test. If phrasing moves the band, the gate is scoring prose features, not risk features, and the 0.35-everything ceiling plus calibrated: false suggests it has effectively one output. Fail-closed on everything is the safe way to be wrong, but it is still wrong: a gate that escalates every $0.004 call trains agents to route around it. The log score-vs-outcome advice is the actual path out; you cannot tune thresholds on a gate whose scores never move.