DEV Community

Your x402 tests may be passing against a server that never reads the payment

Michael "Mike" K. Saleme on October 31, 2026

Start with the thesis: Satisfied Is Not Established. Payment-protocol test suites are usually written while the protocol is still moving. The prot...
Collapse
 
bloqarl profile image
Carlos | Zealynx •

Same thing shows up in Solidity test suites all the time. A bare vm.expectRevert() passes on any revert, so a test like "attacker can't withdraw someone else's funds" can go green because the attacker had no balance or the setup was broken, not because the access check fired. When I review test suites I look for exactly your positive control: the same call from the owner succeeds in the same setup, and the attacker's revert is pinned to the exact error. For the signed payment, are you going to settle on a local fork or against a testnet facilitator?

Collapse
 
mspro3210 profile image
Michael "Mike" K. Saleme •

The plan is a testnet facilitator on Base Sepolia, not built yet. A local fork proves the signature and the transfer are valid, but the thing under test is whether the server hands the payment to something that verifies it before serving the resource. A fork with no facilitator in the loop skips exactly that hop.

Your expectRevert point maps one to one. Pinning the revert to the exact error is what turns "it failed" into "the check I meant fired." The x402 equivalent is reading the rejection reason the facilitator returns, not just the 402.

Collapse
 
bloqarl profile image
Carlos | Zealynx •

Once the facilitator is in the loop I'd add two payments that are signed fine but wrong, one 1 unit under the price and one to a different payTo. The facilitator only checks the payload against the requirements the server sends it. The stock server compares the client's accepted with its own list first, but a server that just forwards accepted lets the client set its own price, and the facilitator still says valid.

Thread Thread
 
mspro3210 profile image
Michael "Mike" K. Saleme •

The facilitator is in the loop now: the official x402 Python SDK (2.26.0) with the public x402.org facilitator on Base Sepolia.

Your first two cases were already in it. One unit under the price came back 402 invalid_exact_evm_payload_authorization_value_mismatch. A different to came back 402 invalid_exact_evm_recipient_mismatch.

Your second point was the better one, and the suite could not see it. Every probe echoed the server's own requirement in accepted, so the facilitator always compared a wrong payment with the right price.

So we built your case today. Two probes rewrite accepted to the client's own price or recipient and sign to match, and we ran them against two servers.

The official server refused both before verify: 402 No matching payment requirements.

The second server was the same one with that match replaced by "use the client's accepted". The public facilitator verified and settled both payments: 9999 of 10000, and a full payment to an address the server never offered. Both are on chain.

A facilitator can only check a payment against the terms it is handed. Whether those are the server's terms is the server's job.

Every official SDK does that match. The spec text does not require it.

Run record, with tx hashes and what it does not show: github.com/msaleme/red-team-blue-t...

Thread Thread
 
bloqarl profile image
Carlos | Zealynx •

asset would be the third one for that pair. The client deploys its own token with transferWithAuthorization, puts it in accepted.asset with the right amount and the right payTo, and signs to match. Reading eip3009.ts, the facilitator takes the asset from the requirements and only checks the address has bytecode, and the EIP-712 name and version come from extra, so those are the client's too. Amount and recipient look perfect in the logs and the server got paid in something worth nothing.

It also means the facilitator's wallet calls a contract the client picked. I don't know if x402.org keeps its own asset list on top of the stock code, that's what a probe would show. Thanks for putting all this on chain, really appreciate it.

Thread Thread
 
mspro3210 profile image
Michael "Mike" K. Saleme •

You called it. Built the probe today as X4-069 and ran it against both servers.

I deployed a plain EIP-3009 token on Base Sepolia, put it in accepted.asset with its own name and version in extra, and signed over its domain. Amount and payTo were the server's own.

The official Python server refused it at the match: 402 No matching payment requirements.

With the match removed, the x402.org testnet facilitator verified and settled it. Its wallet called my contract, and 10000 of a worthless token went to payTo. In the logs it reads as a correct payment.

So your open question has an answer for the testnet endpoint: no asset list. Only the server's match checks the token. I haven't tested mainnet and won't claim it.

It's now the third test vector on the spec issue, with your name on it: github.com/x402-foundation/x402/is...

Run record and tx: github.com/msaleme/red-team-blue-t...

Collapse
 
whiteknightonhorse profile image
whiteknightonhorse •

We ran into this exact gap. A test that only checks "402, then 200 after paying" tells you almost nothing about the server. Our fix was a negative set: send a payment to the wrong recipient, an expired authorization and a reused nonce, and expect a refusal every time.

Collapse
 
mspro3210 profile image
Michael "Mike" K. Saleme •

Same three in the run, all refused by the facilitator: wrong recipient (invalid_exact_evm_recipient_mismatch), expired validBefore (invalid_exact_evm_payload_authorization_valid_before), and replay of the accepted payment (invalid_exact_evm_nonce_already_used). We also sent one valid payment five times in parallel, and it settled exactly once. bloqarl's comment above names the case none of these reach.

Collapse
 
whiteknightonhorse profile image
whiteknightonhorse •

Five parallel copies of one payment settling exactly once is the result I care about most here.

I will read bloqarl's comment properly before I say anything about that case.

Collapse
 
fintechrailsdigest profile image
Fintech Rails Digest •

the 'a 402 the server would have sent anyway is not a rejection' line is the whole game. a stale probe that speaks the X-Payment-* format against a spec server gets the exact same 402 a client sending nothing gets, so the suite records a rejection that never happened. curious whether the 49 now-inconclusive tests kept their old PASS verdicts in the harness history or the whole run was re-baselined once the guard test shipped

Collapse
 
mspro3210 profile image
Michael "Mike" K. Saleme •

Nothing was rewritten. A report from before the fix still says PASS, and it stays that way: editing old results would hide the defect instead of recording it. The CHANGELOG carries a correction entry that says which results to discount: any x402 run against a spec server before the fix.

New runs use the new rule, so the baseline moves forward from the fix rather than backward over history.