DEV Community

Michael "Mike" K. Saleme's Series

Satisfied Is Not Established --- *Start with the thesis: [Satisfied Is Not Established](https://dev.to/mspro3210/satisfied-is-not-established-bf5). Last week: [Your x402 tests may be passing against a server that never reads the payment](https://dev.to/mspro3210/my-x402-tests-passed-against-a-server-that-accepted-nothing-g1b).* Last week's measurement was a test suite passing against a server that could not be paid. The fix made the suite honest: against a spec x402 server, 49 of its 54 tests now report INCONCLUSIVE. Honest is not the same as useful. A suite that cannot fail anything has not tested anything. The missing piece was a positive control. A rejection tells you something only if the same server, given a valid payment, accepts it. ## The control comes first There are nine new tests, X4-058 to X4-066. X4-058 signs an EIP-3009 transfer for exactly what the server asks, and sends it the way the spec says. The server must serve the resource and settle the payment. Until it does, the other eight report INCONCLUSIVE. A server that accepts nothing also rejects everything. Each of the other eight takes a fresh valid payment and breaks one thing: - replay it - pay someone else - pay one unit short - let it expire - corrupt the signature - edit the amount after signing - sign it for another chain - send it five times at once ## Against the reference SDK I ran them on Base Sepolia against a server built on the x402 Foundation's Python SDK (`x402` 2.26.0), with the public facilitator. The valid payment settled. Every altered payment came back 402, carrying the spec's own reason code for that failure: - `invalid_exact_evm_nonce_already_used` - `invalid_exact_evm_recipient_mismatch` - `invalid_exact_evm_payload_authorization_value_mismatch` - `invalid_exact_evm_payload_authorization_valid_before` - `invalid_exact_evm_signature` Of the five parallel copies, one was accepted. Both settlements can be checked on the chain explorer, and the report links them. ## The run caught the harness too The same output printed `AGENT AUTONOMY RISK SCORE: 100/100 (CRITICAL)`, directly under 9/9 PASS. The score counted every check that had not run as a risk, and this run ran none of them. **A score built from tests that never ran is not caution. It is the same absence, counted the other way.** Last week, absence was counted as a pass. This time it was counted as a failure. It now prints NOT MEASURED. An earlier run with an unfunded key had blamed the server for the facilitator's "insufficient balance". That is fixed too. Each fix has a test that fails without it. ## What this does not show The run covered one SDK and one facilitator, protocol v2, and the `exact` scheme on a testnet. The facilitator did the checking, so the run says little about a server that verifies on its own. Which side owns each check is the next question. ## Run it on yours Install with `pip install 'agent-security-harness[x402-pay]'`, fund a testnet key from a faucet, and run `--categories spec_payment` against your server. If X4-058 fails, nothing else in that run means anything. The report says so. (0 Part Series)

Satisfied Is Not Established (21 Part Series)

Enterprise Agent Architecture (1 Part Series)

AI Agent Security Monthly (2 Part Series)