DEV Community

Eric Shepard
Eric Shepard

Posted on

Four bugs shipping a live x402 endpoint

Four bugs shipping a live x402 endpoint

Four bugs reached production in a pay-per-call scraping API that settles USDC on
Base. Every one of them passed a green test suite. In each case the thing meant
to catch the bug was more forgiving than production was.

The dependency nobody declared

Every payment through the Coinbase facilitator had been failing for weeks, and
the error said nothing useful: clients got facilitator_unreachable.

The cause was one line that did not exist. cdp_facilitator.py imported PyJWT
lazily, inside the function that signs the bearer token:

def build_bearer_jwt(...):
    import jwt   # never declared in requirements.txt
Enter fullscreen mode Exit fullscreen mode

PyJWT was installed on every development machine as somebody else's transitive
dependency. It was not in the container. The import only ran when a real payment
arrived, so nothing failed at startup, no health check noticed, and the test
suite passed because the tests ran where the package happened to exist.

A lazy import is exactly the shape that survives local development and dies in
production. It runs on the one code path nobody exercises before shipping.

The fix is a test, not a discipline:

RUNTIME_IMPORTS = {"jwt": "pyjwt", "cryptography": "cryptography", "stripe": "stripe"}

def test_runtime_imports_are_declared(self):
    for module, distribution in RUNTIME_IMPORTS.items():
        assert distribution in declared_distributions()
Enter fullscreen mode Exit fullscreen mode

Worth knowing the limit of that guard: it checks a hardcoded list rather than
scanning source for imports. It protects the dependencies someone remembered to
add to it, which is better than nothing and less than it sounds.

A wallet that can be paid and can never pay

The x402 exact scheme on Base settles with EIP-3009
transferWithAuthorization. The payer signs an authorization off-chain, the
facilitator submits it and pays the gas. That signature is ECDSA, produced by a
private key.

A smart-contract wallet has no private key. It cannot produce that signature, so
it can never be the payer ΓÇö even though it receives USDC perfectly well and
looks identical in a block explorer.

One call tells them apart:

curl -s -X POST https://mainnet.base.org \
  -H 'content-type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"eth_getCode","params":["0xYOUR_ADDRESS","latest"]}'
Enter fullscreen mode Exit fullscreen mode

0x means an EOA, which can sign. Anything longer is a contract, which cannot.
The receiving wallet here returned 48 bytes of code.

Two related things cost real time:

  • A deposit address is not proof of control. An earlier receiving address had been pasted from an exchange deposit screen and treated as verified. On-chain it had never received or sent anything ΓÇö zero transfers, ever. The test deposit it was supposed to have received did not exist. The rule that replaced it: send a dollar in, send it back out. If you cannot send it back, you do not control it.
  • Contract wallets pay gas in tokens. The one here pays via a paymaster, charging roughly 1.5 cents in USDC per outbound transfer. Fine for receiving revenue, expensive to spend from.

The transaction that never committed

This one shipped to production and reported success every time.

transaction = self._client.transaction()
customer_doc = customer_ref.get(transaction=transaction)
...
transaction.update(customer_ref, {"credit_used_micro": new_value})
transaction.set(event_ref, {...})
return DebitResult("ok", remaining)
Enter fullscreen mode Exit fullscreen mode

Transaction.update() and .set() in google-cloud-firestore append to an
internal write buffer. Those writes are sent by commit(), which is called by
the @firestore.transactional decorator or by using the transaction as a
context manager. This code does neither, so the buffer is discarded when the
object is garbage collected.

Nothing raises. The function returns ok, the request is served, and the
customer's balance never moves. Refunds wrote directly rather than through a
transaction, so a balance could only ever go up.

The reason 678 passing tests said nothing about it is the part worth copying.
The fake Firestore client applied transaction writes immediately:

class FakeTransaction:
    def update(self, ref, data):
        ref.update(data)      # real client buffers; this does not
Enter fullscreen mode Exit fullscreen mode

The fake modelled the API I assumed rather than the one that exists. A test
double more forgiving than production hides exactly the class of bug it exists
to catch.
The suite was not weak on coverage; it was confidently wrong about
the dependency.

Two changes, in order of importance:

  1. The fake now buffers until commit(), and a test asserts writes are invisible before it. The regression test for the bug is a runner that never commits, asserting the balance stays put while the call claims success.
  2. Begin, retry and commit moved into a single injected runner, so no individual method drives a transaction by hand and no future method can forget.

It cost nothing, because no customer had a key yet. It would have hit the first
one.

Completed does not mean paid

The card path had the same shape of error, caught before it shipped.

checkout.session.completed sounds like the fulfilment event. For card payments
it is. For delayed payment methods ΓÇö bank debits and similar ΓÇö the session
completes when the customer finishes the flow, which is before the money
arrives. Crediting on that event alone grants credit for funds that may never
settle.

So the handler gated on payment status:

if session.get("payment_status") not in (None, "paid"):
    return None
Enter fullscreen mode Exit fullscreen mode

Correct, and half a feature. Nothing listened for
checkout.session.async_payment_succeeded, the event that fires when the
delayed payment actually settles. A customer paying by bank debit would have
been charged and never credited ΓÇö the failure mode that generates a support
ticket and a chargeback rather than a quiet loss.

Both events fulfil now, both gated on payment status.

Two smaller things from the same reading of Stripe's own guidance:

  • Setting the module-level stripe.api_key on each call is deprecated, and in a concurrent server it is shared mutable state. A StripeClient instance carries the key instead.
  • Passing payment_method_types disables dynamic payment methods, which decide what to show each buyer. Omitting it is the default for a reason.

The webhook is also the most attacker-interesting endpoint in the service: a
public URL that moves money into an account. It verifies the signature before
parsing the body, grants only what the processor reports as paid, and is
idempotent on the event id, because a webhook is retried on any non-2xx. A
verified event that does nothing still returns 200 ΓÇö otherwise it is redelivered
forever.

The pattern

None of these were hard bugs. Each one hid behind something that was supposed to
catch it and was more permissive than production:

Bug What should have caught it Why it didn't
PyJWT undeclared The test suite Ran where the package happened to exist
Contract wallet as payer The address looking valid Only eth_getCode distinguishes them
Transaction never committed 678 passing tests The fake applied writes the real client buffers
Credit on completed Reading the event name The name describes the session, not the money

The cheap checks, in the order I wish I had done them:

  1. Assert your declared dependencies, do not trust your machine. A lazy import is the one that gets you.
  2. Prove wallet control by moving money out, not in. Anyone can send to an address nobody holds the key to.
  3. Make test doubles pessimistic. If the real client buffers, the fake buffers. A double that is kinder than production converts a green suite into false confidence.
  4. Read the payment provider's own guidance before going live, not after. Two of these came straight out of it, and it took ten minutes.

The one that generalises furthest is the third. Coverage was never the problem ΓÇö
the money path had tests for concurrency, idempotency, expiry and refunds, and
they all passed. They were all asking a stub that had been written to agree with
me.


The endpoint these came from is a hosted Crawl4AI service that takes x402
payments: page in, markdown or LLM-extracted JSON out. There is a demo that
needs no signup, if you want something to point an agent at.

curl -X POST https://api.doit2winsolutions.co/demo/scrape \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'
Enter fullscreen mode Exit fullscreen mode

It is new and has no customers yet. I would rather hear that it breaks on your
pages than hear nothing.

Top comments (0)