DEV Community

chovy
chovy

Posted on Originally published at dev.profullstack.com

c0mpute nodes now charge AI crawlers for the files they host

#ai #crypto #llm #web3

c0mpute nodes now charge AI crawlers for the files they host

Every c0mpute node runs a gateway that serves hosted sites to the public. Most of what fetches those files in bulk is AI training crawlers. As of c0mpute v0.3.33, a node can charge them.

How it works

A training crawler (GPTBot, ClaudeBot, CCBot, Bytespider and the rest) that asks a gateway for a hosted file gets HTTP 402 with an x402 offer. It pays USDC on Base, Polygon or Ethereum through CoinPay and gets back a signed day pass. People, search engines and assistant bots like ChatGPT-User and Claude-User are never charged.

The money goes straight to the node operator. The node does not need a wallet in its config: it asks CoinPay which wallet the operator's account uses (a new GET /api/x402/pay-to endpoint) and checks again every hour, so changing the wallet in CoinPay changes it on every node.

It is off by default:

[x402]
enabled = true
# CoinPay key (x402 scope) in $C0MPUTE_X402_COINPAY_KEY
Enter fullscreen mode Exit fullscreen mode

It runs on our own nodes today. You can see the offer a bot gets at https://gateway.c0mpute.com/_x402/crawl

ThreatCrush on every node

c0mpute plugin install threatcrush makes the gateway write an access log with the real client address and registers it with ThreatCrush, which scores requests against attack rules and bans repeat offenders. A test XSS probe against each of our three nodes showed up as a critical event within a second.

Getting there turned up two ThreatCrush fixes. Its docs described extra log paths that the daemon never actually read (fixed in 0.13.77). And its 2 million row event cap had grown one node's database to 3.2 GB, which left the daemon stuck re-reading its own file. The default is now 250k rows (0.13.78).

A CoinPay audit along the way

Wiring the nodes to CoinPay showed that an x402 key, meant only to settle payments, could list every business on the account with its API key. That led to an audit of all 126 routes a business API key can reach. 28 let one business's key act as the whole account, including registering a passkey on the owner's login and creating the secrets vault. All 28 are fixed and deployed.

Source: c0mpute, ThreatCrush, CoinPay. Drafted with AI help, checked by me.

Top comments (0)