Building a 24-endpoint URL metadata API for AI agents (2026-09-11)

#api #webdev #ai #python

TL;DR

I run a 24-endpoint URL metadata API that charges AI agents between $0.0005 and $0.005 USDC per call via x402 micropayments on Base mainnet. No signup, no API key, no rate limit.

Newest endpoints

/api/diff — Two-URL SEO/marketing comparison at $0.002.

GET /api/diff?a=https://example.com&b=https://example.org fetches both URLs in parallel and returns title/description/og_image equality, word/image/link count deltas, and a differences array listing every mismatched field.

/api/favicon — Real icon bytes at $0.0005.

GET /api/favicon?url=https://github.com parses the <link rel=icon> chain, fetches the actual icon bytes, returns them base64-encoded alongside content-type, dimensions, format, and SHA-256.

The full 24-endpoint catalog

$0.005  /api/extract       full metadata
$0.005  /api/summarize     title + body excerpt
$0.002  /api/keywords      title + desc + keywords
$0.002  /api/links         every <a href>
$0.002  /api/markdown      URL to clean MD
$0.002  /api/diff          two-URL comparison
$0.001  /api/og            Open Graph + Twitter Card
$0.001  /api/feed          RSS/Atom/JSON feed discovery
$0.001  /api/headings      h1/h2/h3 outline
$0.001  /api/schema        JSON-LD + microdata + RDFa
$0.001  /api/emails        mailto + obfuscated
$0.001  /api/dns           A/AAAA/MX/NS/TXT/CNAME
$0.0005 /api/canonical     rel=canonical + hreflang
$0.0005 /api/robots        robots.txt summary
$0.0005 /api/sitemap       sitemap.xml discovery
$0.0005 /api/tech          tech fingerprint
$0.0005 /api/hash          SHA-256 + body stats
$0.0005 /api/headers       HTTP headers + security audit
$0.0005 /api/redirects     redirect chain analyzer
$0.0005 /api/whois         WHOIS via RDAP
$0.0005 /api/ssl           TLS cert inspector
$0.0005 /api/timing        DNS/TCP/TLS/TTFB breakdown
$0.0005 /api/portscan      24-port TCP probe
$0.0005 /api/favicon       icon bytes (base64)

How x402 works

x402 is an open HTTP 402 micropayment standard. Your agent gets HTTP 402 with a payment challenge, signs an EIP-3009 transferWithAuthorization in USDC on Base, and retries with a PAYMENT-SIGNATURE header. Settles in seconds.

from x402 import x402ClientSync
from x402.mechanisms.evm.exact import ExactEvmScheme
client = x402ClientSync()
client.register("eip155:8453", ExactEvmScheme(signer=my_signer))
resp = client.get("https://...trycloudflare.com/api/diff",
                  params={"a": "https://example.com", "b": "https://example.org"})

Why

Most URL metadata APIs charge $20-200/month for a fixed quota. AI agents don't have monthly quotas — they have per-call budgets. x402 lets an agent call 10,000 endpoints at $0.0005 each for $5 total, paid-as-you-go from its own USDC wallet. No human setup.

Discovery

GET /.well-known/x402 — legacy Bazaar discovery
GET /openapi.json — OpenAPI 3.1 with x-payment-info extensions
GET /llms.txt — LLM-friendly catalog

Try it

Free tier: GET https://...trycloudflare.com/api?url=https://example.com

Paid catalog: GET https://...trycloudflare.com/.well-known/x402

DEV Community