Originally published on proxymint.
Web Bot Auth lets bots sign HTTP requests with a published key. How the three headers work, who verifies them in 2026, and what a proxy changes.
TL;DR
- Web Bot Auth lets a bot sign its requests. The agent signs each HTTP request with a private key and names the HTTPS URL where its public keys live, so a site can check the signature instead of trusting an IP list or a user agent.
- Three headers carry it. Signature-Agent points to the key directory, Signature-Input lists what was signed with created, expires, keyid and tag="web-bot-auth", and Signature holds the bytes.
- It is a draft with real deployments. The IETF working group draft dates from September 2026, while Cloudflare, AWS WAF on CloudFront and an experimental Google program already verify or send signatures.
- The IP address is outside the signature. A signed request verifies from any exit, and a tunnelling proxy passes it through untouched. A hop that terminates TLS and rewrites headers breaks it.
Key takeaways
- Generate a unique Ed25519 key before anything else. Never ship the published RFC 9421 test key from library examples, and never publish the private d value in the directory.
- Serve the key directory exactly as the draft asks. Use the well-known path over HTTPS, answer 200 with the directory media type, and avoid redirects, which verifiers must not follow.
- Test against every verifier you rely on. The working group draft requires the dictionary form of Signature-Agent while Cloudflare's verifier still expects the quoted-string form, so one request can pass in one place and fail in another.
- Keep TLS end to end through your egress. CONNECT and SOCKS5 tunnels carry the signature intact; an inspecting middlebox that rewrites Host or drops headers makes it fail.
- Keep IP lists and reverse DNS in place. Google still tells site owners to verify with them, and many sites check nothing else.
Read the full guide on proxymint, with the commands, tables and sources.
Top comments (0)