People keep saying that sites treat cryptographically signed AI agents differently, or that they hide extra text aimed at models. Both claims get repeated a lot. Neither seemed to have been measured.
So I built a tool and pointed it at 300 public sites.
What I expected to find
Two things, mainly.
First, that sites would treat a signed agent differently from an unsigned one. There is a standard for this now, HTTP Message Signatures with the web-bot-auth drafts, and Cloudflare shipped support for it. A bot signs its requests with a private key, publishes the public key at a well-known URL, and the site can verify who is actually knocking.
Second, that some sites would be hiding text for models. Invisible spans, text present in the machine-facing response but absent from what a person sees.
Neither one showed up.
Signed identity changed almost nothing. Across 300 sites there were three cases where a signed request was treated differently from an identical unsigned one, and in all three the signed request was treated worse. Never better.
Hidden machine-only text turned up on one page out of 281. It belongs to firecrawl.dev, a 1px transparent span that tells AI agents which onboarding file to read. Real, but rare, and not adversarial.
The standard works. Nobody is using it to decide anything yet.
What I did not expect
The measurement that produced a number on every single page was cost.
The median page costs a machine reader 2592 tokens. That is cl100k_base, measured on extracted text, not raw HTML. Script and style tags are not counted, because no agent sends those to a model.
About half of that is not body content. Roughly a fifth is navigation, header and footer. The rest is repeated text.
Seven of the 300 sites serve a markdown variant to machine readers. Those pages deliver the same content at a median of 962 tokens, about a third of the cost.
The part that bothered me
Six of those seven markdown variants are missing Vary: Accept.
That header tells caches that the response depends on what the client asked for. Without it, a cache sitting in front can store the markdown version and hand it to a browser, or store the HTML and hand it to an agent. It fails quietly and nobody notices.
Three of the seven also drop real body content from the markdown version. Not navigation, actual page copy. All three are on the same platform.
Then I pointed the tool at Cloudflare's own documentation page for the markdown feature.
It has both problems. Missing Vary: Accept, and the markdown version is missing the heading "How to enable" along with the paragraph underneath it.
That one surprised me.
Why this is harder to measure than it looks
Here is the part I did not plan for.
My first attempt compared one unsigned request to one signed request on each site's homepage. Sensible enough. Except that when I fetched the same homepage twice with identical unsigned requests, 8 out of 30 sites returned different content.
Twenty seven percent. Product cards rotating. Timestamps ticking over. Tracking parameters on links.
So a naive signed-versus-unsigned comparison is mostly measuring the page moving underneath you. Any result you get is noise wearing a costume.
What fixed it: measure an interior content page instead of a homepage, fetch the unsigned request three times, and define a stable core as the blocks that are identical across all three. Compare the signed request only against that core.
That brought baseline instability from 27 percent to under 4.
If you take one thing from this, take that. Anyone measuring identity effects on live pages without controlling for the page changing under them is publishing noise.
Things I got wrong
The first cost figures counted raw HTML. Script, style and attributes went in as if a reader sent them to a model. Withdrawn.
The first markdown figure was 6/7 for content loss. Most of what it counted was navigation chrome, which is exactly what a markdown conversion is supposed to drop. Corrected to 3/7.
The injection scan first reported 58 findings. Almost all of them were HTML comments, tooltips, and related-post rails. I went through all 190 dropped findings by hand before tightening the rules, because a detector tuned by looking at its own output is not a measurement.
One of four challenge findings did not reproduce on a second fetch. It is not reported as a finding.
All of that is listed in the writeup with what each number was, what it is now, and why it changed.
The tool
Go, single static binary, four runtime dependencies, no model calls, no API key, nothing leaves your machine. MIT.
It fetches a page under several identities, including one that signs its requests properly, and tells you what each one received. It does not grade sites.
content-parity check https://example.com/
Repo: https://github.com/Zulwatha/content-parity
Full numbers and method: https://github.com/Zulwatha/content-parity/blob/main/docs/results.md
If you can break the methodology, I would rather hear it than not.
Top comments (0)