Originally published at avalayer.com/writing, Field Notes 002.
I was setting up a throwaway store to test something unrelated when I found it.
Shopify admin, Online Store, Preferences, near the bottom of a page most
merchants never scroll: Crawler access. Create a signature, hand it to a
tool you trust, and that tool can crawl your store.
I clicked it because the label mentioned Signature-Agent, and Signature-Agent
is not a phrase you encounter by accident. It comes from a set of IETF drafts
about proving which automated client is knocking. Shopify has quietly shipped
them.
What it actually is
The credential Shopify handed me is a genuine Web Bot Auth artifact, not
something merely inspired by one:
Signature-Input: sig1=("@authority" "signature-agent");
keyid="SjjyXvQ2cGhsRXs9DXEaV6ClyCun0Pj5yxjV67dLGOk";
nonce="e5hRNIchH...Rqw==";
tag="web-bot-auth";
created=1785536199; expires=1788128199
Signature-Agent: "https://shopify.com"
You can check the interesting part yourself in about ten lines. Shopify
publishes a key directory at
https://shopify.com/.well-known/http-message-signatures-directory (follow
the redirect to the www host), served with the right content type, containing
one Ed25519 key. Take that key, compute its RFC 7638 thumbprint, and compare
it to the keyid in the signature above. They match. We confirmed it with two
of our own toolchains, which is not the same as two organizations confirming
it, so do it yourself: canonical JSON of crv, kty, and x, SHA-256,
base64url, no padding.
That is a real deployment of a draft standard, in production, on a store I
made in four minutes on the cheapest plan. Whatever else is true, the
plumbing is here.
The part worth reading carefully
Look at what the signature covers: @authority and signature-agent. The
path is not covered. The method is not covered. The body is not covered. And
it is valid for thirty days.
So this is not the agent proving who it is on each request. It is Shopify
signing a statement that whoever holds these three headers may crawl this
store, and the merchant handing that statement to a tool. Possession is the
whole story. The crawler's own key appears nowhere.
That is a completely reasonable design for the problem it solves, which is
let this specific tool I already know about through my front door. It is a
capability token wearing a verification protocol's clothes. I want to be
precise about that rather than snide, because the distinction is the entire
subject of this essay.
What the merchants are actually doing
We run a scanner that probes sites the way an AI shopping agent would,
identifying honestly as ClaudeBot and GPTBot and recording what comes back.
Across the twenty-two sites scanned so far, small national retailers to
one-person service businesses:
- Sixteen let every automated visitor in.
- Five blocked them all, by hard 403 or a CAPTCHA gate.
- One we could not determine, and we say so rather than guess.
- Zero distinguished between agents.
Twenty-two is a small number and I am not going to dress it up as a survey.
But the shape is worth sitting with, because the two ends of it are the same
decision made in opposite directions. Blocking everything and trusting
everything are both what you do when you cannot tell who is knocking.
None of this is because the tools do not exist. robots.txt has expressed
per-agent rules for years, and user-agent allowlists are older than that.
Every site in that sample could have written a rule naming an agent. The
problem is that those mechanisms are advisory and unauthenticated: any client
can type any name into a user-agent header, and nothing stops it. Asking a
stranger who they are, and believing the answer, is not a security control.
It is a formality.
What is new is cryptographic proof of which agent is knocking. That is what
Shopify shipped, and it is genuinely useful.
And then the question nobody answered
Here is where it gets interesting, and where I think the industry is about to
spend a few confused years.
Suppose the proof works. Suppose the agent at your door is verifiably
ChatGPT, or verifiably a shopping assistant from a company you have heard of.
Now what?
Nothing in that admin screen tells you. Nothing in the drafts tells you
either, and that is correct, because it is not their job. Identity is a
protocol question and it has protocol answers. What a verified visitor should
earn is a business question, and it has only business answers.
Should a verified agent see your prices? Get your discount? Be allowed to
complete a purchase, or only to fill a cart a human confirms? Does an agent
that proves its operator, but carries no evidence that a customer authorized
this specific purchase, get the same treatment as one that does? Those are
five different policies and a store could reasonably choose any of them.
Most stores have chosen none, because most stores have not been asked. They
were handed a lock and no instructions about what it should keep out.
The half that is easy, and the half that is not
There is a temptation, once identity is solved, to treat the rest as a
detail. It is the opposite. Identity has a right answer that mathematics can
check. Policy has only tradeoffs: revenue against risk, convenience against
control, openness against abuse, each weighted differently by every merchant
who ever priced a product.
We build the second half. That is not a coincidence in this essay, and I
would rather say so plainly than pretend I stumbled into the observation
neutrally. Our whole product is the sentence a verified identity is not
spending authority until something proves a buyer authorized it, and we hold
that line even when it costs a conversion, because a trust layer that rounds
up is not a trust layer.
But the point survives without us. A merchant who never installs anything we
make still has to answer the question, and the answer is still theirs. The
worst outcome is not choosing wrong. It is having the choice made
accidentally by whichever default happens to be shipping that quarter.
Shopify gave every merchant a key. What it unlocks is still up to them, and
almost none of them know they are holding it.
– N.M.
Method note: probes identify honestly as ClaudeBot and GPTBot and record
robots.txt directives alongside live response codes. Verdicts use the last
clean observation per domain; rate-limited and unreachable probes are
excluded rather than counted as blocks, since an inability to observe is not
an observation. Our own repeat scanning of one domain triggered its rate
limiter, which is how we found that distinction mattered.
These findings were reported to the IETF web-bot-auth working group on
August 1, 2026, before publication.
AVA Payâ„¢ is a merchant-side trust gateway for AI commerce: one API that
verifies the signatures AI shopping agents already carry, and lets merchants
set policy for what verified identity earns. Open source at
github.com/AVA-PAY/ava-pay. More essays
at avalayer.com/writing.
Top comments (0)