I split my site's AI bot traffic three ways. Only one turns into a citation.

#webdev #seo #ai #buildinpublic

Everyone keeps saying "AI is reading your site now." I wanted the actual number, not the vibe. So I read my own Cloudflare logs and split every AI bot fetch by what the fetch is FOR. One 23.5 hour window, my own site.

Here is what came back.

Live answers: 189. These are ChatGPT-User, Perplexity-User and Claude-User. A real person asked a question, and the assistant fetched my page right then to write the answer. This is the fetch that can become a citation, because there is a human on the other end of it, mid decision.

Search index: 541. OAI-SearchBot, PerplexityBot, bingbot, Applebot. These build the index the assistants search over. Indirect, but it is how you become retrievable in the first place. Bing matters more than it looks here: it grounds Copilot and it receives ChatGPT's search queries.

Training: 1,178. GPTBot, ClaudeBot, Google-Extended, CCBot and friends. Collecting data for a model that ships months from now. The biggest bucket by far, and the least useful to you today.

Here is the honest part of that last number: 974 of the 1,178 training fetches came from crawlers Cloudflare flags as "AI" but does not name. I cannot attribute them precisely, and that is its own finding. A large slice of the AI traffic hitting a site is currently unverifiable, because nobody publishes the list you would need to check it against.

Why the split matters

If you look at a dashboard that says "2,000 AI bots visited you this week," you are mostly looking at training noise. It feels like progress. It is not the thing that sends you a customer.

The 189 live-answer fetches are the ones worth chasing. In one week I measured, 23 to 29 July, six real people arrived on my site from an AI answer: three from ChatGPT, two from Bing, one from Perplexity, across four referring sources. A browser beacon counts them and no bot triggers it, so a referral there is a human. Five of the six landed on a page built from my own measurement data. Zero landed on a page that sells, not the home page, not pricing, not the check. Every assistant that sent me a person sent them to a page of data, and none of those people met an offer. That is the most useful number in this whole post.

Lump all three buckets into "AI traffic" and you lose that story. Split them, and the story is obvious: publish something worth fetching in the middle of an answer, and the live-answer number is the one that moves.

How to read your own, in five minutes

You do not need a tool for this. Match the User-Agent in your logs:

Live answer: ChatGPT-User, Perplexity-User, Claude-User
Search index: OAI-SearchBot, PerplexityBot, Claude-SearchBot, bingbot, Applebot
Training: GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider

One warning: do not trust Cloudflare's own bot category alone. On my site PerplexityBot came back with an empty category while ChatGPT-User was labelled correctly, so leaning on the label would have silently dropped one of the three assistants I care about most. Match the raw User-Agent yourself.

Also filter to successful responses (status under 400). A credential scanner was spoofing PerplexityBot and OAI-SearchBot user agents to probe for /.env and /config files, all 404, and every one of those was inflating my "assistant" count until I filtered them out. It cut the number by a third.

What I did with it

The live-answer fetches cluster on my pages that carry original data, the ones where I measured something and published the raw result. They do not cluster on my pricing page or my homepage. So the move is not "optimise the landing page for AI." It is "publish a number nobody else has, on a page an assistant can quote in one sentence."

I build a tool that measures exactly this, so I run it on myself and publish the numbers, training noise and all. If you want your own three-way split without reading logs by hand, that is what it does, and the first check is free at glotier.com. But everything above is the method, and your Cloudflare logs already have the data. Go look.