DEV Community

Gregory Pellitteri
Gregory Pellitteri

Posted on

Which AI Is Actually Reading Your Website? We Counted 18.9 Million Crawler Visits

Everyone optimizes for ChatGPT. Our server logs say that is the wrong target.

We count AI crawler visits at the server across a large network of business pages, matched against reverse-checked bot signatures so a bot cannot pretend to be something it is not. The counter is now past 18.9 million verified AI crawler visits. When we break it down by which AI is doing the reading, the result surprises almost everyone.

The breakdown

  • ClaudeBot (Anthropic): ~70% of all AI crawling we see, more than 13 million visits
  • Applebot (Siri): ~18%, about 3.4 million visits
  • GPTBot (ChatGPT): ~10%, about 1.9 million visits

Those three are ~98% of AI crawl traffic. ClaudeBot alone does roughly seven reads for every one from GPTBot. And Apple's crawler out-reads OpenAI's, which almost nobody is planning for.

Why this matters

The question is rarely whether AI has read your site. It almost certainly has, and it was probably Claude. The question is what it understood when it did.

The good news: you do not optimize per bot. Claude, Apple, and OpenAI's crawlers all read the same signals: an open robots.txt that lets AI crawlers in, structured data (JSON-LD) they can parse, and an llms.txt that tells them what you are. Get those right and you are readable to all of them at once. Get them wrong (block AI crawlers, ship no schema) and you are invisible to the 13-million-visit reader, not just the two-million one.

The honest limits

This is the crawler mix across our network; the exact split on any single site varies with its age, links, and platform. And a visit is not a citation. Being read is step one; being understood and then named is the rest. What the count settles is narrower and useful: the AI reading your site the most is very likely not the one you have been picturing.


Full write-up and methodology on our blog: Which AI Is Actually Reading Your Website?

You can check which AI crawlers have hit your own site with a free AEO score at engagemii.com/aeo.

Top comments (0)