Which AI crawler fetches mean a real person asked a question?
User-fetch agents such as ChatGPT-User, Claude-User and Perplexity-User fire when a person asks an assistant a question and it needs a live page. Index crawlers such as OAI-SearchBot build a search index ahead of time. A user fetch means you were considered for a real question, so report it separately.
I am Eliran, founder of Cuanto Labs, a software company in Fort Lauderdale, Florida. We log crawler requests on our own marketing site on the server side, because most of these bots never run JavaScript and so never appear in a browser-based tool. This post is what the first eight days of that log showed.
A warning about sample size, up front. Our log starts on October 7, 2026, and this is a small site. Every number below is a count from that window, not a rate and not a trend. Treat them as direction, not proof.
What did the counts look like?
This is the server-side log for our main domain plus an old alias domain that now redirects to it, from October 7 to October 15, 2026. Total user fetches were 90, of which 75 were ChatGPT-User. We saw zero hits from GPTBot or Applebot, which is a fact about our log and not a claim about those bots elsewhere.
| Bot | Type | Requests | Distinct paths |
|---|---|---|---|
| Bingbot | index | 683 | 80 |
| PerplexityBot | index | 448 | 89 |
| OAI-SearchBot | index | 345 | 61 |
| ClaudeBot | index | 99 | 60 |
| Googlebot | index | 86 | 26 |
| ChatGPT-User | user fetch | 75 | 28 |
| Claude-User | user fetch | 13 | 7 |
| Perplexity-User | user fetch | 2 | 1 |
| GPTBot, Applebot | index | 0 | 0 |
When do index crawlers show up?
They arrive in bursts, near deploys. OAI-SearchBot's 345 hits were not spread evenly: nearly all fell into three short windows, the morning of October 10, midday and evening of October 12, and the morning of October 13. Every new page from our October 12 evening release was fetched within minutes of going live.
Which pages do assistants fetch for users?
Mostly the plain-facts pages. Of the 90 user fetches, brand, home and portfolio pages took 59, pricing, buy and service hubs took 12, city pages took 7, AI automation pages took 6, and research and blog took 6. These are counts of fetches, not conversions.
An assistant that fetches only the home page should be able to answer who you are, where you are and what you do. A few plain sentences matter more than a clever one. Say what you do, where you are and what it costs, in words an assistant can lift.
Does the daily curve follow what you publish?
It did for us, though one peak proves little. Daily user fetches in our window were 12 on October 10, 32 on October 12 (the peak), 9 on October 13 and 14 on October 14. The October 12 peak lines up with the release day. The shape is consistent with new content being looked at when it exists.
Can an index hit come before a live fetch?
Yes, and it can come days earlier. One example, without identifying anyone: a page on automation services was first fetched by OAI-SearchBot on the evening of October 12. On the afternoon of October 14, ChatGPT-User fetched the same page. About an hour later, a business owner emailed us directly asking to talk about AI and lead generation.
We cannot prove the two are linked, and nothing contradicts it. We put it at roughly even odds on timing alone.
What it does show is that this channel can leave no trace in the usual places. There was no chatgpt.com referrer, no campaign tag and no form submission, because the person went straight to email. If you only look at referrals and forms, a conversation like this is invisible. The server-side fetch log is the only place the chain shows up at all.
What should you log to see AI bot traffic?
Log four fields on every request whose user agent matches a known AI bot, then report index crawlers and user fetches in two separate tables. You do not need an enterprise tool, and the log costs very little to start keeping.
- Timestamp, converted to the time zone you read reports in.
- User agent family, normalized to a short name.
- Host and path, and whether the response was a 200, a redirect or a 404.
- Whether the request came from a data-center network, which helps separate bot traffic from ordinary visitors.
Two cautions. User agent strings can be spoofed, so where a vendor publishes IP ranges, verify against them before you treat a hit as real. And log server-side only: these agents mostly do not execute your analytics script, so a browser-based tool shows you none of this. Make sure your tracking script does not fire for them either, or your human counts get polluted.
What does this suggest you do?
These follow from the log. They are suggestions, not a record of results. Put the plain facts on the home page and the pricing page, since those were fetched most. Keep your sitemap current so new pages are discoverable on the first crawl.
- If you have an old domain, redirect it to the main one. In our log, 14 of the 75 ChatGPT-User fetches went to the old alias domain, which now redirects, so fetches land on one canonical site.
- Do not read meaning into a probe path that returns a 404. Fix it or filter it out of your crawl report.
What are we not claiming?
We are not claiming that appearing in this log produces customers at any rate. Our sample is small, our window is eight days, and the one possible conversion above is a single timing coincidence. The narrower claim: the log is the only way to see user-fetch behavior at all.
If you want help with AI automation, agents or the visibility side of it, our AI automation page lists what we build and what it costs. Related: SEO for AI businesses.
Written by Eliran Yihye, founder of Cuanto Labs, on October 15, 2026.
Originally published at cuantolabs.com.
Top comments (0)