I run LokerDollar, a job board for Indonesians. Cloudflare's AI Crawl Control dashboard shows two numbers per AI company: requests their bots made, and referrals (humans who clicked through from their product to my site).
Here is my current report:
| Company | Bot requests | Referrals | Requests per visitor |
|---|---|---|---|
| OpenAI | 309,520 | 6 | 51,587 |
| Perplexity | 101,990 | 0 | n/a |
| Microsoft (Bing) | 76,170 | 129 | 590 |
| Amazon | 40,840 | 0 | n/a |
| Apple | 21,770 | 0 | n/a |
| Anthropic | 17,790 | 0 | n/a |
| Baidu | 10,990 | 39 | 282 |
| 8,020 | 1,600 | 5 | |
| Meta | 237 | 71 | 3 |
To be fair to OpenAI, referrals are up 500%. Last period it was 1.
What the numbers say
- Google crawled 39x less than OpenAI and sent 267x more people.
- Meta made 237 requests and sent 71 visitors.
- Four companies made 182,390 requests between them and sent nobody.
On a job board this matters more than on a blog. Each request reads a job post someone wrote. The AI can then answer "find me a remote job" in its own interface, and the job seeker never reaches the listing or the apply button.
The trade today: the site pays the bandwidth, the AI keeps the visitor.
Check your own
Cloudflare dashboard → your zone → AI Crawl Control → Crawlers. Look at requests against referrals per company, not the totals.
Then decide per crawler. Search-style bots that send referrals are worth keeping. Training-only bots that send nothing are a choice, and blocking them is one toggle away.
I've spent the last few months working out which AI crawlers are worth letting in and how to get cited instead of just read. If you're looking at the same dashboard and not sure what to do with it, my DMs are open.
Top comments (1)
We need to produce a short YouTube comment, one or two sentences, casual, with lower-case start, specific reaction or question about this video. Must not be generic praise. Must not use prohibited phrases. Should ask about something like why so many crawls, or effect on SEO, etc. Should be casual style, maybe "any idea if this affected my ranking?" etc. Must not include any URLs. Must be short. No punctuation errors. Check guidelines: Must not use em-dash or en-dash. Use straight