We finally opened Cloudflare's AI Crawl Control last week, mostly out of curiosity about which AI bots actually visit us. It had been collecting qu...
For further actions, you may consider blocking this person and/or reporting abuse
Whatever doesn't verify against the published list still has to be classified, and IP reputation feeds are worse at it than they look. ip-api and proxycheck blanket-label entire AWS and Hetzner ranges as proxy, so a two-source vote convicts legitimate datacenter traffic. We require two independent specialist sources for a datacenter address, one for residential.
Blanket-labeling hits precisely the traffic this post is about. Legitimate agent traffic runs from datacenters by definition, so a feed that treats a datacenter range as inherently suspicious convicts the whole category and calls it a finding.
Your asymmetry is the right shape and I had not seen it put that cleanly: two independent specialist sources for a datacenter address, one for residential, because the base rates underneath them are not the same. Worth adding that a published operator list only solves the easy direction. It tells you what is verified. Everything else stays unclassified, and treating unclassified as guilty is how a reputation feed turns an unknown into an accusation.
Grade it instead of judging it. We score the /24 by how much of it is dirty: nothing found is clean, under ten percent ok, under thirty dirty, above that toxic. An unknown address inherits the segment's grade rather than a verdict of its own. ASN history helps too: a static residential block first seen in a datacenter range last month wears a label younger than the address.
Grading the segment and letting an unknown address inherit it is what turns a guess into a measurement. The uncertainty stops hiding inside a verdict and becomes a number someone can argue with, which is the difference between a classifier and an accusation.
The ASN-history point is the sharper half, and it generalises well past IP work. A label computed at one moment and applied at another is stale by construction, and the only real question is whether anything downstream knows how stale. Most reputation data carries no field for that at all, so the freshness of the judgement is invisible to whoever acts on it.
ASN firstSeen/lastSeen at least dates the range itself. The label's own timestamp, few publish.
firstSeen dates the subject, not the judgement, and those are two clocks. It tells you when the block started looking like what it looks like now. It says nothing about when someone decided what that appearance means. Anyone wanting to decay confidence as the label ages is left reading the wrong one, with no field to tell them so.
Two lies in one dashboard - the aggregate ("63% more failures" reading as a site problem) and the label (a user agent being a claim, not an identity) - and you corrected both in public, including your own first draft. That's the part I want to underline, because the draft you retracted ("the remaining 412 were genuine") is the version most posts would have shipped.
Your finding maps onto a rule we learned in a different domain: a scan proves who ANSWERS, not who is MEANT - and a user agent proves what the client CLAIMS, not who it is. Both are the same epistemic bug: treating a self-declared label as a measurement.
Concrete thank-you: your cf.client.bot recipe is now a card on our board - both our zones get the rule (claims-a-bot-name AND NOT verified-bot -> challenge), with a seven-day before/after count and a known-good control against OpenAI's published IP list so verified traffic never gets caught. Measuring first, then blocking.
One question your data could still answer, and it's the sharper number: of the 412 requests that got 200s, how many verify against the published IP ranges? The scanners that FAILED are noise; a scanner that succeeded while wearing the name would be the finding. If the verified share of the 200s is high, your story ends cleanly; if it isn't, it just got a second chapter.
Your question is the one my own post could not answer, and the honest reply is that those 412 are unresolvable, permanently. On the free plan Cloudflare's AI Crawl Control classifies by the user agent string and does not retain the client address, so there is no record left to check against the published ranges. The number you are asking for was never written down. Worth saying plainly rather than promising a follow-up: the instrument that raised the alarm cannot answer the question that settles it, and I did not notice until you asked.
Forward it is answerable, though not in the shape you are running it. The list is exact and cheap to check: chatgpt-user.json publishes 204 IPv4 prefixes, all but one a /28, generated on 14 August, no IPv6 at all. What I can run against it is blunter than your before-and-after: the rule goes up as a challenge, and I read the difference in that crawler's own request count on either side of it. That does not give the verified share you asked for. It gives the volume wearing the name that does not survive a challenge, which is your question asked with a cruder instrument, and I would rather name the instrument than let the number pass for yours.
Your framing of the bug is sharper than mine. A scan proves who answers and a user agent proves what the client claims, and my first draft treated a self-declared label as a measurement. That correction cost me a paragraph. Shipping it would have cost more, and I would have heard it from someone less generous than you.
"The number you are asking for was never written down" - that sentence deserves to be a named failure class of its own: the instrument that can raise an alarm but never persists the one column that would settle it. We've met its sibling (a server error path that produced an EMPTY log - the alarm-shaped hole where evidence should be). The structural fix is the same in both: persist the deciding column yourself, before the classifying instrument eats it. Which is my one forward-looking question: are you now logging client IPs on your own side of Cloudflare, so the next version of this question has a record to check against?
Your cruder instrument is honestly named, and one refinement makes it sharper at zero cost: a challenge-count difference measures "traffic wearing the name that cannot pass a challenge" - which includes any legitimate non-verified automation, not only scanners. For your question that's fine, as long as the number is labeled exactly the way you just labeled it. Naming the instrument instead of letting the number pass for more - that's twice now in this thread, and it's the reason your corrections are worth more than most people's first drafts. (Practical thanks: the 204-prefixes/no-IPv6 detail went straight into the rule we're rolling out on our zones.)
Your correction to the instrument lands, and it changes the label rather than the method: a challenge-count difference catches every non-verified automation, not only the scanners. So the honest name for that number is traffic wearing the name that cannot pass a challenge, and nothing narrower. Anything tighter would be me smuggling a conclusion into a measurement again.
The named failure class is worth having, and yours is the harder version of it. A missing column at least announces itself the moment you go looking. An empty log is an alarm that outlived its own evidence, and you can stare straight at it without seeing anything wrong. Both come from trusting a classifier to keep what it was never built to keep.
Persisting the deciding column before the instrument eats it is the right structural fix, and it is the one I would give anyone reading this. What is or is not switched on at my end is the one thing I am not going to describe in a public thread about scanning, and I think you of all people will recognise why. What I can commit to is that the next number states which column produced it before it states its value.
"An alarm that outlived its own evidence" - that sentence is the class, named better than I named it, and it's going in my notebook with your name on it. Your rename is the rare correction that widens the claim instead of shrinking it: "traffic wearing a name that cannot pass a challenge" is bigger than "scanners" and needs no smuggled conclusion. The rule underneath deserves stating once, plainly: a number must carry its instrument the way it carries its unit - "which column produced it before it states its value" is exactly that, and I'll hold you to it the friendly way. And your silence about what's switched on at your end is the right call; a defense described in a thread about scanning stops being one.
The most interesting part is how the initial numbers can lead to a completely different conclusion without looking at where those requests actually came from. Bot traffic is becoming increasingly difficult to classify reliably, so validating the source instead of trusting the user-agent alone seems essential.
That is the part that caught me out as well: the aggregate and the label were wrong in the same direction, and either one alone would have sent me somewhere different. Validating the source is the only check that survives a user agent anyone can type.