I measured whether datacenter IPs are actually blocked. Only 4% of the top 1,000 sites refuse them.

"Datacenter proxies are dead" is repeated constantly in scraping circles and I have never seen a number attached to it. So I built one.

Every day, a job takes the Tranco top 1,000 and asks each host for its homepage twice at the same moment — once from a datacenter IP, once through a residential exit. It publishes only the cases where the two answers disagree.

The list is not what it says it is

The first finding had nothing to do with blocking.

228 of the top 1,000 names have no address at their apex at all. akamai.net is rank 8. gtld-servers.net is 27. apple-dns.net is 33. Tranco ranks by DNS query volume, so nameserver and CDN domains — which answer billions of lookups and serve a homepage to nobody — sit near the top of a list everybody uses as "the top websites."

That is roughly a quarter of the sample. If you have ever drawn a crawl target list from Tranco, you were partly crawling infrastructure.

For this measurement it is load-bearing rather than trivia. A name with no address of its own still gets an answer from a residential exit whose resolver replies regardless — and that looks byte-for-byte like "the datacenter request was refused and the residential one succeeded." Before I resolved every name first, that alone had put 42 hosts on the published list that did not belong there.

So the denominator is 772, not 1,000.

The number

Of the 772 that serve a homepage, 698 gave a clean answer on both legs.

Refused datacenter, served residential	31 (4%)
Refused residential, served datacenter	17
Blockers behind Cloudflare	42% (13 of 31)
Blockers announcing no CDN	8
Dialled, no clear answer	74

Four percent. Twenty-four sites in twenty-five still answer a datacenter IP.

Rank barely predicts it

The folk belief is that the big sites are the hard ones.

Band	Block rate
Top 100	4% (3 of 74)
101–500	3% (7 of 263)
501–1,000	6% (21 of 361)

Flat enough that the honest reading is that popularity does not predict whether a site cares where your request came from.

Residential is not strictly better

17 sites did the reverse — served the datacenter request and refused the residential one. A residential-only setup loses those. It is a smaller number than the headline, but it kills the idea that one IP type dominates the other.

Who actually blocks

Refusing a datacenter IP and serving the identical request from a residential one, in the same second:

openai.com (rank 85), medium.com, mit.edu, patreon.com, imgur.com, meta.com, espn.com, tripadvisor.com, character.ai, investopedia.com, ancestry.com, att.com, ikea.com, tencent.com.

Method, and the two things that cost me the first version

Ask twice, at the same moment. A plain GET / from a datacenter IP and the same request through a residential exit, following up to three redirects. Status and headers only — never the body.

Resolve before dialling. Covered above. This is the one that produced 42 false positives.

Confirm before counting. Every refusal is re-asked before it counts: twice from the datacenter, where the IP never changes, and up to three times residentially, where every attempt draws a different exit. A pair that would be published gets measured again from a fresh exit. A difference that will not reproduce is not published.

Publish only the disagreement. A host that refuses both legs is blocking the request, not the IP. That is a measurement whose answer is "does not single out datacenter traffic" — not an exclusion.

Limits, stated up front

• Homepages only. No logged-in pages, no search endpoints, no APIs.
• One datacenter provider and one residential pool. A different pair moves the number; the direction should hold.
• A snapshot per host, not a rate over time. A soft block that fires on the second request looks like a pass here.
• A bot wall that returns 200 with a challenge body counts as served. So 4% is a floor, not a ceiling.
• The crawler identifies itself as ProxmintBench/1.0 rather than impersonating a browser. I checked what that costs: a browser user-agent with realistic Accept headers changed no status on either leg.

Take the data

CC BY 4.0, JSON and CSV, no key, CORS open:

curl 'https://proxmint.com/api/site-blocks'
curl 'https://proxmint.com/api/site-blocks?format=csv'
curl 'https://proxmint.com/api/site-blocks?blocked=all'

Full table and daily-updated figures: https://proxmint.com/free-proxies/blocked-sites

Disclosure: I sell proxies. That is why the measurement exists, and it is why I would rather you checked it than believed it. The data is free either way, including to anyone writing something that concludes you do not need what I sell.

Comments

[szp2005]

The 17 reverse cases are the interesting ones. Reputation gets scored per /24, not per IP: once enough of a block is flagged, the whole range inherits it, and rotating exits inside it changes nothing. Worth checking your pool's ASN first-seen dates too. Plenty of 'static residential' space was datacenter space not long ago.

[P O]

Nice dataset, especially the warning about what the list actually measures. One extra control that may help is to repeat each URL with a warm browser session and a fresh cookie jar, then record status, redirect chain, TLS fingerprint changes, and response body separately. A lot of “blocked” results are challenge pages or rate limits rather than an IP decision. Publishing the timestamp and request headers used would make the CSV easier to reproduce.