I measured whether datacenter IPs are actually blocked. Only 4% of the top 1,000 sites refuse them.

Faceless Satine — Sat, 12 Sep 2026 07:06:50 +0000

"Datacenter proxies are dead" is repeated constantly in scraping circles and I have never seen a number attached to it. So I built one.

Every day, a job takes the Tranco top 1,000 and asks each host for its homepage twice at the same moment — once from a datacenter IP, once through a residential exit. It publishes only the cases where the two answers disagree.

The list is not what it says it is

The first finding had nothing to do with blocking.

228 of the top 1,000 names have no address at their apex at all. akamai.net is rank 8. gtld-servers.net is 27. apple-dns.net is 33. Tranco ranks by DNS query volume, so nameserver and CDN domains — which answer billions of lookups and serve a homepage to nobody — sit near the top of a list everybody uses as "the top websites."

That is roughly a quarter of the sample. If you have ever drawn a crawl target list from Tranco, you were partly crawling infrastructure.

For this measurement it is load-bearing rather than trivia. A name with no address of its own still gets an answer from a residential exit whose resolver replies regardless — and that looks byte-for-byte like "the datacenter request was refused and the residential one succeeded." Before I resolved every name first, that alone had put 42 hosts on the published list that did not belong there.

So the denominator is 772, not 1,000.

The number

Of the 772 that serve a homepage, 698 gave a clean answer on both legs.


Refused datacenter, served residential	31 (4%)
Refused residential, served datacenter	17
Blockers behind Cloudflare	42% (13 of 31)
Blockers announcing no CDN	8
Dialled, no clear answer	74

Four percent. Twenty-four sites in twenty-five still answer a datacenter IP.

Rank barely predicts it

The folk belief is that the big sites are the hard ones.

Band	Block rate
Top 100	4% (3 of 74)
101–500	3% (7 of 263)
501–1,000	6% (21 of 361)

Flat enough that the honest reading is that popularity does not predict whether a site cares where your request came from.

Residential is not strictly better

17 sites did the reverse — served the datacenter request and refused the residential one. A residential-only setup loses those. It is a smaller number than the headline, but it kills the idea that one IP type dominates the other.

Who actually blocks

Refusing a datacenter IP and serving the identical request from a residential one, in the same second:

openai.com (rank 85), medium.com, mit.edu, patreon.com, imgur.com, meta.com, espn.com, tripadvisor.com, character.ai, investopedia.com, ancestry.com, att.com, ikea.com, tencent.com.

Method, and the two things that cost me the first version

Ask twice, at the same moment. A plain GET / from a datacenter IP and the same request through a residential exit, following up to three redirects. Status and headers only — never the body.

Resolve before dialling. Covered above. This is the one that produced 42 false positives.

Confirm before counting. Every refusal is re-asked before it counts: twice from the datacenter, where the IP never changes, and up to three times residentially, where every attempt draws a different exit. A pair that would be published gets measured again from a fresh exit. A difference that will not reproduce is not published.

Publish only the disagreement. A host that refuses both legs is blocking the request, not the IP. That is a measurement whose answer is "does not single out datacenter traffic" — not an exclusion.

Limits, stated up front

Homepages only. No logged-in pages, no search endpoints, no APIs.
One datacenter provider and one residential pool. A different pair moves the number; the direction should hold.
A snapshot per host, not a rate over time. A soft block that fires on the second request looks like a pass here.
A bot wall that returns 200 with a challenge body counts as served. So 4% is a floor, not a ceiling.
The crawler identifies itself as ProxmintBench/1.0 rather than impersonating a browser. I checked what that costs: a browser user-agent with realistic Accept headers changed no status on either leg.

Take the data

CC BY 4.0, JSON and CSV, no key, CORS open:

curl 'https://proxmint.com/api/site-blocks'
curl 'https://proxmint.com/api/site-blocks?format=csv'
curl 'https://proxmint.com/api/site-blocks?blocked=all'

Full table and daily-updated figures: https://proxmint.com/free-proxies/blocked-sites

Disclosure: I sell proxies. That is why the measurement exists, and it is why I would rather you checked it than believed it. The data is free either way, including to anyone writing something that concludes you do not need what I sell.

Update, 23 Sep 2026: what held up over 12 days

The job has now run daily for 12 days (12–23 Sep). The datacenter-only block rate stayed between 3.2% and 5.3% (22 to 37 sites per run), so treat any single day's number as noisy and the band as the result.

The per-site list above came from one run, and not all of it held up. 17 sites refused the datacenter leg in all 12 runs, including reddit.com, mit.edu, patreon.com, imgur.com, espn.com, genius.com, mediafire.com, teamviewer.com, wiley.com, doctolib.fr, kleinanzeigen.de and att.com. Some names above did not: openai.com showed up in 2 of 12 runs, medium.com in 1, meta.com in none, and character.ai, investopedia.com and ancestry.com in 3–4. Those are sites whose answer depends on which exit you draw, not reliable blockers.

The reverse set (residential refused, datacenter served) ran 17 to 23 sites a day. Daily snapshots are mirrored to https://github.com/proxmint/blocked-sites if you want to check the history yourself.

We tested 62,541 free proxies from GitHub. Only 4% actually work.

Faceless Satine — Fri, 03 Jul 2026 13:36:55 +0000

If you've ever grabbed a "free proxy list" off GitHub, you already know the feeling: you paste 10,000 IPs into your scraper, and approximately none of them connect. I wanted to know exactly how bad it is, so I measured it.

Short version: of 62,541 free proxies pulled from public lists, only 2,236 (4%) were alive. And "alive" is generous — it just means the TCP handshake completed once.

How I measured it

I built a liveness checker that:

Ingests the big public free-proxy lists on GitHub (the monosans/proxy-list-style aggregators — HTTP, HTTPS, SOCKS4, SOCKS5).
Dedupes and re-checks every proxy through a controlled endpoint on a rolling 5-minute cycle.
Records success/failure, latency, and protocol over a 7-day window.

No sampling — the full set, continuously. A proxy counts as "alive" onlest in the most recent check.

The results

4% alive. 2,236 of 62,541. That's the number at any given moment; ttly.

Alive-rate by protocol:

Protocol	Live now	Alive rate
HTTP	1,474	6%
SOCKS5	479	2%
SOCKS4	247	2%
HTTPS	36	10%

HTTP dominates the live pool by raw count, but every protocol is brutal. HTTPS has the "best" alive-rate (10%) only because so few are listed to begin with.

It gets worse when you look at stability:

Median latency of live proxies: ~2,900 ms. Nearly 3 seconds per request — before your target server even responds.
Uptime across the live set: 44%. A proxy that answers now has a con the next check. So the usable pool at any instant is a fraction ofeven that 4%.

Put together: you might scrape a 60k list, find ~2k that ping, and be left with a few hundred that are actually usable for more than one request — each one adding ~3s of
latency.

Why free proxy lists are like this

They're recycled instantly. The moment an open proxy is discoverered by everyone until it dies or gets firewalled.
No ownership = no maintenance. Most are misconfigured servers or honeypots, not intentional services. Nobody keeps them up.
Churn is the whole game. By the time a list is committed to GitHud. Lists that "update daily" are mostly re-listing corpses.

The security part nobody mentions

Free proxies aren't just unreliable — some are actively harvesting. An nd your target and sees everything not wrapped in TLS. Treat any traffic through a random free proxy as public. Never send credentials or anything sensitive through one.

What this is actually good for

Free proxies do have a narrow, legit use: throwaway, low-stakes, high-tolerance-for-failure tasks where you can retry infinitely and don't care about latency or getting a
clean IP. Building a hobby scraper to learn on? Fine. Anything you depeill eat you alive.

Data + methodology

Full breakdown — alive-rate by protocol, geography, latency distributiop — is published here and refreshed continuously:

👉 **[Free Proxy Statistics report](https://proxmint.com/free-proxies/statistics

Disclosure: I run Proxmint, a paid proxy servibuilding a free proxy-liveness tool, and the numbers were bleak-worthyenough to write up. The report page is free to read and cite — raw figures on request.

What's your experience been with free proxy lists? Curious if anyone's seen a source with a meaningfully better hit-rate.

webscraping #proxy #opensource #data

DEV Community: Faceless Satine