DEV Community

Flora
Flora

Posted on

That "Residential" IP Might Be a Datacenter in Disguise - A Scriptable Proxy Audit

The scenario nobody plans for

You buy a residential proxy plan. Weeks later your scraper's block rate quietly doubles, and support tells you the pool is "100% real residential IPs, peer-to-peer sourced." You have no way to check that claim, so you either trust it or you churn.

This is uncomfortable to write about as someone affiliated with a proxy vendor — which is exactly why you should learn to verify it yourself. Proxy type is not a marketing adjective. It is a property of an IP address that anyone can look up, because every public IP is registered to an ASN, and ASNs are classified as hosting providers, ISPs, or mobile carriers at the registry level. If your "residential" exit IPs resolve to Hetzner or OVH address space, no amount of branding changes what the target website sees.

In this post I'll walk through what actually distinguishes the three main proxy types, then give you a small Python script that samples egress IPs through your gateway and classifies them. It runs in about twenty minutes and uses only requests plus two free public APIs.

Why the distinction matters mechanically

When a site's anti-bot system scores a visiting IP, the single most informative feature is the reputation of the /24 (or /48) block the IP belongs to. And the dominant driver of block-level reputation is the ASN's registered type:

  • Datacenter (hosting) space — ASN owned by a cloud or hosting provider: AWS, Google Cloud, Hetzner, OVH, Vultr, DigitalOcean, Contabo, and thousands of smaller ones. Every serious bot-management vendor maintains a hosting-space list. Requests from these blocks get elevated scrutiny almost regardless of behavior. These IPs are fast and cheap, which is why datacenter proxies have a legitimate role (more on that later), but they are not residential.
  • Consumer ISP (fixed-line) space — ASN owned by Comcast, AT&T, Deutsche Telekom, Telefónica, Jio, and so on. This is what a true residential pool is: end-user devices behind consumer connections. Traffic looks like normal households, so block rates are much lower — at the cost of latency, because your request now traverses real users' links.
  • Mobile carrier space — IPs from T-Mobile, Vodafone, Airtel and similar, typically behind CGNAT. Hundreds or thousands of real users share the same egress IP, so the blocklists are reluctant to poison a single IP. This is the highest-trust tier, and the most expensive per gigabyte.

There is a fourth category that shows up in audits: static ISP proxies, which are IPs allocated from consumer ISP space but parked in a rack rather than attached to home devices. They register as "residential type" in most geo databases, behave like residential to many blocklists, and offer datacenter-ish speed. That gray zone is precisely where mislabeling is easiest to sell — and easiest to detect, because the ASN owner is public record.

The three independent signals

Any single check can be fooled; three checks agreeing is persuasive.

  1. ASN owner and type. Registry data (RIPE/ARIN/APNIC via whois, or conveniently via ip-api.com) gives the organization behind the IP. "HETZNER-AS" in your residential plan is a red flag; "COMCAST-7922" in your datacenter plan is a surprise in the other direction.
  2. Reverse DNS (PTR record). Consumer ISPs usually assign customer PTR names (cpe-76-xxx.res.spectrum.com, pppoe-..., host-by-...). Datacenter blocks use provider PTR patterns (static.94.23.44.5.ovh.net). Many residential IPs have no PTR at all — absence is normal and not a failure signal, but a PTR naming a hosting provider is a strong one.
  3. Geolocation consistency. Compare the city/country your gateway claims to use against what three-party geo databases report for the egress IP. Frequent city-level disagreement means either sloppy pool metadata or traffic exiting somewhere other than what you paid for. Be careful here: consumer-grade geolocation databases genuinely disagree at city level even for honest IPs, so treat this as evidence, not a verdict.

The audit script

Requirements: Python 3.9+, pip install requests. The free ip-api.com endpoint is HTTP-only and rate-limited to about 450 requests/minute, so we sleep between samples.

A quick manual check first — before you need the whole script:

# what IP do I actually egress as?
curl -s -x http://USER:PASS@gateway.thordata.com:9999 https://api.ipify.org?format=json

# what does the registry think that IP is?
curl -s "http://ip-api.com/json/203.0.113.42?fields=status,country,city,isp,org,as"
Enter fullscreen mode Exit fullscreen mode

If the org field says anything resembling a hosting provider while you're paying for residential, you already have your answer. The script below automates a statistically meaningful version of that check: sample N egress IPs, look up each one, classify, and report the distribution.

"""proxy_audit.py — sample egress IPs through a gateway and classify pool composition.

Usage:
  python proxy_audit.py --proxy http://USER:PASS@GATEWAY:PORT --sessions 30
Most residential/ISP gateways rotate the exit IP per connection (or per
minute); if your gateway supports sticky sessions, do NOT enable them here,
or you will measure one IP thirty times.
"""

import argparse
import json
import socket
import time
from collections import Counter

import requests

DATACENTER_KEYS = [
    "hetzner", "ovh", "vultr", "digitalocean", "contabo", "scaleway",
    "amazon", "microsoft", "google cloud", "fastly", "cloudflare",
    "akamai", "equinix", " Choopa", "linode", "upcloud", "xtom",
    "serverius", "pskz", "hostkey", "bluevps",
]
MOBILE_KEYS = [
    "t-mobile", "telekom mobile", "vodafone", "orange", "airtel",
    "jio", "china mobile", "china unicom", "verizon wireless", "at&ts",
]

def classify(meta: dict, ptr: str | None) -> str:
    blob = f"{meta.get('org', '')} {meta.get('isp', '')} {ptr or ''}".lower()
    if any(k in blob for k in MOBILE_KEYS):
        return "mobile-carrier"
    if any(k in blob for k in DATACENTER_KEYS):
        return "datacenter"
    # An ASN owned by a known consumer ISP with no hosting keyword.
    if any(k in blob for k in ["comcast", "spectrum", "att", "verizon",
                              "telefonica", "movistar", "deutsche telekom",
                              "bt ", "telecom", "broadband", "cable"]):
        return "consumer-isp (residential-like)"
    return "unclassified"   # be honest: unknown is not a verdict

def reverse_dns(ip: str) -> str | None:
    try:
        return socket.gethostbyaddr(ip)[0]
    except (socket.herror, socket.gaierror, OSError):
        return None        # no PTR — common for real residential IPs

def sample_once(proxy_url: str) -> dict | None:
    proxies = {"http": proxy_url, "https": proxy_url}
    try:
        r = requests.get("https://api.ipify.org?format=json",
                         proxies=proxies, timeout=25)
        r.raise_for_status()
        ip = r.json()["ip"]
    except Exception as e:
        return {"error": f"egress failed: {type(e).__name__}"}
    try:
        m = requests.get(
            f"http://ip-api.com/json/{ip}"
            "?fields=status,country,regionName,city,isp,org,as",
            timeout=15).json()
    except Exception as e:
        return {"ip": ip, "error": f"lookup failed: {type(e).__name__}"}
    ptr = reverse_dns(ip)
    return {"ip": ip, "city": m.get("city"), "country": m.get("country"),
            "as": m.get("as"), "org": m.get("org"), "ptr": ptr,
            "class": classify(m, ptr)}

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--proxy", required=True, help="http://user:pass@host:port")
    ap.add_argument("--sessions", type=int, default=30)
    ap.add_argument("--sleep", type=float, default=2.0)
    ap.add_argument("--out", default="audit_results.jsonl")
    args = ap.parse_args()

    rows, tally = [], Counter()
    with open(args.out, "w") as f:
        for i in range(args.sessions):
            row = sample_once(args.proxy)
            row["sample"] = i
            rows.append(row)
            f.write(json.dumps(row) + "\n")
            tally[row.get("class", row.get("error", "error"))] += 1
            print(f"[{i+1:>3}] {row.get('ip',''):<16} "
                  f"{row.get('city',''):<14} {row.get('class', row.get('error'))}")
            time.sleep(args.sleep)   # respect ip-api free-tier limits

    print("\n=== pool composition ===")
    for k, v in tally.most_common():
        print(f"{v:>4}  {100*v/args.sessions:5.1f}%  {k}")
    print(f"\nRaw per-IP evidence saved to {args.out} — keep it; it is the "
          "receipt you show your provider when something looks off.")

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

A couple of implementation notes that will save you a debugging cycle:

  • Pass the proxy under both "http" and "https" keys in the dict — requests needs the explicit https entry to tunnel TLS over the proxy, and api.ipify.org is HTTPS-only.
  • socket.gethostbyaddr throwing is expected behavior for many genuine residential IPs (no reverse record configured). Only treat a PTR that names a hosting provider as adverse evidence.
  • The keyword lists above are a starting point, not a taxonomy. When the script prints unclassified, look up that ASN's whois once by hand; registry "status" fields (ASSIGNED PI vs LEGACY vs allocatable-to-subscribers) tell you more than any keyword list.

Reading the results

What I'd expect a clean run to look like: a residential pool dominated by consumer-isp with a tail of unclassified (that's normal — there are ~60,000 consumer ISPs worldwide, far more than any keyword list covers). Now the failure signatures:

  • Majority datacenter on a residential plan. Either you were resold hosting space, or a mixed pool is configured to prefer the cheap half. Take the JSONL evidence to support with specific IPs and timestamps.
  • A stable handful of IPs repeating across samples. Two causes: sticky-session mode left on (fix your gateway config), or the pool genuinely has a few /24s it keeps reusing — which is also worth knowing, because reuse concentrates blocklists.
  • City mismatch rate above roughly 20%. Escalate gently; geo DB lag is real, but systemic country-level mismatch never is.
  • Mobile plan full of consumer-isp. Mobile proxies are priced at a premium for the carrier-CGNAT property; fixed-ISP IPs in that pool means you are overpaying.

And to be fair about the gray zone: static ISP proxies legitimately occupy the middle. They come from consumer-ISP ASN space (so this audit will happily classify them "residential-like") even though they sit in a rack. If you want datacenter reliability at residential-ASN reputation — dashboard monitoring, ad verification where sessions must persist for weeks — that's the right product, and the pricing gap is large: Thordata, for example, lists its ISP line starting at $2 per IP (fetched from their pricing page on 2026-09-17), a fraction of typical residential per-GB economics. Just buy it knowingly, not by accident.

Pools lie in both directions

The uncomfortable lesson from running audits is that labeling errors go both ways: some "enterprise residential" pools are 70% cloud, and some "cheap datacenter" pools turn out to hold real ISP space someone sold as shared hosting by mistake. The only durable response is to make verification part of your pipeline — sample N exits after each provider change, keep the JSONL, and diff the composition monthly. It costs you thirty lines of cron and thirty minutes of attention, and it converts every vague suspicion into a specific, fixable ticket.

Disclosure: I work with Thordata (sponsored account). The audit script above uses no Thordata-specific tooling — run it against any vendor's gateway. If you want to see what our residential pool composition looks like under this exact script, the product page is here: https://www.thordata.com/products/residential-proxies?ls=dev&lk=DEV — and new signups get a trial allowance, no code needed.

Top comments (0)