DEV Community: Aethyn Team

Your scraper isn't blocked — your proxy is. Stop treating every failure the same way

Aethyn Team — Sun, 02 Aug 2026 11:38:54 +0000

Here's a retry loop. You've written this loop. I've written this loop.

for attempt in range(5):
    try:
        r = requests.get(url, proxies=proxies, timeout=30)
        if r.status_code == 200:
            return r.text
    except Exception:
        pass
    rotate_proxy()          # <- the bug
    time.sleep(2 ** attempt)

It looks defensive. It's actually doing something worse than nothing: it's collapsing seven genuinely different failures into one response. Rotate and retry. Rotate and retry.

Which means when your success rate drops, you have no idea whether you're being blocked, being rate-limited, being fingerprinted, or whether your proxy credentials expired forty minutes ago and every single one of those retries was doomed before it left your machine.

Let's separate them.

The seven failures, and what each one actually means

1. Can't reach the proxy at all. Connection refused, DNS failure, connect timeout to the proxy host. Your request never reached the target. Rotating your exit identity does nothing — you didn't get far enough to have one.
→ Retry the same target through a different upstream. Don't burn a rotation, don't back off aggressively.

2. 407 Proxy Authentication Required. Your credentials are wrong, expired, out of balance, or — the sneaky one — you asked for a feature on the wrong port. Requesting city-level targeting against a country-level endpoint fails here, and it looks exactly like an auth failure because it is one.
→ Never retry. This is a config bug and retrying just burns rate limit while you stay broken. Fail loud, page someone.

3. The CONNECT tunnel fails, or the proxy returns 502/503. You reached the proxy, the proxy couldn't reach the target, or the exit died mid-handshake. This is upstream, not you.
→ New identity, retry. If this class dominates your histogram, it's a provider problem — go read their status page, or change providers.

4. TLS handshake reset / EOF immediately after CONNECT. The tunnel opened and then the target dropped you at the TLS layer. This is usually fingerprinting — your client's TLS signature doesn't look like the browser it claims to be in the User-Agent.
→ Rotating IPs will not fix this. Ever. You need a different HTTP client. Every rotation you spend here is wasted money.

5. 403 / 401 from the target. You got a real HTTP response, and it says no. Identity-level block.
→ New identity + backoff. This is the one case where "rotate" is genuinely the right instinct.

6. 429 Too Many Requests. You're going too fast. Note that this is a statement about your behaviour, not your IP.
→ Back off first, rotate second — and honour Retry-After. Rotating immediately on a 429 is the single most common mistake in this list: you keep the request rate identical, just spread across more addresses, and now you're teaching the target that a whole range is misbehaving instead of one address.

7. 200 OK with an interstitial, a challenge page, or a suspiciously small body. The most expensive one, because your loop counts it as a success and writes garbage to your database.
→ Content check required. The status code lies.

Notice that the correct response differs in all seven cases, and that in two of them (2 and 4) retrying at all is actively harmful.

The classifier — Python

Zero dependencies beyond requests. Drop it in.

from enum import Enum
import requests
from requests.exceptions import (
    ProxyError, ConnectTimeout, ConnectionError as ReqConnectionError,
    ReadTimeout, SSLError,
)


class Failure(str, Enum):
    PROXY_UNREACHABLE = "proxy_unreachable"   # 1
    PROXY_AUTH        = "proxy_auth"          # 2
    UPSTREAM_DEAD     = "upstream_dead"       # 3
    TLS_REJECTED      = "tls_rejected"        # 4
    TARGET_BLOCKED    = "target_blocked"      # 5
    RATE_LIMITED      = "rate_limited"        # 6
    SOFT_BLOCK        = "soft_block"          # 7
    OK                = "ok"


# action: retry? rotate identity? back off? or stop and shout?
POLICY = {
    Failure.PROXY_UNREACHABLE: dict(retry=True,  rotate=False, backoff=1,  fatal=False),
    Failure.PROXY_AUTH:        dict(retry=False, rotate=False, backoff=0,  fatal=True),
    Failure.UPSTREAM_DEAD:     dict(retry=True,  rotate=True,  backoff=2,  fatal=False),
    Failure.TLS_REJECTED:      dict(retry=False, rotate=False, backoff=0,  fatal=True),
    Failure.TARGET_BLOCKED:    dict(retry=True,  rotate=True,  backoff=5,  fatal=False),
    Failure.RATE_LIMITED:      dict(retry=True,  rotate=False, backoff=30, fatal=False),
    Failure.SOFT_BLOCK:        dict(retry=True,  rotate=True,  backoff=10, fatal=False),
}

# tune these to your targets — they are the part you own
BLOCK_MARKERS = ("just a moment", "verify you are human", "access denied",
                 "unusual traffic", "enable javascript and cookies")
MIN_BODY_BYTES = 512


def classify_response(r: requests.Response) -> Failure:
    if r.status_code == 407:
        return Failure.PROXY_AUTH
    if r.status_code == 429:
        return Failure.RATE_LIMITED
    if r.status_code in (401, 403):
        return Failure.TARGET_BLOCKED
    if r.status_code in (502, 503, 504):
        return Failure.UPSTREAM_DEAD
    if r.status_code == 200:
        body = r.text
        low = body[:4000].lower()
        if len(body) < MIN_BODY_BYTES or any(m in low for m in BLOCK_MARKERS):
            return Failure.SOFT_BLOCK
        return Failure.OK
    return Failure.TARGET_BLOCKED


def classify_exception(e: Exception) -> Failure:
    msg = str(e).lower()

    # order matters: ProxyError subclasses ConnectionError in requests
    if isinstance(e, ProxyError):
        if "407" in msg or "authentication" in msg:
            return Failure.PROXY_AUTH
        # "tunnel connection failed" = proxy reached, target didn't answer.
        # everything else wrapped in ProxyError (incl. the generic
        # "cannot connect to proxy") means we never got there at all.
        if "tunnel connection failed" in msg:
            return Failure.UPSTREAM_DEAD
        return Failure.PROXY_UNREACHABLE
    if isinstance(e, SSLError):
        return Failure.TLS_REJECTED
    if isinstance(e, (ConnectTimeout, ReqConnectionError)):
        if "reset by peer" in msg or "eof occurred" in msg:
            return Failure.TLS_REJECTED
        return Failure.PROXY_UNREACHABLE
    if isinstance(e, ReadTimeout):
        return Failure.UPSTREAM_DEAD
    return Failure.PROXY_UNREACHABLE

Two things worth calling out, because they're where hand-rolled versions get it wrong:

ProxyError subclasses ConnectionError in requests. Check it first or every proxy failure lands in the wrong bucket.
ReadTimeout is not ConnectTimeout. Connect timeout means you never reached the proxy; read timeout means the tunnel opened and the far end went quiet. Different problems, different fixes.

Wiring it into a loop that respects the policy

import time
from proxy_builder_sdk import AethynClient

client = AethynClient()   # reads AETHYN_USERNAME / AETHYN_PASSWORD

def fetch(url: str, country: str = "us", max_attempts: int = 5):
    stats, run = {}, 0

    for attempt in range(max_attempts):
        # a new session id == a new exit identity; reuse it to keep one
        p = client.session(country=country, session=f"job-{run}", ttl=10)

        try:
            r = requests.get(url, proxies=p.proxies, timeout=(10, 30))
            outcome = classify_response(r)
        except Exception as e:
            outcome = classify_exception(e)

        stats[outcome] = stats.get(outcome, 0) + 1

        if outcome is Failure.OK:
            return r.text, stats

        policy = POLICY[outcome]
        if policy["fatal"]:
            # 407 or a TLS rejection: retrying cannot help. Say so, loudly.
            raise RuntimeError(f"unrecoverable: {outcome.value} via {p}")   # str(p) redacts the password
        if not policy["retry"]:
            break
        if policy["rotate"]:
            run += 1                       # next loop builds a fresh identity
        if policy["backoff"]:
            time.sleep(policy["backoff"] * (attempt + 1))

    raise RuntimeError(f"exhausted after {max_attempts}: {stats}")

str(p) is safe to log — the SDK redacts the password. Please do not be the person who pipes proxy credentials into Sentry.

The classifier — Node

Same taxonomy, undici/got error codes instead of exception classes.

import got, { HTTPError, RequestError } from "got";
import { AethynClient } from "proxy-builder-sdk";

const Failure = {
  PROXY_UNREACHABLE: "proxy_unreachable",
  PROXY_AUTH:        "proxy_auth",
  UPSTREAM_DEAD:     "upstream_dead",
  TLS_REJECTED:      "tls_rejected",
  TARGET_BLOCKED:    "target_blocked",
  RATE_LIMITED:      "rate_limited",
  SOFT_BLOCK:        "soft_block",
  OK:                "ok",
};

const BLOCK_MARKERS = ["just a moment", "verify you are human", "access denied"];
const MIN_BODY_BYTES = 512;

function classify(err, res) {
  if (res) {
    const s = res.statusCode;
    if (s === 407) return Failure.PROXY_AUTH;
    if (s === 429) return Failure.RATE_LIMITED;
    if (s === 401 || s === 403) return Failure.TARGET_BLOCKED;
    if ([502, 503, 504].includes(s)) return Failure.UPSTREAM_DEAD;
    if (s === 200) {
      const body = res.body ?? "";
      const low = body.slice(0, 4000).toLowerCase();
      return body.length < MIN_BODY_BYTES || BLOCK_MARKERS.some((m) => low.includes(m))
        ? Failure.SOFT_BLOCK
        : Failure.OK;
    }
    return Failure.TARGET_BLOCKED;
  }

  if (err instanceof HTTPError) return classify(null, err.response);

  const code = err?.code ?? "";
  const msg  = (err?.message ?? "").toLowerCase();

  if (msg.includes("407") || msg.includes("proxy authentication")) return Failure.PROXY_AUTH;
  if (code === "ECONNRESET" || code.startsWith("ERR_TLS") || code === "EPROTO") return Failure.TLS_REJECTED;
  if (code === "ECONNREFUSED" || code === "ENOTFOUND" || code === "ETIMEDOUT") return Failure.PROXY_UNREACHABLE;
  if (err instanceof RequestError && msg.includes("tunnel")) return Failure.UPSTREAM_DEAD;
  return Failure.PROXY_UNREACHABLE;
}

const client = new AethynClient();

async function fetchOnce(url, run = 0) {
  const p = client.session({ country: "us", session: `job-${run}`, ttl: 10 });
  try {
    const res = await got(url, { proxy: p.url, throwHttpErrors: false, timeout: { request: 30_000 } });
    return { outcome: classify(null, res), res };
  } catch (err) {
    return { outcome: classify(err), res: null };
  }
}

The payoff: one histogram tells you whose fault it is

This is the reason to bother. Count outcomes per run and print the distribution:

proxy_unreachable   2
upstream_dead      41
target_blocked      6
rate_limited        1
soft_block          3
ok                847

Now the diagnosis is mechanical:

Dominant class	Verdict	What to do
`proxy_unreachable`, `upstream_dead`	Provider problem. Your requests are dying before the target sees them.	Check their status. Escalate. Switch if it persists.
`proxy_auth`	Your config. Nothing else.	Check credentials, balance, and that your targeting features match your plan/port.
`tls_rejected`	Your HTTP client.	Change the client, not the proxies. No amount of rotation fixes a fingerprint.
`target_blocked`, `soft_block`	Your identity strategy.	Rotate more, slow down, use a real browser for JS-heavy targets.
`rate_limited`	Your request rate.	Back off. Honour `Retry-After`. Rotating is not a rate limit fix.

Before I had this, "the scrape is failing" was a two-hour investigation. Now it's a glance. Roughly the first three times I ran it in anger, the answer was a category I'd have bet against.

Where a proxy provider actually fits

The classifier is provider-neutral and I'd rather you kept it that way — it's the instrument, and an instrument that can't return a bad verdict about your vendor is worthless. But three properties genuinely reduce the classes it has to catch, and they're what we built Aethyn around:

Identity is a string, not an API call. A new session id is a new exit; reuse it and you keep one for the lifetime window (1–1440 min). Rotation costs you nothing and needs no round trip. client.session({ country: "us", session: "job-7", ttl: 10 }).
Failures are separable by design. Proxy-layer errors and target-layer responses arrive distinctly, which is what makes classes 1–3 distinguishable from 5–7 at all.
Published per-GB pricing. A retry storm has a cost and you should be able to see it. Ours is on the pricing page — no sales call to find out.

The proxy-builder-sdk used above is MIT-licensed and on npm and PyPI. It also speaks other providers' username formats, so you can run this exact code against whatever you're using today and compare histograms. That comparison is a much better provider evaluation than anybody's marketing page, ours included.

If you want to try it against our network, the trial needs no card.

The obligatory, non-optional caveat

This is tooling for collecting public data: respect robots.txt, respect rate limits, read the terms. Class 6 exists to tell you that you're being rude, and the correct response to it is to slow down, not to find a cleverer way to keep the same pace. And there's no CAPTCHA-solving in any of the above — when a real challenge fires, the honest move is to log it and stop.

Classify your failures. Half of them aren't what you think they are.

Scrape Any Site in 20 Lines with Residential Proxies (Node.js & Python)

Aethyn Team — Sun, 19 Jul 2026 17:06:02 +0000

Cross-posted from the Aethyn blog. I work on Aethyn — but the code here is standard-library stuff you can run against any proxy, so take the useful bits regardless.

Most proxy tutorials bury the code under 800 words of theory. Not this one. Terminal open, five minutes, working residential-proxy scraper by the end. We'll use the Aethyn SDK (proxy-builder-sdk), a thin wrapper over the proxy.aethyn.io gateway that assembles the proxy URL — country, city, sticky sessions, protocol — so you don't hand-build username strings.

1. Install

npm install proxy-builder-sdk     # Node.js
pip install proxy-builder-sdk     # Python

2. Authenticate

Use your aethyn-XXXXX username + password, or set AETHYN_USERNAME / AETHYN_PASSWORD and let the client read them.

from proxy_builder_sdk import AethynClient
client = AethynClient(username="aethyn-XXXXX", password="PASSWORD")

import { AethynClient } from "proxy-builder-sdk";
const client = new AethynClient({ username: "aethyn-XXXXX", password: "PASSWORD" });

3. First proxied request

If the returned IP isn't yours, it works.

import requests
p = client.proxy(country="us")
r = requests.get("https://api.ipify.org?format=json", proxies=p.proxies)
print(r.json())

import { ProxyAgent, request } from "undici";   // npm i undici (axios/got also fine)
const p = client.proxy({ country: "us" });
const res = await request("https://api.ipify.org?format=json", {
  dispatcher: new ProxyAgent(p.url),
});
console.log(await res.body.json());

client.proxy() rotates the IP on every request — ideal for stateless, high-volume reads.

4. Sticky sessions (same IP)

For logins, carts, and anything cookie-bound, pin one IP:

p = client.session(country="de", session="cart42", ttl=10)   # same IP, 10 min

const p = client.session({ country: "de", session: "cart42", ttl: 10 });

5. City / ISP targeting (Elite)

p = client.session(country="us", city="chicago", isp="comcast",
                   session="run1", ttl=30, tier="elite")

const p = client.session({ country: "us", city: "chicago", isp: "comcast",
                           session: "run1", ttl: 30, tier: "elite" });

6. JavaScript sites: Playwright

from playwright.sync_api import sync_playwright
p = client.session(country="fr", session="s1", ttl=15, tier="elite")
with sync_playwright() as pw:
    browser = pw.chromium.launch(proxy=p.for_playwright())
    page = browser.new_page(); page.goto("https://example.com")
    print(page.title()); browser.close()

import { chromium } from "playwright";
const p = client.session({ country: "fr", session: "s1", ttl: 15, tier: "elite" });
const browser = await chromium.launch({ proxy: p.forPlaywright() });
const page = await browser.newPage();
await page.goto("https://example.com");
console.log(await page.title());
await browser.close();

7. SOCKS5

p = client.proxy(country="jp", protocol="socks5")

const p = client.proxy({ country: "jp", protocol: "socks5" });

Full mini-example: one page, three countries

import requests
from proxy_builder_sdk import AethynClient

client = AethynClient()
url = "https://api.ipify.org?format=json"   # swap for your target
for cc in ["us", "de", "jp"]:
    p = client.proxy(country=cc)
    print(cc, "->", requests.get(url, proxies=p.proxies, timeout=30).json())

Time-savers

Use proxy() (rotating) for stateless reads; switch to session() (sticky) the moment a cookie or login is involved. On a 403/429, rotate rather than retry-hammer — a rotating proxy does it on the next call, a sticky one when you change the session label. Verify your exit IP before trusting geo-specific data. And scrape public data only, respecting robots.txt, rate limits, and each site's terms — well-behaved scrapers get blocked far less.

New Aethyn accounts get a bandwidth allowance on both pools with no card and no time limit, so you can run every snippet above for free. Docs: aethyn.io/sdk.

Build a country-by-country price index with an AI agent (one MCP server, no scraping stack)

Aethyn Team — Sun, 12 Jul 2026 12:45:27 +0000

Give an AI agent a browser and it can read the web. But it reads the web from exactly one address — whichever machine it happens to be running on.

That's fine until the thing you're asking about is country-specific. Public prices, currency, regional availability, localized search results — all of it comes back through a vantage point the agent never chose and can't move. The agent doesn't error. It just confidently reports the wrong country's answer.

I wanted an agent that could say "check this from Germany, now from the UK, now from the US" — and then prove it actually did. So I built an MCP server for it, and this post is the walkthrough: we'll build a small country-by-country price index, driven entirely by tool calls.

The gap this fills

If you've used Microsoft's Playwright MCP, you know it's a genuinely good tool — it drives a browser properly. But its proxy is a launch flag: you set it when the server starts, and every call in that session goes out the same door. It wasn't built for per-task geography, and that's a reasonable design choice.

Hosted "scraping" MCPs solve geography by routing through their own cloud and handing you JSON. The geography is real, but it's opaque — you don't pick the exit, and you can't verify where a request actually landed.

What I wanted sat in between: a local browser, a residential exit, and the country as an argument on the tool call — plus a way for the agent to check the exit before trusting the page.

Setup (2 minutes)

It's an npm package. Add it to your MCP client config — claude_desktop_config.json for Claude Desktop, ~/.cursor/mcp.json for Cursor:

{
  "mcpServers": {
    "aethyn-browser": {
      "command": "npx",
      "args": ["-y", "aethyn-browser-mcp"],
      "env": {
        "AETHYN_USERNAME": "aethyn-XXXXX",
        "AETHYN_PASSWORD": "your-proxy-password",
        "AETHYN_DEFAULT_TIER": "premium"
      }
    }
  }
}

Chromium auto-installs on first run (~170 MB, once). The browser runs on your machine — nothing is hosted for you, and your credentials never leave your box.

Restart the client and the agent has 10 tools, all prefixed aethyn_:

Tool	What it does
`aethyn_launch_browser`	Open a proxied session in a chosen country
`aethyn_navigate`	Go to a URL
`aethyn_get_content`	Page content as text / html / markdown
`aethyn_snapshot`	Accessibility tree with `[ref=eNN]` handles
`aethyn_click` / `aethyn_type`	Interact by ref or selector
`aethyn_check_exit_ip`	Verify the exit through the session's proxy
`aethyn_new_identity`	Rotate to a fresh exit IP, clear cookies
`aethyn_close`	Close the session
`aethyn_list_countries`	Discover exit options at runtime

The build: a regional price index

The task: read a public product/pricing page from three countries and put the numbers side by side. Every step below is a real tool call the agent makes on its own.

Step 1 — let the agent discover its options

Rather than hard-coding a country list, the agent asks:

{ "tool": "aethyn_list_countries", "args": { "with_cities": false } }

This matters more than it looks: the agent can now plan a loop over countries it knows are actually available, instead of guessing and failing.

Step 2 — one sticky session per country

For each country, launch a session. The session argument is the important one — it's your task id, and reusing it pins the same exit IP for the whole task:

{
  "tool": "aethyn_launch_browser",
  "args": {
    "country": "de",
    "session": "price_idx_de",
    "lifetime_min": 10,
    "tier": "premium"
  }
}

Note the token is lifetime_min (1–1440 minutes, default 10), not "ttl". It's how long that exit stays pinned to your session. Launch returns a session_id, which every other tool takes.

Step 3 — verify before you trust (the step people skip)

This is the one I'd argue is the whole point:

{ "tool": "aethyn_check_exit_ip", "args": { "session_id": "..." } }

{ "ip": "…", "country": "DE", "city": "…", "org": "…", "is_residential": true }

It runs the check through the session's own proxy, so it tells you where the page you're about to load will actually come from. If the agent asked for de and the exit says something else, it can rotate and re-check instead of quietly reporting a German price it never saw.

(Fair warning: is_residential is a best-effort heuristic derived from the ASN/org — treat it as a signal, not a guarantee.)

Step 4 — read the page

{ "tool": "aethyn_navigate", "args": { "session_id": "...", "url": "https://example-store.com/pricing" } }
{ "tool": "aethyn_get_content", "args": { "session_id": "...", "format": "markdown" } }

markdown is usually the right format for feeding an LLM — it's the page, cleaned, without the script/style noise eating your context window.

Step 5 — close, repeat, aggregate

aethyn_close the session, then run the same sequence for uk and us with their own session ids. Because each country gets its own session, the identities never cross-contaminate.

What you actually type

The nice part is you don't orchestrate any of this by hand. You say:

"Using the aethyn-browser tools: list the available countries, then for de, uk, and us, open a sticky session, verify the exit IP is in the right country, load https://example-store.com/pricing, and pull the plan prices. Close each session when you're done. Put the results in a table with the currency, and flag any country where the exit IP didn't match what you asked for."

…and the agent runs the loop, checks its own work, and hands you the table. From there it's one more step to a chart or a nightly pipeline.

Two gotchas worth your time

1. The 407 that looks like a proxy failure but isn't.
AETHYN_DEFAULT_TIER defaults to premium, which uses HTTP port 2099. Elite uses 5499. If your account is Elite and you leave the default alone, the browser launches perfectly fine and then the proxy rejects auth with a 407. It looks like broken credentials; it's a port mismatch. Fix: set AETHYN_DEFAULT_TIER=elite, or pass tier: "elite" on the call.

Also note city and state targeting are Elite-only (port 5499). Country works on both tiers.

2. HTTP, not SOCKS5.
The server is HTTP-only by design — Chromium can't authenticate SOCKS5 proxies. If you've been burned by that before, this is why.

Bring your own proxy

It defaults to Aethyn, but it isn't locked to it. Point it at any HTTP proxy with PROXY_HOST, PROXY_PORT, PROXY_USERNAME, PROXY_PASSWORD, and a PROXY_USERNAME_TEMPLATE describing how your provider encodes targeting in the username:

PROXY_USERNAME_TEMPLATE="{username}-country-{country}[-city-{city}]-session-{session}-lifetime-{lifetime}"

[optional] segments drop out when empty. For a fixed proxy with no geo targeting at all, the template is just {username}.

Where this stays honest

Worth being direct about the boundaries, because the "agents + browsers" space is full of overclaiming:

Public data only. Respect robots.txt, rate limits, and each site's terms. "The agent can reach it" isn't "the agent should."
There is no CAPTCHA-solving tool, and none is planned. If a challenge fires, that's a signal to back off — not to fight it.
No login/credential-wall automation.
Nothing here auto-detects a block. If a page comes back looking wrong, the agent reads the content, judges it, and calls aethyn_new_identity. That's a deliberate choice — I'd rather the model make that call visibly than have the tool silently paper over it.
A residential exit fixes your network vantage point. It doesn't fix reckless pacing or a client that looks robotic in every other way.

Try it

npx -y aethyn-browser-mcp

Drop the config block in your MCP client, and your agent can pick a country on its very next tool call.

Full walkthrough + runnable examples: docs
The deeper writeup: Residential proxy browser MCP for AI agents
Source (MIT): github.com/aethynio/aethyn-browser-mcp · npm · Glama
Free account, no card: aethyn.io/signup

If you build something with it — or it breaks in an interesting way — I'd like to hear about it.

The Residential Proxy "Pool Size" Myth — Why 100M+ IPs Tells You Almost Nothing

Aethyn Team — Wed, 08 Jul 2026 16:03:38 +0000

Open any residential proxy provider's homepage and the first thing you see is a number. 50 million IPs. 100 million. "The largest residential network in the world." It's the industry's favourite headline — and one of the weakest predictors of whether your scraper actually works.

What the "pool size" number actually measures

That headline figure is almost always a cumulative count of unique IPs seen over a rolling 30-day window — not a live inventory. Three things follow, none of which make the marketing:

It's self-reported and unaudited. No standard for what counts as an IP, over what window. Two networks drawing on the same underlying supply can honestly print wildly different numbers.
The incentive only runs one way. Nobody ever chose a provider for a smaller pool. Every incentive pushes the number up — the exact condition under which a metric stops meaning anything.
Nobody publishes the number that matters: concurrent addresses available in your country, right now. That's a fraction of the headline and you'll never see it on a pricing page.

The headline IP count is a marketing decision, not a measurement.

Why the number can't predict success

The target never sees your pool. It sees one IP — the one you're using right now — and it scores that single address. Modern anti-bot systems judge a request on:

Is this a residential IP at all? The ASN instantly separates a real consumer ISP from a datacenter range. Datacenter IPs are blocked on sight.
What's this IP's reputation? Fraud-score databases flag addresses recently tied to abuse or automation.
Has this IP already misbehaved here? A recycled exit that hammered the target yesterday is treated very differently from a quiet one.
Is the story consistent? An IP geolocating to one country while headers/timezone claim another is a tell.

None of those is pool size. A huge pool full of recycled or flagged addresses performs worse than a smaller, clean one.

The mistake that ruins your data: trusting the status code

Internalise this before you benchmark anything: a 200 is not success. Modern defenses "soft-block" — a 200 with a CAPTCHA interstitial, a stripped decoy, or an empty shell. Measure success on content, not status:

import requests

PROXY = "http://USER:PASS@proxy.example.com:PORT"  # your provider's endpoint
proxies = {"http": PROXY, "https": PROXY}

def is_real_success(resp, positive_token):
    if resp.status_code != 200:
        return False
    body = resp.text.lower()
    block_signals = ["captcha", "unusual traffic", "access denied", "are you a robot"]
    if any(s in body for s in block_signals):
        return False
    return positive_token.lower() in body   # something ONLY a real page has

targets = [...]          # YOUR real URLs — not httpbin, not the vendor demo
TOKEN   = "add to cart"  # a string unique to a genuine page on YOUR target

ok = sum(is_real_success(requests.get(u, proxies=proxies, timeout=30), TOKEN)
         for u in targets)
print(f"content-verified success rate: {ok / len(targets):.1%}")

That percentage — success on your sites, verified on content — is the number the entire industry's marketing exists to distract you from.

Check what you're actually handed

Pull the exit IP and inspect it. This is where "big pool, dirty IPs" gets exposed in thirty seconds:

ip = requests.get("https://api.ipify.org", proxies=proxies, timeout=30).text
print("exit IP:", ip)

# Look that IP up in any public IP-reputation / fraud-score tool and confirm:
#   1. ASN is a consumer ISP (not AWS/GCP/a hosting provider)
#   2. it geolocates where you asked
#   3. it isn't already flagged for abuse

Spot-check the exits you're served — reputation beats raw count every time.

The one question a provider can't dodge

Ask where the IPs come from — and don't accept a non-answer. IPs from users who gave informed, opt-in consent behave like the real people they belong to and carry the reputation to match. IPs harvested from compromised devices are a botnet: already flagged, ethically indefensible, and a legal liability you inherit. A clean sourcing story is either documented or it isn't. A network that leads with a pool-size number and goes quiet on sourcing has told you which one it optimised for.

TL;DR — what to judge a network on

Success rate on your targets, verified on content (not status).
IP reputation — spot-check exits against a fraud-score tool.
Sourcing — consented/opt-in, not mystery pools.
Concurrency where you need it, not a global cumulative total.
Per-GB pricing you can forecast — if you need a sales call to see a price, you can't model cost.

Whatever wins that test is your provider. Pool size never enters the calculation — because it was never in the target's calculation either.

About Aethyn

I help build Aethyn — residential proxies built for engineers, not a marketing slide. One unified endpoint, country/city/session targeting encoded in the username, transparent per-GB pricing (€2.00/GB Premium, €4.50/GB Elite), and a free trial, no card. We deliberately don't lead with an IP-count — we'd rather you run the benchmark above on your own targets.

👉 Try it on your own targets: aethyn.io — free, no card

Originally published on aethyn.io.

Images via Unsplash (free license).