Health-Check Your Proxy Pool From the Exit Side, Not the Gateway

Every team that runs a proxy pool builds a health check eventually, and the first version is almost always the same: ping the proxy gateway, or send a test request to https://httpbin.org/ip through it, and record whether it worked. Green checkmark, pool is healthy.

Then production breaks anyway. Requests time out on the target site, CAPTCHAs spike, but the dashboard says the pool is fine. The reason: a gateway-side health check measures your connection to the proxy provider. It does not measure the exit's ability to reach your actual targets. Those are two different networks, and only one of them is where your failures live.

This post is about building the second kind of check: synthetic canaries that run through each exit against endpoints that resemble real workload conditions, producing a per-exit health table you can route on.

What gateway checks actually verify

When you send a test request to a well-known neutral endpoint through your provider's gateway, you verify:

• Your credentials are valid.
• The gateway host is up.
• The gateway picked some exit that can reach a highly-available, CDN-fronted, bot-friendly site.

What you don't verify:

• Whether the exit's IP is currently rate-limited or soft-blocked by your target.
• Whether the exit's carrier path to your target's region is degraded.
• Whether the exit presents a coherent TLS/HTTP profile to a fingerprinting defense.
• Whether that exit is even in the geography you asked for.

A rotating pool makes this worse: the exit your health check used is likely not the exit your next real request will get. Your green checkmark and your production request share almost nothing.

The design: per-exit synthetic canaries

The fix has three parts:

1. A canary target set that behaves like your real targets — ideally your actual target's own low-stakes pages (homepage, robots.txt), fetched politely and rarely.
2. Exit-pinned sessions so a health verdict attaches to a specific exit IP, not to "the pool."
3. A rolling health table — per exit, exponentially-weighted success rate and latency — that your router consumes before assigning work.

With a provider like Thordata, exit pinning is done via the session ID in the gateway credentials: the same session ID holds the same exit for the session window, and the response tells you which IP you got. That's enough to key the health table.

import requests, time, math
from collections import defaultdict

GATEWAY = "http://user:pass-TD.{sid}-30m.gate.thordata.com:7000"
CANARY_URLS = [
    "https://target.example/robots.txt",   # cheap, static, low-stakes
    "https://target.example/",             # real page, real defense stack
]

class ExitHealth:
    """EWMA success rate and latency, keyed by exit IP."""
    def __init__(self, alpha=0.2):
        self.alpha = alpha
        self.success = {}   # ip -> ewma in [0,1]
        self.latency = {}   # ip -> ewma seconds
        self.last_seen = {}

    def update(self, ip, ok, latency):
        a = self.alpha
        self.success[ip] = a * (1.0 if ok else 0.0) + (1 - a) * self.success.get(ip, 0.8)
        self.latency[ip] = a * latency + (1 - a) * self.latency.get(ip, 2.0)
        self.last_seen[ip] = time.time()

    def score(self, ip):
        # score = success rate, penalized for stale entries
        age = time.time() - self.last_seen.get(ip, 0)
        staleness = max(0.0, 1 - age / 1800)   # decay over 30 min
        return self.success.get(ip, 0.5) * staleness

    def healthy(self, ip, floor=0.6):
        return self.score(ip) >= floor

health = ExitHealth()

def canary(session_id):
    proxy = GATEWAY.format(sid=session_id)
    s = requests.Session()
    s.proxies = {"http": proxy, "https": proxy}
    s.headers.update({"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"})
    ip = None
    for url in CANARY_URLS:
        start = time.monotonic()
        try:
            r = s.get(url, timeout=12)
            ok = r.status_code == 200
            if ip is None:
                # trust the first body that looks like it can tell us the exit
                ip = r.json().get("origin", session_id) if "json" in \
                     r.headers.get("content-type", "") else session_id
        except requests.RequestException:
            ok = False
        latency = time.monotonic() - start
        health.update(ip or session_id, ok, latency)
    return ip

A note on the exit identity: some gateways expose the exit IP via a debug endpoint (/ip-style), others let you read it from response headers. If neither is available, key the table by session_id — it's a stable proxy for "this exit for the next 30 minutes," which is exactly the lifetime your router cares about.

The part that makes it useful: routing on health

A health table nobody consumes is decoration. The consumer is your work assignment loop: before dispatching a job, pick the best-scoring exit; when an exit's score dips below the floor, quarantine it for a cool-down instead of hammering it back to life.

import heapq

QUARANTINE = {}  # ip -> unblock timestamp

def acquire_exit(candidates, n=1):
    """Pick the n highest-scoring non-quarantined exits."""
    now = time.time()
    scored = [
        (-health.score(ip), ip)
        for ip in candidates
        if QUARANTINE.get(ip, 0) <= now and health.last_seen.get(ip, 0) > 0
    ]
    heapq.heapify(scored)
    return [heapq.heappop(scored)[1] for _ in range(min(n, len(scored)))]

def report_failure(ip):
    """Called by workers on a real (non-canary) failure."""
    health.update(ip, False, health.latency.get(ip, 3.0))
    if health.score(ip) < 0.4:
        # quarantine 10 minutes; many per-IP blocks lift in minutes, not hours
        QUARANTINE[ip] = time.time() + 600

This closes the loop that gateway checks can never close: production failures now update the health table, and the health table now steers production traffic. The canary runs provide the baseline; real traffic provides the corrections.

Making the canaries honest

Several practical rules keep the health signal meaningful:

• Canary against your target, not just neutral endpoints. A neutral endpoint can only detect provider-side path failure. It cannot detect that your target started CAPTCHA-ing one of your exits. If you can't afford canary traffic against the target itself, at least include one page behind the same defense stack.
• Keep canary volume negligible. One canary pass per exit per 10–15 minutes is plenty — the EWMA smooths over sparse samples by design. The canary must never become your dominant traffic source.
• Score latency, but weight success. A slow-but-successful exit is workable; a fast-but-blocked one is worthless. The score() function above deliberately ignores latency for routing eligibility and uses it only for tie-breaking.
• Beware correlated failures. If every exit fails simultaneously, the problem is upstream of the pool — your credentials, the gateway, or the target's global posture. Track a pool-level success aggregate too; when it collapses while per-exit scores collapse together, page a human instead of quarantining the entire pool.
• Persist the table. Process restarts shouldn't erase ten minutes of learned health. The dict-of-floats structure serializes to JSON or SQLite trivially.

What this buys you in practice

Before per-exit health routing, a typical pattern in a 50-exit pool: a handful of exits silently degraded (blocked yesterday, still in rotation) absorbing 10–20% of dispatches and failing most of them. Mean retry rate across the pool climbs; everyone blames "the proxy quality."

After: those exits get quarantined within one or two failures, dispatch lands on exits with recent proof of life, and the retry budget goes to genuine target-side volatility instead of known-bad exits. In my pipelines this consistently removed a double-digit percentage of wasted requests — the same win people chase by switching vendors, achieved by routing instead.

The proxy pool is a distributed system, and like any distributed system it needs health checks that observe the actual failure domain. Your failure domain is the exit-to-target path. Check that.

Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele