DEV Community

Greta
Greta

Posted on

Static Residential Is a Promise, Not a Guarantee: Detecting When a Fixed Exit Silently Drifts

You pay a premium for a static residential IP for exactly one reason: the metadata is supposed to be frozen. Not just the address string, but the country, the metro, the autonomous system, and the "this is a real household broadband line, not a rack in Virginia" classification. That bundle is the product. A rotating pool is cheaper precisely because it makes no promises about any of it.

Here is the part nobody puts in the order form: nothing in the deal pins the metadata. Your provider pins the IP string. Everything else is an opinion held by third-party databases that get rebuilt on someone else's release cadence, fed by registries that change when an ISP resells a block or a RIR allocation gets re-delegated. Providers re-provision, rebuy address blocks, and re-map gateway ports after incidents. The result is silent geo drift: same exit, same credentials, same dashboard label, different internet position.

So treat every assumption you bought the exit for as a monitored invariant, and fail closed when it changes. Not "verify once at onboarding" — verify on a schedule, for as long as you own the thing.

What "static" actually guarantees

The provisioning contract is narrow: this IPv4 address is assigned to you, it will not be handed to another tenant, and it will not move between sessions. That is a real and useful guarantee. It says nothing about how MaxMind, ipinfo, or any other geo vendor classifies the prefix that address sits in, and it says even less about how a target's risk engine interprets that classification next Tuesday.

Geo placement is derived from the prefix, not from the wire. Your /24 can be yours while the AS reporting on it changes underneath, and the city label changes with it.

Failure mode 1: ASN reassignment

An ISP sells or consolidates a block. The IRR/RPKI objects get updated, the prefix moves from a consumer brand's AS to a smaller regional's AS, or the RIR record is re-delegated. No packet changes course. But geo feeds key city, region, and connection type off the registration, so your Dallas Comcast exit becomes "downtown Dallas, corporate, different AS" overnight. If your dataset's validity depends on "residential broadband in Dallas," the label on your dashboard is now a lie with good posture.

Failure mode 2: re-provisioning after an incident

Hardware failure, a PoP deprecation, an upstream dispute — a provider re-maps your logical exit ID onto physically different hardware, sometimes a different aggregation site in the same metro. Same city per the database, different RTT, different CDN egress, different cached regional variant from the origin. This one is sneaky because every field you check still passes, and only your conversion rates know something moved.

Failure mode 3: two databases disagree

Vendor geo databases disagree routinely at the edges: carrier-NAT adjacencies, university blocks, military ranges, newly acquired fiber subs. That disagreement is not academic. The bot-detection vendor your target uses buys one of those feeds, and when it flips, the target's classification flips with it. Cross-check at least two independent observers every cycle and persist both answers, even when they agree — you will need the history later to date the drift. One practical wrinkle: ip-api's free tier is HTTP-only, so it can serve you a lie from a hostile network. Use it as a second opinion alongside an HTTPS observer, never as the sole authority.

Encode the contract you bought

If the assumption is not written down, it is not monitored. Put every promised attribute in a typed object with a version number, so a deliberate re-baseline is a code change with a diff, not a quiet edit to config.

from __future__ import annotations

import re
from dataclasses import dataclass
from typing import Any

import requests

CRITICAL, WARNING = "critical", "warning"


@dataclass
class ExitContract:
    """Everything you paid the premium to have pinned."""
    exit_id: str
    proxy_url: str              # http://user:pass@gw.entry:8443
    country: str                # ISO-3166 alpha-2, uppercase
    asn: int | None = None      # bare number, no "AS" prefix
    city: str | None = None
    connection: str = "isp"     # "isp" | "residential"
    version: int = 1            # bump when you knowingly re-baseline


def observe(proxy_url: str, timeout: float = 20.0) -> dict[str, Any]:
    """Two independent opinions, collected through the exit itself."""
    out: dict[str, Any] = {"errors": []}
    try:
        a = requests.get("https://ipinfo.io/json", proxies={"https": proxy_url},
                         timeout=timeout).json()
        m = re.search(r"AS(\d+)", a.get("org", ""))
        out.update(country=(a.get("country") or "").upper(), city=a.get("city"),
                   org=a.get("org", ""), asn=int(m.group(1)) if m else None)
    except (requests.RequestException, ValueError) as exc:
        out["errors"].append(f"ipinfo: {exc}")
    try:
        b = requests.get("http://ip-api.com/json/?fields=status,countryCode,"
                         "city,as,connection", proxies={"http": proxy_url},
                         timeout=timeout).json()
        if b.get("status") == "success":
            m = re.search(r"AS(\d+)", b.get("as", ""))
            out.update(country_b=(b.get("countryCode") or "").upper(), city_b=b.get("city"),
                       asn_b=int(m.group(1)) if m else None,
                       connection=(b.get("connection") or "").upper())
    except (requests.RequestException, ValueError) as exc:
        out["errors"].append(f"ip-api: {exc}")
    return out


def _classify(connection: str, org: str) -> str:
    text = f"{connection} {org}".upper()
    if "CELLULAR" in text or "MOBILE" in text:
        return "cellular"
    if "HOSTING" in text or "CLOUD" in text or connection == "BUSINESS":
        return "non_residential"
    return "isp" if connection == "ISP" else "unknown"


def check_drift(c: ExitContract, obs: dict[str, Any]) -> list[dict[str, Any]]:
    """Structured diff: what we bought vs. what the internet sees now."""
    if not (obs.get("country") or obs.get("country_b")):
        return [{"field": "reachability", "severity": CRITICAL, "observed": None,
                 "note": "; ".join(obs.get("errors", ["silent"]))}]

    asserted = [
        ("country", c.country, obs.get("country") or obs.get("country_b"), CRITICAL),
        ("asn", c.asn, obs.get("asn") or obs.get("asn_b"), CRITICAL),
        ("connection", c.connection,
         _classify(obs.get("connection", ""), obs.get("org", "")), CRITICAL),
        ("city", c.city, obs.get("city") or obs.get("city_b"), WARNING),
    ]
    diffs = []
    for name, expected, actual, severity in asserted:
        if not expected:
            continue      # not asserted => not promised => not monitored
        if name == "connection":
            ok = actual == expected or (expected == "residential" and actual == "isp")
        else:
            ok = str(expected).upper() == str(actual or "").upper()
        if not ok:
            diffs.append({"field": name, "severity": severity,
                          "expected": expected, "observed": actual})
    return diffs
Enter fullscreen mode Exit fullscreen mode

Fail closed, with a grace streak

A scheduler runs observe + check_drift per exit daily; it costs a couple of kilobytes. The semantics that matter:

  • Critical fields: country, ASN, connection class. Any change is drift.
  • Warning fields: city labels, which churn legitimately across feed versions; they alert but do not stop work.
  • Unreachable is not clean. If no observer answers, that is inconclusive; check_drift emits it as a critical, so a silent observer costs you a clean slot rather than buying you a pass. Fail-open is how you end up with six weeks of data collected from the wrong continent.
  • Grace streak of 2-3 consecutive failures before quarantine, so one flaky observer does not page you, but a real rehome still lands inside a day.
import json
import os
from datetime import datetime, timezone

STATE_FILE, QUARANTINE_FILE = "exit_state.json", "quarantined_exits.json"
JOURNAL = "geo_provenance.jsonl"
MAX_FAIL_STREAK = 2


def now() -> str:
    return datetime.now(timezone.utc).isoformat()


def load(path: str, default):
    return json.load(open(path, encoding="utf-8")) if os.path.exists(path) else default


def append_journal(row: dict) -> None:
    with open(JOURNAL, "a", encoding="utf-8") as fh:
        fh.write(json.dumps(row) + "\n")


def stamp(record: dict, c: ExitContract, obs: dict, s: dict) -> dict:
    """Provenance is stamped at collection time, not reconstructed later."""
    record["_geo"] = {"exit_id": c.exit_id, "contract_version": c.version,
                      "observed_country": obs.get("country") or obs.get("country_b"),
                      "observed_asn": obs.get("asn") or obs.get("asn_b"),
                      "observed_city": obs.get("city"), "last_clean_check": s.get("last_ok"),
                      "trust": "suspect" if s.get("drift") else "clean", "stamped_at": now()}
    return record


def monitor(contracts: list[ExitContract], tasks_by_exit: dict) -> dict:
    state, quarantined = load(STATE_FILE, {}), load(QUARANTINE_FILE, {})
    for c in contracts:
        s = state.setdefault(c.exit_id, {"fail_streak": 0, "last_ok": None, "drift": []})
        obs = observe(c.proxy_url)
        diffs = check_drift(c, obs)
        hard = [d for d in diffs if d["severity"] == CRITICAL]
        soft = [d for d in diffs if d["severity"] == WARNING]

        s["fail_streak"] = s["fail_streak"] + 1 if hard else 0
        s["drift"] = diffs
        if not diffs:
            s["last_ok"] = now()
        append_journal({"ts": now(), "exit_id": c.exit_id, "version": c.version,
                        "observed": obs, "critical": hard, "warning": soft})

        if s["fail_streak"] >= MAX_FAIL_STREAK and c.exit_id not in quarantined:
            quarantined[c.exit_id] = {"since": now(), "reason": hard,
                                      "drift_window_start": s["last_ok"],
                                      "affected_tasks": sorted(tasks_by_exit.get(c.exit_id, []))}
            print("QUARANTINE", c.exit_id, quarantined[c.exit_id]["affected_tasks"])
        elif soft:
            print("WARN", c.exit_id, [d["field"] for d in soft])

    json.dump(state, open(STATE_FILE, "w", encoding="utf-8"), indent=2)
    json.dump(quarantined, open(QUARANTINE_FILE, "w", encoding="utf-8"), indent=2)
    return quarantined


if __name__ == "__main__":
    contract = ExitContract(exit_id="tex-01", proxy_url="http://u:p@gw.entry:8443",
                            country="US", asn=7922, city="Dallas", connection="isp")
    monitor([contract], {"tex-01": ["rank-2026-09-21-a", "pricing-2026-09-21-b"]})
Enter fullscreen mode Exit fullscreen mode

Quarantine means the scheduler stops assigning new work to that exit. It does not mean the exit is deleted — you usually want it alive to prove the drift is still there when the provider claims it cannot reproduce it.

What to do with data collected under a broken assumption

This is the step everyone skips. A drifted exit does not just poison the future; it retroactively poisons the past, because you no longer know which geo the past was collected from.

Bisect on the journal. Every record carries exit_id, the observed attributes at fetch time, and the timestamp of the last clean check. drift_window_start is your last known-good moment, so records inside [drift_window_start, now] are suspect and records before it are defensible — subject to the same field being critical. A city-only wobble is a footnote; a country or ASN wobble on a rank-tracking dataset is a re-run.

Then decide with a cheap experiment, not a meeting: re-fetch a sample of the suspect records through a known-good exit and diff. Rank positions, localized prices, and ad creatives are all stable enough that a real geo change shows up immediately. If the sample agrees, keep the window with a flag. If it diverges, drop the window — for a trend series, deleting a week is much cheaper than carrying a permanently wrong slope.

The asymmetry to internalize: geo-dependent datasets fail plausibly. Wrong-city rank tracking returns confident, well-formed, wrong numbers. Localized pricing returns a price that exists, just not in your market. Ad verification confirms delivery to an audience you did not pay to reach. Nothing downstream will throw an exception, which is precisely why the invariant check has to be its own job.

When static residential is still the right call

When geo and identity must be stable for hours — logged-in flows, long sessions, per-account isolation — nothing else in the catalogue does the job, and a rotating pool is actively harmful. The point is not that static exits are unreliable; it is that their reliability is a claim you have to measure, on your cadence, against your contract, rather than a property you get to assume after purchase.

The checklist

  1. One ExitContract per exit, in version control, with a version field.
  2. Assert only what you actually depend on; unasserted fields are unchecked promises.
  3. Two observers, both persisted, daily, through the exit itself.
  4. Unreachable counts as failure. Never as pass.
  5. Grace streak of 2-3, then quarantine new assignments; keep the exit alive for evidence.
  6. Provenance stamp at write time, so a drift event can bisect history.
  7. Re-baseline with a code change and a note, or the drift becomes invisible policy.

Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele

Top comments (0)