You Can't De-bias a Corpus After the Fact: Fix the Geography of Your Proxy Pool First

Most teams discover their corpus is geographically skewed and reach for the wrong lever. They add a "diversity" filter to the preprocessing step, or they up-weight under-represented regions during training, or they simply collect more data and hope the long tail fills in. None of those touch the actual cause. The skew was baked in the moment you decided which exit IPs you would crawl through. If 80% of your requests left from Northern Virginia and Frankfurt, roughly 80% of your corpus encodes how the web looks from those two places, and no amount of clever weighting manufactures the four billion people who see something else.

Here is the thesis, stated plainly: you cannot de-bias a corpus after collection, because after collection the missing perspectives do not exist as rows. You can only bias less at collection time, and the single highest-leverage knob for geographic bias is the geography of your proxy pool. Fix the exit distribution and the corpus distribution follows. Ignore it and you are training a model on a map drawn from three cities and calling it the world.

Why the exit sets the floor, not the ceiling

Every page you fetch is a function of where you fetched it from. The same URL served to a residential exit in Lagos, a datacenter exit in Ashburn, and a mobile exit in São Paulo can return different languages, different currency and unit defaults, different localized product availability, different "no results in your area" states, different regulatory banners, and — most importantly for anyone tracking rankings or prices — genuinely different content. Search results are the canonical example, but news front pages, marketplace inventories, and app-store listings all vary by inferred location.

That means the marginal content your model never sees is not random noise; it is systematically the content that only surfaces from under-covered exits. A datacenter-heavy pool under the same handful of US and EU ASNs has almost no chance of surfacing the Southeast Asian, South Asian, Latin American, and African variants of any given page, because those variants are not what those exits are handed. You do not have "some bias." You have a coverage hole shaped exactly like your exit map.

The uncomfortable corollary: adding volume from the same exits does not help. A million more pages from Northern Virginia is a million more confirmations of the worldview you already over-collected. To buy down geographic bias you have to buy — or route for — geographic coverage, which is an exit-pool property, not a corpus-cleaning property.

Treat the proxy pool as a sampling frame

Statisticians have a name for what you are doing when you collect training data: you are drawing a sample from a population, and the set of units that could have been drawn is the sampling frame. If the frame excludes a stratum, your sample cannot represent it, and every downstream estimate is biased by the amount you under-covered that stratum. Your exit pool is the sampling frame, and its geographic composition determines which strata were even eligible.

So the design task is: choose an exit distribution whose coverage matches the population you claim to represent. If your model serves a global audience, the frame must carry real exits in the regions where that audience lives. If your model is intentionally US-and-EU, say so in the model card and stop pretending the numbers generalize. Both are defensible. A silently-skewed frame is not.

The practical move is to define a target geographic quota — a vector of desired exit-share weights per region or country — and then provision your pool to hit it. Not "get proxies from many places," but "30% North America, 25% Europe, 20% South & East Asia, 10% Latin America, 8% Africa, 7% Middle East," whatever your target population actually is. Then measure what you collected and reconcile.

Stratified routing with a quota controller

The routing layer should decide the exit's geography first, pick a live exit from a per-region pool, and only then make the request. That is a two-level choice: region from the target distribution (adjusted for how far the region currently sits below quota), then a specific exit from the region's healthy set.

Below is a runnable controller. It tracks, per region, how many rows you have collected versus how many the target distribution implies, and biases each routing decision toward the regions you are still under-filling. This is deliberately a plain-stdlib loop you can drop into an existing crawler skeleton.

import random
from dataclasses import dataclass, field


@dataclass
class Region:
    code: str
    target_share: float                 # desired fraction of the corpus
    pool: list[str] = field(default_factory=list)   # live exit endpoints
    collected: int = 0                  # rows fetched through this region


class GeoStratifiedRouter:
    """Pick a region weighted by how far it is below its target quota,
    then a live exit from that region's pool."""

    def __init__(self, regions: list[Region], seed: int | None = None):
        total = sum(r.target_share for r in regions)
        if abs(total - 1.0) > 1e-6:
            raise ValueError(f"target_share must sum to 1.0, got {total}")
        self.regions = [r for r in regions if r.pool]  # skip dead regions loudly
        if len(self.regions) != len(regions):
            dead = {r.code for r in regions if not r.pool}
            print(f"WARN: regions with no live exits excluded from frame: {dead}")
        self._rng = random.Random(seed)

    def _deficit_weight(self, r: Region, total_collected: int) -> float:
        if total_collected == 0:
            return r.target_share
        actual = r.collected / total_collected
        # weight rises with under-coverage; never fully zeroes an over-covered region
        deficit_ratio = (r.target_share - actual) / r.target_share
        base = max(deficit_ratio, -0.5)
        return max(0.01, (0.5 + base) * r.target_share)

    def choose(self) -> tuple[str, str]:
        total = sum(r.collected for r in self.regions)
        weights = [self._deficit_weight(r, total) for r in self.regions]
        region = self._rng.choices(self.regions, weights=weights, k=1)[0]
        exit_ip = self._rng.choice(region.pool)
        return region.code, exit_ip

    def record(self, region_code: str) -> None:
        for r in self.regions:
            if r.code == region_code:
                r.collected += 1
                return

Wiring the router to a real client is a few lines. The key point is that proxy= is chosen by the router, not by a flat global rotation, so the geography of your traffic is under your control rather than an accident of pool ordering.

import asyncio
import aiohttp


async def fetch(session, url, exit_ip):
    try:
        async with session.get(url, proxy=exit_ip, timeout=aiohttp.ClientTimeout(15)) as resp:
            body = await resp.text()
            if resp.status == 200 and len(body) > 2048:
                return True, len(body)
            return False, 0   # soft-block / tiny body: do NOT count toward quota
    except Exception:
        return False, 0


async def crawl(urls, router):
    async with aiohttp.ClientSession() as session:
        for url in urls:
            for _attempt in range(3):
                region, exit_ip = router.choose()
                ok, _ = await fetch(session, url, exit_ip)
                if ok:
                    router.record(region)
                    break

Note the discipline: only a clean, real-content fetch increments the region counter. A 200-that-lies from a blocked exit must not count, or your collected distribution will look balanced while your actual successful corpus is whatever your least-blocked region happened to fill.

Measure the frame you actually drew from

Provisioning a stratified pool is half the job; the other half is proving your successful rows land near target. The failure mode is silent: a region in your config, but so heavily blocked that almost nothing comes back through it, quietly collapses your effective frame back onto the easy regions. Track realized share per region and a scalar divergence between target and realized, so you get paged when coverage drifts.

import math


def realized_shares(regions):
    total = sum(r.collected for r in regions) or 1
    return {r.code: r.collected / total for r in regions}


def tv_distance(target: dict, realized: dict) -> float:
    """Total variation distance in [0,1]; 0 = frame matches target exactly."""
    codes = set(target) | set(realized)
    return 0.5 * sum(abs(target.get(c, 0.0) - realized.get(c, 0.0)) for c in codes)


def report(regions, warn_at=0.10):
    shares = realized_shares(regions)
    tv = tv_distance({r.code: r.target_share for r in regions}, shares)
    line = ", ".join(f"{c}={shares.get(c,0):.1%}/{target:.0%}"
                     for c, target in ((r.code, r.target_share) for r in regions))
    print(f"TV divergence {tv:.3f}  ({'OK' if tv < warn_at else 'SKEWED'})  {line}")
    return tv

A rising TV distance is a diagnostic with a specific remedy, and it is not "collect more." If Africa drifted from its 8% target to 2%, the fix is in the pool: more live African exits, a better ASN mix, or lower per-exit concurrency so exits stop getting soft-blocked. The number tells you which region's frame is leaking; the frame is what you repair.

The cost argument sneaks in sideways

Geo-stratifying feels like a purity virtue until you price it. Residential exits in under-represented regions are usually pricier and slower per GB than a datacenter block next to your object store, so the naive read is "coverage costs money, bias is free." But bias is never free — you just pay it later and less transparently. A geographically skewed corpus degrades exactly on the users you under-covered, so the model's worst cohorts correlate with your frame's thinnest strata, and you find out through retention, through eval failures you cannot reproduce, through "why does it hallucinate for Indonesian addresses." Re-collecting to fix that after the fact, once product and eval expectations exist, dwarfs the marginal GB premium you saved by routing everything through Ashburn. The cheap exit geography is a loan; the corpus tax comes due with interest.

The honest version

Skew is not a cleaning problem. It is a frame problem, and the frame is your proxy pool's map. Define the target distribution your model claims to represent. Provision and route for it region-by-region, choosing exit geography first and a live exit second. Count only clean fetches toward the quota, measure realized-versus-target every run, and treat drift in that divergence as a pool incident, not a data-science incident. Do this and the geographic honesty of your model stops being a hope and starts being a configuration you can read off a dashboard. Skip it and no downstream trick recovers the perspectives your exits never reached — because you never fetched them, and you cannot weight into existence a page you never pulled.

Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele