DEV Community

Greta
Greta

Posted on

Round-Robin Is a Correlation Bug: Weight-Aware Account-to-Exit Assignment for Multi-Account Fleets

Round-Robin Is a Correlation Bug: Weight-Aware Account-to-Exit Assignment for Multi-Account Fleets

When you have 40 accounts and a pool of 200 sticky residential exits, the natural instinct is to hand out exits round-robin: account 1 gets exit 1, account 2 gets exit 2, wrap around, done. It looks fair. It is quietly wrong in two directions at once, and both directions show up in your data as the same symptom — accounts getting challenged together, in patterns, at rates that feel random but are not.

The core point of this article: round-robin over a pool of non-identical exits is not load balancing, it is a correlation generator. You need a weighted allocator that treats every exit as having a decaying reputation score, and hands each account the least-loaded, freshest exit that still fits the account's needs.

Why round-robin fails

Round-robin carries one hidden assumption: that every exit is interchangeable and that "even distribution" is the same as "safe distribution." Both are false for a real residential pool.

First, exits are not interchangeable. A residential exit is a real ISP connection with an observable history. Some have been used recently and heavily, some have been idle, some sit behind an ASN that a given origin is currently scoring aggressively. If you walk the pool in index order, you will hand a freshly-burned exit to account 7 today and to account 27 tomorrow, because they happen to be 20 apart. Round-robin creates phase relationships between accounts and exits. Anti-bot systems that correlate "these two accounts always share infrastructure characteristics" do not need your secret — they read it off your allocator's periodicity.

Second, even distribution is not the goal. The goal is that no account repeatedly touches a low-reputation exit and that no exit repeatedly touches accounts whose behavior the origin can link. A round-robin pass through a pool where 15% of exits are degraded means some accounts inherit those degraded exits on a strict schedule. Those accounts fail together, every cycle, like clockwork. Clockwork is a fingerprint.

Here is the naive version, and how fast it produces the pattern:

class NaiveRoundRobin:
    def __init__(self, exits):
        self.exits = list(exits)
        self.i = 0

    def assign(self, account):
        e = self.exits[self.i % len(self.exits)]
        self.i += 1
        return e

# 40 accounts, 200 exits -> after 5 logins you have already
# given every account an exit, and on login #201 the exact same
# account/exit pairing repeats. The origin only needs to watch
# two cycles to see your assignment function.
Enter fullscreen mode Exit fullscreen mode

The right mental model: exits as weighted, decaying resources

Replace the index counter with a priority queue keyed on a score. Each exit has a score that combines how recently it was used, how heavily, and a reputation prior. Lower score means "more attractive right now." An account is assigned the top of the queue, then its score is bumped and it sinks.

import heapq
from time import monotonic


class Exit:
    __slots__ = ("id", "asn", "last_used", "use_count", "prior")

    def __init__(self, eid, asn, prior=0.0):
        self.id = eid
        self.asn = asn
        self.last_used = 0.0
        self.use_count = 0
        self.prior = prior   # provider/observed reputation; higher = worse


class WeightedAllocator:
    """Least-loaded, freshest-exit-first assignment with decay.

    score = recency + volume + asn_spread + prior
    recency  : a freshly-reused exit is risky -> penalize recent use
    volume   : total touches on this exit      -> spread the burn
    asn_spread: don't give two live accounts the same ASN at once
    """

    RECENCY_TAU = 90.0   # seconds; recency penalty half-life

    def __init__(self, exits):
        self.exits = {e.id: e for e in exits}
        self.asn_live = {}          # asn -> number of currently-bound accounts
        self._seq = 0

    def _score(self, e: Exit) -> float:
        age = monotonic() - e.last_used
        recency = self.RECENCY_TAU / (age + 1.0)      # recent => high penalty
        volume = e.use_count * 0.15
        asn = self.asn_live.get(e.asn, 0) * 5.0        # avoid ASN collisions
        return recency + volume + asn + e.prior

    def assign(self, account):
        # rebuild the heap each call; O(n) is fine for pools in the hundreds
        heap = [(self._score(e), i, e) for i, (k, e) in enumerate(self.exits.items())]
        heapq.heapify(heap)
        score, _, chosen = heapq.heappop(heap)
        chosen.last_used = monotonic()
        chosen.use_count += 1
        self.asn_live[chosen.asn] = self.asn_live.get(chosen.asn, 0) + 1
        self._seq += 1
        return chosen

    def release(self, e: Exit):
        self.asn_live[e.asn] = max(0, self.asn_live.get(e.asn, 1) - 1)
Enter fullscreen mode Exit fullscreen mode

The asn term is the piece that actually kills the correlation. Two accounts that resolve out of the same ASN look similar to an origin even if their IPs differ, because routing, latency, and sometimes reverse-DNS all rhyme. By counting how many accounts are currently bound to each ASN and heavily penalizing the fourth such exit, you keep the accounts you are actively running off each other's infrastructure, then release the penalty as sessions die.

You are choosing a minimum, not a fair share

The difference between the two allocators is philosophical as much as mechanical. Round-robin optimizes for "everyone gets a turn," which is a fairness objective. The weighted allocator optimizes for "the next account gets the exit that is least likely to be already-flagged or already-linked to a running peer," which is a risk objective. For a scraping fleet you do not care whether every exit is used equally. You care that no account is standing next to a neighbor on shared ground. Those are different distributions, and the second one is deliberately unequal: hot, recently-burned exits sit unused while cool ones do the work.

That inequality is the feature. Watch it directly:

def health(alloc):
    counts = sorted((e.use_count for e in alloc.exits.values()))
    hot = sum(1 for e in alloc.exits.values() if e.use_count >= 20)
    # a healthy weighted pool has a long flat tail and few 'hot' exits;
    # round-robin produces near-identical counts and lets hot exits
    # be re-served to fresh accounts on schedule.
    return {"median_uses": counts[len(counts)//2],
            "p95_uses": counts[int(len(counts)*0.95)],
            "hot_exits": hot}
Enter fullscreen mode Exit fullscreen mode

If your use_count distribution looks almost perfectly flat, you have accidentally rebuilt round-robin and you are leaking periodicity again. A good run shows median well below p95, meaning genuinely fresh exits are being preferred over merely-owed-a-turn ones.

Recency decay is not optional

One subtlety in the score: the recency term uses a half-life, not a hard "was it used recently, yes/no." A binary recency gate oscillates — every exit becomes eligible at the same time on the same schedule, which reintroduces exactly the periodicity we are removing. The continuous RECENCY_TAU / (age + 1) makes exits become attractive at staggered, effectively random-feeling moments. Combined with per-account jitter in your request timing, it means the origin watching your fleet cannot recover your assignment function from a couple of observation windows. That is the whole point: break the phase relationship.

Cost, because this is still a bill

A weighted allocator naturally under-uses degraded exits, which is good for success rate and bad for "did I get my money's worth from the whole pool" accounting. Resist the urge to force even utilization to feel productive. An exit the origin already distrusts costs you the same per-GB whether it succeeds or not, and a failed authenticated request that trips a challenge is far more expensive than the bytes — it can burn the account. Spend the pool unevenly, on the exits that are actually winning, and let freshness be the currency.

Putting it in the rotation loop

Call assign() when you bind an account to a sticky session, call release() when that session goes DEAD. Keep the ASN map in the same place as your session state machine (the previous article's Session.exit_id) so that affinity and correlation-avoidance share one source of truth. When the two agree, your logged-in success rate stops looking like weather.

Stop counting exits. Start scoring them. Round-robin is not lazy — it is a leak.

Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele

Top comments (0)