The Birthday Problem in Proxy Pools: Why Two of Your Accounts Will Share an IP
Everyone in multi-account scraping eventually meets the rule "one account, one IP" and then meets its price tag. Real pools are finite, residential sessions rotate, and sooner or later you're running 200 accounts through a pool sized for 50 concurrent IPs — at which point someone on the team asks the question this article answers: when two of my accounts share an exit IP, how bad is it, and how often will it happen?
The intuition most engineers have is wrong in a specific, predictable way. They assume that if the pool is 50 IPs and you have 200 accounts, the sharing is somehow "4 accounts per IP, evenly spread" — a tidy deterministic mapping. It isn't. IP assignment from a rotating residential pool is effectively random draws with replacement, which means the distribution of collisions is lumpier than you think: some IPs serve zero accounts while others serve five, and the probability that at least two specific accounts co-occur on one IP follows the same math as the birthday problem. With 23 people in a room, there's a >50% chance two share a birthday. Your proxy pool is the room, your accounts are the people, and the odds are worse than they look.
The core claim: in a finite rotating pool, cross-account IP collisions are governed by birthday mathematics, they happen far more often than naive division suggests, and what actually endangers your accounts is not sharing an IP — it's temporally correlated sharing: two identities with different fingerprints and different purposes presenting from one address in overlapping time windows. Size and schedule around that, and a pool smaller than 1:1 becomes safe. Ignore it, and even a 1:1 pool will eventually burn you.
The math, concretely
Birthday problem, adapted. Pool of n distinct exit IPs (over the window you care about — more on that in a second), k accounts each drawing one IP for the current session. Probability that all draws are distinct:
from math import comb, factorial
def p_all_distinct(n: int, k: int) -> float:
"""P(no two of k accounts share an IP) — birthday math, n = pool size."""
if k > n:
return 0.0
return factorial(n) / (factorial(n - k) * n**k)
for pool in (50, 100, 400):
for accts in (20, 50, 100):
p = 1 - p_all_distinct(pool, accts)
print(f"pool={pool:4d} accounts={accts:4d} P(collision)={p:.3f}")
Run it and look at the numbers without flinching: 50 IPs and 20 accounts gives a ~93% chance of at least one collision. 50 IPs and 50 accounts? 100%. Even 400 IPs and 50 accounts — a comfortable-looking 8× headroom — still gives a ~95% chance that some pair of accounts shares an IP somewhere in the window. If a single collision were fatal, multi-account scraping with rotating pools would be impossible. It isn't, so a single collision isn't fatal — and pretending it is leads to massively oversized pools.
What a single collision actually does is add one edge to a graph the anti-bot system maintains: nodes are identities, edges are "seen from same IP". One edge is noise — real households have multiple users, mobile carrier NATs put thousands behind one address, roommates share Wi-Fi. The danger threshold is structural: when one account's failure signature can traverse the graph to its neighbors. The account that gets flagged for scraping behavior, then its IP gets reputation-flagged, and the flag then lands on every other identity recently seen from that IP — that's graph traversal, and it's how one burned account takes three friends down.
So the sizing question is really: what's the expected number of collision pairs, and how concentrated do they get? That's computable directly:
def expected_colliding_pairs(n: int, k: int) -> float:
"""E[number of account pairs sharing an IP] = C(k,2) / n."""
return comb(k, 2) / n
# 50 IPs, 20 accounts: expected 190/50 = 3.8 colliding pairs
# 400 IPs, 50 accounts: expected 1225/400 ≈ 3.06 colliding pairs
Notice what this says: the expected number of colliding pairs scales with accounts squared over pool size. Doubling your accounts quadruples the collision load unless you double the pool. That's the real arithmetic of "one account one IP" — not a binary rule but a budget: you're buying expected-collision headroom, and each new account costs more than the last.
Where the birthday math breaks — and what's worse
The model above assumes independent uniform draws. Two real-world effects make reality worse, and both are worth engineering around.
Effect 1: the pool isn't uniform. Residential pools are heavily stratified by geography. If your 200 accounts all target one country, your effective n is the country sub-pool, not the marketing number on the provider's pricing page. A "55M IP pool" is 55M globally; the /16s available in your target metro might number in the low thousands. Always compute the birthday math against the sub-pool you actually draw from.
Effect 2: time windows, not eternity. Two accounts shared an IP — in March and in August. Irrelevant. The anti-bot join that matters is concurrent or near-concurrent co-occurrence: same IP, overlapping wall-clock, ideally with similar request patterns. So the operative quantity is collision probability per scheduling window, not per lifetime. This is also your lever: you can't afford a bigger pool, but you can afford to not schedule two accounts through the same IP at the same time — if you know about the collision.
Making the invisible visible: a collision-aware scheduler
You can't avoid what you don't measure. In practice I run every multi-account pipeline through an assignment layer that tracks the current IP behind each sticky session and logs co-occurrence:
import time
from dataclasses import dataclass, field
from collections import defaultdict
@dataclass
class Assignment:
account_id: str
ip: str
started: float
ended: float | None = None
class CollisionMonitor:
"""Tracks which accounts are behind which IP and flags co-occurrence."""
def __init__(self, alert_window_s: float = 300):
self.active: dict[str, Assignment] = {} # ip -> current assignment
self.alert_window_s = alert_window_s
self.recent: list[Assignment] = [] # ended assignments (ring)
self.alerts: list[tuple[str, str, float]] = []
def assign(self, account_id: str, ip: str) -> dict:
now = time.time()
conflicts = {}
# 1. concurrent collision: someone else is on this IP right now
if ip in self.active:
conflicts["concurrent_with"] = self.active[ip].account_id
self.alerts.append((account_id, self.active[ip].account_id, now))
# 2. near-time reuse: an account *recently* ended on this IP
for a in self.recent:
if a.ip == ip and now - a.ended < self.alert_window_s:
conflicts.setdefault("recent_users", []).append(a.account_id)
# track regardless — you want the data, not just the alert
self.active[ip] = Assignment(account_id, ip, now)
return conflicts
def release(self, account_id: str) -> None:
for ip, a in list(self.active.items()):
if a.account_id == account_id:
a.ended = time.time()
self.active.pop(ip)
self.recent.append(a)
self.recent = self.recent[-500:]
def collision_report(self) -> dict:
pairs = defaultdict(int)
for x, y, _ in self.alerts:
pairs[tuple(sorted((x, y)))] += 1
return dict(pairs)
The design principle: assign() doesn't refuse collisions — it records and reports them. When the daily report shows account shop_ops_14 and shop_ops_31 co-occurring on one IP four times, you have options: stagger their schedules, put them in different geo sub-pools, or accept it because they run different traffic profiles. The point is that the decision moves from "the pool surprised us" to an explicit, data-backed trade-off. Pairs that co-occur and present similar fingerprints should go to the top of the fix list — that combination is exactly the correlated-sharing signature that turns one flag into a graph traversal.
Two operational rules fall out of the monitor's data. First, stagger release and reuse: an account ending on an IP and a different account starting on the same IP within a minute is the worst co-occurrence pattern, because it looks like an identity hand-off; the 300-second cooldown window in the code above exists purely for this. Second, separate account classes onto disjoint sub-pools when you can: your high-risk experimental accounts and your revenue-critical accounts should never draw from the same birthday room, because the expected-collision math guarantees they'll eventually meet.
The sizing formula that replaces the rule
Putting it together, I'd replace "one account one IP" with a three-line sizing procedure:
-
Measure the effective sub-pool
n— the distinct IPs you actually see over a representative day, not the provider's global number. Log the distinct exit IPs; providers vary more than their dashboards admit. -
Budget expected collision pairs:
E[pairs] = k(k-1)/2n. Pick a threshold you can mitigate operationally (for most pipelines, keepingE[pairs] < 1per scheduling window is comfortable; concentrated high-value accounts deserve≪ 1). - Verify with a simulation before scaling: Monte Carlo your actual schedule (accounts, session durations, release times) against the measured pool and read off the concurrent-collision rate — the closed form assumes uniform draws and your schedule isn't uniform.
import random
def simulate_concurrent(n: int, accounts: list[tuple[str, float]], trials: int = 2000):
"""accounts: [(id, session_duration_s)]. Returns mean concurrent-collision pairs."""
total = 0
for _ in range(trials):
start = {a[0]: random.uniform(0, 3600) for a in accounts}
ends = {a[0]: start[a[0]] + a[1] for a in accounts}
ips = {a[0]: f"ip-{random.randrange(n)}" for a in accounts}
pairs = 0
ids = [a[0] for a in accounts]
for i in range(len(ids)):
for j in range(i + 1, len(ids)):
a, b = ids[i], ids[j]
overlap = min(ends[a], ends[b]) - max(start[a], start[b])
if ips[a] == ips[b] and overlap > 0:
pairs += 1
total += pairs
return total / trials
The rule "one account, one IP" is a special case of this math where E[pairs] = 0 because k ≤ n and assignment is sticky-permanent. It's the right answer when accounts are few and precious. But past a few dozen accounts it's a budget question, and budgets are engineered, not obeyed. Buy the headroom you need, measure the collisions you can't avoid, and spend your paranoia on the temporal correlation — that's the part that actually burns fleets.
Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele
Top comments (0)