"How many concurrent requests can I run?" doesn't have a single number answer — it depends on your pool size, target tolerance, and what you're actually optimizing for. Here's how to think about it instead of guessing.
The pool-size math nobody does upfront
If you have 1,000 IPs in your pool and you're running 500 concurrent threads with rotation, you're not actually getting 1,000-IP-worth of distribution — you're hitting a huge chunk of your pool within seconds, meaning many requests land on IPs that were JUST used moments earlier. Effective distribution depends on the ratio of concurrent threads to pool size, not just pool size alone.
A rough starting rule: keep concurrent threads at roughly 5-10% of your active pool size if you want each IP to have meaningful "rest time" between requests to the same target. Push much higher than that and you're relying on the target not noticing rapid reuse, not on genuine distribution.
Per-target tolerance varies enormously
The same concurrency level that's invisible to one target gets you blocked in minutes on another:
Static content, light protection: high concurrency tolerated, target largely doesn't track request velocity per IP
Search/listing pages with basic rate limiting: concurrency matters less than requests-per-IP-per-minute — you can run many threads as long as each individual IP stays under the target's per-IP threshold
Account-gated or aggressively fingerprinted targets: concurrency across the whole pool matters less than behavior per session — this is where sticky sessions and human-paced timing matter more than raw thread count
Testing concurrency in isolation, without accounting for which of these three categories your target falls into, produces numbers that don't transfer to a different target at all.
Ramping instead of jumping straight to max concurrency
Starting a new target at your theoretical max concurrency is how you find out the hard way that the theoretical max was wrong. A ramp pattern works better:
Start at a low concurrency (5-10 threads), measure ban/failure rate over a meaningful sample (hundreds of requests, not tens)
Increase incrementally, re-measuring failure rate at each step
Stop increasing once failure rate starts climbing measurably above your baseline — that's your real ceiling for this specific target, today
This ceiling isn't permanent. Targets change their tolerance over time (sometimes because of your own traffic pattern, sometimes independently), so the ramp is worth re-running periodically rather than treating a number you found once as permanent truth.
Concurrency per IP vs concurrency per pool
These are different numbers and conflating them causes confusion:
Concurrency per IP: how many simultaneous requests one specific IP is handling right now. For most targets, this should be 1 — an IP handling multiple simultaneous requests to the same target looks nothing like normal user behavior.
Concurrency per pool: how many total simultaneous requests you're running across your whole IP pool. This can be much higher, since it's spread across many different IPs each behaving like a normal single user.
If you're seeing high ban rates and you're not sure why, check whether you've accidentally let concurrency-per-IP creep above 1 somewhere in your request queue logic — this is a common, easy-to-miss bug that looks like a "bad pool" problem but is actually a scheduling problem.
Practical takeaway
Don't ask "what's the max concurrency" as a fixed number — ask "what's the max concurrency for this pool size, against this specific target, today." Re-derive it periodically instead of hardcoding a value you found once and trusting it forever.
This kind of pool-sizing and concurrency planning is exactly what we help teams work through when architecting proxy infrastructure for scraping and automation at scale — happy to compare notes if you're mid-tuning something like this.
Top comments (0)