Building data pipelines that hit multiple crypto exchange APIs taught us a lot about rate limiting. Here is what works at kkinvesting.io.
The Problem
Every exchange has different rate limit rules, and they are enforced differently:
- Binance: IP-based, 1200 weight/min
- OKX: per-endpoint, 60 requests/2s
- Bybit: tier-based, 120 requests/min
- Kraken: call counter that decays over time
Our Solution: Adaptive Rate Limiter
class AdaptiveRateLimiter:
def __init__(self, base_delay=0.5):
self.delay = base_delay
self.consecutive_429s = 0
def wait(self):
time.sleep(self.delay)
def on_success(self):
self.consecutive_429s = 0
self.delay = max(self.delay * 0.9, 0.1)
def on_rate_limit(self):
self.consecutive_429s += 1
self.delay = min(self.delay * 2, 60)
Key Patterns
- Per-exchange queues — do not share rate limiters across exchanges
- Exponential backoff on 429s, but cap at 60 seconds
- Cache aggressively — fee data does not change every minute
- Batch requests where the API supports it (Binance batch endpoints)
More technical content: kkinvesting.io
Top comments (0)