Membangun Local API Proxy dengan Token Bucket untuk Rate Limit v2

Banyak layanan API menerapkan rate limit untuk melindungi server. Ketika aplikasi melebihi batas, respons 429 Too Many Requests muncul. Solusi klasik retry dengan exponential backoff tetap membiarkan burst request ke upstream. Pendekatan lebih baik: local proxy yang menampung request, mengatur keluar ritmis, dan menyimpan cache.

Arsitektur Proxy Lokal

Penjelasan Alur

1. Client kirim request ke Local Proxy (bukan langsung upstream).
2. Proxy cek Token Bucket: token ada ? lepas ke upstream; tidak ada ? masuk Request Queue.
3. Retry Manager tangani 429 dengan header Retry-After + jitter.
4. Cache Store simpan respons GET sukses; request identik berikutnya ambil dari cache.

Implementasi Minimal (FastAPI + httpx)

# proxy.py
import asyncio
import time
import hashlib
from collections import deque
from fastapi import FastAPI, Request, Response
import httpx

app = FastAPI()
UPSTREAM = "https://api.example.com"
RATE = 10
BURST = 20
CACHE_TTL = 300

class TokenBucket:
    def __init__(self, rate: float, burst: int):
        self.rate = rate
        self.burst = burst
        self.tokens = float(burst)
        self.last = time.monotonic()
        self.lock = asyncio.Lock()

    async def take(self) -> bool:
        async with self.lock:
            now = time.monotonic()
            self.tokens = min(self.burst, self.tokens + (now - self.last) * self.rate)
            self.last = now
            if self.tokens >= 1:
                self.tokens -= 1
                return True
            return False

bucket = TokenBucket(RATE, BURST)
cache: dict[str, tuple[bytes, float]] = {}

async def forward(req: Request) -> Response:
    key = f"{req.method}:{req.url.path}:{req.query_params}"
    if req.method == "GET" and key in cache:
        cached, ts = cache[key]
        if time.time() - ts < CACHE_TTL:
            return Response(content=cached, media_type="application/json")

    async with httpx.AsyncClient(base_url=UPSTREAM, timeout=10) as client:
        while True:
            if await bucket.take():
                break
            await asyncio.sleep(0.1)

        upstream_req = client.build_request(
            req.method, req.url.path,
            params=req.query_params,
            headers=dict(req.headers),
            content=await req.body()
        )
        resp = await client.send(upstream_req)

        if resp.status_code == 429:
            retry = int(resp.headers.get("Retry-After", "1"))
            await asyncio.sleep(retry + 0.1 * (hash(key) % 10))
            continue

        if req.method == "GET" and resp.status_code == 200:
            cache[key] = (resp.content, time.time())

        return Response(
            content=resp.content,
            status_code=resp.status_code,
            media_type=resp.headers.get("content-type")
        )

@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
async def proxy(path: str, request: Request):
    return await forward(request)

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

Menjalankan Proxy

pip install fastapi httpx uvicorn
uvicorn proxy:app --port 8000

Arahkan client ke http://localhost:8000/... sebagai pengganti upstream.

Checklist Produksi

• [ ] Tambahkan metrics (Prometheus) untuk bucket & queue
• [ ] Implementasikan circuit breaker jika upstream gagal
• [ ] Gunakan Redis untuk cache & bucket terdistribusi
• [ ] Tambahkan authentication (API key) pada proxy
• [ ] Tulis unit & integration test untuk retry logic

Kesimpulan

Dengan local proxy token-bucket, aplikasi tidak khawatir 429. Proxy menyerap burst, atur throughput, kurangi beban upstream lewat caching. Cocok untuk microservices, scraper, integrasi pihak ketiga.