DEV Community

Odd_Background_328
Odd_Background_328

Posted on

Stabilize AI Smoke Tests With a Token-Bucket Proxy Before the Free Server Bites Back

The CI dashboard is green. One merge request arrives, fans out into 14 parallel jobs, and every job calls the same AI endpoint to summarize a diff. Twelve jobs fail after a 60-second timeout. Green goes red in less than a minute.

The common reaction is to blame the AI provider. The actual failure sits one layer above: unbounded concurrency. Free shared capacity looks like a fire hose until everyone opens their valve at once.

This article shows a small, dependency-free proxy that turns an OpenAI-compatible endpoint into a rate-limited, circuit-broken service suitable for CI smoke tests, batch analysis, and other throwaway workloads.

What the free tier really gives you

MonkeyCode is an open-source project that provides free model access and a free server option for developers who want to experiment without provisioning their own GPU. That combination is valuable for non-critical pipelines. The fine print is operational, not contractual: the server is shared, latency fluctuates, and your jobs are not the only tenants.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Free infrastructure changes how you design clients. A paid API can absorb 20 concurrent requests and still return in 500 ms. A free shared server cannot, and it should not be expected to. The correct response is to protect the shared resource from your own burst, and to protect your pipeline from the shared resource's inevitable hiccups.

The architecture: a local egress proxy

The pattern is simple. Every AI client in your build writes to http://localhost:8765 instead of the real endpoint. The proxy applies two rules before forwarding.

  1. Token bucket – caps call rate so your concurrent jobs cannot stampede the upstream.
  2. Circuit breaker – stops traffic for a cool-down window after repeated 5xx responses or timeouts.

This gives you a reproducible boundary. You can tune it locally, watch the failure modes, and decide whether the free tier is a good fit for a specific workload.

The proxy code

The implementation uses only the Python standard library. Save it as ai_proxy.py and run it with python3 ai_proxy.py.

#!/usr/bin/env python3
"""Rate-limited, circuit-broken proxy for OpenAI-compatible endpoints."""
import json
import threading
import time
import urllib.request
import urllib.error
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

UPSTREAM = "http://localhost:8080/v1/chat/completions"  # change to MonkeyCode free server endpoint
RATE = 2        # steady requests per second
BURST = 5       # max burst size
FAIL_THRESHOLD = 3
COOLDOWN = 30   # seconds

class TokenBucket:
    def __init__(self, rate, burst):
        self.rate = rate
        self.burst = float(burst)
        self.tokens = float(burst)
        self.lock = threading.Lock()
        self.updated = time.monotonic()

    def acquire(self, timeout=10):
        deadline = time.monotonic() + timeout
        with self.lock:
            while True:
                now = time.monotonic()
                self.tokens = min(self.burst, self.tokens + (now - self.updated) * self.rate)
                self.updated = now
                if self.tokens >= 1:
                    self.tokens -= 1
                    return True
                if now >= deadline:
                    return False
                wait = (1 - self.tokens) / self.rate
                self.lock.release()
                time.sleep(min(wait, 0.1))
                self.lock.acquire()

class CircuitBreaker:
    def __init__(self, threshold, cooldown):
        self.threshold = threshold
        self.cooldown = cooldown
        self.failures = 0
        self.open_until = 0
        self.lock = threading.Lock()

    def allow(self):
        with self.lock:
            if time.monotonic() < self.open_until:
                return False
            return True

    def record(self, ok):
        with self.lock:
            if ok:
                self.failures = 0
                return
            self.failures += 1
            if self.failures >= self.threshold:
                self.open_until = time.monotonic() + self.cooldown
                self.failures = 0

bucket = TokenBucket(RATE, BURST)
breaker = CircuitBreaker(FAIL_THRESHOLD, COOLDOWN)

class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        if not breaker.allow():
            self.send_error(503, "Circuit open; upstream cooling down")
            return
        if not bucket.acquire():
            self.send_error(429, "Rate limit exceeded; try again soon")
            return

        length = int(self.headers.get("Content-Length", 0))
        body = self.rfile.read(length)
        req = urllib.request.Request(UPSTREAM, data=body, headers={"Content-Type": "application/json"})
        try:
            with urllib.request.urlopen(req, timeout=60) as resp:
                data = resp.read()
                self.send_response(200)
                self.send_header("Content-Type", "application/json")
                self.end_headers()
                self.wfile.write(data)
                breaker.record(True)
        except urllib.error.HTTPError as e:
            breaker.record(e.code >= 500)
            self.send_error(e.code)
        except Exception:
            breaker.record(False)
            self.send_error(502, "Upstream unavailable")

if __name__ == "__main__":
    print(f"Proxy running on :8765 -> {UPSTREAM}")
    print(f"Rate={RATE} req/s, burst={BURST}")
    ThreadingHTTPServer(("127.0.0.1", 8765), Handler).serve_forever()
Enter fullscreen mode Exit fullscreen mode

The proxy does not parse the prompt or track tokens. It is intentionally dumb. That keeps it trustworthy enough to put in a build pipeline.

Step-by-step: put it into a CI job

  1. Start the proxy on the runner: python3 ai_proxy.py &
  2. Point your AI client at http://127.0.0.1:8765/v1/chat/completions.
  3. Keep the same API key the model provider expects; the proxy forwards headers only if you extend it, which this example does not do for brevity.
  4. Run a small batch of 10 requests to verify the bucket works.
  5. Launch the 14 parallel jobs again and record completion times.

That last step is the whole point. With RATE=2, the proxy admits no more than 5 requests in the first second, then 2 per second. The 14 jobs queue locally instead of piling onto the shared server. Some finish later, but far fewer fail.

Reading the numbers

The proxy gives you two explicit failure signals: HTTP 429 and HTTP 503. Deciding what to do with them depends on your workload.

Workload Free model + free server Self-hosted Paid API
CI smoke test, low volume, tolerant to minutes Yes, with proxy Overkill Acceptable
Interactive chat, sub-second SLO No Maybe Best
Batch analysis, hours of runtime Yes, with long cooldown Best if stable Expensive
User-facing feature, 2-second SLO No Yes Yes

Use the decision table as a starting point. The proxy gives you numbers to adjust the thresholds: if you see 429s, lower RATE or increase the client's backoff. If you see 503s, the server is unhealthy; raise COOLDOWN or switch workloads entirely.

Limitations and who should skip this

This proxy is a guardrail, not a scaler. It does not make the free server faster. It does not turn the shared capacity into a contract. It only prevents your own clients from acting like a herd.

Do not use this approach when:

  • Your SLO requires sub-second latency. The local queue adds milliseconds, but the upstream variance will still dominate.
  • You have zero tolerance for dropped jobs. Use a paid or dedicated path.
  • You expect the proxy to solve upstream outages. The circuit breaker stops the bleeding, not the cause.

The example also omits API-key forwarding to keep the code short. In production, copy the Authorization header from the incoming request.

The operational takeaway

Free model access and a free server are a real way to keep experimentation costs at zero. They become hostile when your client design ignores their shared nature. A 40-line proxy restores the boundary.

Next time the CI goes red, check the concurrency before blaming the model. Your own burst is usually the first failure to fix.

If you want a quick, honest way to test MonkeyCode's free layer, run this proxy in front of it and push 100 requests. The output will tell you whether the free server fits your workflow or whether you need the more predictable option.

Top comments (0)