The Retry Loop That Ate a Free Tier: A Rate-Limit Postmortem

The webhook handler passed every local test, deployed cleanly, and then took down the free server at 2:14 AM. The symptom looked like a provider quota problem, but the root cause was a single AI-generated loop that retried too eagerly. This post walks through the debugging path from the first 429 to the actual fix, because the same failure pattern is hiding in a lot of generated code.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The Symptom: 429s That Appeared Out of Nowhere

The setup was deliberately small: a webhook endpoint generated with MonkeyCode's free model access, hosted on its free server, and connected to a third-party API that occasionally returned 503. MonkeyCode is an open-source project, and its free tier includes a 10-million-token allowance plus a free server instance, which is plenty for a small integration but tight enough to expose amplification bugs. The handler worked for two days, and then legitimate requests started failing with 429 Too Many Requests even though the app was nowhere near its monthly token budget.

The first debugging instinct was to blame the third-party provider, since 429 is a rate-limit status code and that provider had been flaky all week. A quick check of the provider's status page showed no incident, which shifted the suspicion back to the free server. The server's own logs showed nothing unusual, because the requests that failed were outgoing retries, not incoming traffic.

The turning point came when the developer replayed the exact failing request sequence locally against the same API and got 200 responses. The code was identical, the payload was identical, and the only difference was the server environment. That local success strongly suggested an environmental cause, so the investigation moved to the server's outgoing traffic.

The Investigation: Following the Retry Trail

The access logs on the free server revealed the pattern: every time the downstream API returned a 503, the handler fired off a new request immediately, then another, then another, with no delay between attempts. A single downstream hiccup of three seconds produced forty outgoing requests in under a minute, and the third-party API responded by rate-limiting the server's IP. The logs made the amplification visible in one command:

grep "POST /api" access.log | awk '{print $4}' | cut -d: -f2 | uniq -c

The AI-generated retry loop looked harmless at first glance because it was short and readable. Here is the simplified version that caused the incident:

def call_api(payload):
    while True:
        response = requests.post(API_URL, json=payload)
        if response.status_code < 500:
            return response.json()
        # keep retrying until the API accepts the request

The loop retries forever, it retries instantly, and it treats every 5xx as worth another attempt. On a free tier with a per-minute request cap, that behavior converts a transient downstream error into a self-inflicted outage, because the retries themselves consume the quota that legitimate requests need. The second problem was that the loop ignored the Retry-After header that the API included in its 429 responses; the API was explicitly saying "wait 30 seconds," and the handler was responding with "how about right now."

The Root Cause: The Happy Path Is Generated, the Failure Path Is Not

The real lesson is not that the AI model wrote a bad loop, because plenty of human-written loops have the same flaw. The lesson is that generated code tends to be optimized for the happy path, and the failure path is where the assumptions live: retries need backoff, backoff needs jitter, and jitter needs a maximum ceiling. This pattern matters more now that AI coding tools produce more of the codebase, because the happy path is exactly what models generate confidently and the failure path is exactly what they gloss over.

The fix for this incident had three parts:

1. Respect the Retry-After header whenever the downstream service provides one.
2. Use exponential backoff with jitter so retries spread out instead of clustering.
3. Cap the total retry budget so a stuck service cannot consume the entire free tier.

import random
import time

MAX_RETRIES = 5
BASE_DELAY = 1.0

def call_api(payload):
    for attempt in range(MAX_RETRIES):
        response = requests.post(API_URL, json=payload)
        if response.status_code < 500:
            return response.json()
        retry_after = response.headers.get("Retry-After")
        if retry_after is not None:
            delay = float(retry_after)
        else:
            delay = BASE_DELAY * (2 ** attempt) + random.uniform(0, 0.5)
        time.sleep(min(delay, 30))
    raise RuntimeError("downstream API still failing after retries")

The decision table that now lives next to this code is simple: retry on 429 with the header's delay, retry on 5xx with exponential backoff, and never retry on 4xx validation errors because the request itself is the problem. That last rule matters more than it looks, because retrying a bad payload only multiplies the damage.

The Reusable Debugging Workflow

The debugging path that found this bug is worth keeping as a checklist for any AI-generated code that talks to external services. Start with the symptom and verify whether it reproduces outside the failing environment, because a local success strongly suggests an environmental cause. Then trace the outgoing traffic rather than the incoming traffic, since the failing requests were the ones the server initiated, not the ones it received.

The next step is to look for amplification loops, which are any code path where one external failure produces multiple internal attempts. A quick grep for while True or retry in the generated code usually reveals the loop, and counting the outgoing requests per minute confirms whether the loop is amplifying the problem. The final step is to apply the retry decision table and re-run the exact failure scenario, which in this case meant simulating a downstream 503 and watching the outgoing request rate drop from forty per minute to five.

Limitations and Who Should Skip This Approach

This retry strategy is not a universal fix, and there are setups where it is the wrong tool entirely. If the downstream service requires exactly-once delivery, retries alone are insufficient because a retry can duplicate a side effect, so the handler needs idempotency keys or a deduplication layer on top of the backoff logic. The approach also assumes the caller controls the retry budget, which is not true when the client is a third party that retries aggressively on its own; in that case the server needs rate limiting on the receiving side, not just polite retries on the sending side.

Teams with a generous paid tier might never see this bug, because the quota is large enough to absorb the amplification, which is exactly why the free tier is a better place to learn the lesson. The constraint exposes the flaw early, and fixing it on a constrained budget produces code that behaves well when the traffic eventually grows.

The Practical Takeaway

AI-generated code is fast to produce and slow to trust, and this incident is a concrete example of where that trust needs to be earned. The fix was not more code but more restraint: a retry cap, a delay that respects the server's instructions, and a rule that validation errors never retry. The constrained setup that produced the bug also made the debugging cheap, because the free server's logs and quota told the whole story without any paid observability tooling.

For anyone who wants to test this pattern without risking a production account, the workflow is to generate a webhook handler, point it at a flaky endpoint, and watch what happens under a per-minute limit. The free 10-million-token allowance and free server make that experiment cost nothing to run, and the failure mode is educational precisely because it is cheap to trigger.