Respecting GPT‑OSS Free Tier Limits: Adding a 15 s Delay Between Repository Calls in Content‑Automation

---

TL;DR:

I added a 15 s pause between consecutive GPT‑OSS calls in src/main.py to stay within the free tier’s request limits. This simple change eliminated intermittent 429 errors and made the content‑generation pipeline stable across all repositories.

The Problem

Our content‑automation pipeline pulls data from every repository in the organization, feeds it to the GPT‑OSS‑20B model, and writes the generated markdown files back to disk.

When the number of repos grew past 30, the script started throwing:

HTTPError: 429 Too Many Requests

or, in some cases, a ConnectionResetError because the model service closed the socket.

The GPT‑OSS free tier allows one request per 15 seconds per IP address. The original loop sent requests back‑to‑back, so we were hitting the limit almost immediately.

I tried adding a ThreadPoolExecutor to parallelize the calls, thinking that a few workers would speed up the run. That only amplified the problem, as the service throttled us even harder.

What I Tried First

1. No delay, no concurrency – The baseline script sent requests as fast as possible.
2. ThreadPoolExecutor – Spawning 4 workers to hit the model in parallel.
3. Back‑off on 429 – On a 429, I retried after a 30 s sleep.
4. Increasing the timeout – Adjusted the requests timeout to 120 s.

None of these approaches solved the underlying issue: the free tier enforces a hard per‑IP rate limit, and the only reliable way to stay within it is to throttle the request rate itself.

The Implementation

I opted for the simplest solution: add a time.sleep(15) between each repository call.

The change lives in src/main.py. Here’s the relevant diff:

@@
-    for repo_info in all_repos:
+    for i, repo_info in enumerate(all_repos):
+        if i > 0:
+            # Respect GPT‑OSS free tier: 1 request per 15 seconds
+            print(f"Sleeping 15s before processing repo #{i+1} ({repo_info['name']})")
+            time.sleep(15)
         # Build the AI config
         ai_cfg = AIConfig(
             model=repo_info['model'],
             prompt=repo_info['prompt'],

Why this works

• Deterministic pacing – We guarantee no more than one request every 15 s, which matches the service’s limit.
• Simplicity – No need for external libraries or complex back‑off logic.
• Transparency – The print statement logs when the script sleeps, making it easy to audit the pacing.

Minor refactor

While adding the delay, I also made a small refactor to the loop variable naming (i, repo_info) for clarity. The rest of the script remains unchanged.

Key Takeaway

When working with rate‑limited APIs, the most reliable strategy is to throttle the request rate yourself, rather than relying on retry logic or concurrency. A fixed sleep interval can be implemented with zero dependencies and will keep your pipeline within the service’s constraints.

This pattern is broadly applicable:

• Any third‑party AI service with hard limits (OpenAI, Cohere, Anthropic).
• Public APIs that enforce per‑IP throttling (GitHub, Twitter).
• Internal services that have strict quotas.

What's Next

1. Exponential back‑off – Instead of a fixed 15 s sleep, we could use a jittered back‑off algorithm that adapts to the actual quota usage.
2. Queue system – Move the repo list into a message queue (e.g., RabbitMQ) and let workers process jobs with built‑in rate limiting.
3. Monitoring – Add metrics to Prometheus to track request counts and sleep durations, so we can alert if we hit the limit again.

For now, the 15 s delay keeps the pipeline running smoothly, and the logs confirm that we’re never over the threshold.

---

Tags: #vibecoding #buildinpublic #python #gpt #rate‑limiting #automation

---

Part of my Build in Public series — sharing the real process of building SaaS projects from Playa del Carmen, México.

Repo: zaerohell/content-automation · 2026-07-30

#playadev #buildinpublic