Free infrastructure is unpredictable. That is not a bug — it is a testing opportunity. Chaos engineering is usually described as a luxury for teams with dedicated SREs and production clusters. The opposite is true. The systems that need fault injection the most are the ones running on free servers and free model access, because their failure modes are the most varied and the least documented.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Why free hardware is the best chaos lab
A paid API gives you a contract. A free tier gives you a probability distribution. Rate limits appear without warning. Timeouts stretch past your patience. The occasional 5xx arrives just when your batch job reaches hour three. You cannot fix these failures — they are the price of free access. But you can learn exactly how your code behaves when they happen.
That learning is what chaos testing is for. You deliberately break things in a controlled way so that when they break for real, the recovery path is already muscle memory. The trick is that you do not need a dedicated test cluster. The free server you already have is the perfect place to break things on purpose.
A fault-injection proxy in one file
The cleanest way to inject failures is not to modify your application code. It is to stand between your code and the model API with a small proxy that decides, per request, whether to fail and how. This keeps your production code untouched while giving you full control over the chaos.
# chaos_proxy.py — inject failures into any OpenAI-compatible API call
# usage: python chaos_proxy.py --port 8765 --upstream https://api.example.com/v1
# then point your client at http://localhost:8765/v1
import argparse
import json
import random
import time
import urllib.request
from http.server import BaseHTTPRequestHandler, HTTPServer
class ChaosHandler(BaseHTTPRequestHandler):
rate_limit_prob = 0.0
timeout_prob = 0.0
error_prob = 0.0
upstream = ""
def do_POST(self):
r = random.random()
if r < self.rate_limit_prob:
self.send_response(429)
self.end_headers()
self.wfile.write(b'{"error": {"message": "rate limit (injected)"}}')
return
if r < self.rate_limit_prob + self.timeout_prob:
time.sleep(15) # longer than your client's timeout
return
if r < self.rate_limit_prob + self.timeout_prob + self.error_prob:
self.send_response(500)
self.end_headers()
self.wfile.write(b'{"error": {"message": "internal error (injected)"}}')
return
length = int(self.headers.get("Content-Length", 0))
body = self.rfile.read(length)
req = urllib.request.Request(
self.upstream + self.path,
data=body,
headers=dict(self.headers),
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=30) as resp:
data = resp.read()
self.send_response(resp.status)
for k, v in resp.headers.items():
self.send_header(k, v)
self.end_headers()
self.wfile.write(data)
except Exception as e:
self.send_response(502)
self.end_headers()
self.wfile.write(str(e).encode())
def log_message(self, fmt, *args):
print(f"[chaos] {self.address_string()} {fmt % args}")
def main():
p = argparse.ArgumentParser()
p.add_argument("--port", type=int, default=8765)
p.add_argument("--upstream", required=True)
p.add_argument("--rate-limit-prob", type=float, default=0.05)
p.add_argument("--timeout-prob", type=float, default=0.03)
p.add_argument("--error-prob", type=float, default=0.02)
args = p.parse_args()
ChaosHandler.rate_limit_prob = args.rate_limit_prob
ChaosHandler.timeout_prob = args.timeout_prob
ChaosHandler.error_prob = args.error_prob
ChaosHandler.upstream = args.upstream
server = HTTPServer(("localhost", args.port), ChaosHandler)
print(f"chaos proxy on :{args.port} -> {args.upstream}")
print(f"probabilities: rate_limit={args.rate_limit_prob} timeout={args.timeout_prob} error={args.error_prob}")
server.serve_forever()
if __name__ == "__main__":
main()
Run it with --rate-limit-prob 0.2 and watch your agent loop stumble. That stumble is the information you need.
Three experiments worth running
Start with the rate limit experiment. Set the probability to 0.2 and run your normal workload. Watch what happens. Most clients retry immediately, which makes the 429 worse. The correct response is exponential backoff with jitter — a random delay that grows with each retry. If your code does not do that, the proxy will show you exactly how fast the retry storm builds.
The timeout experiment is more subtle. A 15-second delay does not always trigger your client's timeout. Some HTTP libraries default to 30 seconds, some to 60, and some to infinity. The proxy makes the mismatch visible. You will learn your real timeout budget, not the one you assumed.
The 500 experiment tests your checkpointing. When a call fails mid-run, does your job resume from the last completed step or restart from zero? On a free server, this distinction is the difference between a 10-minute recovery and a 3-hour redo.
What the failures teach you
Run the proxy for an hour against a representative workload and record three numbers: the percentage of requests that failed, the time to recover from each failure, and the tokens wasted on retries. The last number is the one nobody measures. A retry re-sends the full conversation history, so a single 429 can cost you ten times the tokens of the original call.
That is the hidden economics of free access. The quota is not consumed by the work itself. It is consumed by the failures. A 5% failure rate with naive retries can inflate your token burn by 20-30%. Chaos testing turns that invisible cost into a number you can see and fix.
Who should skip this
If your workload is a single synchronous call with no retry logic, a proxy adds nothing — you already know what a failure looks like. If you are building a user-facing assistant where a 15-second delay is itself a product failure, chaos testing will only confirm what you already know: you need paid access with a latency guarantee. And if you cannot tolerate any artificial failures in your pipeline, run the experiments in a separate environment, not against your live job.
Chaos testing is not about making free infrastructure look bad. It is about making your code honest about what it can survive. A free server plus free model access gives you a place to run that honesty check without spending a dollar. Break things on purpose, measure the recovery, and fix the retry logic before the real rate limit arrives.
If you want a place to run these experiments, MonkeyCode's free server and free model access are a practical starting point. Point the proxy at their API, set the probabilities, and see what your code actually does under pressure.
Top comments (1)
I think the most interesting failure mode here is retry amplification. A pipeline can recover perfectly from individual faults and still fail as a system because every retry consumes shared capacity rate limits, tokens, queues, or downstream concurrency and makes the next failure more likely. So I’d measure not just recovery time, but how much additional work a single injected failure creates. A resilient agent should have a bounded blast radius: one failed call shouldn't be able to turn into 20 extra requests, a full context resend, and a backlog for unrelated runs.