Your Free Endpoint Is Fine. Your Retry Loop Is the Anti-Pattern.

Free model access and a free server remove the billing argument. They do not remove the engineering. Your loop is still the part that breaks at 3 a.m.

Here is my anti-pattern catalog for that loop. Four symptoms, four root causes, four replacements. Then a harness you can actually run.

The setup I am assuming

• You call a hosted model over an OpenAI-compatible HTTP API.
• You run a worker on a small free server instance.
• Your job is a loop: plan, call, check, repeat.

MonkeyCode is one option in that space. The operator markets free model access and a free server option. The operator also advertises a promotional token allowance; the figure I was given is 10M tokens. I have not verified that number, and quotas move. Check the current page before you plan around it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Anti-pattern 1: The Retry Storm

Symptom

One bad response triggers five retries. Each retry spawns its own retries. Your worker spends an hour asking the same question.

Root cause

Retry logic lives at three layers. The HTTP client, the SDK, and your loop each think they own the problem. Nobody caps the total.

Replacement

One owner, one budget, full jitter. Count attempts once, across everything.

import random
import time

MAX_ATTEMPTS = 4  # total budget, not per layer

def call_with_budget(fn, attempts=MAX_ATTEMPTS):
    for i in range(attempts):
        try:
            return fn()
        except Exception as err:  # narrow this in real code
            if i == attempts - 1:
                raise
            # Full jitter beats fixed backoff on a shared pool.
            time.sleep(random.uniform(0, min(2 ** i, 8)))
    raise RuntimeError("unreachable")

Add a circuit breaker on top. After N consecutive failures, stop for 60 seconds. Mark the job failed loudly instead of quietly.

Anti-pattern 2: The Sample of One

Symptom

You run the prompt once. It looks right. You call it validated.

Root cause

You measured an output, not a distribution. Shared free capacity is noisy by design.

Replacement

Repeat the call N times and record what actually came back.

import csv, json, os, statistics, time
from urllib import request

BASE   = os.environ["LLM_BASE_URL"].rstrip("/")  # any OpenAI-compatible endpoint
KEY    = os.environ["LLM_API_KEY"]
MODEL  = os.environ["LLM_MODEL"]
N      = int(os.environ.get("N", "20"))
PROMPT = "Return one JSON object with the single key 'ok'."

def one_call(prompt):
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0.2,
    }).encode()
    req = request.Request(
        f"{BASE}/chat/completions",
        data=body,
        headers={"Authorization": f"Bearer {KEY}",
                 "Content-Type": "application/json"},
    )
    t0 = time.perf_counter()
    try:
        with request.urlopen(req, timeout=60) as resp:
            payload = json.loads(resp.read())
            status = resp.status
    except Exception as err:  # 429, timeouts, 5xx all land here
        payload, status = {}, f"error:{type(err).__name__}"
    latency_ms = (time.perf_counter() - t0) * 1000
    text = ""
    if payload.get("choices"):
        text = payload["choices"][0]["message"]["content"] or ""
    return status, latency_ms, text

def schema_ok(text):
    try:
        return isinstance(json.loads(text), dict)
    except Exception:
        return False

def main():
    latencies = []
    with open("runs.csv", "w", newline="") as f:
        writer = csv.writer(f)
        writer.writerow(["i", "status", "latency_ms", "chars", "schema_ok"])
        for i in range(N):
            status, latency_ms, text = one_call(PROMPT)
            latencies.append(latency_ms)
            writer.writerow([i, status, round(latency_ms), len(text),
                             schema_ok(text)])
    latencies.sort()
    print("p50_ms", round(statistics.median(latencies)))
    print("p95_ms", round(latencies[int(len(latencies) * 0.95) - 1]))
    print("max_ms", round(latencies[-1]))

if __name__ == "__main__":
    main()

Run it twice: once in your morning, once in your evening. Compare the two CSVs. If p95 shifts a lot, you learned something the demo never showed you. This snippet uses only the standard library, so you can paste it and go.

Anti-pattern 3: The Pet Server

Symptom

Your loop keeps progress in memory. A restart loses three hours of work.

Root cause

You treat the free instance like a pet. Free capacity is not a promise of uptime, and preemption is normal.

Replacement

Idempotent steps plus external checkpoints. Design for kill -9 at any moment.

import json, os, tempfile

CKPT = "checkpoint.json"

def load_state():
    if not os.path.exists(CKPT):
        return {"step": 0, "done": []}
    with open(CKPT) as f:
        return json.load(f)

def save_state(state):
    fd, tmp = tempfile.mkstemp(dir=".")
    with os.fdopen(fd, "w") as f:
        json.dump(state, f)
    os.replace(tmp, CKPT)  # rename is atomic on the same filesystem

The rule that matters: every step must be safe to run twice. If step 7 charges a card, step 7 needs a key.

Anti-pattern 4: Context Assumed, Not Measured

Symptom

Long input gets truncated. Your agent then answers confidently about text it never saw.

Root cause

You assumed a context size instead of measuring your own payload.

Replacement

Estimate before you send, then fail loudly.

• Estimate tokens with len(text) / 4 as a rough heuristic, not a truth.
• Refuse requests above your budget. Do not let the server pick for you.
• Log the estimate next to the request ID so you can debug later.
• Split long documents yourself, and track which chunk produced which claim.

Decision table

Situation	Free model + free server	Paid endpoint	Self-host
Prototype a loop, one developer	Good fit	Fine	Overkill
20-run evaluation harness	Good fit, expect noise	Better for tight p95	Fine
Overnight batch with retries	Workable with checkpoints	Simpler	Fine
Latency-sensitive user traffic	Wrong tool	Usually yes	Possible
Data that cannot leave your network	No	Depends on contract	Yes

Read the left column honestly. Free is excellent for learning a loop's shape. It is a bad place to hide a missing circuit breaker.

Who should not use this approach

• You need a written SLA or a fixed quota. Free tiers change without notice.
• You process regulated data that cannot go to a third party.
• You need guaranteed concurrency for user-facing traffic.
• You want a benchmark number you can quote. Run your own, on your workload.

What I would measure before committing

1. p50 and p95 latency, from your own runs.
2. Error rate split by status code, not one blended number.
3. Schema failure rate on structured output.
4. Recovery time after a forced restart.
5. Cost per completed job, priced at paid rates for comparison.

Numbers 1 through 4 tell you whether the loop is sound. Number 5 tells you whether the free tier was ever the point.

Where MonkeyCode fits, and where it does not

MonkeyCode is relevant at two steps in this workflow. Free model access lets you build the harness without a payment method. A free server option lets you test checkpoint recovery under real restarts. Both are the cheap part of the exercise, which is exactly why they are useful.

Neither one fixes a retry storm. Neither one makes a single sample meaningful. If you want a place to run the harness above, the free model access and free server option are the two things I would test first.

Keep the loop boring. Count attempts once. Checkpoint every step. Measure twenty runs before you trust one.