Myth FAQ: Retry Is Not Free — Five Duplicate-Work Traps

A worker of mine died mid-task at 2am. The retry ran clean. The customer still got two invoices.

Nothing in the logs looked broken. Both attempts returned HTTP 200. So which attempt actually did the work?

This FAQ lists five claims I keep hearing about retries, free model calls, and cheap compute. Each claim has a local test you can run in about ten minutes. You need no provider account for the first four.

The five myths at a glance

1. A 200 response means the work happened.
2. Retrying a call costs nothing that matters.
3. An idempotency key dedupes everything.
4. An in-memory list is a queue.
5. Free or shared compute can hold your state.

Let me take them one at a time.

Myth 1: A 200 response means the work happened

A status code describes the last hop, not the side effect. A worker can write the row and crash before replying. Your retry then writes the row a second time.

Can you say, right now, whether your handler writes state before or after it responds? Most people cannot answer that about their own code.

The fix is boring. Make the effect idempotent at the receiver. Then retries become safe by construction.

Myth 2: Retrying costs nothing

Every retry re-sends the same prompt and the same context. If you pay per token, the bill doubles. If tokens are free, you still pay in latency and duplicated effects.

So what does a retry actually cost you? Count attempts per task, not calls per minute. A one-line ledger answers it:

# calls.jsonl: one JSON object per attempt, tagged with a task id
jq -r '[.task, (.attempt|tostring), (.tokens|tostring)] | @tsv' calls.jsonl \
  | awk '{n[$1]++; t[$1]+=$3} END {for (k in n) printf "%s attempts=%d tokens=%d\n", k, n[k], t[k]}'

Run that after a bad night. If one task shows attempts=4, you have a retry loop, not a flaky network.

Myth 3: An idempotency key dedupes everything

An idempotency key is a contract, not a header. Both sides must honor it.

• The sender must reuse the same key on every retry. A fresh UUID per attempt dedupes nothing.
• The receiver must persist the key before performing the effect. After the effect is too late.
• That store must survive a restart of the process holding it.

Break any one of the three and you have decoration, not deduplication.

Myth 4: An in-memory list is a queue

A Python list is a data structure. Durability is a property you add on top.

Ask yourself where pending work lives when the process is killed at step 3 of 7. If the answer is "in RAM", the answer is "nowhere".

Use a claim table or an external queue. One row per job, one state column, one unique index.

Myth 5: Free or shared compute can hold your state

Treat any free or shared compute as evictable. Assume the process dies between every step.

That assumption is cheap to design around. Keep the process stateless. Keep the ledger, the keys, and the queue outside it. Memory becomes a cache, never a source of truth.

Where the free tier fits my workflow

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option, as stated by the operator. I have not verified quotas, hardware, or how long anything stays up, so I design as if it could vanish.

That makes the split simple for me. The model call does the thinking and holds no state. The small server runs the worker and the ledger. Every artifact that matters gets written somewhere I control.

I keep exactly one rule: nothing I cannot re-create lives only on free compute.

The harness: prove the duplicate yourself

Here is the smallest reproduction I know. It runs a stub endpoint that fails after performing its side effect, then retries once.

# retry_ledger.py  —  python retry_ledger.py
import json, sqlite3, threading, urllib.request, urllib.error
from http.server import BaseHTTPRequestHandler, HTTPServer

DB, EFFECTS = "ledger.db", "effects.log"

def reset():
    con = sqlite3.connect(DB)
    con.execute("CREATE TABLE IF NOT EXISTS seen(key TEXT PRIMARY KEY)")
    con.execute("DELETE FROM seen")
    con.commit(); con.close()
    open(EFFECTS, "w").close()

def claim(key):
    con = sqlite3.connect(DB)
    try:
        con.execute("INSERT INTO seen(key) VALUES (?)", (key,))
        con.commit()
        return True
    except sqlite3.IntegrityError:
        return False
    finally:
        con.close()

def effect(key):
    with open(EFFECTS, "a") as f:
        f.write(key + "\n")

class Stub(BaseHTTPRequestHandler):
    hits = {}
    def do_POST(self):
        raw = self.rfile.read(int(self.headers["Content-Length"]))
        body = json.loads(raw or b"{}")
        key, path = body["key"], self.path
        Stub.hits[path] = Stub.hits.get(path, 0) + 1
        if path == "/guarded" and not claim(key):
            self.send_response(200); self.end_headers(); return
        effect(key)
        code = 500 if Stub.hits[path] == 1 else 200
        self.send_response(code); self.end_headers()
    def log_message(self, *args):
        pass

def worker(url, key, attempts=3):
    for _ in range(attempts):
        req = urllib.request.Request(
            url,
            data=json.dumps({"key": key}).encode(),
            headers={"Content-Type": "application/json"},
        )
        try:
            with urllib.request.urlopen(req) as r:
                if r.status == 200:
                    return 200
        except urllib.error.HTTPError as e:
            if e.code == 200:
                return 200
    return None

if __name__ == "__main__":
    reset()
    server = HTTPServer(("127.0.0.1", 8099), Stub)
    threading.Thread(target=server.serve_forever, daemon=True).start()
    for path in ("/naive", "/guarded"):
        worker("http://127.0.0.1:8099" + path, "task-" + path.strip("/"))
    print(sum(1 for _ in open(EFFECTS)), "effects recorded")
    server.shutdown()

The stub fails on the first hit per path, after writing the effect. Retries reuse the same key, so the guarded path should dedupe.

Test plan

1. Run python retry_ledger.py. Expect 3 effects recorded.
2. Read effects.log. The task-naive key should appear twice.
3. Confirm task-guarded appears once. That is the deduplication working.
4. Delete ledger.db and re-run. The count should return to 3.
5. Swap /guarded for your own handler and repeat step 3.

These are predicted outcomes for a deterministic stub, not measured production numbers. Your handler is the real variable.

Decision table: which guard goes where

Failure mode	Guard	Where it lives	Cost if missing
Worker dies after the effect	Idempotency key plus unique index	Receiver's store	Duplicate invoices, emails, rows
Retry storm	Attempt counter plus backoff	Sender	Wasted tokens, rate-limit bans
Process eviction	External queue or claim table	Storage layer	Silently lost tasks
Double scheduling	Single-writer lock	Scheduler	Two workers, one job

Print it. Tape it to the monitor. It answers most 2am questions in ten seconds.

Limitations and who should not use this

• This is a design pattern, not a product guarantee. It says nothing about any provider's uptime or quotas.
• SQLite is fine for a demo and one worker. Use a real store for concurrent writers.
• If your effect is already idempotent by nature, a read-only cache lookup, you need none of this.
• If you cannot change the receiver, dedupe on your side with a local ledger and accept the extra state.
• Skip the guarded path entirely if duplicate work is harmless and cheap. Sometimes it is.

The harness is the point. Everything else is a decision about how much duplicate work you can tolerate.

So which of the five myths is still true in your codebase tonight? Run the harness against one real handler and find out.

If you want a place to run the generator and the worker for free while you test this, MonkeyCode's free model access and free server option are where I would start. Keep the ledger local anyway.