I was in the Killam library, headphones on, watching my homework helper “just retry once more.” It did. Then it did it again. My fake dropbox stored two copies of lab3, both marked accepted. The script below should print one accepted row for the timeout fixture. Mine printed two. How do you budget a retry loop when the first call might already have succeeded?
That is the whole case study. Not a tour of agents. Not a framework quickstart. One swallowed ack, one bad key, one expected printout.
Background
I had been treating retries like a personality trait. Slow response? Try again. Empty JSON? Try again. The helper sounded confident, so I let the loop keep walking. It felt like work. It was a leak.
A retry loop is a while-loop with hope glued on. If you cannot say when it must halt, you do not have a tool wrapper. You have a second submit hiding inside a timeout. I wanted a lab small enough to finish between classes, with a store I could dump to stdout and argue with.
People keep calling every retrying helper an “agent.” I wanted to feel the failure without the costume. The model was never the interesting part. The interesting part was the identity of the attempt.
Goal
Simulate a course dropbox that sometimes accepts a file and then fails to send the ack. Prove that a naive client double-submits. Then add two boring brakes: an idempotency key — a stable id that means “this is the same attempt, not a new lab” — and a hard attempt cap. If those two checks do not hold, the loop is not allowed to talk to anything, including a model.
Success for this lab is not a clever prompt. Success is a store with one row after a swallowed ack.
Prerequisites
You need Python 3.11 or newer and the standard library. I ran this on 3.11.9. No pip install. No API key for the core experiment. The dropbox is a dictionary. The “network” is a function that raises TimeoutError after it already wrote the record. If that sounds fake, good. Fake is how you see the bug before it hits a real assignment system.
Implementation
Save this as lab_dropbox.py. Read naive_retry first and predict the printout. Do not skip that. If you cannot predict fixture A, the rest of this post will feel like a magic trick instead of a receipt.
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
class Status(str, Enum):
PENDING = "pending"
ACCEPTED = "accepted"
REJECTED = "rejected"
@dataclass
class Receipt:
key: str
student: str
assignment: str
status: Status
writes: int
class Dropbox:
"""In-memory dropbox. The first ack can vanish after a successful write."""
def __init__(self, swallow_first_ack: bool = True) -> None:
self._rows: dict[str, Receipt] = {}
self._ack_calls = 0
self.swallow_first_ack = swallow_first_ack
def submit(self, key: str, student: str, assignment: str) -> str:
self._ack_calls += 1
if key in self._rows:
return self._rows[key].status.value
row = Receipt(
key=key,
student=student,
assignment=assignment,
status=Status.ACCEPTED,
writes=1,
)
self._rows[key] = row
if self.swallow_first_ack and self._ack_calls == 1:
raise TimeoutError("accepted, but the ack never came back")
return row.status.value
def dump(self) -> list[dict[str, object]]:
return [
{
"key": r.key,
"student": r.student,
"assignment": r.assignment,
"status": r.status.value,
"writes": r.writes,
}
for r in self._rows.values()
]
def naive_retry(box: Dropbox, student: str, assignment: str, attempts: int = 4) -> list[str]:
events: list[str] = []
for i in range(attempts):
# Broken on purpose: a new key every try looks like a new lab.
key = f"{student}:{assignment}:try{i}"
try:
events.append(box.submit(key, student, assignment))
break
except TimeoutError as exc:
events.append(f"timeout:{exc}")
return events
def budgeted_retry(
box: Dropbox,
student: str,
assignment: str,
*,
key: str,
max_attempts: int = 3,
) -> list[str]:
if not key.strip():
raise ValueError("empty idempotency key")
if max_attempts < 1:
raise ValueError("max_attempts must be >= 1")
events: list[str] = []
for i in range(max_attempts):
try:
events.append(box.submit(key, student, assignment))
break
except TimeoutError as exc:
events.append(f"timeout:{exc}")
if i == max_attempts - 1:
events.append("stop:budget_exhausted")
return events
def main() -> None:
print("=== fixture A: naive retry after a swallowed ack ===")
naive_box = Dropbox(swallow_first_ack=True)
naive_events = naive_retry(naive_box, "achen", "lab3")
print("events:", naive_events)
print("store:", naive_box.dump())
print("=== fixture B: same key, same budget ===")
budget_box = Dropbox(swallow_first_ack=True)
budget_events = budgeted_retry(
budget_box,
"achen",
"lab3",
key="achen:lab3:v1",
max_attempts=3,
)
print("events:", budget_events)
print("store:", budget_box.dump())
print("=== fixture C: error input, empty key ===")
try:
budgeted_retry(Dropbox(False), "achen", "lab3", key=" ")
except ValueError as exc:
print("caught:", exc)
if __name__ == "__main__":
main()
Run it from the same folder:
python3 lab_dropbox.py
Results
Expected output on 3.11, flattened the way Python prints these lists:
=== fixture A: naive retry after a swallowed ack ===
events: ['timeout:accepted, but the ack never came back', 'accepted']
store: [{'key': 'achen:lab3:try0', 'student': 'achen', 'assignment': 'lab3', 'status': 'accepted', 'writes': 1}, {'key': 'achen:lab3:try1', 'student': 'achen', 'assignment': 'lab3', 'status': 'accepted', 'writes': 1}]
=== fixture B: same key, same budget ===
events: ['timeout:accepted, but the ack never came back', 'accepted']
store: [{'key': 'achen:lab3:v1', 'student': 'achen', 'assignment': 'lab3', 'status': 'accepted', 'writes': 1}]
=== fixture C: error input, empty key ===
caught: empty idempotency key
Fixture A is the library incident. The first write worked. The ack died. The client invented a fresh key, so the server invented a fresh lab. Two accepted rows. Same student, same assignment, same night. Did you predict two rows, or did you predict one row and a brave retry?
Fixture B looks almost identical in the event log. That is the trap. The events still say timeout, then accepted. The store does not. One key. One write. The second call is not a second lab. It is a reprint of the first receipt.
Fixture C is the mean one I actually typed while “just testing.” An empty key is not a creative id. It is an error input. If your wrapper silently fills that gap with str(time.time()), you have rebuilt fixture A with extra confidence.
What actually broke
I had blamed the model. Slow models make slow acks, sure. But the duplicate was my key function. A loop that changes identity on every attempt cannot be safe, no matter how calm the rationale sounds.
Think of a bus ticket. If you lose the paper and buy another ticket with a new serial number, you now have two rides on the books. If you ask the clerk to reprint serial 4412, you still have one ride. Idempotency is the reprint. The attempt cap is the clerk refusing to stand there all night.
The model never had a chance to be the hero here. I did not even call one in the core test. That was the point. If your stop condition only exists in natural language, it does not exist. A dictionary will not applaud you. It will just grow a second row.
Where a shared box actually helped
I wanted a lab partner to hit the same dropbox without borrowing my laptop in Killam. Localhost dies when I close the lid. So I put this same in-memory server behind a tiny HTTP wrapper on MonkeyCode’s free server option, and I used the free model access only to draft the human-readable “retry or stop” notes that sit next to a receipt. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The model did not own the budget. If the notes disagreed with budgeted_retry, the Python function won. If you cannot describe that rule without a product, you should not add the product yet.
I am not going to quote a quota, a model name, or a speed number I did not measure. The useful part was boring: a URL my partner could curl while I was on the bus, plus a local test that still failed the same way on my machine. The remote box did not make retries honest. The key did.
Common mistakes from the first hour
I hashed student + assignment + time.time(). That key is unique forever, which is another way of saying it is useless. I also treated TimeoutError as REJECTED. A timeout is unknown. You may retry an unknown only with the same key. You may not invent a new lab because you got impatient.
I set max_attempts=99 “just in case.” That is not a budget. That is a shrug. For this lab, three is enough to see the swallowed ack and still halt. If three feels tight, your network story is bigger than this script, and you should stop pretending a while-loop will save the deadline.
Another miss: I logged only events, not the store. Fixture A and fixture B can print the same event shape and still disagree about reality. If your demo cannot dump the store, you are reviewing vibes.
What you should understand after running it
A retry is a second send of the same intent, not a second intent. If you cannot point to the key that makes those the same, you are duplicating work. A stop condition has two halves: a maximum number of sends, and a terminal receipt you are willing to trust. Timeout is not terminal. accepted is. An empty key is an error, not a missing feature.
If you wrap a model around homework tools, put this check outside the model. Ask the model for a sentence if you want. Do not ask it whether it already submitted. It will sound sure. The dictionary will not.
Limitations, and who should skip this
This dropbox forgets everything when the process dies. It is not your university’s real assignment system. It does not handle two clients racing with two different keys for the same human intent. It does not prove exactly-once delivery. It proves a smaller, meaner thing: naive retries duplicate, stable keys do not.
Do not use this as a graded submitter. Do not use it as evidence that “agents work.” Do not host even a toy dropbox with real student files on a shared server. If you need crash recovery, you need a disk and a write-ahead log, not a blog post. If you need a production queue, this script is the wrong object.
Skip this approach if you cannot run the local fixtures first. A remote URL will only hide the second row until someone grades it.
Extension
Add a close time. Pass closes_at as a Unix timestamp and refuse to retry when time.time() is past it, even if attempts remain. Then feed it a fixture where the first ack is swallowed at closes_at - 1 and the second attempt happens at closes_at + 1. Predict the store before you run it. If your loop still writes, your budget ignored the only clock that matters in a course: the deadline.
If you try the lab, tell me which fixture you failed to predict. I still flinch at fixture A.
Top comments (0)