I was in the Halifax lab after the last bus, staring at a classmate's terminal. The same prompt had been sent five times. Each response began with "Sure, that looks right." He pointed at the free endpoint and said the server was lying. Was it?
Here is the fixture I made him freeze before the sixth try.
[
{"name": "yes-man", "status": 200, "body": "Sure, that looks right.", "elapsed_ms": 640, "schema_ok": false}
]
The learning question is small. Before I spend another free model call, can I name who is at fault?
Background
Last week the group chat kept mixing three different failures into one sentence: "the AI is down." Sometimes the socket never opened. Sometimes the key was rejected. Sometimes the model answered in a cheerful paragraph when the assignment required a JSON object with a label field. Those are not the same bug. Retrying them with the same curl is like tapping a locked door because the lights flickered in the hallway.
I am an AI/CS student, not an SRE. I do not have a production dashboard, and I should not pretend a course project is one. What I do have is a habit of blaming the tool that is easiest to see. A free model and a free server make that habit cheaper, which is exactly why it gets worse. Cheap retries feel like science. They are often just noise.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. The only product claims I will use are the ones I was given for this draft: free model access, and a free server option. I am not quoting a token quota, a hardware spec, a duration, or a benchmark, because I have not verified those against a primary page in this write-up. If those numbers move, a blog post should not be the place you memorize them.
Goal
I wanted one tiny project that ends before the network begins. Feed it a local record of an attempt. Get back a blame receipt. The receipt may say retry once, or it may say stop and fix your own check.
The goal was not to grade the model. It was to stop me from using a free call as a fidget spinner. If the receipt says the failure is schema, a free server will not rescue me. If the receipt says transport, then one carefully logged call on a free endpoint is a fair next step. That is the whole decision.
What you need
Python 3.11 or newer, so int | None is valid syntax. No pip install. No API key in the script. If python3 --version prints something below 3.11, stop and use the lab machine rather than "fixing" the file with a random backport. The run below is a local exercise. I am not reporting a timed shootout against a live host.
Implementation
Save this as blame_receipt.py. It reads a JSON list from stdin and writes receipts to stdout. schema_ok is your judgment, not the model's. You set it after you try to parse the body. The script refuses to guess.
#!/usr/bin/env python3
"""Label a failed model attempt before you retry it.
Python 3.11+. Standard library only. No network calls.
"""
from __future__ import annotations
import json
import sys
from dataclasses import asdict, dataclass
@dataclass(frozen=True)
class Attempt:
name: str
status: int | None
body: str
elapsed_ms: int
schema_ok: bool
@dataclass(frozen=True)
class Receipt:
name: str
label: str
retry: bool
why: str
def label_attempt(attempt: Attempt) -> Receipt:
if attempt.status is None:
return Receipt(
attempt.name,
"transport",
True,
"No status means the request never landed. Retry once, then stop.",
)
if attempt.status in {401, 403}:
return Receipt(
attempt.name,
"auth",
False,
"A key or permission failed. The same call will fail the same way.",
)
if attempt.status == 429:
return Receipt(
attempt.name,
"rate",
True,
"The server asked you to wait. A blind retry is another complaint.",
)
if attempt.status >= 500:
return Receipt(
attempt.name,
"server",
True,
"The far side failed. One retry is a question. Five retries are a habit.",
)
if attempt.status != 200:
return Receipt(
attempt.name,
"client",
False,
"The request shape was rejected. Fix the call before you spend another token.",
)
if not attempt.body.strip():
return Receipt(
attempt.name,
"empty",
False,
"HTTP 200 with an empty body is a silent failure. Inspect the prompt.",
)
if not attempt.schema_ok:
return Receipt(
attempt.name,
"schema",
False,
"The model answered, but not in the shape you required.",
)
if attempt.elapsed_ms > 8000:
return Receipt(
attempt.name,
"slow",
False,
"It worked, slowly. A retry will not make the lesson clearer.",
)
return Receipt(
attempt.name,
"ok",
False,
"Nothing here is the server's fault. If the answer is wrong, fix your check.",
)
def main() -> int:
raw = json.load(sys.stdin)
if not isinstance(raw, list):
print("expected a JSON list of attempts", file=sys.stderr)
return 2
receipts = [label_attempt(Attempt(**item)) for item in raw]
json.dump([asdict(item) for item in receipts], sys.stdout, indent=2)
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())
The 8000 millisecond cut is a teaching threshold, not a measured SLO. I picked a number a student can feel. If your lab network is slower than that on a good day, change the constant and write down why. Do not treat it as a fact about any vendor.
The run
I saved four attempts as fixtures.json. They are invented records, not a log scraped from a live host. That is the point. You can diff them in git without leaking a key.
[
{
"name": "yes-man",
"status": 200,
"body": "Sure, that looks right.",
"elapsed_ms": 640,
"schema_ok": false
},
{
"name": "dropped",
"status": null,
"body": "",
"elapsed_ms": 30,
"schema_ok": false
},
{
"name": "overloaded",
"status": 503,
"body": "unavailable",
"elapsed_ms": 1200,
"schema_ok": false
},
{
"name": "locked",
"status": 401,
"body": "unauthorized",
"elapsed_ms": 80,
"schema_ok": false
}
]
python3 blame_receipt.py < fixtures.json
Expected output, which you can regenerate on any laptop with Python 3.11:
[
{
"name": "yes-man",
"label": "schema",
"retry": false,
"why": "The model answered, but not in the shape you required."
},
{
"name": "dropped",
"label": "transport",
"retry": true,
"why": "No status means the request never landed. Retry once, then stop."
},
{
"name": "overloaded",
"label": "server",
"retry": true,
"why": "The far side failed. One retry is a question. Five retries are a habit."
},
{
"name": "locked",
"label": "auth",
"retry": false,
"why": "A key or permission failed. The same call will fail the same way."
}
]
Read that slowly. The cheerful "Sure" is not a server outage. It is a schema miss. The dropped call is the only one where a free server even enters the story, and even then the receipt allows one retry, not a loop. The 503 is a question you may ask once. The 401 is a key problem. Asking a free model to "try harder" will not mint a new credential.
One bad input
What if the fixture itself is the bug? Pass an object instead of a list.
echo '{"name":"oops"}' | python3 blame_receipt.py
You should see expected a JSON list of attempts on stderr and exit code 2. A missing field is louder. Attempt(**item) raises TypeError because I did not swallow bad notes into a fake "server" label. If your lab journal is missing schema_ok, the script should crash. A crash is more honest than a receipt that blames the model for your incomplete row.
What changed in the lab
I did not time a vendor, and I will not pretend I did. The result I can stand behind is local and boring. Four fixtures, four labels, zero network calls, and one classmate who stopped looping curl. He rewrote the prompt to demand {"label":"...", "evidence":"..."} and only then considered a single confirmation call.
That is where free model access and a free server option actually help a student. Use them for the confirmation you can explain, not for the five retries you cannot. I would rather spend one deliberate call on a free server than five accidental ones that all return the same yes. If you already have another endpoint in the course VM, use that. The receipt does not care whose logo is on the other side of the socket.
Lessons
The analogy that stuck is a coat check, not a courtroom. You hand over the attempt, you get a tag, and you do not get a verdict on the whole industry. Transport, auth, rate, server, client, empty, schema, slow, ok. Nine tags. Most of my "the model is broken" nights were schema, or ok with a weak checker.
I keep making the same three mistakes. I set schema_ok to true because the prose "looked fine." I treat 429 as a reason to hammer. I hide a 401 inside a retry loop until the free access looks flaky. None of those are model-quality questions. They are bookkeeping. After this exercise you should be able to point at a failed call and say which tag it wears before you touch the network again.
Who should skip this
Do not use this script as an outage monitor, a security review, or a quota calculator. It never sees headers such as Retry-After, and it never opens a socket. If your job is streaming partial tokens, this receipt is the wrong tool. A timer on the stream is a different lab. If you need a published rate limit, read the vendor's current docs, not a student post. If you cannot explain how you set schema_ok, do not run the script and then argue with its label.
Free access is also a bad place to hide sloppy prompts. A free server does not make an untested parser serious. If your course forbids external endpoints, stay on the fixture file. The lesson does not require an account.
A small extension
Add a fifth fixture named slow-ok with status 200, a valid body, schema_ok true, and elapsed_ms 9000. Predict the label before you run it. I think you already know. Then break the prediction on purpose. What label should a 200 with the body {"label":"yes"} get if your parser accepts it and your rubric does not? That gap is the next project, and it is still local.
If you want to replay one fixture against a live model after the receipt says the shape is the bug, keep it to a single call and log the status beside the prompt. A free server is enough for that experiment. I am not asking you to migrate a project, and I am not claiming a permanent allowance. Predict the failing fixture first. The call can wait.
Top comments (0)