DEV Community: Charlie Zhu

The Doorman Workshop: Keeping a Model Key Off the Client

Charlie Zhu — Mon, 09 Nov 2026 20:40:32 +0000

Illustrative scenario. A workshop cohort of six people has a browser page that turns a meeting note into a one-line summary. The page calls the model directly, which means the key sits in a bundled JavaScript file. One participant opens devtools in ninety seconds and reads it aloud. The room goes quiet. That silence is the first exercise.

Free model access makes this mistake cheaper to make and more expensive to fix. The constraint is not the price of a token. It is that a leaked free key is shared capacity, and shared capacity has an owner. So the workshop teaches one narrow move: put a small server between the client and the model, and let that server enforce size, shape, and concurrency.

MonkeyCode is relevant here because the project offers free model access and a free server option, which is enough to rehearse the whole pattern without a budget conversation. Both are operator-supplied availability claims and both are the kind of thing that changes; check the current allowance and server terms on the project's own site before you plan a class around them, rather than trusting a figure quoted in any blog post, this one included.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why the free tier changes the design

A paid endpoint lets you be lazy. You retry on failure, you fan out ten parallel calls, you send the whole document because trimming feels like work. A free endpoint punishes all three, and it punishes them unevenly, because the two limits you will hit first are the ones nobody documents in advance: how long a cold instance takes to answer the first request, and what happens to your request when the shared pool is busy.

The workshop does not try to answer those questions. It teaches students to measure them once, write the numbers down, and design against the numbers they saw instead of the numbers they hoped for. That is the entire intellectual content. The rest is plumbing.

The 75-minute outline

Minutes 0-5, framing. Show the leaked-key story above. Ask nobody to confess. State the rule for the session: no client code ever holds a model credential, and no request from the internet reaches the model without passing a shape check first.

Minutes 5-15, glass box. Students call the model once from a throwaway script and log the wall-clock time and the size of the response. This is the only exercise that needs a real key, and it is deliberately the shortest. Anyone without a key runs the fake model described below and continues.

Minutes 15-30, move the key. Students take the illustrative doorman.py file at the end of this article and run it locally with DOORMAN_FAKE=1. Then they change one function, call_model, to hit the real endpoint using the request shape from the provider's current documentation. The point of the exercise is that only one function changes. If a student ends up editing the handler, their design is wrong and the room should discuss why.

Minutes 30-45, put a ceiling on it. Each student picks a maximum body size, a maximum note length, and an allow-list of two templates. Then they attack their own server: a 5 MB body, a template name that does not exist, a note that is 10,000 characters. All three must be refused before any model call happens. The check is timing, not logging. A refusal that takes twenty seconds is a refusal that already called the model.

Minutes 45-55, error taxonomy. The handler in this workshop returns exactly four codes, and students must argue for each one. The useful argument is always about retries, not about HTTP purity.

Server code	Meaning	Client should
422	shape rejected, note too long, unknown template	fix the request, never retry
413	body over the ceiling	shrink and resend once, then stop
429	both local slots busy	wait, then retry once with jitter
502	model call failed or timed out	retry with backoff, surface to the user after two failures

The tempting mistake is to collapse 429 and 502 into one "try again" path. They have different causes: one is your own admission control doing its job, the other is the upstream refusing or stalling. Students who merge them will spend the next week unable to tell which half of their system is broken.

Minutes 55-70, rehearse on the free server. Deploy the file, point a probe at it, and run concurrency 1, then 2, then 3. Record three columns per run: wall-clock for the batch, status codes seen, and whether the first request after idle was slower than the rest. Leave the cells blank in the shared document. Filled-in numbers from someone else's session are the most common way a workshop teaches the wrong lesson.

Concurrency	Wall clock	Statuses	First request slower?
1
2
3

Minutes 70-75, debrief. Each student states one thing their server refuses and one thing it still permits that worries them. Write the second list on the board. It is usually longer than expected and always more interesting.

The worked example

This file is illustrative and was not executed for this article. It uses only the Python standard library, so python doorman.py works on a bare machine, and it runs offline until a student edits one function. Replace the request body and response field names with whatever the provider's current documentation specifies; the placeholder comments mark the two places where guessing would be wrong.

# doorman.py - illustrative gateway for a workshop, not production code.
import json, os, threading, urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

MAX_BODY = 2_000
MAX_NOTE = 800
MAX_CONCURRENCY = 2

TEMPLATES = {
    "summary": "Summarise the note in one sentence. Note: {note}",
    "rewrite": "Rewrite the note for clarity without adding facts. Note: {note}",
}

slots = threading.BoundedSemaphore(MAX_CONCURRENCY)

def call_model(prompt: str) -> str:
    if os.environ.get("DOORMAN_FAKE") == "1":
        return "[fake] " + prompt.split("Note: ", 1)[-1][:60]
    # Two edits below: request field names and response field name.
    # Take both from the provider's current documentation.
    body = json.dumps({"input": prompt}).encode()          # verify
    req = urllib.request.Request(
        os.environ["MODEL_BASE_URL"], data=body, method="POST",
        headers={"content-type": "application/json",
                 "authorization": "Bearer " + os.environ["MODEL_API_KEY"]})
    with urllib.request.urlopen(req, timeout=20) as resp:
        return json.loads(resp.read())["output"]           # verify

class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        if self.path != "/v1/draft":
            return self.reply(404, {"error": "not_found"})
        length = int(self.headers.get("content-length") or 0)
        if length > MAX_BODY:
            return self.reply(413, {"error": "too_large"})
        try:
            payload = json.loads(self.rfile.read(length))
            name, note = payload["template"], payload["note"]
            if name not in TEMPLATES or not isinstance(note, str) or len(note) > MAX_NOTE:
                raise ValueError
        except Exception:
            return self.reply(422, {"error": "rejected"})
        if not slots.acquire(blocking=False):
            return self.reply(429, {"error": "busy"})
        try:
            return self.reply(200, {"text": call_model(TEMPLATES[name].format(note=note))})
        except Exception:
            return self.reply(502, {"error": "upstream"})
        finally:
            slots.release()

    def reply(self, code, obj):
        raw = json.dumps(obj).encode()
        self.send_response(code)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(raw)))
        self.end_headers()
        self.wfile.write(raw)

if __name__ == "__main__":
    ThreadingHTTPServer(("127.0.0.1", 8080), Handler).serve_forever()

A student checks it in two steps, both of which fail loudly if the shape check is missing rather than quietly costing a model call.

DOORMAN_FAKE=1 python doorman.py &
curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:8080/v1/draft \
  -d '{"template":"summary","note":"budget review moved to friday"}'   # expect 200
curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:8080/v1/draft \
  -d '{"template":"translate","note":"hello"}'                        # expect 422
curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:8080/v1/draft \
  -H 'content-type: application/json' --data-binary @5mb.json           # expect 413

The transaction boundary worth explaining out loud: a slot is held only around the model call, and it dies with the process. That is adequate for one instance and wrong for two. Anyone who deploys a second copy has two independent semaphores and a concurrency ceiling of four, which is exactly the kind of detail a workshop should surface rather than hide.

What fits on a small free server, and what does not

Workload	Fits	Reason
One operator drafting summaries on demand	yes	bursty, tolerant of a slow first request
A classroom of six running timed probes	yes	concurrency is bounded by the room, not by traffic
A public demo page anyone can find	no	strangers share your ceiling and your key
A nightly job that must finish by 06:00	probably not	no committed capacity, so the deadline is a wish
Anything with a latency promise in a contract	no	a free tier cannot back an SLA

Do not send data you would not hand to a stranger, and read the provider's current terms before pointing the gateway at real notes. The free option is for rehearsal, teaching, and side projects whose worst outcome is a slow page.

Who should skip this

Teams with a latency commitment, a compliance review, or more than a handful of concurrent users should build the same doorman against a paid endpoint with a real deployment story, because the gateway pattern transfers and the free tier does not. Anyone who needs to prove throughput under load should also skip it; one probe run on a shared pool proves nothing about Tuesday.

For everyone else, the file above is small enough to be read in one sitting, which is the actual requirement. If you want somewhere to rehearse exercise five before writing your own version, MonkeyCode's free server is a reasonable place to put the doorman for an afternoon and watch what the first request after idle actually does.

The Half-Sentence Workshop: Making Streamed Model Output Safe to Stop

Charlie Zhu — Sun, 08 Nov 2026 19:22:52 +0000

A support ticket arrived with a screenshot of a form that said status: complete. Underneath it sat a JSON blob that ended in "steps": [1, 2,. The retry counter read three. The database held half a record, and nothing in the pipeline had complained.

That is the failure this workshop teaches people to prevent. Streaming turns a document into a sequence of arrivals, and a sequence can be interrupted in more places than most code is written to survive. Ninety minutes, one file, no network required until the last exercise.

Why the happy path lies

Non-streaming calls give you one of two outcomes: a body or an error. Streaming gives you a body that may become an error halfway through, and the intermediate states look exactly like success to a parser that runs on every chunk. The instinct is to call json.loads inside the loop and catch the exception. That works until the prefix happens to be valid JSON on its own, at which point the loop commits garbage and moves on.

The fix is structural rather than clever. Separate the buffer from the commit, and make the commit conditional on a proof that the document is finished. The rest of the workshop is spent making that proof honest.

The 90-minute plan

Minutes	Block
0–10	Reproduce a cut with a scripted stream
10–30	Build the two-phase gate
30–50	Break the gate with prefixes that parse
50–70	Add abort and commit-once semantics
70–90	Swap in a live endpoint and re-run the suite

Exercise 1: cut the stream at every byte

The first artifact is a harness, not a client. It takes a finished payload and replays it in slices, stopping at a chosen offset to simulate a dropped connection or a user pressing stop.

# harness.py — no network, runs in under a second
import json

PAYLOAD = json.dumps({"title": "half", "steps": [1, 2, 3], "done": True})

def replay(payload: str, cut_at: int, size: int = 5):
    """Yield the payload in slices, then stop dead at cut_at."""
    for i in range(0, cut_at, size):
        yield payload[i:i + size]

Students run replay(PAYLOAD, 12) and print what a naive json.loads does with it. Some offsets raise. Others return a dictionary with missing keys and no error at all. The room usually goes quiet at the second case.

Exercise 2: buffer, then commit

Now the gate. Two methods matter: feed accumulates, finish decides. Until finish says yes, nothing downstream can see the document.

class StreamGate:
    def __init__(self, required):
        self._buf = []
        self._required = set(required)
        self._committed = None

    def feed(self, piece: str) -> None:
        if self._committed is not None:
            raise RuntimeError("feed after commit")
        self._buf.append(piece)

    def abort(self) -> tuple:
        self._buf.clear()
        return ("aborted", 0)

    def finish(self) -> tuple:
        raw = "".join(self._buf)
        try:
            doc, end = json.JSONDecoder().raw_decode(raw)
        except json.JSONDecodeError:
            return ("partial", len(raw))
        if end != len(raw):
            return ("partial", len(raw))
        if not self._required.issubset(doc):
            return ("partial", len(raw))
        self._committed = doc
        return ("ok", doc)

raw_decode returns the index where parsing stopped, which is the piece the naive version is missing. Requiring end == len(raw) rejects a valid prefix followed by anything else. Requiring the declared keys rejects the smaller class of prefixes that are both valid and complete-looking.

Exercise 3: try to fool it

Here is the assertion that carries the whole workshop. Walk every cut point, and the gate must report ok exactly once — at the full length.

def sweep(payload: str, required, size: int = 5) -> list:
    ok_at = []
    for cut in range(len(payload) + 1):
        gate = StreamGate(required)
        for piece in replay(payload, cut, size):
            gate.feed(piece)
        if gate.finish()[0] == "ok":
            ok_at.append(cut)
    return ok_at

assert sweep(PAYLOAD, ["title", "steps", "done"]) == [len(PAYLOAD)]

Then hand out a payload of a bare integer, "12345", and watch the assertion fail. Every prefix is valid JSON, and raw_decode will happily consume 123 and stop. This is the teaching moment: no parser can prove completeness from syntax alone, so the required-key check and an explicit terminator in the protocol are doing real work, not decoration.

Exercise 4: aborts are not retries

The gate needs to distinguish three endings, and conflating them causes duplicate writes. A partial buffer should be discarded and retried from scratch. An abort requested by the user should not be retried at all, because retrying a cancelled request burns tokens on work nobody wants.

The commit-once guard in feed is deliberately noisy — it raises rather than silently ignoring the second stream. During the exercise, students remove the guard and watch two concurrent streams interleave into a buffer that validates against nothing.

Exercise 5: real chunk boundaries

Everything above runs against a fake stream, which is the point: the suite is deterministic and each student can rerun it after the workshop. The final block swaps the fake for a live endpoint so people see how their provider actually slices tokens, which is usually neither per character nor per line.

# Pseudocode for the shape only. Adapt to the SDK you actually use.
def live_pieces(client, prompt):
    with client.stream(prompt) as events:
        for event in events:
            if event.text:
                yield event.text

The adapter is ten lines; the gate underneath does not change. That separation is what makes the workshop portable across providers, and it is the reason the last exercise is a swap rather than a rewrite.

This is where free model access earns its keep. MonkeyCode, as an open-source project, offers a free model tier and a free server option, which means a room of students can run the live exercise without each one provisioning a card or a personal key. The operator also advertises a large free token allowance; treat that figure as perishable, and check the project page on the morning you print the syllabus rather than copying a number out of a blog post.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The honest limitation is that a free tier is a shared resource with terms that can change. If your workshop depends on a specific model or a specific latency, pin it in advance and bring a recorded fallback. The suite runs offline either way, so a bad morning on the network costs you one exercise, not the session.

What this does not fix

Buffering the whole document defeats the purpose if your UI is supposed to paint tokens as they arrive. Everything here applies to the commit boundary, not to the rendering path; a chat surface can stream freely and still refuse to write to the database until finish agrees. Likewise, tool-call streaming usually carries its own completion signal, and you should use that instead of guessing from JSON shape.

Semantic truncation is out of scope too. A payload that parses cleanly but contains two of the four expected steps will pass the gate. Catching that needs business rules, not a parser, and pretending otherwise is how teams end up trusting a green test suite.

Skip this approach if your output is binary, if the provider already gives you a reliable terminal event you can trust, or if you need sub-second first paint and full buffering cannot meet it. For everyone else shipping a form, a job queue, or a database write behind a model call, the two-phase gate is cheap insurance.

Going home with something runnable

Keep the harness and the sweep assertion. The model is replaceable, the chunk sizes are arbitrary, and the payload is a toy — but the property under test is exactly the one that failed in that support ticket. A stream is finished when you can prove it is finished, and nothing else counts.

If you want to run the last exercise against real chunk boundaries instead of mine, the free tier is enough to try it with a handful of prompts.

The Roll Call Workshop: Telling Throttling Apart from an Outage

Charlie Zhu — Sat, 07 Nov 2026 18:06:33 +0000

Picture a nightly batch job that has been green for eleven nights. On the twelfth it dies at 03:10 with one line in the log: HTTP 429. The retry loop treats it like any other error, backs off politely, tries again, and drains the whole window. By morning the queue is behind and nobody can say whether the endpoint was busy, the key had expired, or the model name had been retired the week before.

To a caller that only asks if response.ok, those three failures are identical. That blindness is the subject of this workshop. It is not a question about answer quality — benchmarking covers that ground well — but about what the caller should do in the ninety seconds after an answer fails to arrive.

The frame is a roll call. At a fixed interval the caller checks in, records enough context to classify what came back, and applies one of three policies: retry, degrade, or stop. A hundred minutes, four exercises, and three artifacts a team keeps afterwards: a probe, a classifier with fixtures, and a decision table.

Why this drill, this week

The loudest AI threads on DEV this week were about organizations rather than syntax: whether teams are building the thing that replaces them, and whether a single internal presentation can move a company. Strip the framing away and both arguments land on the same engineering question. Can the pipeline still behave when the model endpoint does not?

Most teams answer that from memory instead of from data. They know the 429s are occasional and the timeouts are rare, because the only record is a log line that scrolled away. A roll call replaces the memory with a table.

0:00–0:10 — Four failure families

The opening ten minutes go to naming what can go wrong, because a caller cannot classify what it has never listed. Four families matter in practice: throttling, credentials, routing, and transport. Throttling arrives as 429 and usually carries Retry-After. Credentials arrive as 401 or 403 and never fix themselves. Routing arrives as 404, or as a 200-wrapped error body naming a model the account cannot reach. Transport is everything the socket layer reports — timeouts, DNS, TLS, resets — and it is the family most often misread as a provider outage.

Students write one sentence per family describing what their current code does about it. Those sentences are the first artifact, and they are usually humbling.

0:10–0:30 — A probe with no dependencies

Exercise one builds a single-shot probe using only the standard library, so nothing in the drill depends on a framework version. The request path below is illustrative; use whatever path the provider documents.

# canary.py — one request, one JSON line
import json, time, urllib.error, urllib.request

def probe(url, key, model, prompt='ping', timeout=20):
    body = json.dumps({
        'model': model,
        'messages': [{'role': 'user', 'content': prompt}],
        'max_tokens': 8,
    }).encode('utf-8')
    req = urllib.request.Request(url, data=body, method='POST', headers={
        'Content-Type': 'application/json',
        'Authorization': 'Bearer ' + key,
    })
    started = time.monotonic()
    rec = {'ts': time.time(), 'model': model}
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            rec['status'] = resp.status
            rec['retry_after'] = resp.headers.get('Retry-After')
            rec['raw'] = resp.read(512).decode('utf-8', 'replace')
    except urllib.error.HTTPError as err:
        rec['status'] = err.code
        rec['retry_after'] = err.headers.get('Retry-After') if err.headers else None
        rec['raw'] = err.read(512).decode('utf-8', 'replace')
    except Exception as err:
        rec['status'] = 0
        rec['retry_after'] = None
        rec['raw'] = type(err).__name__ + ': ' + str(err)
    rec['ms'] = round((time.monotonic() - started) * 1000, 1)
    return rec

The 512-byte clamp is deliberate. A probe that stores full completions becomes a data-retention problem on day two, and the first 512 bytes carry everything the classifier needs.

Run it once from the shell, and look at the raw record before writing any logic:

BASE_URL=... API_KEY=... MODEL=... python3 canary.py

0:30–0:50 — The classifier and its fixtures

Classification should be a pure function over the record, tested without a network. That constraint keeps the exercise fast and makes the result reviewable by someone who is not in the room.

def classify(rec):
    status = rec.get('status', 0)
    raw = (rec.get('raw') or '').lower()
    if status == 0:
        return 'transport'
    if status in (401, 403):
        return 'auth'
    if status == 429:
        return 'throttled'
    if status == 404 or 'unknown model' in raw:
        return 'routing'
    if 500 <= status < 600:
        return 'upstream'
    if status == 200:
        return 'ok'
    return 'unknown'

Fixtures come next, one short entry per family. These are constructed examples for the test, not measurements from any service:

FIXTURES = [
    ({'status': 429, 'retry_after': '12', 'raw': 'rate limit exceeded'}, 'throttled'),
    ({'status': 401, 'retry_after': None, 'raw': 'invalid api key'}, 'auth'),
    ({'status': 404, 'retry_after': None, 'raw': 'unknown model: legacy-v1'}, 'routing'),
    ({'status': 0,   'retry_after': None, 'raw': 'TimeoutError: timed out'}, 'transport'),
    ({'status': 200, 'retry_after': None, 'raw': 'pong'}, 'ok'),
]

for rec, expected in FIXTURES:
    assert classify(rec) == expected, (rec, classify(rec))

Work the first fixture by hand in front of the room. Status 429, a Retry-After of twelve seconds, an error string that adds no information — the function returns throttled, and the detail that matters is the twelve, because the gate will spend it. The second fixture is the one that changes behaviour: auth is a permanent class, and a caller that backs off on a 401 has only made the incident slower.

The moment worth protecting is when someone adds a sixth family because their own log shows a shape the classifier calls unknown. That is the drill working as intended.

0:50–1:10 — A gate and a written policy

The gate is a token bucket with jitter, and its only interesting feature is that the server can overrule it.

import random, time

class Gate:
    def __init__(self, per_minute, burst=3):
        self.rate = per_minute / 60.0
        self.burst = burst
        self.tokens = float(burst)
        self.last = time.monotonic()

    def _refill(self):
        now = time.monotonic()
        self.tokens = min(self.burst, self.tokens + (now - self.last) * self.rate)
        self.last = now

    def wait(self, retry_after=None):
        self._refill()
        if self.tokens >= 1:
            self.tokens -= 1
            return 0.0
        delay = (1 - self.tokens) / self.rate
        if retry_after:
            delay = max(delay, float(retry_after))
        delay += random.uniform(0, 0.5)
        time.sleep(delay)
        return delay

Then the table, which is the artifact most teams actually keep after the workshop:

Class	Retry?	First action	Give up after
throttled	yes	wait `Retry-After`, then jitter	3 attempts, then degrade
auth	never	stop, page the key owner	immediate
routing	never	fail fast, check config	immediate
upstream	yes	backoff from 1s, double each time	5 attempts, then degrade
transport	yes	retry with doubled timeout	2 attempts
unknown	once	record everything, then stop	1 attempt

Degrade gets defined before the exercise ends: serve the last known good answer, or return an explicit unavailable marker. Both beat a stack trace at 03:10.

1:10–1:40 — Leaving the timer running

The last exercise moves the probe off a laptop. A dozen characters of cron is enough, and a systemd timer works just as well:

*/15 * * * * cd /srv/rollcall && /usr/bin/python3 canary.py >> roll-call.jsonl

Two practical blockers stop this step in most workshops: an API key, and somewhere to leave a timer running. The operator of MonkeyCode states that the project offers free model access and a free server option, so students without a paid key can run the whole drill and let the canary accumulate a day of records. Treat both as operator-supplied availability claims rather than guarantees, and read the current allowances, model availability, and terms in the project's own documentation before designing anything around them — free-tier numbers move.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The report at the end is deliberately dull:

import json, sys
from collections import Counter

rows = [json.loads(line) for line in open(sys.argv[1])]
print(Counter(classify(r) for r in rows).most_common())

A day of output turns the endpoint is flaky into 41 throttled, 3 transport, 0 auth — a sentence someone can argue with.

1:40–1:50 — Debrief, and what the numbers cannot say

Four questions close the session. Did any class appear that the table does not cover? Did any retry make things worse? Would the caller have survived the slot using the cached answer it already had? And which of the six rows does nobody on the team own?

The honest limits matter as much as the artifact. A fifteen-minute interval misses short bursts and can misread a provider's rolling window as a pattern. A single key cannot separate the account is throttled from the endpoint is throttled. Free-tier behaviour is not production behaviour, so a table built there should be re-run before anyone trusts it for capacity planning. Nothing in this drill measures answer quality, and no amount of classification will tell you whether the model is good enough for the task.

Skip the drill if you are choosing a model, tuning prompts, or load-testing — the first two need an evaluation harness and the third needs concurrency this probe deliberately avoids. Skip it too if your pipeline already emits per-class error metrics with a named owner for each class, because the workshop exists to create that instrumentation rather than replace it.

For a team that has none of it, a hundred minutes buys the difference between a 429 that pages nobody and a 429 that means something. If the outline is useful, point it at MonkeyCode's free tier, leave the timer on overnight, and read the table in the morning — your own rows will teach more than this page can.

The Twin Bench Workshop: Proving a Self-Hosted Model Can Take Over

Charlie Zhu — Fri, 06 Nov 2026 16:39:10 +0000

A prototype passes demo on Thursday. On Monday at 02:40 the batch job that feeds the morning report starts collecting retries, and the queue drains at half speed because the hosted endpoint is throttling. The obvious fix is to point the client at a self-hosted server and move on. The non-obvious part is proving that nothing quietly regressed in the three days since anyone last read a raw response.

This is the workshop I would run for that Monday-morning team. It takes ninety minutes, produces one reusable artifact, and — done properly — answers a question that a vibe check cannot: does the second bench pass the same cases as the first, and where does it fail?

Why a second bench is not a diff

Two endpoints that speak the same chat-completions dialect are not interchangeable. Prompt templates get rewritten by the server, stop sequences land differently, JSON modes are optional in practice, and a tokenizer boundary can turn a clean extraction into a trailing explanation. Swapping the base URL is a five-line change. Trusting the result is a different project.

MonkeyCode's free model access and free server option make the cheap half of this genuinely cheap: both benches can exist before lunch, so the workshop can spend its budget on adjudication instead of provisioning. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The ninety minutes

The first ten minutes are framing only — write the swap decision on a whiteboard and leave it there. From minute ten to twenty-five, the room builds a frozen golden set. The next twenty minutes go to the harness, the twenty after that to hard-fail checks, and the last twenty-five to running both benches and writing the verdict card. Nobody is allowed to look at model output during the first forty-five minutes; that is the discipline that keeps the bench honest.

Exercise 1: freeze twelve cases (15 min)

Twelve is the right number for a workshop because everyone can read all twelve out loud. Pull them from the existing test suite and the bug tracker, not from imagination — the cases that bit you in production are the only ones with predictive value. Cover schema-following, numeric extraction, refusal boundaries, one long-context needle, one unicode-and-code-fence case, and at least one deliberately ambiguous prompt.

golden.jsonl, one object per line:

{"id":"amount-01","kind":"number","prompt":"Extract the total from: 'Widgets x 12 @ 103.375 = 1,240.50'","expect":"1240.50"}
{"id":"schema-03","kind":"json","prompt":"Return JSON with keys id and status for ticket 9912, status closed.","required":["id","status"]}

The kind field matters more than it looks. It decides which hard check runs, and it stops the room from arguing about scoring before anyone has collected data.

Exercise 2: one harness, two output files (20 min)

Resist the temptation to write two scripts. One script, two environment variables, two JSONL files — that symmetry is what makes the later diff readable.

import hashlib, json, os, time, urllib.request

BASE = os.environ["BENCH_BASE_URL"]
KEY = os.environ.get("BENCH_API_KEY", "not-needed")
OUT = os.environ["BENCH_OUT"]  # bench-hosted.jsonl or bench-server.jsonl

def call(prompt: str) -> dict:
    body = json.dumps({
        "model": os.environ["BENCH_MODEL"],
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0,
    }).encode()
    req = urllib.request.Request(
        BASE.rstrip("/") + "/v1/chat/completions",
        data=body,
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {KEY}"},
    )
    t0 = time.perf_counter()
    with urllib.request.urlopen(req, timeout=120) as r:
        payload = json.loads(r.read())
    choice = payload["choices"][0]
    return {
        "text": choice["message"]["content"],
        "finish_reason": choice.get("finish_reason"),
        "latency_ms": round((time.perf_counter() - t0) * 1000),
        "usage": payload.get("usage", {}),
    }

with open(OUT, "w", encoding="utf-8") as sink:
    for line in open("golden.jsonl", encoding="utf-8"):
        case = json.loads(line)
        rec = {"id": case["id"], "kind": case["kind"],
               "prompt_sha": hashlib.sha256(case["prompt"].encode()).hexdigest()[:16]}
        rec.update(call(case["prompt"]))
        sink.write(json.dumps(rec, ensure_ascii=False) + "\n")

Two details earn their keep. The prompt hash proves both benches saw the same input even if someone edits the golden set mid-run. And temperature: 0 is a request, not a guarantee — write that on the whiteboard, because a self-hosted stack may sample differently and the room needs to expect that.

Exercise 3: hard checks before fuzzy ones (20 min)

The most common workshop mistake is reaching for similarity scoring first. A hard check is a veto; fuzzy scoring only ranks cases that already pass. Keep them in separate files so nobody can soften a failure by accident.

import json, re
from decimal import Decimal

def normalize_amount(text: str):
    m = re.search(r"-?\d[\d,]*\.\d{2}", text)
    return str(Decimal(m.group(0).replace(",", ""))) if m else None

def check_number(case, rec):
    got = normalize_amount(rec["text"])
    want = str(Decimal(case["expect"]))
    return ("PASS" if got == want else "FAIL", got, want)

def check_json(case, rec):
    try:
        obj = json.loads(rec["text"])
    except json.JSONDecodeError as e:
        return ("FAIL", f"unparseable: {e.msg}", None)
    missing = [k for k in case["required"] if k not in obj]
    return ("PASS", None, None) if not missing else ("FAIL", f"missing {missing}", None)

The normalizer in check_number is the worked example worth slowing down for. One bench returns 1,240.50 and the other returns 1240.50; compared as strings they look like a regression, compared as Decimal they are the same number. Half the "failures" in a first parity run are bench bugs, not model bugs — so before the room blames the server, it must prove the checker can pass the hosted output.

Then run both benches and diff:

BENCH_BASE_URL=https://hosted.example/v1 BENCH_MODEL=free-tier BENCH_OUT=bench-hosted.jsonl python run_bench.py
BENCH_BASE_URL=http://localhost:8000 BENCH_MODEL=local BENCH_OUT=bench-server.jsonl python run_bench.py
python checks.py bench-hosted.jsonl bench-server.jsonl --fail-fast

Exercise 4: the verdict card (25 min)

Sort every case into one of three buckets and write them down, not into a chat thread.

Signal	Verdict	Action
Hard check fails on either bench	Veto	do not swap
Pass sets match, latency 2x+ worse	Acceptable	revisit with batching
Free-text drift only	Note	human rubric, not code

The card is one page and lives next to the harness: which cases passed on both, which failed on both (pre-existing — do not blame the swap), which failed only after the swap, the p50 latency of each run, and the finish-reason histogram. Anything that only drifted in wording goes to a human reviewer with three examples, because a script that scores prose will score the wrong thing.

Limitations and who should skip this

Twelve cases are a regression net, not a benchmark. They tell you whether your known failure modes survived the swap; they say nothing about capability in general, and a clean parity run is not evidence that one model matches another. Free access and free server capacity are availability claims at a moment in time, so re-run the harness rather than trusting last month's card. Self-hosting also moves operations onto you — cold starts, model download time, and whatever hardware you actually got — and a shared free server is not a place to assume throughput.

Skip this workshop if you need a throughput guarantee today, or if your product output is subjective prose that a human reads anyway; for that, three reviewers and a rubric are cheaper than a harness. Skip it too if the swap is a temporary experiment you will revert on Friday.

For everyone else, the order is the lesson: freeze the cases, build the symmetric harness, let hard checks veto, and only then look at the output. If you want both benches running before you commit to any provisioning, MonkeyCode's free model access and free server option are enough to get from empty repo to two JSONL files in an afternoon.

The Fault Injection Workshop: Teaching a Model Call to Fail on Purpose

Charlie Zhu — Thu, 05 Nov 2026 15:17:27 +0000

The Friday demo

Picture a review session in a room with twelve laptops. One student's assistant has answered correctly all week. On demo day the endpoint returns a body that stops mid-sentence, response.json() raises, and the try/except that was wrapped around the network call catches nothing at all. The demo ends early, and the lesson arrives a week late and in public.

That failure is cheap to manufacture. It is a truncation, it takes one line of code, and it needs no flaky provider — only a proxy the students control.

The outline below is a teaching plan, not a report from a delivered class. All code here was written for this article and should be run once by the instructor before anyone teaches from it.

The skill being taught is naming, not handling

"Handle errors" is not a lesson; it is a shrug. Naming them is a lesson, because the name decides the recovery. A rate limit wants a wait. A truncated body wants a re-ask. A 500 wants one bounded retry. A stalled stream wants a shorter cap and a log line. A student who cannot tell a 429 from a truncated 200 will apply the wrong recovery with perfect confidence.

So the exercise gives students a proxy that breaks responses on demand, and asks them to classify what came back before they fix anything. Validation of the schema is a different exercise; here the only deliverable is a label plus an elapsed time.

The artifact: a proxy that corrupts responses on request

Sixty lines, standard library only, no dependencies to install on lab machines. It forwards JSON POSTs upstream and then damages the reply according to an X-Fault header.

#!/usr/bin/env python3
# faultproxy.py — forward model calls, then break them on demand
import http.server
import os
import sys
import time
import urllib.error
import urllib.request

UPSTREAM = os.environ.get("UPSTREAM", "http://127.0.0.1:9000").rstrip("/")
DEFAULT_FAULT = os.environ.get("FAULT", "none")
PORT = int(os.environ.get("PORT", "8787"))


class Handler(http.server.BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def do_POST(self):
        fault = self.headers.get("X-Fault") or DEFAULT_FAULT
        body = self.rfile.read(int(self.headers.get("Content-Length") or 0))

        if fault == "ratelimit":
            return self._send(429, b'{"error": "rate_limited"}', {"Retry-After": "2"})
        if fault == "stall":
            time.sleep(30)  # longer than any sane client timeout
            return self._send(504, b'{"error": "stalled"}')

        req = urllib.request.Request(
            UPSTREAM + self.path,
            data=body,
            headers={"Content-Type": "application/json"},
        )
        try:
            with urllib.request.urlopen(req, timeout=20) as resp:
                payload = resp.read()
        except urllib.error.HTTPError as exc:
            return self._send(exc.code, exc.read())
        except (urllib.error.URLError, TimeoutError):
            return self._send(502, b'{"error": "upstream_unreachable"}')

        if fault == "truncate":
            payload = payload[: max(1, len(payload) // 3)]
        elif fault == "quotes":
            payload = payload.replace(b'"', b"'", 3)
        self._send(200, payload)

    def _send(self, status, payload, extra=None):
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(payload)))
        for key, value in (extra or {}).items():
            self.send_header(key, value)
        self.end_headers()
        self.wfile.write(payload)

    def log_message(self, fmt, *args):
        sys.stderr.write("proxy: " + fmt % args + "\n")


if __name__ == "__main__":
    print(f"faultproxy :{PORT} fault={DEFAULT_FAULT} upstream={UPSTREAM}", file=sys.stderr)
    http.server.ThreadingHTTPServer(("127.0.0.1", PORT), Handler).serve_forever()

Three design choices carry the lesson. The fault arrives by header, so nobody restarts a process between runs. truncate returns HTTP 200 with a short body, which is the point: the status code is not the health check. stall sleeps for thirty seconds, longer than the client cap, so students measure a timeout instead of reading about one.

The student client: one call, one label

#!/usr/bin/env python3
# classify.py — send one call and name the outcome
import json
import socket
import sys
import time
import urllib.error
import urllib.request

URL = sys.argv[1] if len(sys.argv) > 1 else "http://127.0.0.1:8787/v1/chat/completions"
FAULT = sys.argv[2] if len(sys.argv) > 2 else None
BODY = {"messages": [{"role": "user", "content": "reply with the single word: ready"}]}


def call(timeout=8.0):
    headers = {"Content-Type": "application/json"}
    if FAULT:
        headers["X-Fault"] = FAULT
    req = urllib.request.Request(URL, data=json.dumps(BODY).encode(), headers=headers)
    started = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            status, raw = resp.status, resp.read()
    except urllib.error.HTTPError as exc:
        return {"class": "http_error", "status": exc.code, "ms": _ms(started)}
    except (socket.timeout, TimeoutError):
        return {"class": "timeout", "ms": _ms(started)}
    except urllib.error.URLError as exc:
        return {"class": "transport", "detail": str(exc.reason), "ms": _ms(started)}

    try:
        json.loads(raw)
    except json.JSONDecodeError as exc:
        return {"class": "bad_json", "status": status, "bytes": len(raw),
                "detail": exc.msg, "pos": exc.pos, "ms": _ms(started)}
    return {"class": "ok", "status": status, "bytes": len(raw), "ms": _ms(started)}


def _ms(started):
    return round((time.monotonic() - started) * 1000)


if __name__ == "__main__":
    print(json.dumps(call(), indent=2))

A student can also drive the proxy with curl, which is useful for students who want to see the raw bytes first:

curl -s -H 'X-Fault: truncate' -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"say ready"}]}' \
  http://127.0.0.1:8787/v1/chat/completions | head -c 200

The run sheet

Ninety minutes, pairs at one laptop, instructor circulating.

Minutes	What happens	What the instructor checks
0–10	Baseline call with `X-Fault: none` through the proxy	every laptop prints `"class": "ok"`
10–25	Students write a prediction, then run `truncate` and `quotes`	predictions exist before the run
25–40	`ratelimit` and `stall` with the client cap left at eight seconds	elapsed time comes from measurement
40–60	Each pair adds one new fault to the proxy	their fault breaks a neighbouring client
60–75	Pairs ask which faults a plain retry would fix	answers name the fault, not the fix
75–90	The decision table gets filled from the students' own output	table is evidence, not lecture notes

The prediction step is not decoration. A student who guesses "it will throw an exception" and then sees "class": "bad_json", "status": 200 has learned something a slide cannot teach.

A worked example for the instructor

Start the proxy against any endpoint that speaks JSON, then run the baseline:

UPSTREAM=https://your-endpoint.example python3 faultproxy.py        # terminal 1
python3 classify.py http://127.0.0.1:8787/v1/chat/completions none # terminal 2

The result is "class": "ok" with a byte count worth writing on the board, because every later run is compared against it. Now send the truncation:

python3 classify.py http://127.0.0.1:8787/v1/chat/completions truncate

The reply is "class": "bad_json" with "status": 200, a byte count near a third of the baseline, and a detail that usually reads as an unterminated string or an unexpected end of input. That single result is the whole lesson: the transport succeeded, the payload did not. Run quotes next and the byte count barely moves while the parse still fails, which is how students learn to read bytes and detail together instead of trusting either alone.

Finish on stall. The client returns in roughly eight seconds with "class": "timeout", and the proxy keeps sleeping behind it — a cheap demonstration that an abandoned request does not stop work on the other side.

Where a free option changes the lesson

The exercise needs two things that budget paperwork usually blocks: a model endpoint every student can hit, and one machine that can host the proxy so twelve laptops do not each run their own copy. MonkeyCode's free model access and free server option cover exactly those two needs, which is why the workshop can be run without a card or a quota meeting.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Limits and quotas on a free tier move, so check the project page on the morning you plan a session and keep a fallback endpoint configured in UPSTREAM. If you host the proxy on a shared machine, bind it to a private address or put an access rule in front of it; an open proxy that forwards credentials is a liability, not a teaching aid.

The debrief question that does the work

Ask which faults a blind retry would have fixed. Rate limits and a single upstream 5xx qualify, with a wait and a cap. A truncation never does — retrying a request that produced half a body tends to produce another half a body, and the student who writes except Exception: retry() has just paid twice for the same corruption. That question, answered out loud by the pairs themselves, is worth more than any summary slide at the end.

Limits, and who should skip this

This is a teaching proxy, not a chaos platform. It binds to loopback by default, it does not emulate DNS failures, TLS handshakes, or slow token-by-token streaming, and its faults are deterministic rather than statistically realistic. Anyone who needs production-grade failure testing should reach for established fault-injection tooling instead.

Skip the workshop if the group has never written an HTTP client, and absolutely skip pointing the proxy at an endpoint you do not own or have permission to test. Also skip it if the goal is load testing; thirty seconds of sleeping threads is not a benchmark.

For a classroom-sized first run, the free model access and the free server option are a reasonable place to start, with the current limits checked on the project page before the lesson is planned around them.

The Cassette Workshop: Making Free Model Calls Rerunnable in Two Hours

Charlie Zhu — Tue, 27 Oct 2026 12:44:53 +0000

The Cassette Workshop: Making Free Model Calls Rerunnable in Two Hours

A room of fourteen people, one shared endpoint, and a two-hour slot. By minute forty, half the room has results that disagree with the other half, nobody can say which call produced which row, and the person who owns the key is watching a dashboard instead of the projector.

The failure is not the model. It is that the workshop never separated two different activities — spending a call, and reading a result — so both became unrepeatable at the same time. This outline fixes that with one small artifact: a cassette layer that records each request/response pair to disk and replays it forever after. Students leave with a harness they can rerun on a laptop, in CI, or on a free box, without asking anyone for a key.

Every block below has a time budget, a deliverable, and a failure the instructor should expect. The code is short enough to type live.

0:00–0:12 — Put the failure on the table

Open with a deliberately bad run, not a slide. Take three prompts, call the model once each with temperature=0, save the raw JSON to live/, and print the outputs on the projector. Then run the same three prompts again and show the second folder. Two files differ, or one request times out, and the room now has a concrete reason to care about replay.

Ask nobody to explain. Just write the two timestamps on the whiteboard and move on; the discomfort does the teaching. The point students should absorb is that a model call is an event with a time, a cost, and a shape, and events need logs before they need opinions.

0:12–0:35 — Exercise one: the recorder

The first deliverable is a single module that owns every outbound call. Nothing else in the workshop is allowed to import urllib or an SDK directly. That constraint is the whole exercise; if a student bypasses it, the second exercise will fail in a way that looks mysterious.

The code assumes only that the endpoint speaks the common chat-completions request shape. Which endpoint you point it at is an environment variable, so the room can mix providers without forking the file.

# cassette.py — record once, replay forever
import hashlib, json, os, pathlib, urllib.request

CASSETTE_DIR = pathlib.Path(os.environ.get("CASSETTE_DIR", "cassettes"))
MODE = os.environ.get("MODE", "replay")            # "record" | "replay"
BASE_URL = os.environ["PROVIDER_BASE_URL"].rstrip("/")
API_KEY = os.environ.get("PROVIDER_API_KEY", "")

def _key(model, messages, params):
    blob = json.dumps([model, messages, params], sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(blob.encode("utf-8")).hexdigest()[:16]

def call(model, messages, params=None):
    params = params or {}
    path = CASSETTE_DIR / f"{_key(model, messages, params)}.json"

    if MODE == "replay":
        if not path.exists():
            raise LookupError(f"missing cassette {path.name} — record it first")
        record = json.loads(path.read_text())
    else:
        body = json.dumps({"model": model, "messages": messages, **params}).encode()
        headers = {"Content-Type": "application/json"}
        if API_KEY:
            headers["Authorization"] = f"Bearer {API_KEY}"
        req = urllib.request.Request(f"{BASE_URL}/chat/completions", data=body, headers=headers)
        with urllib.request.urlopen(req, timeout=60) as resp:
            record = json.loads(resp.read())
        path.parent.mkdir(parents=True, exist_ok=True)
        path.write_text(json.dumps(record, indent=2, sort_keys=True))

    return record["choices"][0]["message"]["content"]

The key is the hash of the request, not a counter, which means a reordered prompt is a different cassette and an identical prompt is a cache hit. Students discover this when they edit one word and get a LookupError instead of a stale answer.

For a small group, the free model access offered by the MonkeyCode project is enough to record the whole exercise set once — the operator describes it as an open-source project with a free token allowance, currently stated as ten million tokens, alongside a free server option. Treat those numbers as a claim to verify on the project page on the day you teach, because allowances move. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

0:35–1:00 — Exercise two: a gate that exits non-zero

Replay is only half a harness. The second deliverable turns cassettes into a pass/fail signal, so the room can argue about a test result rather than a paragraph of prose. Give students five cases with an exact expected string and one temperature=0 parameter, and let the runner print misses without stopping at the first one.

# run_workshop.py
import json, os, sys
from cassette import call

cases = json.load(open("cases.json"))
misses = 0
for case in cases:
    out = call(os.environ["MODEL"], [{"role": "user", "content": case["prompt"]}],
               {"temperature": 0})
    if out.strip() != case["expect"].strip():
        misses += 1
        print(f"MISS {case['id']}\n  want: {case['expect'][:70]}\n  got : {out[:70]}")
print(f"{len(cases) - misses} pass / {misses} miss")
sys.exit(1 if misses else 0)

Run it with MODE=replay. It should be instant and it should pass. Then someone changes one expected string and the room watches the exit code go to 1 — that single integer is what makes the harness usable in a pipeline later.

Add a live-call ceiling before anyone records again. A module-level counter that raises when MAX_LIVE_CALLS is exceeded costs four lines and prevents the classic accident where a student wraps the recorder in a loop over five hundred prompts. The instructor should demo the raise, not describe it.

1:00–1:22 — Exercise three: put the replayer on a free box

A replay runner needs no GPU and no key, so it belongs somewhere cheap and always on. This is where the project's free server option is genuinely relevant: the students deploy the replayer, not the recorder, which means the box never holds a provider credential.

Ship .env with MODE=replay, MODEL, and PROVIDER_BASE_URL only — no PROVIDER_API_KEY. Then a oneshot unit plus a timer gives the room a scheduled, credential-free check.

# /etc/systemd/system/workshop-replay.service
[Unit]
Description=workshop replay runner
After=network-online.target

[Service]
Type=oneshot
User=workshop
WorkingDirectory=/srv/workshop
EnvironmentFile=/srv/workshop/.env
ExecStart=/srv/workshop/.venv/bin/python run_workshop.py

sudo systemctl daemon-reload
sudo systemctl enable --now workshop-replay.timer
journalctl -u workshop-replay.service -n 20 --no-pager

The deliverable is a journal line that says pass, produced by a machine that cannot spend money. Students who finish early should copy the cassettes/ directory into a second folder, point CASSETTE_DIR at it, and confirm the run is byte-identical.

1:22–1:42 — Exercise four: break it on purpose

Deliberate damage teaches more than a clean demo. Ask each pair to delete one cassette and rerun in replay mode. They get a LookupError naming a sixteen-character hash, which is exactly the information they need: the missing request is identified without reading any log.

Then have them record that one missing case with MODE=record, and run git diff --stat cassettes/. One file appears. That diff is the artifact the room will actually use in review — a prompt change becomes a reviewable file change instead of an invisible behavioral drift.

The last instruction in this block matters for honesty. Have students flip one prompt in cases.json, replay, and watch the result. Replay does not prove the new prompt is good; it proves the harness notices. Saying that out loud prevents the next hour from turning into a false sense of coverage.

1:42–2:00 — Debrief, and where the method breaks

Close by naming the limits rather than defending the design. Replay hides model drift by construction: a cassette recorded in October will keep returning October's answer in December, which is a feature for grading and a hazard for evaluation. Non-zero temperature breaks exact-match assertions, so this harness rewards constrained outputs and punishes free-form ones.

Cassettes also store prompts as plaintext. Any workshop running real customer text through the recorder is writing that text to disk under a hash that is not encryption. Strip, synthesize, or relocate the data before the first record run.

Finally, the availability assumptions are exactly that. Free tiers and free boxes change shape, and a class that depends on a live endpoint the morning of the session has a single point of failure. Record the cassettes a week early, keep them in the repository, and the two hours survive any outage.

Teams running production traffic, regulated data, or anything with a latency contract should not borrow this pattern wholesale. Neither should anyone who needs a stable, versioned evaluation of model quality — that needs a pinned snapshot and many samples, not one call per case. This workshop is for making a small claim reproducible in front of other people, which is a narrower and more useful goal than it sounds.

The cassettes are the souvenir. Everything else can be regenerated.

The Overnight Shift Workshop: Batch Jobs That Survive a Crash

Charlie Zhu — Mon, 26 Oct 2026 11:25:35 +0000

A reading group met on a Tuesday evening to look at a nightly job that labeled support tickets with a model. The job had run for three weeks without complaint, then a container restart landed mid-batch and the next morning's report showed forty-one tickets tagged twice, eleven not tagged at all, and one row where two different labels sat in the same column. Nobody had written a bug. The job simply had no memory of what it had already done, and neither did anyone watching it.

That evening turned into the workshop below. It is ninety minutes long, it needs one laptop and one model endpoint, and its whole point is that the process will die without warning. Students do not debug a broken script. They build a job that can be killed on purpose and resumed by retyping the same command.

Why a free tier forces this lesson

The account's operator runs the drill on MonkeyCode's free model access and its free server option, and the program currently advertises 10 million free tokens. Treat those numbers as program terms rather than guarantees; the terms page, not a blog post, is the source of truth for what is left on any given day. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server matters more than the tokens here. Cheap compute is usually preemptible in practice, so the honest design assumption is that the worker disappears at an arbitrary second. A script that only works when nothing interrupts it is not a script, it is a demo. The workshop makes that difference visible in twenty minutes of wall-clock time.

The 90-minute outline

The first ten minutes are framing. The instructor puts the duplicate-ticket story on the board and asks one question: what would the job have to write down so that a restart changed nothing? Students shout "the id", "a timestamp", "whether it finished" — and the board slowly fills with the shape of a ledger.

Minutes ten through forty are the build. Each student writes a SQLite table with a stable job key, a hash of the input, a state column, and an attempt counter. They then write a claim function and a finish function, and they wire a two-line model adapter. The rule for this stretch is that no one is allowed to call the model until the ledger round-trips on fake data.

Minutes forty through sixty-five are the drill. The instructor calls out kills. Half the room loses its worker at item nine, the other half at item thirty, and everyone restarts with the identical command. Success is boring: the count of finished rows equals the count of inputs, and no output appears twice.

The last twenty-five minutes are the post-mortem and one extension. Students add a deadline, then argue about which failures should be retried and which should be parked for a human. The argument is the lesson.

The worked example

The ledger is three functions and one table. Nothing here is clever, which is the point — students should be able to explain every line out loud.

# ledger.py
import sqlite3, json, time, pathlib

DB = pathlib.Path("shift.db")

def connect():
    con = sqlite3.connect(DB, isolation_level=None)
    con.execute("PRAGMA journal_mode=WAL")
    con.execute("""CREATE TABLE IF NOT EXISTS jobs (
        key        TEXT PRIMARY KEY,
        input_sha  TEXT NOT NULL,
        state      TEXT NOT NULL,   -- running | done | failed
        attempt    INTEGER NOT NULL DEFAULT 0,
        output     TEXT,
        updated_at REAL NOT NULL
    )""")
    return con

The job key is derived from the input, not from the loop counter and not from a UUID. That single choice is what makes the restart safe: a resumed run regenerates the same keys and collides with the rows already on disk.

def claim(con, key, sha):
    con.execute("BEGIN IMMEDIATE")
    row = con.execute("SELECT state FROM jobs WHERE key=?", (key,)).fetchone()
    if row and row[0] == "done":
        con.execute("COMMIT")
        return False
    con.execute(
        """INSERT INTO jobs (key, input_sha, state, attempt, updated_at)
           VALUES (?, ?, 'running', 1, ?)
           ON CONFLICT(key) DO UPDATE SET
             state='running', attempt=attempt+1, updated_at=excluded.updated_at""",
        (key, sha, time.time()),
    )
    con.execute("COMMIT")
    return True

def finish(con, key, output):
    con.execute(
        "UPDATE jobs SET state='done', output=?, updated_at=? WHERE key=?",
        (json.dumps(output), time.time(), key),
    )

The worker itself is a loop with a clock attached. The deadline is not decoration; without it a slow endpoint turns a batch into an all-night process that the free tier may reclaim halfway through the last item.

# worker.py — pseudocode shape; the model call is your adapter
DEADLINE_S = 45 * 60

def run(items, budget_s=DEADLINE_S):
    con, start = connect(), time.monotonic()
    for key, sha, prompt in items:
        if time.monotonic() - start > budget_s:
            print("deadline reached, stopping cleanly")
            break
        if not claim(con, key, sha):
            continue                        # already done, skip
        try:
            finish(con, key, call_model(prompt, timeout=60))
        except Exception as exc:
            con.execute("UPDATE jobs SET state='failed', updated_at=? WHERE key=?",
                        (time.time(), key))
            print("failed", key, type(exc).__name__)

call_model is an adapter students fill in against whatever endpoint and model id their own account page shows. The workshop deliberately never hardcodes a model name, because half the confusion in these sessions comes from copying a string out of a blog post that was accurate a month ago.

The run command and the resume command are the same command. That is the punchline:

python worker.py --input tickets.jsonl
# instructor kills it here, or you just close the laptop
python worker.py --input tickets.jsonl

How students verify the drill honestly

A batch job that "looks finished" is worthless. Two queries close the loop, and both belong in the post-mortem rather than the happy path.

sqlite3 shift.db "select state, count(*) from jobs group by state;"
sqlite3 shift.db "select count(*) from jobs where state='done' and output is null;"

The first shows running, done, and failed side by side, so an abandoned running row cannot hide behind a cheerful log line. The second catches the quiet failure mode where a model returns an empty string, the worker marks the row done, and nobody notices until a week of labels turns out to be blank. Adding a schema check on the parsed output — one small function, ten lines — is the natural extension exercise for faster students.

Limitations, and who should skip this

The pattern costs a write before every model call and a read after every restart, which is fine for overnight batches and wrong for anything a user is waiting on. It also assumes the model call is deterministic enough that retrying a failed row does not silently change its meaning; when outputs are consequential, the failed row should park for a human instead of retrying.

Skip this workshop if you already run Temporal, Celery with a proper result backend, or any queue with at-least-once delivery and idempotency keys built in — you would be hand-rolling a worse version of a tool you already have. Skip it too if the workload touches regulated data, since a free-tier server is the wrong place to discover your retention policy. And skip it if the batch is under a hundred items run once; a shell script and a forgiving attitude is honestly enough.

For everyone else, the value is not the code. It is the twenty minutes where the room watches a process die mid-sentence, restarts it with the same command, and sees the totals line up. If you want to run the drill with a group, the operator of this account uses MonkeyCode's free model access and free server option for it; read the current program terms first, then go kill your own worker on purpose and count the rows.

Tags: ai, python, tutorial, opensource

The Changeover Workshop: A Schema Gate for Free Model Output

Charlie Zhu — Tue, 20 Oct 2026 09:30:42 +0000

A rented room, eighteen laptops, one shared power strip. The exercise was an invoice extractor. It worked on three PDFs and then, on the fourth, returned a paragraph of cheerful prose where a total should have been. The student blamed the model. The model was fine. Nobody had written down what a finished answer looks like, so nothing could catch the moment the answer stopped being finished.

That is the whole premise of this workshop. Before students touch a free endpoint, they build a changeover gate: a contract that says what "done" means, a repair loop that gives the model one honest chance to fix itself, and an escalation path that fails loudly instead of quietly. The gate is boring. Boring is the point.

The session runs ninety minutes and produces one file students can rerun at home. MonkeyCode, which offers free model access and a free server option, is where most of the room sends its requests, because eighteen people cannot each need a credit card to learn a validation pattern. Those two availability claims come from the project operator; current limits belong on the project's own pages, not in a workshop handout.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The clock

Minutes	Segment	What students hold at the end
0–10	Framing: the fourth PDF	A one-sentence definition of a finished answer
10–25	Exercise 1: write the contract	A JSON Schema with five to seven fields, no more
25–45	Exercise 2: wire one call	A working `ask()` and a dry run that passes
45–65	Exercise 3: break it on purpose	Three mutations, three distinct error paths
65–80	Exercise 4: escalate and log	A shift report and a `rejects.jsonl`
80–90	Debrief: who should not use this	A named boundary for the technique

Exercise 1 is deliberately small. Students who write a schema for the whole invoice format spend the next hour debugging their own strictness instead of learning the gate. Five fields. invoice_id, currency, total_cents, line_items, and one item shape. Everything else is homework.

The artifact

One file, no framework, standard library plus jsonschema. The transport is left as an exercise on purpose, since request shapes drift and a workshop should not hard-code a shape it cannot verify on the day.

"""changeover.py — the workshop artifact.

Dry run, no network:  python changeover.py --dry-run --tasks 12
Live, after exercise 2: python changeover.py --tasks 12
"""

from __future__ import annotations

import json
import sys
import time
from pathlib import Path

import jsonschema

CONTRACT = {
    "type": "object",
    "required": ["invoice_id", "currency", "total_cents", "line_items"],
    "additionalProperties": False,
    "properties": {
        "invoice_id": {"type": "string", "minLength": 3},
        "currency": {"enum": ["USD", "EUR", "CNY"]},
        "total_cents": {"type": "integer", "minimum": 0},
        "line_items": {
            "type": "array",
            "minItems": 1,
            "items": {
                "type": "object",
                "required": ["sku", "qty", "unit_cents"],
                "additionalProperties": False,
                "properties": {
                    "sku": {"type": "string"},
                    "qty": {"type": "integer", "minimum": 1},
                    "unit_cents": {"type": "integer", "minimum": 0},
                },
            },
        },
    },
}

SYSTEM = (
    "You extract invoice data. Reply with one JSON object and nothing else. "
    "Money is integer cents. Never wrap the object in prose or code fences."
)

DRY_RUN = "--dry-run" in sys.argv
MAX_REPAIRS = 2

CANNED = [
    '{"invoice_id": "INV-1042", "currency": "USD", "total_cents": "12.00", '
    '"line_items": [{"sku": "A-1", "qty": 2, "unit_cents": 600}]}',
    '```

json\n{"invoice_id": "INV-1042", "currency": "USD", "total_cents": 1200, '
    '"line_items": [{"sku": "A-1", "qty": 2, "unit_cents": 600}]}\n

```',
]


def ask(prompt: str, attempt: int) -> str:
    if DRY_RUN:
        return CANNED[min(attempt, len(CANNED) - 1)]
    # Exercise 2: replace this with exactly one HTTP call.
    # Read the base URL, key and model name from the environment, send
    # SYSTEM + prompt, return the assistant text.
    raise SystemExit("live mode not wired yet: fill in ask() from the provider docs")


def strip_fence(text: str) -> str:
    t = text.strip()
    if t.startswith("```

"):
        t = t.split("\n", 1)[1] if "\n" in t else t
        t = t.rsplit("

```", 1)[0]
    return t.strip()


class GateFailure(RuntimeError):
    def __init__(self, reasons: list[str]) -> None:
        super().__init__("; ".join(reasons))
        self.reasons = reasons


def repair_note(reasons: list[str]) -> str:
    lines = ["", "Your previous reply was rejected by a JSON Schema validator."]
    lines += [f"- {r}" for r in reasons[-3:]]
    lines.append("Return the corrected object only. Integers, not strings.")
    return "\n".join(lines)


def extract(prompt: str) -> tuple[dict, int]:
    reasons: list[str] = []
    for attempt in range(MAX_REPAIRS + 1):
        text = ask(prompt + repair_note(reasons) if reasons else prompt, attempt)
        try:
            doc = json.loads(strip_fence(text))
            jsonschema.validate(doc, CONTRACT)
            return doc, attempt
        except json.JSONDecodeError as exc:
            reasons.append(f"not JSON: {exc.msg} at char {exc.pos}")
        except jsonschema.ValidationError as exc:
            path = "/".join(str(p) for p in exc.absolute_path) or "<root>"
            reasons.append(f"{path}: {exc.message}")
    raise GateFailure(reasons)


def tasks(count: int) -> list[str]:
    sample = "Invoice INV-1042, 2 x A-1 at 6.00, total 12.00 USD."
    return [sample] * count


def main() -> None:
    count = 12
    if "--tasks" in sys.argv:
        count = int(sys.argv[sys.argv.index("--tasks") + 1])
    shift = {"tasks": 0, "first_pass": 0, "repaired": 0, "escalated": 0}
    started = time.perf_counter()
    for prompt in tasks(count):
        shift["tasks"] += 1
        try:
            _, repairs = extract(prompt)
        except GateFailure as exc:
            shift["escalated"] += 1
            with Path("rejects.jsonl").open("a", encoding="utf-8") as fh:
                fh.write(json.dumps({"input": prompt, "reasons": exc.reasons}) + "\n")
            continue
        shift["first_pass" if repairs == 0 else "repaired"] += 1
    shift["seconds"] = round(time.perf_counter() - started, 2)
    print(json.dumps(shift))


if __name__ == "__main__":
    main()

The dry run prints {"tasks": 12, "first_pass": 0, "repaired": 12, "escalated": 0, "seconds": 0.0}. Students read that and immediately ask whether the model is bad. It is not a measurement. The canned replies are fixtures that prove the wiring, and the whole first-pass number is an artifact of the fixture list. Real numbers require real calls, and even then twelve tasks is a smoke test, not a benchmark.

Exercise 3: break it on purpose

Reading an error message is not the same as causing one. Students mutate a known-good document three ways and record which part of the contract fires.

python - <<'PY'
import json
import changeover as c
from jsonschema import ValidationError, validate

base = json.loads(c.strip_fence(c.CANNED[1]))
mutations = [
    ("drop currency", lambda d: d.pop("currency")),
    ("zero qty", lambda d: d["line_items"][0].__setitem__("qty", 0)),
    ("extra field", lambda d: d.__setitem__("tax_cents", 99)),
]
for name, mutate in mutations:
    doc = json.loads(json.dumps(base))
    mutate(doc)
    try:
        validate(doc, c.CONTRACT)
        print(f"{name:14} -> passed, which means the contract has a hole")
    except ValidationError as exc:
        path = "/".join(map(str, exc.absolute_path)) or "<root>"
        print(f"{name:14} -> {path}: {exc.message}")
PY

The third mutation is the teaching moment. additionalProperties: False turns a harmless extra field into a rejection, and students who left it out discover their gate was a suggestion rather than a gate. A schema with holes behaves like a turnstile with a gap at knee height: it looks like control and it is not.

Exercise 4: the shift report

Escalation is the part everyone skips. When the repair loop exhausts its budget, the gate raises, the input and the last three rejection reasons land in rejects.jsonl, and the loop keeps going. Nobody's batch dies because one document was hostile.

The rejects file is the deliverable worth keeping. Reading twenty of them tells a team more about its prompts than any percentage, because the reasons cluster: string money, missing currency, an item list that came back as a single object. Those clusters become contract edits or prompt edits, and the decision is made from evidence rather than from a hunch about which model is smart.

A word on what the free lane buys and does not buy. Free model access and the free server option make a room of eighteen viable, and the free server means no one spends the first hour installing anything. They do not buy throughput guarantees, and a workshop should never imply otherwise. Students who need a latency contract should read current terms before they design around them.

Who should not use this

Teams that need a guaranteed rate or a latency SLA should not put a free lane in the request path, gate or no gate. Anyone extracting values that move money, clear a person, or feed a regulator needs a human in the loop regardless of how green the shift report looks, because schema-valid is not the same as correct: total_cents: 1300 validates perfectly and can still be wrong.

The repair loop also costs calls. It doubles the worst case and triples it under MAX_REPAIRS = 2, which is why the budget is a named constant and not a hidden default. And anyone who cannot store raw rejected inputs should not run this at all, since the rejects file is where the learning lives.

Run the workshop once and the room leaves with three things: a contract they wrote, a failure they caused on purpose, and a file of rejects. The last one is the useful one. Check the project's own pages for what the free lane currently allows before you plan the next session around it.

Bench-Testing a Free Model Endpoint in 90 Minutes

Charlie Zhu — Mon, 19 Oct 2026 08:12:58 +0000

A nightly refactor job begins failing at 2 a.m., and nobody touched the code. The team had moved one summarization step onto a free model endpoint the week before, and the only thing that changed was the response shape and the latency tail. The details differ from team to team; the shape recurs. Free access answers whether a developer can try a model. It does not answer whether an automated loop can depend on one.

This is a workshop outline for that second question. It teaches participants to build a small probe, run it against whatever endpoint they actually have credentials for, and turn the results into a go/no-go decision. The whole thing fits in ninety minutes and produces an artifact the team keeps.

Why measure before wiring

Most fatigue around AI tooling is not fatigue with models. It is fatigue with unmeasured claims. A team adopts an endpoint because a demo looked fast, then discovers during an incident that timeouts arrive without a status code, that usage counters are sometimes missing, and that rate limits are signaled in a header nobody parsed.

The probe below does not benchmark a provider. It measures one account, one model, one network path, on one afternoon. That is a deliberately narrow claim, and it is the only claim the workshop defends.

The schedule

The first ten minutes are framing: the facilitator draws the failing-job story on a whiteboard and asks the room which property they would have wanted to know in advance. Latency, error taxonomy, and token accounting usually win.

Minutes ten through thirty are an inventory exercise. Participants list every place their own code calls a model, then mark which calls sit inside an automated loop rather than behind a human. Only the automated ones matter for this workshop.

Minutes thirty through fifty-five are the build. Two files, roughly ninety lines total, run from a terminal.

Minutes fifty-five through seventy are analysis: percentiles, error counts, and the awkward question of what happens when the usage object is absent. Minutes seventy through eighty-five turn the numbers into a decision with the table further down. The last five minutes are spent writing a stop rule into the repository, because a decision nobody wrote down is a preference.

Exercise 1: the probe

The probe sends the same fixed prompt twelve times, sequentially, and records one JSON line per run. Sequential matters. Concurrency looks impressive and destroys the signal, because a rate limiter will fold your latency distribution into a step function.

// probe.mjs — node probe.mjs > runs.jsonl
import { performance } from 'node:perf_hooks';

const BASE  = process.env.PROBE_BASE_URL;
const PATH  = process.env.PROBE_PATH ?? '/chat/completions';
const KEY   = process.env.PROBE_API_KEY;
const MODEL = process.env.PROBE_MODEL;
const N     = Number(process.env.PROBE_N ?? 12);

if (!BASE || !KEY || !MODEL) {
  console.error('set PROBE_BASE_URL, PROBE_API_KEY, PROBE_MODEL');
  process.exit(2);
}

The path above is the common OpenAI-compatible shape, not a guarantee. Participants confirm the base URL and path against the provider's current documentation before the first run, and write that link into the file header.

const PROMPT = [
  'Summarize this build log in exactly three bullet lines.',
  'No preamble. Do not invent line numbers.',
  'line 12: warning: unused import React',
  'line 44: error: TS2345 argument of type string',
  'line 91: warning: deprecated API call'
].join('\n');

async function once(i) {
  const started = performance.now();
  const ctrl = new AbortController();
  const timer = setTimeout(() => ctrl.abort(), 60_000);
  let attempts = 0, status = 'ok', http = 0, usage = null, text = '';

  while (attempts < 4) {
    attempts++;
    try {
      const res = await fetch(BASE + PATH, {
        method: 'POST',
        signal: ctrl.signal,
        headers: {
          'content-type': 'application/json',
          authorization: `Bearer ${KEY}`
        },
        body: JSON.stringify({
          model: MODEL,
          messages: [{ role: 'user', content: PROMPT }]
        })
      });
      http = res.status;
      if (res.status === 429 || res.status >= 500) {
        if (attempts < 4) {
          const ra = Number(res.headers.get('retry-after') ?? 0);
          const wait = ra > 0 ? ra * 1000 : 500 * 2 ** attempts + Math.random() * 250;
          await new Promise(r => setTimeout(r, wait));
          continue;
        }
        status = http === 429 ? 'rate_limited' : 'server_error';
        break;
      }
      if (!res.ok) { status = 'client_error'; break; }
      const body = await res.json();
      usage = body.usage ?? null;
      text = body.choices?.[0]?.message?.content ?? '';
      break;
    } catch (err) {
      status = err.name === 'AbortError' ? 'timeout' : 'network_error';
      if (attempts >= 4) break;
      await new Promise(r => setTimeout(r, 400 * 2 ** attempts));
    }
  }

  clearTimeout(timer);
  return {
    i, status, http, attempts,
    total_ms: Math.round(performance.now() - started),
    usage, chars: text.length, sample: text.slice(0, 120)
  };
}

for (let i = 0; i < N; i++) console.log(JSON.stringify(await once(i)));

Running it is one command. Participants export credentials into the shell rather than pasting them into the file, and the facilitator checks for that habit before the first execution.

PROBE_BASE_URL="$MC_BASE_URL" PROBE_PATH="/chat/completions" \
PROBE_API_KEY="$MC_API_KEY" PROBE_MODEL="$MC_MODEL" PROBE_N=12 \
node probe.mjs > runs.jsonl

Exercise 2: reading the numbers

The summarizer is short on purpose. Percentiles over twelve samples are descriptive, not inferential, and it should say so out loud when it prints.

// summarize.mjs — node summarize.mjs runs.jsonl
import { readFileSync } from 'node:fs';

const rows = readFileSync(process.argv[2], 'utf8')
  .trim().split('\n').map(l => JSON.parse(l));
const ok = rows.filter(r => r.status === 'ok');
const lat = ok.map(r => r.total_ms).sort((a, b) => a - b);
const q = p => lat.length ? lat[Math.floor(p * (lat.length - 1))] : NaN;
const usageSeen = ok.filter(r => r.usage).length;

console.log({
  runs: rows.length,
  ok: ok.length,
  errors: rows.length - ok.length,
  statuses: [...new Set(rows.map(r => r.status))],
  p50_ms: q(0.5),
  p95_ms: q(0.95),
  max_ms: lat.at(-1),
  usage_reported: `${usageSeen}/${ok.length}`
});

The interesting column is rarely the median. If usage_reported is not equal to ok, the accounting layer cannot be trusted, and any per-request cost model the team built becomes a guess. Participants who find that should stop tuning prompts and start reading the provider's terms.

Exercise 3: a collector that stays put

A laptop on office Wi-Fi is a bad measurement instrument, and a laptop on hotel Wi-Fi is worse. Participants who want the twelve runs to be comparable across weeks should place the collector somewhere stable.

MonkeyCode offers free model access and a free server option, which makes it a reasonable place to park this harness for a first pass: the probe runs from a fixed machine, writes runs.jsonl, and the team reads the file. Confirm the endpoint, the current limits, and the terms in the provider's documentation before the session rather than during it. No quota, hardware specification, or duration is assumed here, because those details change and a workshop should not teach stale numbers.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The same instruction applies to any free tier: the free server is a place to run a measurement, not a substitute for an availability guarantee. A team that needs one should be talking to a vendor, not reading a workshop outline.

Exercise 4: what the room usually gets wrong

One participant will want to add concurrency. Let them, in a separate file, and then compare. The step function that appears is the lesson. Another will want to tune the prompt until output quality looks good; that is a different workshop, and mixing it in here contaminates the latency data.

A third will ask whether twelve runs are enough. They are enough to catch a missing usage field, a 429 storm, or a two-second tail. They are not enough to estimate a p99, and the summarizer should print a warning when someone asks it to.

The decision table

The room fills this in together, then commits it next to the probe script.

Signal from the run	Reading	Action
p95 well under the loop's timeout, no errors, usage present	Predictable enough for a guarded trial	Wire behind a timeout and a kill switch, log every response
p50 fine, p95 close to or above the timeout	Tail will page someone at night	Add a timeout below the observed p95 and a fallback path first
`usage` missing on most successes	Cost and token budgets are unmeasurable	Do not build budget enforcement on this endpoint yet
Any 429s at concurrency one	Limits are tighter than the loop assumes	Batch, cache, or move the call out of the loop

Limitations and who should skip this

Twelve sequential calls from one machine measure one path at one moment. Provider behavior changes, free-tier conditions change, and a good afternoon proves nothing about next month. The probe also says nothing about output quality, data handling, or licensing, which are separate reviews with separate owners.

Teams under a service-level obligation, teams handling regulated data without a reviewed agreement, and teams planning production volume should not adopt this workflow as a substitute for procurement. The probe is a first filter, not a contract.

For groups that want to rerun the harness weekly without maintaining a box, standing the collector up on MonkeyCode's free server and pointing it at the free model access is the shortest path the organizers know — bring the current terms and read them in the room, out loud.

The Receiving Inspection Workshop: A Gate for AI-Written Code

Charlie Zhu — Fri, 16 Oct 2026 04:57:23 +0000

Three agent-written patches landed on a Friday afternoon. Two were nine-line diffs, one touched a helper nobody remembered owning, and all three were reviewed in the ten minutes between a standup and a release. On Monday the nightly seed job died, and nobody could say which candidate produced the line that killed it.

That team did not have a review problem. It had a receiving problem. A candidate sat in a pull request, looked plausible, and crossed into the trunk without anything recording what had actually been checked.

The last few weeks of DEV discussion keep circling a familiar split: model capability climbs, and the software wrapped around it stays about as fragile as the day it was written. That gap is not a model problem. It is a testing problem, and it is teachable in ninety minutes.

This workshop builds a receiving dock. Students leave with a gate that parses a candidate, runs it against a fixture, writes a verdict to a ledger, and refuses to be the last word on anything.

The invariant

The whole workshop hangs on one sentence, written by the students in the first ten minutes and never edited afterward: nothing reaches the trunk without a receipt.

A receipt is one JSON line. It names the task, the model that produced the candidate, the verdict, a machine-readable reason, and a timestamp. It is boring on purpose. When the nightly job breaks three days later, the receipt is what lets someone say this candidate passed a three-case fixture and nothing else, instead of reconstructing the week from chat scrollback.

The run sheet

Clock	Segment	Artifact at the end
00–10	The Monday failure story, told once	The written invariant
10–25	Fixture before gate	`tasks/03_slugify/test_solution.py`, three cases
25–45	Build the gate	`gate.py`, first `accept` receipt
45–60	Break it deliberately	Two quarantines, one false accept
60–75	Boundary sweep	A property test that catches the fake
75–90	Yield accounting and transport swap	`receipts.jsonl` and a yield number

Every segment ends with a file. If a segment runs long, the artifact gets cut down, never skipped, because a workshop that finishes with a feeling instead of a file teaches nothing rerunnable.

Fixture before gate

Students almost always want to write the gate first. That order produces a gate that grades whatever it happens to be shown, which is the exact failure mode of human review under deadline pressure.

The fixture comes first. A task is a directory with a prompt and a test file, and the candidate is expected to expose one function.

{
  "id": "03_slugify",
  "model": "fill-from-current-provider-docs",
  "prompt": "Write slugify(text: str) -> str"
}

# tasks/03_slugify/test_solution.py
from solution import slugify


def test_basic():
    assert slugify("Hello World") == "hello-world"


def test_collapses_runs():
    assert slugify("a   b---c") == "a-b-c"


def test_empty_stays_empty():
    assert slugify("") == ""

Three cases is the right number for the first pass. A fixture large enough to be impressive is a fixture nobody in the room can explain, and an unexplained fixture fails silently the day it is edited.

The gate

The gate is deliberately small: it parses, it runs, it records, it stops.

#!/usr/bin/env python3
"""gate.py - receiving inspection for model-written candidates."""
import ast, json, pathlib, subprocess, sys, time

ROOT = pathlib.Path(__file__).parent
MAX_BYTES = 8192


def parse_ok(path):
    try:
        ast.parse(path.read_text())
    except SyntaxError as exc:
        return False, f"syntax_error_line_{exc.lineno}"
    return True, "parsed"


def behavior(test_path, scratch):
    proc = subprocess.run(
        [sys.executable, "-m", "pytest", "-q", str(test_path)],
        cwd=scratch, capture_output=True, text=True, timeout=60,
    )
    lines = proc.stdout.strip().splitlines()
    return proc.returncode == 0, (lines[-1] if lines else "no_output")


def main(task_dir):
    task = json.loads(pathlib.Path(task_dir, "task.json").read_text())
    scratch = ROOT / "scratch" / task["id"]
    scratch.mkdir(parents=True, exist_ok=True)
    candidate = ROOT / "candidates" / f'{task["id"]}.py'
    verdict, reason = "quarantine", "no_candidate"

    if candidate.exists() and candidate.stat().st_size <= MAX_BYTES:
        ok, why = parse_ok(candidate)
        if ok:
            (scratch / "solution.py").write_text(candidate.read_text())
            (scratch / "test_solution.py").write_text(
                pathlib.Path(task_dir, "test_solution.py").read_text()
            )
            ok, why = behavior(scratch / "test_solution.py", scratch)
        verdict, reason = ("accept" if ok else "quarantine"), why
    elif candidate.exists():
        reason = f"oversize_{candidate.stat().st_size}b"

    receipt = {
        "task": task["id"],
        "model": task.get("model", "unspecified"),
        "verdict": verdict,
        "reason": reason,
        "ts": int(time.time()),
    }
    with open(ROOT / "receipts.jsonl", "a") as fh:
        fh.write(json.dumps(receipt) + "\n")
    print(json.dumps(receipt))


if __name__ == "__main__":
    main(sys.argv[1])

A candidate for the slug task, dropped at candidates/03_slugify.py, looks like this:

import re


def slugify(text):
    return re.sub(r"[^a-z0-9]+", "-", text.lower()).strip("-")

$ pip install pytest
$ python gate.py tasks/03_slugify
{"task": "03_slugify", "model": "fill-from-current-provider-docs", "verdict": "accept", "reason": "3 passed", "ts": 1792000000}

If pytest reports an ImportError on solution, the instructor has hit the classic path problem; run the gate with PYTHONPATH=. set and the scratch directory as the working directory, and the failure disappears. That detour is worth five minutes, because everyone meets it eventually in their own repository.

Break it deliberately

This is the segment that changes how students think about green checks. The instructor supplies a candidate that hardcodes the fixture's answers:

def slugify(text):
    return {
        "Hello World": "hello-world",
        "a   b---c": "a-b-c",
        "": "",
    }.get(text, "")

The gate accepts it. Three tests, three passes, one receipt that says accept, and a function that will return an empty string for every input the product actually receives. The fixture graded the fixture.

The correction is a property check that ties output back to input instead of to remembered examples:

# tasks/03_slugify/test_boundary.py
import random, string
from solution import slugify


def test_alnum_digest_matches():
    rng = random.Random(7)
    for _ in range(200):
        raw = "".join(
            rng.choice(string.printable) for _ in range(rng.randint(1, 40))
        )
        digest = "".join(ch for ch in raw.lower() if ch.isalnum())
        assert slugify(raw).replace("-", "") == digest

The hardcoded candidate now fails within a handful of iterations, while the real implementation passes all two hundred. Students see the difference between evidence and a memory test, and they see it in code rather than in a slide.

Where the compute comes from

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two hundred generated candidates per cohort is small by industry standards and painful in a classroom, because it forces every student to hold a paid key before they have learned anything worth paying for. MonkeyCode, the project behind this article's operator, describes a free tier and a free server option; the operator currently reports the free allowance at ten million tokens. Those terms live on the project's own page and can change, so this workshop treats them as a starting point to check rather than constants to quote in a syllabus, and it asserts nothing about which models are included, how large the server is, or how long either offer lasts.

The harness itself assumes no vendor at all. The transport is one function, and the shape below is what students fill in from whatever endpoint they are actually using:

import json, os, urllib.request


def ask(prompt):
    request = urllib.request.Request(
        os.environ["BASE_URL"] + "/chat/completions",
        data=json.dumps({
            "model": os.environ["MODEL"],
            "messages": [{"role": "user", "content": prompt}],
        }).encode(),
        headers={
            "Authorization": f'Bearer {os.environ["API_KEY"]}',
            "Content-Type": "application/json",
        },
    )
    with urllib.request.urlopen(request, timeout=120) as response:
        return json.load(response)["choices"][0]["message"]["content"]

The free server option earns its place in the boundary-sweep segment, where a loop of two hundred candidates would otherwise compete with whatever else is on the student's laptop. It is a convenience for the exercise, not a claim about production throughput.

Yield accounting

The last artifact is a number students compute themselves:

import collections, json

rows = [json.loads(line) for line in open("receipts.jsonl")]
counts = collections.Counter(row["verdict"] for row in rows)
total = sum(counts.values())
print(counts, "yield=", round(counts["accept"] / total, 3))

Yield is acceptance rate per task, and it belongs to the task, not to the model. A slug function with a clear contract collects accepts; a task whose prompt hides its error handling collects quarantines and tells the room nothing about model quality. That distinction is worth more than any leaderboard.

The verdict table

The gate produces four outcomes, and each maps to one action.

Verdict	Signal	Action
Accept	Parses, fixture green, sweep green	Merge with a link to the receipt
Repair	Parses, one named test fails with a local cause	One bounded revision, re-run the gate, two attempts maximum
Quarantine	Syntax error, oversize candidate, timeout, or missing file	Move the candidate to `quarantine/` and stop
Escalate	Sweep green but the diff touches files, network, or credentials	Human review of record, no exceptions

What this gate cannot see

The gate checks exactly what the fixture encodes and nothing more. A candidate that passes one afternoon is not guaranteed to pass tomorrow, because sampling variance means the same prompt can produce a different artifact; that is why the candidate bytes get stored next to the receipt and re-run on demand. The byte limit is arbitrary and exists to keep reviews short. The sixty-second timeout catches hangs, not deadlocks that resolve in ninety seconds. Concurrency, license headers, and secret handling are invisible to it.

Teams in regulated pipelines should not treat this gate as a control. It is a teaching device and a first filter, and a named human reviewer of record is still required wherever the rules say so. Teams with no fixture-writing habit should fix that habit before building the gate, or they will automate the grading of nothing.

A cohort that wants to see the sweep finish before deciding whether the gate belongs in its pipeline can start on the free tier and the free server option; the current terms page is the only place worth trusting for what is included today.

MonkeyCode provides free models that can run this workflow.

The Fare Gate Workshop: A Token Budget That Fails Loudly

Charlie Zhu — Thu, 15 Oct 2026 11:13:59 +0000

The Fare Gate Workshop: A Token Budget That Fails Loudly

The room goes quiet at the same moment every time. A scripted client loops through forty prompts, the provider stops answering somewhere around the thirty-first, and the traceback on the projector says nothing useful, because the real cause lives in an account nobody in the room can inspect. The failure is not a bug in the loop. It is a missing gate.

This workshop runs ninety minutes, uses one repository, and needs no paid key. Students write the component that refuses to spend before the refusal becomes real, and they leave with a ledger file they can hand to the next cohort. The framing comes from a complaint that keeps resurfacing in developer discussions: AI demos are easy to stage and hard to keep honest once quota and money enter the picture.

Think of a subway turnstile. Nobody expects the gate to explain the fare table to the crowd behind it. They expect it to close cleanly at the right moment and to leave a record of every entry. A token budget behaves the same way, and the record is the part students actually keep.

What the class builds

Three files, standard library only, no network required by default. ledger.py holds the spend record and the projection check. gate.py chooses a route and refuses when the projection crosses the budget. test_gate.py pins the behaviour so the exercises cannot be faked by accident. The ledger is append-only JSONL, which matters more than it sounds: a student who kills the process mid-batch and reruns the script should see the earlier spend still there.

# ledger.py — token ledger and preflight gate. Standard library only.
from __future__ import annotations
import json, pathlib, time

LEDGER_PATH = pathlib.Path("runs/ledger.jsonl")

def estimate_tokens(text: str) -> int:
    """Deliberately conservative: 4 chars per token, rounded up, +8 per message."""
    return max(1, -(-len(text) // 4)) + 8

class BudgetExceeded(RuntimeError):
    def __init__(self, spent: int, projected: int, budget: int):
        super().__init__(f"spent {spent} + projected {projected} > budget {budget}")
        self.spent, self.projected, self.budget = spent, projected, budget

class Ledger:
    def __init__(self, budget: int, path: pathlib.Path = LEDGER_PATH):
        self.budget, self.path = budget, path
        self.path.parent.mkdir(parents=True, exist_ok=True)

    def spent(self) -> int:
        if not self.path.exists():
            return 0
        rows = (json.loads(line) for line in self.path.read_text().splitlines() if line.strip())
        return sum(row["tokens"] for row in rows)

    def preflight(self, prompt: str, max_output: int) -> int:
        projected = estimate_tokens(prompt) + max_output
        if self.spent() + projected > self.budget:
            raise BudgetExceeded(self.spent(), projected, self.budget)
        return projected

    def record(self, route: str, tokens: int, note: str = "") -> None:
        row = {"ts": round(time.time(), 3), "route": route, "tokens": tokens, "note": note}
        with self.path.open("a") as fh:
            fh.write(json.dumps(row, sort_keys=True) + "\n")

    def report(self) -> str:
        spent = self.spent()
        pct = 100 * spent / self.budget if self.budget else 0.0
        return f"spent={spent} budget={self.budget} ({pct:.1f}%)"

The -(-len(text) // 4) trick is ceiling division, and the constant four is a guess that the calibration exercise later punishes. That is intentional. Students should feel the estimate drift before they trust it.

The gate, and the route table

Two routes exist: a local stub that never touches the network, and a hosted route that students wire to a real client themselves. The stub is what makes the class reproducible on a train with no signal, and it is also the honest default in argparse.

# gate.py — route selection; dry-run friendly by design.
import argparse
from ledger import Ledger, estimate_tokens, BudgetExceeded

def stub(prompt: str) -> tuple[str, int]:
    return f"[stub] {prompt[:40]}", estimate_tokens(prompt)

def hosted(prompt: str) -> tuple[str, int]:
    raise SystemExit(
        "Wire the provider client here. Read the key from the environment, "
        "never from source, and return (text, tokens_reported_by_provider)."
    )

ROUTES = {"stub": stub, "hosted": hosted}

def run(route: str, prompts: list[str], budget: int, max_output: int) -> int:
    led, refused = Ledger(budget=budget), 0
    for i, prompt in enumerate(prompts, 1):
        try:
            led.preflight(prompt, max_output)
        except BudgetExceeded as exc:
            print(f"{i:02d} refused: {exc}")
            refused += 1
            continue
        text, tokens = ROUTES[route](prompt)
        led.record(route, tokens, note=f"prompt={i}")
        print(f"{i:02d} {route} {tokens}t :: {text}")
    print(led.report(), f"refused={refused}")
    return refused

if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--route", choices=sorted(ROUTES), default="stub")
    ap.add_argument("--budget", type=int, default=600)
    ap.add_argument("--max-output", type=int, default=64)
    ap.add_argument("--prompts", type=int, default=40)
    args = ap.parse_args()
    batch = [f"Explain checkpointing, variation {i}." for i in range(1, args.prompts + 1)]
    run(args.route, batch, args.budget, args.max_output)

A class-sized demo looks like this, and the important line is the refusal, not the success.

python gate.py --route stub --budget 600 --max-output 64 --prompts 40
# 31 stub 15t :: [stub] Explain checkpointing, variation 31.
# 32 refused: spent 592 + projected 20 > budget 600
# spent=592 budget=600 (98.7%) refused=9

python -c "from ledger import Ledger; print(Ledger(600).report())"
# spent=592 budget=600 (98.7%)   <- survives a fresh process

The tests students must not skip

The two assertions below cover the only two failures that matter in production: the gate that never closes, and the ledger that forgets.

# test_gate.py — run with: pytest -q
import pytest
from ledger import Ledger, BudgetExceeded

def test_gate_refuses_when_projection_exceeds_budget(tmp_path):
    led = Ledger(budget=50, path=tmp_path / "l.jsonl")
    led.record("hosted", 40)
    with pytest.raises(BudgetExceeded):
        led.preflight("x" * 400, max_output=64)

def test_ledger_survives_process_restart(tmp_path):
    path = tmp_path / "l.jsonl"
    Ledger(budget=1000, path=path).record("stub", 12)
    assert Ledger(budget=1000, path=path).spent() == 12

The ninety minutes, block by block

The first ten minutes are the staged incident and nothing else; the class watches the loop die, then writes down three things the traceback failed to say. From ten to twenty-five, students copy ledger.py, run pytest -q, and watch both tests pass, which buys credibility for a file that looks trivial. The middle half hour is the gate itself: wire the refusal path, force BudgetExceeded on purpose with --budget 200, then open runs/ledger.jsonl and read the rows aloud. Students who finish early delete the ledger file and rerun, and the failure of that run is the lesson about persistence.

At the forty-five minute mark the class switches to the hosted route for roughly twelve minutes. The goal is not volume; it is the calibration step. Each student records estimate_tokens(prompt) + max_output before the call, compares it against the token count the provider reports, and writes the drift into a shared column on the board. Drift of twenty percent is common in a chatty batch, which is why the last fifteen minutes belong to the post-mortem: which of the three numbers would you trust in an alerting rule, and what would you set the budget to if the free allowance disappeared tomorrow. The final five minutes exist for one question only, and it is not about the code.

The hosted route and a free tier

Exercise four needs a real provider response, and not every student arrives with a key. For that block the workshop points the hosted route at MonkeyCode, whose free tier the operator describes as including ten million free tokens and a free server option; the ledger does not care who pays for the tokens, so the exercise survives either way. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat the free allowance as a teaching budget, not a production plan. Nothing here depends on specific model names, throughput, or hardware, because the class never needs them — the gate, the JSONL rows, and the drift column are provider-agnostic, and swapping the hosted client for a different vendor is a five-line change in gate.py. A reader who wants to run exercise four next week can claim the free tier and point the hosted route at it, then keep the ledger as the artifact.

Where this breaks, and who should skip it

The char-divided-by-four estimate is a heuristic, not a tokenizer. On code-heavy prompts it can be off by a wide margin, and the calibration block is the only honest defence against that. The gate is also per-process: two students running the same ledger file concurrently will interleave writes, and the file has no locking, so a team doing parallel batch work needs a real counter in a database instead. Refusals are coarse. The gate stops a whole prompt; it does not shrink the output budget or degrade gracefully to the stub, and adding that fallback is a legitimate second workshop.

Skip this entirely if the workload is interactive and single-shot, where a gate costs more attention than it saves. Skip it if quota exhaustion is already handled by platform-side limits you do not control. And skip it if the goal is throughput measurement, because a ledger records spend, not speed.

What remains after the room empties is a directory with one JSONL file in it, a few hundred lines of number, and a refusal message that reads like a sentence instead of a stack trace. That file is the part worth keeping. It is also the cheapest possible answer to the question every AI demo eventually raises, which is who is paying, and how much, and what happens when the answer runs out.

MonkeyCode provides free models that can run this workflow.

The Red Pencil Workshop

Charlie Zhu — Tue, 13 Oct 2026 00:18:32 +0000

The workshop opens on a staged failure, because a visible row count teaches faster than a definition. On the projector, a fake ledger shows three captures for one invoice after a short reply suggested a three-try loop around a gateway timeout. The teaching assistant does not start a lecture. She slides a red pencil across the bench and holds the next patch until the pencil mark and the test log agree.

That hour is the whole lesson. It is aimed at people who already write small Python services and who review a partner's patch before it lands. A fluent answer counts as a draft. A draft earns a mark only after a local test writes evidence a neighbor can rerun on a clean machine.

The pencil is not a stand-in for taste. It is a script exit line the pair reads aloud before anyone talks about merging. If the line and the test disagree, the paragraph loses.

The shop picture is a traveler card, not a courtroom. On a metal part, red pencil means a second person stood at the gauge and marked the card. The mark is worthless if the gauge never moved. Here the gauge is a unit test that counts capture attempts, and the mark is one stamped line.

Eighty minutes, one process

Eighty minutes is enough when the files already sit on the instructor machine. The first ten minutes replay the double charge, with the attempt list on screen, so the failure is a row count. The next fifteen minutes are local setup: a virtual environment, the modules below, and one command that prints three attempts for the bad policy.

The following twenty minutes are the first exercise. Pairs write a charge function that records an invoice key before the call and refuses a second call in the same process, even when the first call raised a timeout. They are not designing a payment network. They are stopping one process from repeating a side effect whose result is unknown.

Fifteen minutes go to the countercheck. A neighbor may paste a model reply into a text file, but only that neighbor runs the stamp, and the stamp does not read confidence. The last twenty minutes are a clean-directory rerun, including a cheat where someone types a passing line by hand. The neighbor deletes that file, runs the test, and makes the pencil follow the gauge.

Files the room can rerun

The modules are a lab design, written to be copied and executed. They are not a transcript of a timed class and they are not a measured production incident. ledger.py appends the attempt before it decides to time out. That order is the trap.

A retry loop can sound careful and still leave three rows, because the side effect happens before the exception. Students replace the loop with charge_once. The seen set lives only in memory, and that weakness is part of the lesson. A later week can put the same key in a journal file. This week stops at one process.

class Ledger:
    def __init__(self):
        self.attempts = []

    def capture(self, invoice_id, cents):
        self.attempts.append((invoice_id, cents))
        if str(invoice_id).endswith("-flaky"):
            raise TimeoutError("gateway timeout")
        return {"invoice_id": invoice_id, "cents": cents, "status": "captured"}


def charge_once(ledger, invoice_id, cents, seen):
    if invoice_id in seen:
        return {
            "invoice_id": invoice_id,
            "status": "already-attempted",
            "attempts": len(ledger.attempts),
        }
    seen.add(invoice_id)
    try:
        return ledger.capture(invoice_id, cents)
    except TimeoutError:
        return {
            "invoice_id": invoice_id,
            "status": "unknown-do-not-retry",
            "attempts": len(ledger.attempts),
        }


def charge_with_blind_retry(ledger, invoice_id, cents, _seen):
    last = None
    for _ in range(3):
        try:
            return ledger.capture(invoice_id, cents)
        except TimeoutError as exc:
            last = exc
    raise last

demo_blind.py is the opening scene reduced to a command. It does not pass or fail a suite. It prints the row count, and a correct copy prints blind attempts=3. If a pair hides the failure by catching the error inside capture, the count drops and the instructor puts the append back. The bad behavior has to stay visible, or the pencil has nothing to contradict.

from ledger import Ledger, charge_with_blind_retry

def main():
    ledger = Ledger()
    try:
        charge_with_blind_retry(ledger, "inv-9-flaky", 1250, set())
    except TimeoutError:
        pass
    print("blind attempts=%s" % len(ledger.attempts))

if __name__ == "__main__":
    main()

The gauge does not fail the suite in order to scold the specimen loop. It proves the once-only policy and writes one evidence line. The specimen stays in the other file so the demo command can still show three rows after the suite is green. A room that starts from a red bar learns to chase the first assertion. This room learns to produce a line a second script can read.

import unittest
from pathlib import Path
from ledger import Ledger, charge_once

EVIDENCE = Path("evidence.txt")

class RedPencilTest(unittest.TestCase):
    def test_timeout_does_not_authorize_a_second_capture(self):
        ledger = Ledger()
        seen = set()
        result = charge_once(ledger, "inv-9-flaky", 1250, seen)
        self.assertEqual(result["status"], "unknown-do-not-retry")
        self.assertEqual(ledger.attempts, [("inv-9-flaky", 1250)])
        self.assertEqual(result["attempts"], 1)
        EVIDENCE.write_text("PASS once-only attempts=1\n", encoding="utf-8")

    def test_same_key_is_not_sent_again(self):
        ledger = Ledger()
        seen = set()
        charge_once(ledger, "inv-9-flaky", 1250, seen)
        again = charge_once(ledger, "inv-9-flaky", 1250, seen)
        self.assertEqual(again["status"], "already-attempted")
        self.assertEqual(len(ledger.attempts), 1)

if __name__ == "__main__":
    unittest.main()

stamp.py lowercases the reply and rejects a few workshop phrases. The phrases are the ones a helpful draft tends to offer: always retry, safe to retry, ignore the test, trust this. The script then requires the evidence file to start with PASS and to contain attempts=1. A pair that wants the stamp rewrites the reply until it describes the once-only stop, then reruns the test so the evidence is fresh.

The script does not compare file clocks. The neighbor does. That human step is deliberate. The lab is a review habit, not a trust root, and a clock check would hide the social part of the pencil.

import sys
from pathlib import Path

BANNED = (
    "always retry",
    "safe to retry",
    "ignore the test",
    "trust this",
)

def stamp(reply_path, evidence_path):
    reply = Path(reply_path).read_text(encoding="utf-8").lower()
    evidence_file = Path(evidence_path)
    if evidence_file.exists():
        evidence = evidence_file.read_text(encoding="utf-8")
    else:
        evidence = ""
    for phrase in BANNED:
        if phrase in reply:
            return "REJECT: reply tells the class to retry or to skip tests"
    if not evidence.startswith("PASS"):
        return "REJECT: no local PASS evidence"
    if "attempts=1" not in evidence:
        return "REJECT: evidence does not record a single attempt"
    return "STAMP: red-pencil-ok"

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print("usage: python stamp.py reply.txt evidence.txt")
        raise SystemExit(2)
    print(stamp(sys.argv[1], sys.argv[2]))

A clean pass uses ordinary commands. The lab assumes Python 3 with the standard library and nothing else. No third-party package is required, which keeps the first fifteen minutes from turning into an installer debate.

python -m venv .venv
source .venv/bin/activate
python demo_blind.py
python test_red_pencil.py
printf '%s\n' "charge_once records the key, then stops after a timeout" > reply.txt
python stamp.py reply.txt evidence.txt

The expected stamp line is STAMP: red-pencil-ok. The counterexample should run first. A reply that says it is safe to retry on timeout must print a REJECT line even when evidence.txt already contains PASS. Windows machines can activate with .venv\Scripts\activate and then use the same Python commands. The lesson does not depend on bash.

A draft is not a stamp

Only the second exercise needs a hosted model, and it still works without one. One student may paste the timeout text and ask for a patch. The partner saves that answer, unchanged, as reply.txt, and runs the stamp before anyone applies a diff. A local file that already contains the bad paragraph teaches the same refusal when the network is down. The model supplies drafts. The test supplies the stamp.

Free model access is a practical fit when a department will not issue paid keys for one afternoon, and a free server option is a practical fit when the instructor wants one shared copy of these files instead of a handful of zip attachments. MonkeyCode is one place those two options are offered for this kind of lab. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Neither option is required for the stamp to mean anything.

Terms should be read on the morning of the class, in the project documentation, because availability language moves. This article does not pin a token ceiling, a machine shape, a region, or an end date. A workshop sheet that freezes an unverified number becomes the outdated handout the next cohort has to unlearn.

The cheat stays on the clock because someone will find it. Hand-typing PASS into evidence.txt makes a careless reading look finished. The workshop names that move in the same tone as a skipped gauge. The neighbor deletes the file, runs the test module, and only then runs the stamp script.

If the script and the test disagree, the test wins. A screenshot of the stamp without that rerun is decoration. The pencil is there to slow the conversation, not to decorate the slide.

Who should leave this lab on the bench

This lab should not be pointed at a real charge path, a mail sender, or an inventory decrement. The in-memory set dies with the process, so two workers can still capture the same invoice. It should not be the only review on a pull request. It is a poor first week for students who have not met exceptions, because the timeout will hide the attempt list.

It is also the wrong artifact for anyone comparing model vendors. No latency, cost, quality, or uptime figure is collected. A team that needs those numbers should measure its own traffic against its own success criteria, rather than borrow a classroom stamp and treat it as a score.

A later hour can append the invoice key to a journal and refuse capture when the line already exists. That problem is closer to crash recovery than to reply hygiene, and it needs its own timer. This workshop ends when every pair can show three attempts, then one attempt, and can refuse a smooth paragraph that disagrees with the evidence file.

Instructors who want the fixture on a no-cost shared box can check the current free-server note in the MonkeyCode project before they book the date. If that note no longer matches the plan, the same files stay on the laptops and the stamp does not change.