Workshop: Run a 20-Student AI Lab on a Free Model Endpoint in 90 Minutes

A shared free model endpoint works fine as workshop infrastructure, provided you stop treating it as an API and start treating it as a scarce, shared, non-deterministic resource. The plan below moves twenty students through four timed exercises against one endpoint and produces three artifacts you can actually grade. The failure that ruins these sessions is rarely model quality; it is contention plus grading noise. Recent DEV threads about AI cost and about intermediaries sitting between developers and models point the same direction: give people direct, measured access and the complaints mostly disappear.

Three failure modes to design against

Before the agenda, name the problems out loud so the room recognizes them when they appear at minute 12.

1. Thundering herd. Twenty students times five prompts is one hundred requests in a two-minute window, and nobody in the room knows the endpoint's real concurrency ceiling. Latency spikes, students blame the model, and the exercise stops teaching anything.
2. Non-determinism as grading noise. Default temperature, provider-side routing, and truncation all change output text between two students running the identical prompt. If you grade on exact strings, half the room fails for reasons unrelated to the lesson.
3. Invisible budget. Nobody knows the remaining allowance, so the lab ends abruptly mid-exercise and the last ten minutes become support triage instead of debrief.

The artifact that fixes all three is a small cassette harness: record every request-response pair once, replay it offline, and meter every live call against a per-student ledger.

The 90-minute agenda

Minute	Block	Concrete output
0–8	Framing: the three failure modes	shared vocabulary
8–20	Setup and health check	green check, usage object printed
20–35	Exercise 1: record a baseline cassette	cassettes/baseline.jsonl
35–50	Exercise 2: ledger and admission control	ledger.csv with observed or estimated tokens
50–65	Exercise 3: inject 429/503 and back off	reports/backoff.md
65–80	Exercise 4: determinism diff and grade	reports/diff.json
80–90	Debrief: fill the decision table	completed rubric

Notice that only minutes 8–65 touch the network. Everything after minute 65 runs from cassettes, which means the last exercise still completes if the endpoint degrades.

Instructor prep: 30 minutes before the room opens

Use one repository per cohort and put the recorder on a single host so students share state instead of duplicating it. MonkeyCode is a reasonable substrate here because it offers free model access and a free server option, so the cohort gateway and the recorder run on one machine without a billing conversation. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If your account exposes a single credential, keep it in the environment, never in a public repository or a slide, and rotate it after the session.

lab/
  record.py        # cassette recorder + fingerprinting
  ledger.py        # per-student token accounting
  inject.py        # fake server: 429 / 503 with Retry-After
  diff.py          # determinism comparison
  exercises/       # one prompt per exercise, fixed text
  cassettes/       # committed after Exercise 1

Set three variables and never hardcode them anywhere else:

export AI_BASE_URL="https://<your-endpoint-host>/v1"
export AI_API_KEY="<handed out at t-5, rotated at t+95>"
export AI_MODEL_ID="<the model id your account can actually select>"

Then run the health check and read the usage object, because Exercise 2 depends on what you find there:

# health.py — reference implementation, not executed against the hosted endpoint
import json, os, urllib.request

req = urllib.request.Request(
    os.environ["AI_BASE_URL"].rstrip("/") + "/chat/completions",
    data=json.dumps({"model": os.environ["AI_MODEL_ID"],
                     "messages": [{"role": "user", "content": "ping"}],
                     "max_tokens": 8}).encode(),
    headers={"Content-Type": "application/json",
             "Authorization": "Bearer " + os.environ["AI_API_KEY"]})
with urllib.request.urlopen(req, timeout=60) as r:
    body = json.loads(r.read())
print(json.dumps(body.get("usage", {}), indent=2))

If usage is present, you have an authoritative count for the ledger. If it is absent or zero, tell the room immediately and switch to estimation mode in the next exercise.

Exercise 1 — Record a baseline cassette (15 min)

Every request gets a fingerprint over model, prompt, temperature, and max tokens, so two students asking the same question collide on one cassette entry instead of spending two calls.

import hashlib, json, os, time, urllib.request

BASE = os.environ["AI_BASE_URL"].rstrip("/")
KEY = os.environ["AI_API_KEY"]
MODEL = os.environ["AI_MODEL_ID"]

def fingerprint(prompt, max_tokens, temperature=0.0):
    raw = f"{MODEL}|{prompt}|{temperature}|{max_tokens}".encode()
    return hashlib.sha256(raw).hexdigest()[:16]

def call(prompt, max_tokens=256, temperature=0.0):
    body = json.dumps({"model": MODEL, "temperature": temperature,
                       "max_tokens": max_tokens,
                       "messages": [{"role": "user", "content": prompt}]}).encode()
    req = urllib.request.Request(BASE + "/chat/completions", data=body,
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {KEY}"})
    start = time.monotonic()
    with urllib.request.urlopen(req, timeout=90) as r:
        payload = json.loads(r.read())
    return payload, int((time.monotonic() - start) * 1000)

with open("cassettes/baseline.jsonl", "a") as out:
    for prompt in open("exercises/ex1.txt").read().split("\n---\n"):
        payload, ms = call(prompt.strip())
        out.write(json.dumps({"fp": fingerprint(prompt.strip()),
                              "prompt": prompt.strip(), "ms": ms,
                              "response": payload}) + "\n")

Students commit baseline.jsonl before the break. From that point on, a replay is a dictionary lookup, and grading no longer needs the network.

Exercise 2 — Ledger and admission control (15 min)

This is where you replace every published number with a measured one. Do not put a quota figure on the slide, including a figure from an article: have each student total the tokens their cassette consumed and write it to a shared CSV.

def tokens_used(payload):
    usage = payload.get("usage") or {}
    total = usage.get("total_tokens")
    if isinstance(total, int) and total > 0:
        return total, "reported"
    approx = len(json.dumps(payload).split())  # crude, label it
    return approx, "estimated"

Then enforce a per-student ceiling of your own choosing, not the endpoint's, with two lines of scheduling: a threading.Semaphore(2) around every live call and a hard stop once the local counter passes the ceiling. Mark estimated rows as estimated in the CSV, because chat payload estimation drifts noticeably versus the provider's own count. The debrief question is the real lesson: how many students fit before the cohort total exceeds what you measured at minute 10?

Exercise 3 — Inject failure and back off (15 min)

Sixteen lines of http.server returning 429 with Retry-After: 2 for the first two calls, then 503, gives every student a reproducible failure without touching the real endpoint.

import time, urllib.error, urllib.request, random

def with_retry(fn, attempts=3, cap=8.0):
    for i in range(attempts):
        try:
            return fn()
        except urllib.error.HTTPError as e:
            if e.code not in (429, 503) or i == attempts - 1:
                raise
            hinted = float(e.headers.get("Retry-After") or 0)
            time.sleep(min(cap, hinted or random.uniform(0, 2 ** i)))

Have students record, per attempt, whether they waited the hinted interval or fell back to jitter. The report should state how many requests survived three attempts and what fraction of the exercise budget the retries consumed.

Exercise 4 — Determinism diff and grading (15 min)

This is not drift detection across weeks, which is a different problem with a different harness. Here you only assert a within-session contract: two students replaying the same fingerprint get observably comparable results.

diff = {"identical": 0, "whitespace_only": 0, "divergent": 0}
for fp, a, b in pairs:                       # pairs built from two runs
    if a == b:
        diff["identical"] += 1
    elif " ".join(a.split()) == " ".join(b.split()):
        diff["whitespace_only"] += 1
    else:
        diff["divergent"] += 1

Grade on four checks, each pass/fail: cassette committed and replayable offline, ledger rows labeled reported or estimated, backoff report names every attempt, and divergent count explained rather than reduced to zero by hand-editing. A student who documents three divergent answers with a hypothesis scores higher than one who reports none.

Decision table

Situation	Use this harness?	Reason
20 students, 90 minutes, one endpoint	Yes	contention and grading reproducibility dominate
Three-person internal demo	No	setup cost exceeds the failure cost
Measuring the endpoint's capacity	No	this schedules your lab; it does not characterize the service
Take-home assignment	Partly	ship instructor cassettes for offline replay
Production evaluation	No	use pinned models and versioned fixtures instead

Limitations and who should not use this

Cassettes go stale the moment the model or the endpoint changes, so re-record at the start of each term rather than trusting last quarter's fixtures. The token counts are either the endpoint's own claim or a crude estimate, and neither is an invoice. This harness creates no capacity; if your measured cohort total exceeds what you can verify at minute 10, split the room into two sessions instead of pushing harder.

Skip this entirely if you are a solo developer prototyping, if you need per-user attribution or an SLA, or if you were hoping to load-test a free endpoint, which is both technically misleading and likely against acceptable use. Instructors who cannot commit to thirty minutes of prep before the session should run a demo, not a lab.

Run the four exercises once with a colleague before the room fills, and write your measured ceiling on the whiteboard rather than any number you read online, this article included. If you want a single host for the recorder and the gateway, the free server option is the least friction I have found for that first cohort.