A shared free model path can support a classroom exercise, yet it should never become the grading oracle. Students should freeze a replay packet that stores the task, the fixture output, and the assertions before any live call. Instructors can then mark the same submission when that free path is slow, capped, or temporarily unreachable. This 75-minute workshop builds that packet and places a free server in an execution lane rather than in the score.
Why the packet comes before the call
Generated output is useful as a draft, and it is a poor source of truth for a grade. A later run can differ because the endpoint is busy, the prompt is underspecified, or the service returns a shorter completion. If the class waits for a live call before writing assertions, every outage becomes an incomplete lab. The packet reverses that order so the expected evidence already exists on disk before the first request leaves the room.
Recent public writing has spent more attention on agent demos than on how a reviewer repeats a result the next morning. That gap matters in a teaching lab, where a full room may share one free path and still need a comparable score. This workshop does not claim a benchmark, a token quota, a hardware profile, or a permanent free offer. It claims only that a frozen fixture plus three explicit lanes makes the same exercise rerunnable on a later date.
What students leave with
By the end of the session, each student can regenerate a local packet and pass three assertions without a network. A second lane may call a free model endpoint only after the packet already contains a frozen fixture. A third lane may run a small command on a free server, then compare stdout with the fixture hash. The live lanes stay optional, so a missing key does not block grading or a later replay.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is relevant here only as one place an instructor may obtain free model access and a free server option for those optional lanes. This draft does not name models, quotas, or hardware, because those details change and were not verified for this class date. Students should read the current offer on the class day before they export an endpoint or share a host.
75-minute timing
Use the blocks below as a wall clock, not as a suggestion to skip the fixture lane. Each block names an output so a facilitator can see whether the room is still on the fixture. These minutes are planning targets for a single 75-minute room, not timings measured from any previous cohort. If a block overruns, cut the optional live lanes first and keep the deliberate assertion-failure exercise intact.
- Minutes 0–10: State the rule that the fixture lane is the only required score, then list the three fields every packet must contain.
- Minutes 10–22: Students write a three-line fixture and three assertions for a status note that must contain the word READY.
- Minutes 22–40: Students run the packing script, inspect the printed hash, and repair one deliberate assertion failure before moving on.
- Minutes 40–52: Pairs review a neighbor packet and reject any file whose stored hash does not match the normalized fixture text.
- Minutes 52–65: Optional lanes run only after the instructor confirms current terms for free model access and the free server option.
- Minutes 65–75: Each student records one limitation, then reruns the local script with the network disabled and keeps the passing JSON.
Packet fields
Keep the schema small enough that a grader can read every field without asking a model for help. Version 1 stores the human task, the normalized fixture, its digest, and a short list of assertions. Optional lanes default to skipped, so a packet remains valid when no endpoint has been configured yet. Unknown assertion kinds fail closed, which stops a student from smuggling an unchecked rule into the file.
-
versionis an integer, and this workshop uses version 1 only. -
taskstores the human instruction so a replay does not depend on chat history. -
fixture_textis the expected body, normalized to one trailing newline. -
fixture_sha256is the hex digest of that normalized body. -
assertionsholdscontains,max_lines, orexact_hashchecks, and no other kinds. -
lanes.fixtureis required, whilelanes.free_modelandlanes.free_serverdefault to skipped.
Worked example students can rerun
The script below is a teaching artifact that writes replay-packet.json and exits zero when the fixture lane passes. The optional lanes are not called in this published example, and they should stay skipped on the first run. Treat any live client as a later exercise that must follow provider documentation you verified during that same week. Do not add network code until exercise 4, because an early call would hide whether the local packet is sound.
#!/usr/bin/env python3
"""Pack a replay packet. Live lanes stay skipped unless you add them later."""
import hashlib
import json
import sys
from pathlib import Path
def normalize(text):
return text.strip() + "\n"
def digest(text):
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def build_packet(task, fixture, assertions):
body = normalize(fixture)
return {
"version": 1,
"task": task,
"fixture_text": body,
"fixture_sha256": digest(body),
"assertions": assertions,
"lanes": {
"fixture": {"required": True, "status": "pending"},
"free_model": {"required": False, "status": "skipped"},
"free_server": {"required": False, "status": "skipped"},
},
}
def check(packet):
errors = []
text = packet["fixture_text"]
if digest(text) != packet["fixture_sha256"]:
errors.append("fixture hash mismatch")
for item in packet["assertions"]:
kind = item["kind"]
if kind == "contains" and item["value"] not in text:
errors.append("missing substring: " + item["value"])
elif kind == "max_lines" and len(text.splitlines()) > int(item["value"]):
errors.append("line count above cap")
elif kind == "exact_hash" and digest(text) != item["value"]:
errors.append("exact hash assertion failed")
elif kind not in ("contains", "max_lines", "exact_hash"):
errors.append("unknown assertion kind")
return errors
def main():
task = "Emit a three-line status note that includes the word READY."
fixture = "status: READY\nlane: fixture\nexit: 0"
packet = build_packet(task, fixture, [])
packet["assertions"] = [
{"kind": "contains", "value": "READY"},
{"kind": "max_lines", "value": 3},
{"kind": "exact_hash", "value": packet["fixture_sha256"]},
]
errors = check(packet)
packet["lanes"]["fixture"]["status"] = "pass" if not errors else "fail"
Path("replay-packet.json").write_text(
json.dumps(packet, indent=2) + "\n", encoding="utf-8"
)
print("errors", len(errors), "hash", packet["fixture_sha256"][:12])
return 0 if not errors else 1
if __name__ == "__main__":
sys.exit(main())
Local commands
Run the three commands from a clean directory, and disable the network if you want a stricter proof. The expected local result is a zero exit, a printed error count of zero, and two lanes left as skipped. If you change the fixture text after the hash is stored, the check fails and the process exits one. Students should trigger that failure once, restore the matching text, and keep both JSON files for the pair review.
python3 pack_replay.py
python3 -m json.tool replay-packet.json > /dev/null
python3 -c "import json; p=json.load(open('replay-packet.json')); assert p['lanes']['fixture']['status']=='pass'"
Failure you should force
After the first successful run, edit replay-packet.json and change one character inside fixture_text without updating the digest. Recompute the check in a short shell snippet, and confirm that the hash mismatch is reported before any lane is trusted. Students should restore the file by rerunning the packing script, rather than editing the digest to bless the broken text. That sequence shows why the digest is a guard against silent edits, not a decorative field copied from a tutorial.
python3 -c "import json,hashlib; p=json.load(open('replay-packet.json')); t=p['fixture_text']; print(hashlib.sha256(t.encode()).hexdigest()==p['fixture_sha256'])"
Exercise sequence
Work these steps in the listed order, and do not open a model client until the fourth exercise. Skip the fourth and fifth exercises when the instructor has not confirmed the current free-path terms. The first three exercises are the grade, and the last exercise is a peer check rather than a new feature. A student who finishes only the local steps has still completed the workshop outcome described above.
- Write the fixture by hand, and do not ask a model for the expected text, because that would hide the oracle inside the tool you are trying to constrain.
- Add one
containscheck, onemax_linescheck, and oneexact_hashcheck, then run the script and keep the JSON. - Break the stored fixture on purpose, confirm a failed comparison, and write one sentence naming which guard fired.
- Optional: if free model access is available, send only the task string, store the completion beside the fixture, and never overwrite
fixture_text. - Optional: if a free server is available, print the fixture there and mark pass only when the stdout hash matches.
- Exchange packets, and reject any file that lacks a hash, uses an unknown kind, or marks a live pass without a digest.
Decision table
Use the table when the room must choose a lane without turning a missing host into a failed grade. The entries are class rules written for this workshop, not measured limits taken from any provider. A match means the compared digest equals fixture_sha256, and a diverge means the side file differs. Never relax an assertion after reading a live completion, because that move lets the model rewrite the test.
| Situation | Lane to score | What to record | Do not do |
|---|---|---|---|
| Network off or key missing | fixture | both live lanes skipped | Invent a completion |
| Free model returns text | fixture, plus a side file | match or diverge against the fixture digest | Replace the fixture with the completion |
| Free server prints stdout | fixture, then server | pass only on a hash match | Treat server uptime as the grade |
| Completion exceeds three lines | fixture | keep the original max_lines cap | Relax the cap after seeing model text |
| Offer terms unread today | fixture only | note that live lanes were not authorized | Copy a quota from an older article |
How the free options fit, and where they stop
Free model access is a reasonable way to let students inspect a live completion after the oracle already exists. A free server is a reasonable place to run the print command when local machines differ and logs have been reviewed. Neither option should set the score, because availability and limits are not treated as fixed facts in this article. If either path is down on class day, the fixture lane still finishes the lab and preserves a comparable result.
Re-read the current terms before minute 52, since a saved screenshot is not evidence for a later cohort. Do not send credentials, personal data, or unpublished student work to a shared host until retention and access are checked. The packing script does not open a socket, which keeps the required lane independent of whichever host you add later. That separation is the lesson, and a branded endpoint does not change the order of fixture first and call second.
Limitations
Hash equality is strict, so a valid paraphrase looks like a failure if you grade the live completion instead of the fixture. The schema does not estimate tokens, latency, or cost, and it should not be reused as a billing or capacity report. Version 1 also ignores tool calls, file edits, and multi-turn chats, so it will not catch an agent that mutates a repository. The example was not executed against a live endpoint in this draft, so a local pass proves only the packing script.
Who should not use this approach
Skip this workshop if the learning goal is open-ended writing, where many different answers remain equally acceptable. Skip it when the lab requires production credentials, private customer data, or a shared host that nobody can audit. Skip it if students need a statistical quality score, because three string checks will not measure taste or coverage. Also skip it when the instructor cannot commit to a fixture before class, since the method fails if the expected text is still moving.
Close
Pack the fixture, hash it, and rerun the three local commands before the room discusses any endpoint. Keep the free model lane and the free server lane as optional evidence that you compare, not as text you trust. If a cohort already has MonkeyCode free model access and the free server option, use both only after a same-day terms check. Leave the fixture status as the grade either way, including when those optional lanes are skipped or unavailable.
Top comments (0)