A run that only works on your laptop is not a result. It's a screenshot with extra steps.
That's the whole lab. Students build a small agent loop, then they have to prove it works on a machine they don't own — the free server option, a classmate's box, whatever the grader hands them. The grade comes from the second host, not the first.
Why so strict? Because "it ran on my machine" is the most expensive sentence in software, and bootcamp demos are where it breeds.
The brief, in one screen
Each team ships a repo that a stranger can run without asking a single question. The repo must contain three things:
-
env.manifest.json— what the run needs, in machine-readable form. -
preflight.py— a script that reports whether the host matches the manifest. -
checkpoint.json— a resumable state file plus the code path that resumes from it.
No chat logs. No "ask me for the API key". No 12-minute screen recording. If the grader has to read your history to run your code, that's a zero on reproduction, and there is no partial credit for enthusiasm.
Artifact 1: the manifest you can diff
Keep it boring and declarative. Boring survives a handoff.
{
"entrypoint": "python run_agent.py --steps 5",
"python": ">=3.10,<3.13",
"env_vars": ["MODEL_BASE_URL", "MODEL_NAME"],
"expected_outputs": ["out/run.jsonl", "out/trace.md"],
"network": "outbound https required during run",
"time_box_seconds": 300
}
The env_vars list holds names only, never values. That one rule kills most key-leak incidents before they happen.
Artifact 2: preflight that prints evidence
This is the core of the lab. Run it on both hosts, diff the JSON, and explain the diff in your README. Here's the version I hand out (labeled: not yet executed against your environment — run it before you trust it):
#!/usr/bin/env python3
"""preflight.py — run on BOTH hosts, then diff the two reports."""
import json, os, platform, shutil, subprocess, sys
from pathlib import Path
REQUIRED_FILES = ["run_agent.py", "checkpoint.json", "requirements.lock"]
REQUIRED_ENV = ["MODEL_BASE_URL", "MODEL_NAME"] # names only, never values
def sh(cmd: str) -> str:
p = subprocess.run(cmd, shell=True, capture_output=True, text=True)
return p.stdout.strip() if p.returncode == 0 else "error"
report = {
"host": platform.node(),
"os": f"{platform.system()} {platform.release()}",
"python": platform.python_version(),
"cwd": str(Path.cwd()),
"files_present": {f: Path(f).exists() for f in REQUIRED_FILES},
"env_present": {k: bool(os.environ.get(k)) for k in REQUIRED_ENV},
"free_disk_gb": round(shutil.disk_usage(".").free / 1e9, 1),
"packages": sorted(sh(f"{sys.executable} -m pip freeze").splitlines()),
}
Path("preflight.json").write_text(json.dumps(report, indent=2))
print(json.dumps({k: v for k, v in report.items() if k != "packages"}, indent=2))
Then the diff — this is the step teams skip and regret:
# laptop
python preflight.py && mv preflight.json preflight.laptop.json
# server
python preflight.py && mv preflight.json preflight.server.json
diff <(jq -S 'del(.host,.cwd)' preflight.laptop.json) \
<(jq -S 'del(.host,.cwd)' preflight.server.json)
Two outcomes are acceptable: no diff, or a diff you predicted in writing. An unexplained diff is a finding, and findings cost points only when they're a surprise.
Artifact 3: a checkpoint the grader can resume
Small schema, strict rules:
{
"run_id": "lab-042",
"step": 3,
"messages": [{"role": "user", "content": "..."}],
"pending_tool": null,
"budget": {"steps_left": 2},
"written_at": "2026-10-17T09:00:00Z"
}
Write it with an atomic replace so a crash can't leave half a file — tmp file, then os.replace. And resume with an explicit cap:
python run_agent.py --resume checkpoint.json --steps 2
If resuming twice changes the output bytes, that's a different lab. Here, we only ask that resuming once produces out/run.jsonl that the grader can read and attribute to a step number.
The three checkpoints
Gate 1, end of day one: manifest committed, preflight.py runs locally, both files parse.
Gate 2, end of day two: the run has been executed on the free server option, preflight.server.json exists, and the README names every diff line.
Gate 3, submission: a classmate you've never paired with runs your entrypoint verbatim, on a host you didn't touch, and gets your documented output shape.
Gate 3 is the one that hurts. That's the point.
A rubric that can't be gamed
| Weight | Criterion | What counts as evidence |
|---|---|---|
| 40 | Reproduces on grader host | documented entrypoint, clean exit, expected files |
| 20 | Preflight diff explained | two JSON reports + prose that matches them |
| 15 | Resumable checkpoint | one resume, attributable output |
| 15 | Honest limitations | says what did not work, in its own words |
| 10 | README clarity | stranger can start within 5 minutes |
Fairness rules the graders follow, not the students:
- Grader runs only the documented command. Improvising is not allowed, even if it would work.
- Same fixture, same time box (300 s), same step cap for every team.
- Timeouts count as failures. A hung run is not "almost working".
- No live demos. The repo is the submission; your laptop stays closed.
Stretch goals that don't buy back a failure
Stretch work is capped at 5 bonus points and cannot rescue reproduction. Otherwise the fast teams just do more features and skip the boring part — which is exactly the behavior the rubric exists to price.
- Add
--dry-runthat emits the planned tool calls without calling a model. - Emit a
trace.mda non-programmer can follow, one line per step. - Add a
preflight --strictmode that fails on any package drift. - Write a one-page incident note for the run that broke, in the format of a real post-mortem.
Where the free tier actually earns its place
This lab needs a second host and a lot of throwaway model calls, and that's the honest reason the free model access and free server option matter here: students get a foreign host and a budget to fail on, without a credit card conversation in week one. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
I'm deliberately not quoting a token number or a hardware spec. Quotas and server terms move, and a stale number in a lab handout is worse than no number — check the project's own current pages before you plan a cohort around them.
Limits worth stating up front: a free host is not your production baseline, model behavior can shift between hosts so pin what you can, and this rubric measures reproducibility, not model quality. Nobody here is claiming your agent is good. Only that a stranger can run it.
Who should skip this lab: teams shipping to a fixed, already-provisioned environment, and anyone whose run legitimately needs more compute than a shared free box. Don't contort the exercise — write your own gate.
If you teach this, run preflight.py on your own machine first. Then ask yourself the only question that matters: could a stranger reproduce your green checkmark without you in the room?
If you try the lab on the free server option, send me the diff that surprised you — that's the part I want to read.
Top comments (0)