Rubric First, Box Second: A Bootcamp Lab Where the Setup Is the Deliverable

The short version: in a bootcamp lab, the environment is part of the answer. If a classmate can't rebuild your box from your repo and get the same bytes back, then your clever prompt never mattered. So this lab grades setup, checkpoints, and a rubric — before it grades the agent at all.

Why flip the order?

Most AI labs hand students a working sandbox and a problem. The sandbox is invisible, so the lesson students actually absorb is "the sandbox was just there." That's a bad lesson to carry into a job.

Free tiers make it worse. A free model sitting behind a shared queue is not guaranteed to be the same model tomorrow. A free server is not the same machine next week. Neither of those is a defect — it is the honest shape of free infrastructure. But it means the reproducible part of your work has to be the contract: what you pinned, what you recorded, and what you can rebuild from scratch.

Hence the flip. Deliverable one is a shell script. Deliverable four is a rubric. The agent run sits in the middle, where it belongs.

The lab contract

Four artifacts, one repo:

1. bootstrap.sh — rebuilds the box from nothing.
2. manifest.json — pins inputs, model identity, and budget.
3. checkpoints/ — three named checks a stranger can run.
4. rubric.json — the scoring rules, written by the student.

Nothing here is exotic. That's the point. If it feels boring, you're doing it right.

Artifact 1: a bootstrap that assumes amnesia

#!/usr/bin/env bash
set -euo pipefail
# lab/bootstrap.sh — rebuild from an empty box. Bash 5+, Debian-ish target.
LAB_ROOT="${LAB_ROOT:-$HOME/lab}"
python3 -m venv "$LAB_ROOT/.venv"
"$LAB_ROOT/.venv/bin/pip" install --quiet -r "$LAB_ROOT/requirements.txt"
mkdir -p "$LAB_ROOT/runs"
git -C "$LAB_ROOT" rev-parse --short HEAD > "$LAB_ROOT/runs/git-sha.txt"
date -u +%Y-%m-%dT%H:%M:%SZ        > "$LAB_ROOT/runs/built-at.txt"
echo "built $(cat "$LAB_ROOT/runs/git-sha.txt") at $(cat "$LAB_ROOT/runs/built-at.txt")"

Two rules for students:

• It must run on a box where $LAB_ROOT does not exist yet.
• It must print a build stamp. No stamp, no credit.

Why so strict? Because "it works on my machine" is a sentence, not evidence.

Artifact 2: pin the model, don't assume it

This is where free access actually changes the design. The project this lab is built around — the open-source coding agent from MonkeyCode — is described by its operator as offering free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I am deliberately not quoting a token quota or a machine spec here. Those numbers move, and a bootcamp that hard-codes them teaches students to trust a stale README. Check the project's own page the morning you run the lab, and write down what you found. That check is itself worth a point.

What you do pin is which model actually answered:

{
  "schema": 1,
  "runtime": "python3.11",
  "model": { "alias": "student-default", "recorded_id": "fill-me-in", "fallback": "fill-me-in" },
  "inputs": { "fixture": "fixtures/issue-0007.md", "sha256": "fill-me-in" },
  "budget": { "max_calls": 40, "max_wall_clock_s": 900 },
  "rebuild": { "script": "bootstrap.sh", "verified_at": null }
}

rebuild.verified_at stays null until the student has run bootstrap.sh twice, on two fresh boxes. That one field kills more hand-waving than any rubric line I've ever written.

Artifact 3: three checkpoints

Not one checkpoint. Three, and each must be cheap enough to run on a whim.

1. Cold — bootstrap.sh on an empty box, then print the build stamp.
2. Frozen — re-run the agent with the fixture hash unchanged; assert the output hash equals run one's output hash.
3. Fenced — assert the diff touches only paths listed in allowed_paths.

A checkpoint is a command, not a paragraph:

$ ./checkpoints/frozen.sh
frozen: output hash 3f9c...  == run-001   OK

If a checkpoint needs three sentences of explanation to run, it isn't a checkpoint. It's a wish.

Artifact 4: stretch goals

Stretch goals are where the lab stops being a tutorial and starts being work:

• Swap the model alias and keep the same score.
• Cut wall clock by a third without losing a single checkpoint.
• Add one rubric criterion that would catch a specific cheat, then explain who it punishes unfairly by accident.

That third one is the best hour of the week. Students grumble, then quietly rewrite the criterion twice.

The fair rubric

A rubric is only fair if two people applying it in the same room land on the same number. Otherwise you're not grading work, you're grading mood.

{
  "criteria": [
    {"id": "R1", "name": "rebuild", "weight": 3, "pass": "bootstrap.sh runs on an empty box and prints a build stamp"},
    {"id": "R2", "name": "repro",   "weight": 3, "pass": "two runs from the same manifest produce identical output bytes"},
    {"id": "R3", "name": "log",     "weight": 2, "pass": "runs/latest.json names model id, fixture hash, and call count"},
    {"id": "R4", "name": "scope",   "weight": 2, "pass": "no edits outside allowed_paths"}
  ],
  "disagreement_rule": "if two graders differ on any criterion, neither score counts until both re-read the manifest"
}

And a twenty-line grader, so the rubric is executable instead of decorative:

import json, sys

def grade(rubric_path, run_path):
    rubric = json.load(open(rubric_path))
    run    = json.load(open(run_path))
    earned = 0
    for c in rubric["criteria"]:
        ok = bool(run.get("checks", {}).get(c["id"]))
        print(f"{c['id']:>3}  {'PASS' if ok else 'FAIL'}  {c['name']}")
        earned += c["weight"] if ok else 0
    total = sum(c["weight"] for c in rubric["criteria"])
    return earned, total

if __name__ == "__main__":
    earned, total = grade(sys.argv[1], sys.argv[2])
    print(f"score {earned}/{total}")
    sys.exit(0 if earned == total else 1)

Then the drill that makes it real: two students grade the same run file, separately, no talking. Different score? The rubric is broken, not the run. Fix the wording of the criterion and try again. That loop — not the agent output — is the skill being taught.

Limitations, honestly

• Free infrastructure recycles. A box can vanish between checkpoint two and three. That is the lesson, but it also eats lab time.
• Shared queues make wall clock noisy. So don't grade speed. Grade recorded call counts.
• The grader reads logs, not code. Pair it with a human read, or students optimize for the log format.
• bootstrap.sh assumes network access and a Debian-ish base. Porting it elsewhere is part of the assignment, not a footnote.
• These scripts are a skeleton. I haven't validated them across a distro matrix — adapt, then stamp your own manifest.

Who should not use this

• Teams that need one fixed production model with compliance paperwork. Pin something stable instead.
• Courses where the environment isn't the lesson. A pure algorithms unit gets overhead and nothing back.
• Anyone who can't tolerate a free tier that changes shape mid-semester. Grade the contract, or pay for predictable hardware.

Try it, then break it

Run the lab once with a classmate grading you blind. Then ask the only question that survives contact with a deadline: could a stranger rebuild this from your repo on a Tuesday you didn't plan for?

If you'd rather not provision anything yourself just to find out, the free server option and the free model access are the cheapest way to start — check the project page for what's actually available today.