DEV Community

Charlie Xu
Charlie Xu

Posted on

If It Only Runs on Your Laptop, It's a Zero: A Bootcamp Lab on Clean-Box Agents

A passing agent homework is not “it compiled on my Mac.” It is a replay on a box that has never seen the student’s .env, global node_modules, or leftover Docker layers. If the run cannot be reproduced on a clean remote server, I score it as a zero. Harsh? Maybe. Fair? That is the whole point.

Bootcamp laptops are crime scenes. Last week’s API key is still in the shell history. A model “just knows” the database URL because it lived in a file nobody gitignored. Then the demo looks like magic. Then CI dies on a machine that does not love you. Sound familiar?

This week’s lab is about environment isolation, not prompt poetry. Students still get an agent. They do not get to grade themselves on a contaminated laptop.

The assignment in one sentence

Ship a tiny HTTP helper, prove an agent helped, and prove the same helper still works after we wipe the laptop out of the picture.

No chat screenshots. No “trust me, the sidebar did it.” A receipt from a remote run, or it did not happen. We already graded halt conditions and write-sets in other labs. Those assumed the machine was honest. This one assumes the machine is lying.

What students actually turn in

A zip is an automatic fail. I want a repo with four files, no more:

  1. src/retryFetch.js — the implementation
  2. src/retryFetch.test.js — tests the human owns
  3. agent/run-contract.json — what the agent was allowed to touch
  4. agent/clean-box-receipt.json — evidence from a remote run

If a fifth file looks like a dumped transcript, I drop it from the grade folder. Chat is not homework.

Lab setup

Students clone a stub. I hand them a broken retryFetch that retries forever on HTTP 500s. That is the bug. The agent may patch the implementation. The human must keep the tests frozen.

Commands I expect on the laptop, only to author tests:

git clone <lab-stub>
cd clean-box-lab
npm test   # must fail on purpose
Enter fullscreen mode Exit fullscreen mode

They may think locally. They may not submit a local-only agent session as proof.

Need a model and a machine that is not a bedroom laptop? Disclosure: This article was prepared as part of MonkeyCode's product outreach. I point students at MonkeyCode’s free model access and free server option so the clean-box run does not require a paid key or a home GPU. That is why the product shows up here: a shared remote box for the receipt step, not a tour. Already have a VPS? Fine. The rubric grades the receipt, not the vendor.

The run contract

{
  "task": "Make retryFetch give up after 3 attempts on HTTP 500, then throw.",
  "allowed_writes": ["src/retryFetch.js"],
  "forbidden_reads": [".env", ".env.*", "~/.npmrc", "~/.ssh"],
  "forbidden_writes": ["src/retryFetch.test.js", "agent/run-contract.json"],
  "runtime": "remote-clean-box",
  "network": "test-fixtures-only"
}
Enter fullscreen mode Exit fullscreen mode

Read that again. The agent cannot “improve” the tests. The agent cannot slurp secrets. The agent cannot pretend localhost is production.

A tiny receipt schema

{
  "runtime": "remote-clean-box",
  "box_id": "student-supplied-remote",
  "started_at": "2026-09-17T00:00:00Z",
  "git_sha": "abc123",
  "command": "npm test",
  "exit_code": 0,
  "stdout_sha256": "replace-me",
  "wrote": ["src/retryFetch.js"],
  "read": ["src/retryFetch.js", "package.json"],
  "secret_scan": "pass"
}
Enter fullscreen mode Exit fullscreen mode

Is this cryptographically bulletproof? No. It is a bootcamp receipt, not an audit log. It still beats a screenshot of a green terminal.

Checkpoint 1 — Tests stay human

I ship this fixture as lab material, not as a library you should paste into production.

// src/retryFetch.test.js
import { retryFetch } from "./retryFetch.js";
import assert from "node:assert/strict";
import { test } from "node:test";

test("stops after three 500s and throws", async () => {
  let calls = 0;
  const fake = async () => {
    calls += 1;
    return { ok: false, status: 500 };
  };
  await assert.rejects(() =>
    retryFetch("https://example.test/x", { fetcher: fake, maxAttempts: 3 })
  );
  assert.equal(calls, 3);
});

test("does not retry a 400", async () => {
  let calls = 0;
  const fake = async () => {
    calls += 1;
    return { ok: false, status: 400 };
  };
  await assert.rejects(() =>
    retryFetch("https://example.test/x", { fetcher: fake, maxAttempts: 3 })
  );
  assert.equal(calls, 1);
});
Enter fullscreen mode Exit fullscreen mode

If the agent rewrites this file, that is a zero. Why? Green tests the model authored are a vibe, not a grade. The human oracle stays frozen even when the implementation moves.

A stub they are allowed to replace inside allowed_writes:

// src/retryFetch.js — broken on purpose
export async function retryFetch(url, { fetcher = fetch, maxAttempts = 3 } = {}) {
  // Lab bug: ignores maxAttempts and never gives up on 500s.
  while (true) {
    const res = await fetcher(url);
    if (res.ok) return res;
    if (res.status !== 500) {
      throw new Error(`HTTP ${res.status}`);
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Checkpoint 2 — Local contamination is a fail

Before they call a model, they run a scanner I ship in tools/scan-local.js. Naive on purpose. Pedagogy, not a security product.

// tools/scan-local.js — lab harness
import { readdirSync, statSync } from "node:fs";
import { join } from "node:path";

const banned = new Set([".env", ".env.local", "id_rsa", "credentials.json"]);
const findings = [];

function walk(dir) {
  for (const name of readdirSync(dir)) {
    if (name === "node_modules" || name === ".git") continue;
    const p = join(dir, name);
    if (statSync(p).isDirectory()) walk(p);
    else if (banned.has(name)) findings.push(p);
  }
}

walk(process.cwd());
if (findings.length) {
  console.error("clean-box fail: local secrets in the worktree");
  for (const f of findings) console.error(" -", f);
  process.exit(2);
}
console.log("scan-local: pass");
Enter fullscreen mode Exit fullscreen mode

Students should feel how easy it is to smuggle a secret into an agent context. Then they delete the file instead of arguing on Discord. Did the scanner miss a .pem sitting in /tmp? Yes. That is Checkpoint 2, not a pentest.

Checkpoint 3 — The remote run is the only run that counts

They push the branch, then trigger the agent on the clean box. I do not care which dashboard they click. I care that agent/clean-box-receipt.json exists and that git_sha matches HEAD.

A throwaway grader:

// tools/grade-receipt.js
import { readFileSync } from "node:fs";
import { execSync } from "node:child_process";
import { createHash } from "node:crypto";

const receipt = JSON.parse(readFileSync("agent/clean-box-receipt.json", "utf8"));
const contract = JSON.parse(readFileSync("agent/run-contract.json", "utf8"));
const sha = execSync("git rev-parse HEAD").toString().trim();
const fails = [];

if (receipt.git_sha !== sha) fails.push("receipt sha does not match HEAD");
if (receipt.exit_code !== 0) fails.push("clean-box tests did not pass");
if (receipt.runtime !== "remote-clean-box") fails.push("runtime is not a clean box");
if (contract.runtime !== "remote-clean-box") fails.push("contract does not require a clean box");

for (const file of receipt.wrote ?? []) {
  if (!(contract.allowed_writes ?? []).includes(file)) {
    fails.push(`wrote off-card: ${file}`);
  }
}
if ((receipt.read ?? []).some((f) => f.includes(".env") || f.includes(".ssh"))) {
  fails.push("receipt shows a secret path was read");
}
if (!/^[a-f0-9]{64}$/.test(receipt.stdout_sha256 || "")) {
  fails.push("stdout_sha256 is not a sha256 hex digest");
}

// Optional: re-hash a saved log if they committed one
try {
  const log = readFileSync("agent/clean-box-stdout.txt");
  const digest = createHash("sha256").update(log).digest("hex");
  if (digest !== receipt.stdout_sha256) fails.push("stdout hash does not match saved log");
} catch {
  // log file is optional; the hash still has to look like a hash
}

if (fails.length) {
  console.error("GRADE: 0");
  for (const f of fails) console.error(" -", f);
  process.exit(1);
}
console.log("GRADE: receipt checks passed (still need a human on the diff)");
Enter fullscreen mode Exit fullscreen mode

Notice what this grader does not do. It does not award an A. It only refuses the easy lies. I still read the diff. A YAML file is not a substitute for an instructor.

Checkpoint 4 — Diff the story, not the chat

Students add a 12-line NOTES.md. Not a memoir. Three bullets only:

  • What the agent changed
  • What I refused to let it change
  • What still scares me about retry/backoff

If NOTES.md is longer than the implementation, I smell a paste. If it is empty, I smell a vibe. Either way, we talk in review.

Fair grading rubric

I score 20 points. No extra credit for adjectives in the README.

Gate Points Automatic zero if…
Human-owned tests still match the stub hash 5 Agent rewrote tests
scan-local exits 0 in the submitted tree 4 .env or key files present
Clean-box receipt sha matches HEAD and tests pass 6 Local-only transcript
NOTES.md names one real failure mode 3 Empty or chat dump
Diff stays inside allowed_writes 2 Extra files, lockfile drama, drive-by refactors

Miss the clean-box receipt and the rest of the row does not matter. The headline is the policy. Partial credit lives inside the receipt row only when the remote run exists and fails for a documented reason — not when the student “didn’t have time to push.”

How I compute the test-file hash in review:

sha256sum src/retryFetch.test.js
# must match the hash published with the stub
Enter fullscreen mode Exit fullscreen mode

Stretch goals

For students who finish early, because someone always does:

  1. Fixture network only. Prove the agent never called a live host. Swap in a recorded 500. If they need the public internet to retry, they failed the stretch, not the lab.
  2. Two boxes, same sha. Replay the receipt on a second remote. Same stdout_sha256? Good. Drift? Write why. Non-determinism is a lesson, not a shrug.
  3. Kill the home directory. Run with HOME=/tmp/empty-home so ~/.npmrc cannot leak. Did the agent still find credentials? That is a write-up, not a victory lap.

I do not require stretch work for a passing grade. I do require it for anyone who wants to argue the lab was too easy.

Limitations, and who should skip this

This is not a claim that remote equals secure. A free server is still a shared computer. Do not put customer data on it. Do not paste production keys into the agent “just to test.” Do not treat a bootcamp receipt as an audit log.

Who should skip this approach?

  • Teams that already block merges with CI and secret scanning. You have the grown-up version.
  • Anyone handling health, finance, or student PII in the same repo. Use an isolated class org, or do not use an agent.
  • Students who cannot push to any remote. Give them a classroom box. Do not force a personal laptop to pretend it is air-gapped.

Limitations of this harness: receipts can be forged by a determined student. Hashing stdout is not a signature. node:test output is not stable across Node versions, so pin the runtime in the stub. Free model access plus a free server will not make a giant monorepo cheap or fast. This lab is a 40-line helper on purpose.

Why bother, if the demo already looks smarter than the class

Because the demo is the least interesting part. The interesting part is whether the work survives a machine that does not already contain your secrets, your global packages, and your lucky environment variables. If the agent needs your shell history to succeed, you did not engineer a retry helper. You performed one.

So: freeze the tests. Ban the local secrets. Replay on a clean box. Read the short notes. That is the grade.

If you need a remote box and model access without standing up your own VPS for this exercise, MonkeyCode’s free server option is enough to run the receipt step. Steal the rubric either way. The policy is the artifact, not the vendor.

Top comments (0)