DEV Community

Charlie Xu
Charlie Xu

Posted on

Grade the Guesses: A Bootcamp Lab With an Assumption Gate

#ai #beginners #tutorial #javascript

Cheap AI code is not the hard part. Unstated assumptions are.

A student types "add a waitlist endpoint." The agent returns JWT, Redis, Postgres, and a welcome email. Looks senior. It is mostly fiction. This lab grades the guess log first. The diff comes second.

If that sounds harsh, good. Bootcamps keep shipping features that never had a chance to be true.

Why this lab exists

I wrote it because pull requests that solve an unassigned product are a grading nightmare.

The brief said POST /waitlist with an email. That is it. No auth story. No provider. No queue. The model filled every silence with a vendor. Students accepted the story because the code compiled.

Sound familiar?

Agents are autocomplete with confidence. Confidence is not evidence. This lab teaches one mechanical habit: no implementation until every material assumption is tagged.

We will use a tiny Node checker, a frozen requirements file, and a four-checkpoint rubric. You can run the whole thing on a laptop. The lab still works if you delete the optional infra paragraph below.

The trap feature

Students get one page. Instructors do not "clarify" in Slack. Ambiguity is the point.

Product brief (frozen):

  • Add POST /waitlist
  • JSON body: { "email": string }
  • Response: 201 with { "ok": true }
  • In-memory storage is acceptable
  • No login, no billing, no email send
  • Deliver in Node.js

That is the entire product. Everything else is a guess.

Lab setup

  1. Fork a starter that contains server.js with a health check only, plus REQUIREMENTS.md (the brief above) and an empty ASSUMPTIONS.md.
  2. Pick any coding agent you already use. Paid, local, browser tab, whatever.
  3. Optional infra for a zero-invoice agent loop: point the agent at MonkeyCode. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The only capabilities I am using here are free model access and a free server option. No model names, quotas, or hardware claims. If those matter for a syllabus, verify them on the product page before you publish dates.
  4. Install Node 20+. The grader needs no extra packages. 
node -v
npm init -y
node scripts/check-assumptions.mjs
Enter fullscreen mode Exit fullscreen mode

If the checker fails on an empty repo, that is correct. An empty assumption log is a failing lab, not a blank canvas.

The artifact: an assumption budget

Students must keep ASSUMPTIONS.md in this shape:

# Assumptions

| id | claim | status | evidence |
|----|-------|--------|----------|
| A1 | POST /waitlist accepts JSON `{ email }` | CONFIRMED | REQUIREMENTS.md |
| A2 | In-memory storage is allowed | CONFIRMED | REQUIREMENTS.md |
| A3 | Duplicate emails should return 409 | GUESSED | not in brief |
| A4 | We should send a confirmation email | REJECTED | out of scope |
Enter fullscreen mode Exit fullscreen mode

Status is a closed enum:

  • CONFIRMED — quoted from REQUIREMENTS.md or a written instructor answer
  • REJECTED — considered and explicitly out of scope
  • GUESSED — the agent (or the student) filled a silence

Budget rule: submitted code may depend on CONFIRMED rows only. GUESSED rows live in the table. They must not appear as branches, dependencies, or env vars. REJECTED rows are not a backlog. They are a fence.

Why a table instead of a vibe-y architecture note? Because I can grade a table. I cannot grade "we thought about architecture."

Decision table

If you see this in the diff Required row Else
jsonwebtoken, sessions, API keys auth assumption, almost always REJECTED fail checkpoint 2
Redis, queues, workers durability / rate-limit assumption fail unless CONFIRMED
nodemailer, SendGrid, SMTP email-send assumption fail
Postgres, Prisma, SQLite file persistence assumption fail unless the brief changed
409 on duplicates uniqueness assumption allowed only if CONFIRMED, or parked as GUESSED with no code

The checker (run this in CI)

Save as scripts/check-assumptions.mjs. It is deliberately picky. Treat it as a lab tool, not a production linter.

#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";

const ROOT = process.cwd();
const MAX_GUESSED_IN_CODE = 0;
const SMELLS = [
  /jsonwebtoken/i,
  /express-session/i,
  /redis/i,
  /nodemailer/i,
  /sendgrid/i,
  /mongoose|prisma|sequelize/i,
  /postgres|mongodb/i,
  /process\.env\.[A-Z0-9_]+/,
];

function read(file) {
  return fs.readFileSync(path.join(ROOT, file), "utf8");
}

function parseAssumptions(md) {
  const rows = [];
  for (const line of md.split("\n")) {
    if (!/^\|\s*A\d+/i.test(line)) continue;
    const cols = line.split("|").map((c) => c.trim()).filter(Boolean);
    if (cols.length < 4) continue;
    rows.push({
      id: cols[0],
      claim: cols[1],
      status: cols[2].toUpperCase(),
      evidence: cols[3],
    });
  }
  return rows;
}

function walk(dir, acc = []) {
  for (const ent of fs.readdirSync(dir, { withFileTypes: true })) {
    if (["node_modules", ".git", "scripts"].includes(ent.name)) continue;
    const p = path.join(dir, ent.name);
    if (ent.isDirectory()) walk(p, acc);
    else if (/\.(js|mjs|cjs|ts)$/.test(ent.name)) acc.push(p);
  }
  return acc;
}

const rows = parseAssumptions(read("ASSUMPTIONS.md"));
if (rows.length < 3) {
  console.error("Need at least 3 assumption rows. Silence is not a design.");
  process.exit(1);
}

const allowed = new Set(["CONFIRMED", "REJECTED", "GUESSED"]);
for (const r of rows) {
  if (!allowed.has(r.status)) {
    console.error(`${r.id} has illegal status ${r.status}`);
    process.exit(1);
  }
  if (r.status === "CONFIRMED" && !/REQUIREMENTS\.md|instructor/i.test(r.evidence)) {
    console.error(`${r.id} is CONFIRMED without evidence`);
    process.exit(1);
  }
}

const guessed = rows.filter((r) => r.status === "GUESSED");
const sourceFiles = walk(ROOT);
const smellHits = [];
for (const file of sourceFiles) {
  const txt = fs.readFileSync(file, "utf8");
  for (const re of SMELLS) {
    if (re.test(txt)) smellHits.push({ file, re: String(re) });
  }
}

if (smellHits.length) {
  console.error("Infrastructure smells need a CONFIRMED or REJECTED row, not hope.");
  for (const h of smellHits) console.error(`  ${h.file} ~ ${h.re}`);
  process.exit(1);
}

if (guessed.length > MAX_GUESSED_IN_CODE) {
  // GUESSED rows are allowed in the table. They are not allowed to drive code.
  // The smell pack above is the cheap proxy for "drove code."
}

console.log(
  `ok: ${rows.length} rows, ${guessed.length} parked guesses, ${sourceFiles.length} source files`
);
Enter fullscreen mode Exit fullscreen mode

Run it like a unit test:

node scripts/check-assumptions.mjs
echo $?   # 0 is the only passing grade for checkpoint 2
Enter fullscreen mode Exit fullscreen mode

Is the smell list complete? No. It is a teaching fence. Students who rename nodemailer to mailer.mjs and call SMTP anyway still fail the human review. The script exists so the obvious fiction dies in CI.

Checkpoints

Checkpoint 0 — freeze the brief

Paste REQUIREMENTS.md into the repo. Do not edit it. If the agent rewrites the brief, that is an automatic zero for this checkpoint.

Why so strict? Because "helpful" rewrites are how a waitlist becomes a growth stack.

Checkpoint 1 — log before code

Fill ASSUMPTIONS.md with at least three rows before server.js gains a route. Commit that file alone. I want a git log that proves the log came first.

Ask the agent a rude prompt and stop there:

Read REQUIREMENTS.md. List every assumption you would need
to implement POST /waitlist. Tag each CONFIRMED, GUESSED,
or REJECTED. Do not write code. Do not add dependencies.
Enter fullscreen mode Exit fullscreen mode

If it still opens a Prisma schema, you have a process bug, not a model bug.

Checkpoint 2 — implement under the budget

Add POST /waitlist. Keep storage in a module-level array. Return 201. No extra packages unless a CONFIRMED row names them. Then run the checker.

A legal happy path looks boring. Boring is the point.

// server.js — labeled example for the lab, not a framework recommendation
import http from "node:http";

const waitlist = [];

const server = http.createServer(async (req, res) => {
  if (req.method === "POST" && req.url === "/waitlist") {
    const chunks = [];
    for await (const c of req) chunks.push(c);
    let body;
    try {
      body = JSON.parse(Buffer.concat(chunks).toString("utf8"));
    } catch {
      res.writeHead(400, { "content-type": "application/json" });
      res.end(JSON.stringify({ error: "invalid_json" }));
      return;
    }
    if (typeof body?.email !== "string" || !body.email.includes("@")) {
      res.writeHead(400, { "content-type": "application/json" });
      res.end(JSON.stringify({ error: "invalid_email" }));
      return;
    }
    waitlist.push({ email: body.email, at: Date.now() });
    res.writeHead(201, { "content-type": "application/json" });
    res.end(JSON.stringify({ ok: true }));
    return;
  }
  res.writeHead(404);
  res.end();
});

server.listen(3000);
Enter fullscreen mode Exit fullscreen mode

Did that @ check invent a validation rule? Yes. Park it as GUESSED or strip it. See how fast the habit shows up?

Smoke test without extra libraries:

node server.js &
curl -sS -D - -o /tmp/wl.json \
  -H 'content-type: application/json' \
  -d '{"email":"dev@example.com"}' \
  http://127.0.0.1:3000/waitlist
cat /tmp/wl.json
Enter fullscreen mode Exit fullscreen mode

You want 201 and {"ok":true}. Anything else is either a broken student or a broken brief. Do not "fix" it by adding Redis.

Checkpoint 3 — mutate the brief

Instructor adds one sentence: duplicate emails return 409. Students must:

  1. Move uniqueness from GUESSED to CONFIRMED with evidence pointing at the new sentence
  2. Change the code
  3. Re-run the checker

If they change code without touching the table, they fail even if the HTTP behavior is right. We are grading the coupling, not the status code.

Stretch goals

  • Add a second smell pack for Python or Go if your cohort is polyglot. Same table. Different walker.
  • Replace regex smells with a lockfile diff: any new dependency needs a CONFIRMED row that names the package.
  • Record a 3-minute walkthrough of the student arguing with the agent: "Do not add Redis." Use that as evidence for checkpoint 1.
  • Turn REJECTED rows into tests that must not pass. A rejected welcome-email is a test that asserts no SMTP call happened.

Fair grading rubric (100 points)

Area Points Pass bar
Checkpoint 0: unmodified REQUIREMENTS.md 10 file hash matches the handout
Checkpoint 1: assumption commit before route commit 20 git log order is checkable
Table quality 20 ≥3 rows, legal statuses, CONFIRMED has evidence
Checkpoint 2: checker exit 0 + 201 path 25 curl shown in the README
Checkpoint 3: brief mutation reflected in table and code 15 409 only after the table update
Human review: no silent vendors 10 reviewer can name every extra import

Automatic zeros: rewriting the brief, committing secrets, adding a paid API call to "finish" the lab, or deleting the checker.

I do not grade prose style. I do not grade how "production ready" the waitlist looks. Production-ready was how we got a fake email pipeline.

Limitations (read this before you copy the rubric)

This is a teaching protocol. It is not an architecture review board.

  • The checker is heuristic. Clever students can hide SMTP behind https.request. That is why 10 points stay human.
  • An assumption table is not threat modeling. POST /waitlist still needs abuse thinking if you ever put it on the internet. This lab should stay on localhost.
  • Free model access and a free server option can change. Do not print a course catalog promise that depends on one vendor remaining free. Keep a local-model backup.
  • English-only tables punish some cohorts. Translate the status enum if you need to. Do not drop the enum.
  • Who should not use this: seniors who already write ADRs; capstone teams talking to real users (they need discovery, not a guess budget of zero); anyone treating a free server as production hosting.

If your class is "build anything," this lab will feel like a muzzle. Use it in week 2, not week 12.

What I want students to feel

The agent will still guess. That is its job. Your job is to make the guess visible, cheap, and fireable.

Run the checker. Fail on purpose once. Then ask the model to list assumptions and stop. That prompt is the whole course if you are short on time.

Steal the checker either way. If you need a zero-invoice box for the agent half of the exercise, MonkeyCode's free model access and free server option are how this lab keeps the infra row of the syllabus empty.

Top comments (0)