DEV Community

Charlie Xu
Charlie Xu

Posted on

If the Tool Isn't on the Card, It's a Zero: A Bootcamp Lab on Agent Allowlists

#javascript #ai #beginners #testing

Stop grading the chat. Grade the tools the agent was allowed to touch.

If a student agent can shell out, hit the network, or install packages that were never on the assignment card, I do not care how pretty the final function looks. That is not a lab. That is a jailbreak with extra steps.

Why this lab exists

A lot of the current “AI already codes better than us” noise skips the classroom problem. Models can write a reducer. They can also curl | bash while they are at it. You want engineering? Then you want a capability card.

Write-set contracts catch files. Halt contracts catch infinite loops. Token receipts catch silent overspend. This lab is none of those. It catches undeclared tools.

What good is a green test suite if the agent fetched the answer from a gist? What good is a refactor if npm install mutated the lockfile behind your back?

The assignment in one sentence

Ship a tiny agent wrapper. The wrapper may only dispatch tools listed on capability-card.json. One undeclared call, and the run is a zero. No debate.

I am not asking you to build AutoGPT. I am asking you to prove the model cannot wander.

Lab setup

You need Node 20+, a frozen fixture repo, and a model that can emit tool calls as JSON. Local Ollama is fine. A classmate’s laptop is fine. A shared box with free model access is fine. The grade does not care which model you used. The grade cares whether the dispatcher obeyed the card.

node -v   # v20 or newer
npm init -y
npm pkg set type=module
mkdir -p src test fixtures/safe-lib
Enter fullscreen mode Exit fullscreen mode

Put a tiny library in the fixture so the agent has real work to do:

// fixtures/safe-lib/sum.js
export function sum(a, b) {
  if (typeof a !== "number" || typeof b !== "number") {
    throw new TypeError("sum() expects two numbers");
  }
  return a + b;
}
Enter fullscreen mode Exit fullscreen mode

The student agent’s job: add mean.js, keep tests green, and never leave the allowlist. Sounds easy. Watch what happens when the model “helpfully” runs curl.

The capability card

This file is the syllabus. If a tool is not here, it does not exist.

{
  "assignment": "mean-from-sum",
  "tools": [
    { "name": "read_file", "args": ["path"] },
    { "name": "write_file", "args": ["path", "contents"] },
    { "name": "run_tests", "args": [] }
  ],
  "pathAllow": ["fixtures/safe-lib/**", "test/**"],
  "denied": ["shell", "http", "git", "install_pkg"]
}
Enter fullscreen mode Exit fullscreen mode

Denied is not decorative. I want students to name the temptations out loud. If you cannot name the dangerous tool, you will not notice when the model invents it.

The dispatcher (this is the whole lab)

Do not let the model call functions directly. Parse a JSON tool call. Route it. Log it. Reject everything else.

// src/dispatch.js
import { readFile, writeFile } from "node:fs/promises";
import { spawn } from "node:child_process";
import { minimatch } from "minimatch";
import card from "../capability-card.json" with { type: "json" };

const allowed = new Map(card.tools.map((t) => [t.name, t]));

export class UndeclaredToolError extends Error {
  constructor(name) {
    super(`undeclared tool: ${name}`);
    this.name = "UndeclaredToolError";
  }
}

function assertPath(path) {
  const ok = card.pathAllow.some((pattern) => minimatch(path, pattern));
  if (!ok) throw new UndeclaredToolError(`path:${path}`);
}

export async function dispatch(call, log = []) {
  if (!call || typeof call.name !== "string") {
    throw new UndeclaredToolError("malformed-call");
  }
  if (!allowed.has(call.name)) {
    log.push({ t: Date.now(), name: call.name, ok: false, reason: "not-on-card" });
    throw new UndeclaredToolError(call.name);
  }

  const spec = allowed.get(call.name);
  for (const key of spec.args) {
    if (!(key in (call.args || {}))) {
      throw new UndeclaredToolError(`missing-arg:${call.name}.${key}`);
    }
  }

  let result;
  if (call.name === "read_file") {
    assertPath(call.args.path);
    result = await readFile(call.args.path, "utf8");
  } else if (call.name === "write_file") {
    assertPath(call.args.path);
    await writeFile(call.args.path, call.args.contents, "utf8");
    result = { wrote: call.args.path.length };
  } else if (call.name === "run_tests") {
    result = await runNodeTests();
  }

  log.push({ t: Date.now(), name: call.name, ok: true });
  return { result, log };
}

function runNodeTests() {
  return new Promise((resolve, reject) => {
    const child = spawn("node", ["--test", "test/mean.test.js"], {
      stdio: ["ignore", "pipe", "pipe"],
    });
    let out = "";
    child.stdout.on("data", (c) => (out += c));
    child.stderr.on("data", (c) => (out += c));
    child.on("close", (code) => resolve({ code, out }));
    child.on("error", reject);
  });
}
Enter fullscreen mode Exit fullscreen mode

Notice run_tests is a named tool, not a generic shell. That is the whole trick. If you expose shell, the allowlist is a sticker on a wrecking ball.

Install the one extra dependency the harness needs:

npm install minimatch
Enter fullscreen mode Exit fullscreen mode

Checkpoint 0 — prove the cheat fails

Before any model is involved, write a fixture that tries to be naughty. If this test does not fail the cheater, your lab is theater.

// test/allowlist.test.js
import { test } from "node:test";
import assert from "node:assert/strict";
import { dispatch, UndeclaredToolError } from "../src/dispatch.js";

test("shell is a zero even if the model begs", async () => {
  const log = [];
  await assert.rejects(
    () => dispatch({ name: "shell", args: { cmd: "curl https://example.com" } }, log),
    UndeclaredToolError,
  );
  assert.equal(log.at(-1).ok, false);
});

test("http is a zero", async () => {
  await assert.rejects(
    () => dispatch({ name: "http", args: { url: "https://example.com" } }),
    UndeclaredToolError,
  );
});

test("write outside the fixture is a zero", async () => {
  await assert.rejects(
    () => dispatch({
      name: "write_file",
      args: { path: ".env", contents: "stolen=1" },
    }),
    UndeclaredToolError,
  );
});

test("declared read still works", async () => {
  const { result } = await dispatch({
    name: "read_file",
    args: { path: "fixtures/safe-lib/sum.js" },
  });
  assert.match(result, /export function sum/);
});
Enter fullscreen mode Exit fullscreen mode

Run it:

node --test test/allowlist.test.js
Enter fullscreen mode Exit fullscreen mode

Four tests. Three punches. One allowed read. If that file is green, you have a lab. If it is not, you have a blog post.

Checkpoint 1 — the agent loop is boring on purpose

The loop is not the product. The card is.

// src/loop.js
import { dispatch } from "./dispatch.js";

export async function runTurn(modelCall) {
  // modelCall is already-parsed JSON from whatever client you use.
  // Label: this is a lab stub, not a production agent runtime.
  const log = [];
  const { result, log: next } = await dispatch(modelCall, log);
  return { result, log: next };
}
Enter fullscreen mode Exit fullscreen mode

Want a fake model for CI? Good. Do not hit a paid API from GitHub Actions just to prove shell is banned.

// src/fake-model.js
export function scriptedModel(turns) {
  let i = 0;
  return async function next() {
    if (i >= turns.length) return { type: "halt" };
    return turns[i++];
  };
}
Enter fullscreen mode Exit fullscreen mode

Then feed it a cheating transcript and a honest transcript. Grade both. The honest one should write mean.js and call run_tests. The cheating one should die on http.

Checkpoint 2 — a real student test file

The oracle stays human. The model does not get to invent the rubric.

// test/mean.test.js
import { test } from "node:test";
import assert from "node:assert/strict";
import { mean } from "../fixtures/safe-lib/mean.js";

test("mean of two numbers", () => {
  assert.equal(mean(2, 4), 3);
});

test("rejects non-numbers", () => {
  assert.throws(() => mean("2", 4), TypeError);
});
Enter fullscreen mode Exit fullscreen mode

Yes, mean.js does not exist yet. That is the point. The agent must write it through write_file. If it shells out to echo instead, the card fails the run even if the file appears.

How did the file get there? If the log cannot answer, the grade is a zero.

Where a free model and a free server actually help

I do not need a frontier model to teach allowlists. I need every student to run the same dispatcher. Paid keys turn the lab into a wallet check. That is a bad rubric.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

If a cohort has no API budget, MonkeyCode’s free model access and free server option is one way to put the harness on a shared box so the allowlist tests run for everyone. Use it as a lab machine, not as a substitute for the card. The product does not grade the assignment. Your node --test output does.

Would I ship production secrets to a shared classroom server? No. This is a fixture repo with sum.js. Keep it that way.

Stretch goals (pick one, not all)

  1. Argument schemas. Reject write_file if contents is not a string, or if path contains ...
  2. Host pins. If you later add a narrow http tool, allow only https://registry.npmjs.org and still fail curl https://pastebin.com. Most labs should not add http at all.
  3. Call budgets. read_file at most 20 times. Infinite “just one more read” is a loop problem wearing a tool costume.
  4. Dry-run mode. Print the would-be writes without touching disk. Then compare the dry-run log to the real log. Drift is a zero.

Stretch is optional. The card is not.

Fair grading rubric

I grade the log, not the vibes.

Score What I need to see
0 Any undeclared tool call, including malformed JSON treated as a tool
0 Write outside pathAllow, even if tests later pass
40 allowlist.test.js green on the cheater fixtures
20 Honest scripted transcript writes mean.js only via write_file
20 test/mean.test.js green after the honest run
10 capability-card.json names at least three denied tools in plain language
10 README explains why shell is not an alias for run_tests

Partial credit exists. Sympathy for “the model did it” does not. The wrapper is the student work. The model is a noisy peripheral.

Late work: I accept a late card. I do not accept a late shell call.

What this lab is not measuring

It does not measure code taste. A clumsy mean that stays inside the card still beats an elegant mean that curl’d a solution. It does not measure model ranking. Swap the model tomorrow; the cheater fixtures must still fail.

It is also not a security audit. minimatch plus a JSON file is a teaching fence, not a sandbox. A determined process can still do process things if you hand it shell. So don’t.

Who should not use this approach

  • Teams handling real credentials, customer data, or deploy keys. Use a real sandbox, not a lab dispatcher.
  • Instructors running a beauty contest between models. This rubric will look unfair to the “smarter” model that likes extra tools.
  • Students who want to vibe a whole app in one sitting. Cool. That is a different course. This one fails undeclared tools.

If your agent platform already has OS-level isolation, keep it. This lab is for people who currently let a chat UI call whatever function the JSON mentioned.

Closing the loop

Run the cheater tests. Run the honest transcript. Read the log out loud.

Did every call appear on the card? Then the student shipped engineering. Did the model improvise a tool because it felt helpful? Then it is a zero, and the chat transcript is not homework.

Need a shared box so nobody is blocked on API keys? MonkeyCode’s free model access and free server option can host this harness for a cohort. Fork the dispatcher either way. The card is the lesson.

Top comments (0)