DEV Community

Charlie Xu
Charlie Xu

Posted on

If You Can't Show the Receipt, It's a Zero: A Bootcamp Lab on Agent Spend Caps

An agent that finishes the kata and cannot show a receipt did not finish the kata. It hid a bill. In this bootcamp that is a zero. Not a vibe. A gate.

Why? Because "it works on my key" is not a skill. It is a subsidy. Someone is paying, and it is usually a personal credit card, a leftover cloud coupon, or a teammate's token pasted into chat. I do not grade subsidies. I grade the ledger.

The failure I keep seeing

A pair ships a tool-calling loop. Looks sharp. Then I ask three questions.

  1. Which step spent, and on what kind of call?
  2. How many retries happened after the first miss?
  3. What happens when the next call would cross the cap?

Silence. Or a shrug. Or "we used the default." Default of what?

Tool calling made this worse, not better. Every extra tool is another round trip. Another chance to retry. Another silent spend. If your agent can search, read, and patch in a loop, the cost is the loop, not the prompt. You already knew that. Did you record it?

I want the ledger in the repo. Same folder as the tests.

This is not the allowlist lab

Different zero. Different artifact.

Allowlists answer "may this tool run at all?" Spend caps answer "even if it may run, is the next unit priced, and does the runner refuse when the sum would blow the ceiling?" An allowed tool with no price is still a zero here. Unpriced is unbounded. Unbounded is a blank check with extra steps.

We are not hunting secrets. We are not mutating tests. We are not asking whether the patch still passes after I delete it. We are asking whether the run can explain its own bill.

What you are building

This lab is not "use a cheaper model." Cheap is not the point. Bounded is the point.

You will ship four files:

  • spend-cap.json — the contract the grader trusts
  • ledger.ndjson — one line per model or tool call
  • agent.mjs — a runner that refuses the next charge when it would exceed the cap
  • grade.mjs — the staff grader you run before you ping me

If I delete ledger.ndjson and your grade still passes, you cheated the lab. The receipts are the work.

Lab fixture note

Everything below is a classroom fixture. It is not a vendor benchmark, not a production billing pipeline, and not a claim about any model's real price. We grade the shape of the evidence. Units in the cap file are cohort placeholders you set on day one.

Setup

You need Node 20+. You do not need a paid API key for the happy path. You do need a writable directory and the habit of treating spend as a test double.

mkdir -p spend-lab && cd spend-lab
node -v
# expect v20 or newer
Enter fullscreen mode Exit fullscreen mode

Write the cap first. Not the agent. The cap is the spec. The agent is just a thing that must lose to the spec.

{
  "schema": "bootcamp.spend-cap.v1",
  "assignment": "week-07-tool-loop",
  "unit": "lab-tokens",
  "cap": 8000,
  "fail_closed": true,
  "weights": {
    "model_call": 400,
    "tool:search": 50,
    "tool:read_file": 10,
    "tool:apply_patch": 80
  },
  "notes": "lab-tokens are classroom units, not a vendor invoice"
}
Enter fullscreen mode Exit fullscreen mode

Read that cap out loud. Eight thousand lab-tokens. If your loop retries a model call twenty times, you are done before you are clever. That is the lesson. Want a bigger brain? Fine. Price it. Then watch the ceiling catch you.

The ledger format

One JSON object per line. No commentary. No trailing commas. If a line fails JSON.parse, it is a zero. I will not "almost" parse your diary.

{"ts":"2026-09-23T15:04:11Z","kind":"model_call","step":"plan","units":400,"running_total":400,"cap":8000}
{"ts":"2026-09-23T15:04:12Z","kind":"tool:search","step":"find-todo","units":50,"running_total":450,"cap":8000}
Enter fullscreen mode Exit fullscreen mode

Notice what is missing. Vendor invoice fields. Provider-specific usage blobs. Five SDKs. We do not parse those in week seven. We parse our cap. If a vendor dashboard and this file disagree later, the lab still grades the file. That is intentional. Classroom evidence has to be local and boring.

Checkpoint 0: the runner refuses

Here is a stub you can actually run. It does not call a network model. It records a would-be call and dies if the next line would cross the cap. Fail-closed is the whole lab. A diary without teeth is journaling.

// agent.mjs
import { readFileSync, appendFileSync } from "node:fs";

const cap = JSON.parse(readFileSync("./spend-cap.json", "utf8"));
const ledgerPath = process.env.LEDGER_PATH || "./ledger.ndjson";

function runningTotal() {
  try {
    return readFileSync(ledgerPath, "utf8")
      .trim()
      .split("\n")
      .filter(Boolean)
      .map((line) => JSON.parse(line))
      .reduce((sum, row) => sum + row.units, 0);
  } catch {
    return 0;
  }
}

export function charge(kind, step) {
  const units = cap.weights[kind];
  if (units == null) {
    throw new Error(`unpriced kind: ${kind}`);
  }
  const current = runningTotal();
  const next = current + units;
  if (cap.fail_closed && next > cap.cap) {
    throw new Error(`spend trip: ${next} > ${cap.cap} on ${kind}`);
  }
  const row = {
    ts: new Date().toISOString(),
    kind,
    step,
    units,
    running_total: next,
    cap: cap.cap,
  };
  appendFileSync(ledgerPath, JSON.stringify(row) + "\n");
  return row;
}

// demo loop — labeled fixture, not a live agent
const steps = [
  ["model_call", "plan"],
  ["tool:search", "find"],
  ["tool:read_file", "open"],
  ["model_call", "plan-again"],
];

for (const [kind, step] of steps) {
  console.log(charge(kind, step));
}
Enter fullscreen mode Exit fullscreen mode

Run it.

: > ledger.ndjson
node agent.mjs
Enter fullscreen mode Exit fullscreen mode

Then crank a retry storm. Change the loop to twenty model_call steps. Does it throw? If it keeps going, you built a diary, not a cap. Caps have teeth. Show me the stack trace in your notes.

The grader I actually run

Students run this before they @ me. I run the same file. No private tests for the zero conditions. Fairness is the point. If the zero is secret, it is not a teaching zero.

// grade.mjs
import { readFileSync, existsSync } from "node:fs";

const cap = JSON.parse(readFileSync("./spend-cap.json", "utf8"));
const ledgerPath = process.env.LEDGER_PATH || "./ledger.ndjson";

if (!existsSync(ledgerPath)) {
  console.error("ZERO: no ledger");
  process.exit(2);
}

const rows = readFileSync(ledgerPath, "utf8")
  .trim()
  .split("\n")
  .filter(Boolean)
  .map((line, i) => {
    try {
      return JSON.parse(line);
    } catch {
      console.error(`ZERO: ledger line ${i + 1} is not JSON`);
      process.exit(2);
    }
  });

if (rows.length === 0) {
  console.error("ZERO: empty ledger");
  process.exit(2);
}

if (cap.fail_closed !== true) {
  console.error("ZERO: fail_closed must be true");
  process.exit(2);
}

let sum = 0;
for (const row of rows) {
  const expected = cap.weights[row.kind];
  if (expected == null) {
    console.error(`ZERO: ${row.kind} is unpriced`);
    process.exit(2);
  }
  if (row.units !== expected) {
    console.error(`ZERO: ${row.kind} units ${row.units} != cap ${expected}`);
    process.exit(2);
  }
  sum += row.units;
  if (sum > cap.cap) {
    console.error(`ZERO: running total ${sum} exceeded cap ${cap.cap}`);
    process.exit(2);
  }
  if (row.running_total !== sum) {
    console.error(`ZERO: row running_total ${row.running_total} != ${sum}`);
    process.exit(2);
  }
}

console.log(`PASS: ${rows.length} rows, ${sum}/${cap.cap} lab-tokens`);
Enter fullscreen mode Exit fullscreen mode
node grade.mjs
Enter fullscreen mode Exit fullscreen mode

Three zero conditions are load-bearing. Missing file. Unpriced kind. Arithmetic that does not match the running total. I do not want a vibe-based "we were probably under." Probably is how invoices happen.

Checkpoints

Work in order. Do not skip to a bigger remote model because the stub feels toy. The stub is the test harness. The remote call is optional clothing.

  1. Cap validates. cap is a positive number, fail_closed is true, every weight is a non-negative number.
  2. One honest row. A single model_call line, grader says PASS.
  3. Fail-closed demo. Force a trip. Paste the throw. A green run with a deleted ledger is not a demo.
  4. Kinds subset of the cap file. If you invent tool:shell, the grader zeros you. Unpriced tools are how bills happen.
  5. Running total is monotonic and exact. No "approximately." No hand-edited last line.

Pass those five and you have a C. Yes, a C. The letter grade lives in the rubric below, not in the checkpoint list. Checkpoints get you into the room. The rubric decides whether you leave with the credit.

Where a no-personal-key lane actually helps

I used to tell students to "just use your own key." That leaked into every other assignment. It also meant the student with a corporate account could retry forever, and the student on a phone hotspot could not. Same rubric, different wallet. That is not a lab. That is an accident.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

For this lab I point the cohort at MonkeyCode's free model access and free server option so the runner can live on a shared box instead of a personal credit card. The spend cap still wins. The product is a lane, not the grade. Strip the product name out of this writeup and grade.mjs does not change. That is on purpose.

I am not going to quote a token quota, a hardware spec, a model menu, or a permanence promise. Those numbers move. Your cap file should not. Pin what the assignment allows. Ceiling what the assignment allows. Receipt what actually ran.

A free server also makes the noisy-neighbor problem real. Two students writing ledger.ndjson in the same directory will corrupt each other's evidence. So namespace it. If you skip this, I cannot tell whose retries I am reading.

export LAB_NS="xu-week07"
mkdir -p "runs/$LAB_NS"
export LEDGER_PATH="runs/$LAB_NS/ledger.ndjson"
: > "$LEDGER_PATH"
node agent.mjs
LEDGER_PATH="$LEDGER_PATH" node grade.mjs
Enter fullscreen mode Exit fullscreen mode

If your ledger lives in a homedir you cannot show me, I cannot grade you. Shared box, shared evidence path, per-student namespace. Boring. Correct. Want to be fancy with databases? Not this week. Files I can cat beat dashboards I cannot open.

Stretch goals

Do these after the zeros are gone. Not before. Stretch on top of a missing ledger is fan fiction.

  • Dry-run estimator. A --estimate flag that prints the would-be total without appending rows. Wrong estimates are fine. Missing flags are not, if you claim the stretch.
  • Per-kind ceiling. model_call may cost 400, but you may not plan more than three times. Encode max_count next to weights. Exceed it and throw a different error than the sum trip. I want to see that you can tell "too expensive" from "too often."
  • Replay from the ledger only. Given just ledger.ndjson, print a timeline of kinds and steps. If you cannot explain the loop from the file, the file is decoration.
  • Adapter swap without cap swap. Keep lab-tokens. Change only the function that would have called a vendor. If the weights change when the provider changes, you were billing the vendor, not the assignment.

Fair grading rubric

I publish this on day one. No surprise zeros. If a gate is binary, you cannot average your way out of it.

Gate Points Zero if
Ledger exists and parses 20 file missing, empty, or invalid JSON
Kinds are priced on the cap 20 any unpriced kind
Arithmetic matches 20 sum, running_total, or silent cap breach
Fail-closed demonstrated 20 retries continue past cap
Namespace on the shared box 10 two students clobber one file
Stretch (estimator or max_count) 10 claimed but not runnable

Binary zeros on the first four gates. You cannot average your way out of an unbounded loop. Partial credit exists only on namespace and stretch. I will not grade prompt poetry. I will not grade "we would have stopped." Show the throw.

How do you demonstrate fail-closed without failing the grader? Two artifacts. ledger.ndjson is the successful bounded run. trip.txt is the stderr from a forced over-cap loop. I read both. If you only give me the green path, you taught the agent to stay lucky.

node agent.mjs 2> trip.txt || true
# trip.txt must contain "spend trip" when you crank retries
Enter fullscreen mode Exit fullscreen mode

Limitations, and who should skip this

This fixture does not know the real price of a completion. Provider dashboards disagree with local counters. Streaming tokens, cached prompts, tool-result size, retries inside an SDK you did not wrap — all of that is out of scope. If you need invoice-grade accounting, stop. Use the vendor bill and a finance person.

Free model access and a free server are availability options, not SLAs. Do not put a customer-facing agent on a classroom cap and call it FinOps. Do not use this rubric for production incident review. Do not use it to shame a teammate who exceeded a number you never wrote down. A cap that lived in your head is not a cap.

Skip this lab if you are not allowed to share traces, if your org forbids third-party model lanes, or if the assignment already has a hard offline rule with no network. In that last case, keep the cap, drop the remote calls, and charge model_call as a local stub. The discipline transfers. The vendor does not have to.

Also skip it if your "agent" is a single completion with no loop. A one-shot script with a hardcoded unit of 400 teaches almost nothing. The interesting bug is the second plan, the third search, the patch you applied twice. No loop, no lab.

What I want in your writeup

Three short answers. Not a blog post. If you write me six pages, I will still only score these.

  1. Where did the runner throw, and which ledger line was last?
  2. What unpriced tool were you tempted to add, and why did you refuse?
  3. If I cut the cap in half, which step dies first?

If you can answer those, you understood the lab. If you cannot, you ran a demo. Steal grade.mjs either way. If you need a no-personal-key lane so the whole cohort hits the same ceiling, MonkeyCode's free model access and free server option is the lane I send people to — then I still grade the receipts.

Top comments (0)