Workshop: Teach Token Budgets With a Reserve–Commit–Refund Ledger (80 Minutes)

Admission control, not caching, is the cheapest reliability feature you can add to a shared model budget. A naive "check the counter, then call the model" loop overshoots under concurrency, and the overshoot stays silent until the provider starts returning quota errors midway through a class. This workshop teaches an 80-minute fix: a reserve–commit–refund ledger that fails closed, plus four exercises students can rerun on their own machines.

The lab targets a shared free-tier grant rather than a paid account. Operator-supplied context for this workshop: MonkeyCode is presented as an open-source project that offers free model access, a free server option, and a 10-million-token grant for evaluation.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat the grant size, the model list, and the quota rules as operator-supplied planning inputs, and re-check the project page before you teach. Free-tier terms change, and a workshop built on stale numbers teaches the wrong lesson. Everything else below is code you can run and arithmetic you can audit.

Workshop at a glance

Format: one 80-minute block, or two 40-minute blocks with the schema walkthrough moved to homework. Audience: developers who already make HTTP calls to a model endpoint but have never had to enforce a budget themselves. Prerequisites: Python 3.10+, SQLite 3.35+ (sqlite3 --version), and a terminal where a small threading script can run.

Deliverable: ledger.py, a short module with reserve(), settle(), and a stale-hold sweep, plus a test that proves the naive version overshoots and the ledger version does not.

Agenda with timing

Minutes	Block	Output
0–8	Framing: why counters fail under concurrency	A written definition of "fail closed"
8–22	Exercise 1: reproduce the overshoot	Failing test, recorded overshoot
22–32	Schema walkthrough	Two tables, two invariants
32–52	Exercise 2: implement reserve and settle	Passing concurrency test
52–64	Exercise 3: expiry and crash recovery	Stale reservations reclaimed
64–74	Exercise 4: reconcile against a usage export	Drift report per scope
74–80	Debrief: decision table, when not to use this	One page of notes

These timings are teaching estimates, not measurements. Add ten minutes if students install SQLite tooling from scratch.

Exercise 1 — reproduce the overshoot (14 min)

Students write the loop almost everyone writes first:

# naive.py — intentionally broken
import sqlite3
GRANTED = 10_000_000

def naive_take(conn, estimate):
    used = conn.execute("SELECT committed FROM budget").fetchone()[0]
    if used + estimate > GRANTED:            # read
        return False
    conn.execute("UPDATE budget SET committed = committed + ?", (estimate,))  # write
    return True

The read and the write are not one atomic step. Run it from twenty threads with estimate=1200 and a grant small enough to bind, and the final committed value lands above GRANTED.

The recorded overshoot becomes the baseline the rest of the workshop beats. Ask students to write the overshoot count down rather than the phrase "it broke"; the number is what makes the next exercise feel earned.

Schema walkthrough (10 min)

Two tables carry the whole idea, and the instructor states two invariants that students repeat back before writing any code.

1. reserved + committed <= granted at every commit boundary.
2. A reservation in state held is charged against the budget even though no tokens have been spent yet.

CREATE TABLE budget (
  scope TEXT PRIMARY KEY, granted INTEGER NOT NULL,
  reserved INTEGER NOT NULL DEFAULT 0, committed INTEGER NOT NULL DEFAULT 0);
CREATE TABLE reservation (
  id TEXT PRIMARY KEY, scope TEXT NOT NULL, estimate INTEGER NOT NULL,
  state TEXT NOT NULL, created_at REAL NOT NULL, expires_at REAL NOT NULL);

Open the connection with isolation_level=None and wrap each mutation in BEGIN IMMEDIATE. SQLite then serialises writers instead of letting two transactions both read the old counter.

Exercise 2 — reserve and settle (20 min)

def reserve(conn, scope, estimate, ttl=120.0):
    now = time.time()
    conn.execute("BEGIN IMMEDIATE")
    try:
        stale = conn.execute(
            "SELECT COALESCE(SUM(estimate),0) FROM reservation "
            "WHERE scope=? AND state='held' AND expires_at < ?", (scope, now)).fetchone()[0]
        conn.execute("UPDATE reservation SET state='expired' "
                     "WHERE scope=? AND state='held' AND expires_at < ?", (scope, now))
        conn.execute("UPDATE budget SET reserved = reserved - ? WHERE scope=?", (stale, scope))
        granted, reserved, committed = conn.execute(
            "SELECT granted, reserved, committed FROM budget WHERE scope=?", (scope,)).fetchone()
        if reserved + committed + estimate > granted:
            conn.execute("COMMIT")
            return None                     # fail closed, by design
        rid = uuid.uuid4().hex
        conn.execute("INSERT INTO reservation VALUES (?,?,?,'held',?,?)",
                     (rid, scope, estimate, now, now + ttl))
        conn.execute("UPDATE budget SET reserved = reserved + ? WHERE scope=?", (estimate, scope))
        conn.execute("COMMIT")
        return rid
    except Exception:
        conn.execute("ROLLBACK")
        raise

settle() moves the estimate out of reserved and books the actual usage into committed, so an over-estimate is refunded automatically at the end of the call.

def settle(conn, rid, actual):
    conn.execute("BEGIN IMMEDIATE")
    try:
        row = conn.execute(
            "SELECT scope, estimate, state FROM reservation WHERE id=?", (rid,)).fetchone()
        if row is None or row[2] != "held":
            conn.execute("ROLLBACK")
            return False
        scope, estimate, _ = row
        conn.execute("UPDATE reservation SET state='committed' WHERE id=?", (rid,))
        conn.execute(
            "UPDATE budget SET reserved = reserved - ?, committed = committed + ? WHERE scope=?",
            (estimate, actual, scope))
        conn.execute("COMMIT")
        return True
    except Exception:
        conn.execute("ROLLBACK")
        raise

Students rerun the twenty-thread test and expect zero overshoot plus a handful of None returns. A None is the correct answer here, not a bug, and counting refusals shows the room the trade: strictness costs throughput.

Exercise 3 — expiry and crash recovery (12 min)

A process that dies between reserve() and settle() leaks budget forever, which is the failure mode that surprises students most. The sweep in reserve() already reclaims stale holds in the same transaction as the admission check, so crashes resolve on the next request rather than on a cron job.

Set ttl above your p99 request latency and below your patience; 120 seconds is a defensible default for chat-sized calls. Kill a worker mid-request, wait out the TTL, and show the reclaimed budget with SELECT state, COUNT(*) FROM reservation GROUP BY state;.

Exercise 4 — reconcile against actuals (10 min)

Estimates are guesses, so the local ledger drifts from provider-reported usage. Students export a usage summary, diff it against committed, and record drift as a percentage per scope.

If drift is consistently one-directional, the fix belongs in the estimator rather than the ledger. Under-counting is the dangerous direction, because it hides headroom you do not actually have.

A worked example students can rerun

Twenty seats, a 10,000,000-token grant as the planning input, 500,000 tokens per seat, and 1,200 tokens per estimated request. The arithmetic below is illustrative; none of it is a benchmark of any endpoint.

• Naive counter, twenty concurrent writers: with a per-seat grant of 500,000 and requests estimated at 1,200 tokens, several threads can pass the check on the final slot before any of them writes, so the excess equals whatever your test prints.
• Ledger, same twenty threads: the excess is structurally zero, because the check and the decrement share one BEGIN IMMEDIATE transaction.
• Cost of correctness: refusals arrive earlier and the room should record how many requests were refused before exhaustion.

Students rerun the whole thing with GRANTED=60_000, estimate=1_200, and twenty threads in under a minute. Small grants make the race visible, while large grants hide it comfortably.

Decision table: what to do when the budget runs out

Situation	Policy	Reason
Classroom demo on a shared grant	Fail closed with a clear error	Predictability beats one extra request
Interactive product with a paid fallback	Soft degrade: route to fallback, keep the ledger	The ledger stays the source of truth
Overnight batch job	Queue and retry after refresh	Refusals are cheap when latency is free
Anything billing-grade	Do not use this ledger	Provider-side accounting is authoritative

Limitations and who should not use this

SQLite with BEGIN IMMEDIATE is a single-node answer, and it stops being sufficient the moment you run multi-writer clusters or demand sub-10ms admission checks. Cross-region deployments need a server-side quota or a transactional store instead.

The ledger tracks your estimate of usage and cannot see tokens the provider counts differently. If invoices must match to the token, reconcile against provider reports and treat local numbers as advisory rather than authoritative.

Finally, skip this entirely if the provider already enforces a hard quota with clear errors. Wrapping an existing enforcement layer in a second one adds a failure mode without removing any.

If you want to run the exercises without provisioning a machine or a paid key, the free server option and the free model access in MonkeyCode are the two operator-supplied pieces that make the lab runnable on day one; the ledger code itself stays provider-agnostic.