Reserve, Then Reconcile: A Token Ledger That Stops a Free-Tier Side Project at the Cap

A free throughput allowance behaves like a budget only when a program refuses to spend past it. Everything else is a dashboard that reports the overspend after the fact.

This build log covers a one-weekend side project: a reservation-based token ledger, roughly 90 lines of Python over SQLite, wrapped around every model call. Two findings came out of the exercise — one about concurrent retries, one about the free server that hosts the demo.

What shipped, and what was cut

A weekend is enough for a guard that fails closed. It is not enough for a platform.

Shipped:

1. ledger.py — a reserve/settle/sweep API over a single SQLite table.
2. guard.py — a wrapper that reserves before a request and reconciles after it.
3. Three pytest cases, including a thread-based overspend test.
4. smoke.sh — five wake-ups against the deployed demo, reporting time-to-first-200 and whether the ledger survived.

Skipped on purpose:

• Authentication and multi-tenant caps. One operator, one cap, one file.
• Dollar pricing. Tokens are counted; currency conversion is a separate problem that changes weekly.
• Dashboards. The sqlite3 CLI answered every question asked that weekend.
• Distributed caps. One host, one ledger. Cross-machine coordination was deferred, not half-built.

The core idea: charge the estimate, reconcile with the actual

A reservation system needs one row per call, not two. Each row starts charged at its estimate and is rewritten to the actual count when the response lands.

CREATE TABLE IF NOT EXISTS ledger (
    id       TEXT PRIMARY KEY,
    ts       REAL    NOT NULL,
    estimate INTEGER NOT NULL,
    actual   INTEGER,            -- NULL means the call is still in flight
    note     TEXT    NOT NULL
);

The charged total is SUM(COALESCE(actual, estimate)). In-flight calls therefore count against the cap immediately, which is the entire point: a retry storm cannot slip through a window between request and billing.

# ledger.py
import os, sqlite3, time, uuid
from pathlib import Path

DB_PATH = Path(os.environ.get("LEDGER_DB", "ledger.sqlite3"))

DDL = """
CREATE TABLE IF NOT EXISTS ledger (
    id TEXT PRIMARY KEY, ts REAL NOT NULL,
    estimate INTEGER NOT NULL, actual INTEGER, note TEXT NOT NULL
);
"""

def connect(path: Path = DB_PATH) -> sqlite3.Connection:
    conn = sqlite3.connect(path, timeout=5, isolation_level=None)
    conn.execute("PRAGMA journal_mode=WAL")
    conn.execute("PRAGMA busy_timeout=5000")
    conn.execute(DDL)
    return conn

def charged(conn: sqlite3.Connection) -> int:
    return conn.execute(
        "SELECT COALESCE(SUM(COALESCE(actual, estimate)), 0) FROM ledger"
    ).fetchone()[0]

def reserve(conn, cap: int, estimate: int, note: str) -> str:
    conn.execute("BEGIN IMMEDIATE")          # serialise writers
    try:
        used = charged(conn)
        if used + estimate > cap:
            raise RuntimeError(f"budget: {used + estimate} > cap {cap}")
        rid = uuid.uuid4().hex
        conn.execute(
            "INSERT INTO ledger VALUES (?,?,?,NULL,?)",
            (rid, time.time(), estimate, note),
        )
        conn.execute("COMMIT")
        return rid
    except Exception:
        conn.execute("ROLLBACK")
        raise

def settle(conn, rid: str, actual: int) -> None:
    conn.execute(
        "UPDATE ledger SET actual = ? WHERE id = ? AND actual IS NULL",
        (actual, rid),
    )

def sweep(conn, older_than_s: float = 900) -> int:
    """Crashed reservations are charged at their estimate, never forgotten."""
    cur = conn.execute(
        "UPDATE ledger SET actual = estimate WHERE actual IS NULL AND ts < ?",
        (time.time() - older_than_s,),
    )
    return cur.rowcount

sweep is the least elegant function in the file and the most important one. A process that dies mid-call leaves an in-flight row forever; sweeping charges the estimate so the cap stays conservative across restarts.

The guard wrapper and the timeout rule

The wrapper reserves first, then calls. The hard part is not the happy path — it is deciding what an exception costs.

# guard.py
TIMEOUT_IS_CHARGED = True   # conservative default, see the table below

def guarded(client, prompt: str, cap: int, estimate: int):
    conn = connect()
    sweep(conn)
    rid = reserve(conn, cap, estimate, note=prompt[:60])
    try:
        resp = client.complete(prompt, max_tokens=estimate)
    except TimeoutError:
        settle(conn, rid, estimate if TIMEOUT_IS_CHARGED else 0)
        raise
    except ConnectionError:
        settle(conn, rid, 0)        # request never left the process
        raise
    settle(conn, rid, resp.usage.total_tokens)
    return resp

Failure	Did the provider charge?	Ledger default
Connection refused / DNS failure	No	Settle at 0
Timeout after the request body was sent	Unknown	Settle at the estimate
HTTP 4xx before generation starts	Usually no	Settle at 0
2xx with truncated usage data	Yes	Settle at the estimate

The timeout row is the honest one. No client can know whether a remote generator billed tokens for a request that died in transit, so the conservative choice is to assume it did. Overcharging a local ledger costs headroom; undercharging costs the cap.

Pair max_tokens = estimate with the reservation. That bound is what keeps the gap between estimate and actual from turning into a silent overshoot.

Tests that a dashboard cannot replace

Three cases cover the behaviour that matters. Each worker thread opens its own connection, because the locking is part of what is being tested.

# test_ledger.py
import pytest, threading
from ledger import connect, reserve, settle, charged

def test_refuses_over_cap(tmp_path):
    conn = connect(tmp_path / "l.db")
    reserve(conn, cap=1000, estimate=600, note="a")
    with pytest.raises(RuntimeError):
        reserve(conn, cap=1000, estimate=600, note="b")

def test_settle_releases_the_difference(tmp_path):
    conn = connect(tmp_path / "l.db")
    rid = reserve(conn, cap=1000, estimate=800, note="a")
    settle(conn, rid, 120)
    assert charged(conn) == 120
    reserve(conn, cap=1000, estimate=800, note="b")   # now it fits

def test_concurrent_reservations_do_not_overspend(tmp_path):
    db = tmp_path / "l.db"
    ok = []

    def worker(i):
        conn = connect(db)                  # own connection, shared file
        try:
            reserve(conn, cap=1000, estimate=300, note=str(i))
            ok.append(i)
        except RuntimeError:
            pass

    threads = [threading.Thread(target=worker, args=(i,)) for i in range(10)]
    [t.start() for t in threads]
    [t.join() for t in threads]
    assert len(ok) == 3                     # 3 x 300 <= 1000, the 4th is refused

Run with python -m pytest -q test_ledger.py. The third test is the one that fails on a naive SELECT-then-INSERT implementation: without BEGIN IMMEDIATE, several threads read the same total and all reserve.

Deploying to the free server: one smoke test, one uncomfortable finding

A cap that resets on redeploy is not a cap. That is precisely what the first deployment did.

#!/usr/bin/env bash
# smoke.sh — five wake-ups, 60 s apart, then a ledger survival check
set -u
URL="https://${DEMO_HOST}/health"

for i in $(seq 1 5); do
  sleep 60
  curl -s -o /dev/null -w "wake %{http_code} %{time_total}s\n" \
       --max-time 30 "$URL"
done

curl -s --max-time 30 "$URL" | tee /dev/stderr | grep -q '"ledger_rows"' \
  || echo "WARN: ledger_rows missing from health payload"

The health endpoint reports ledger_rows and charged, so the smoke test checks two things at once: whether the host wakes within the timeout, and whether the ledger was still there afterwards. Five wake-ups produce a spread, not a benchmark; a single cold-start number from one run says almost nothing.

The finding: after a redeploy, ledger_rows was back to 0. The default container filesystem is ephemeral, and a fresh instance had started with a fresh, empty cap. Two fixes exist — mount a volume and point LEDGER_DB at it, or accept a per-instance cap and document it loudly. The first costs a configuration line; the second costs trust in the number on the dashboard.

Choosing between three controls

Control	Catches	Misses	Effort
max_tokens per request	One runaway completion	Cumulative drift over hundreds of retries	Zero
Provider usage dashboard or alert	Eventual overspend	Tonight's batch job	Minutes, with lag
Reservation ledger (this)	Spend before it happens, including concurrent calls	Caps shared across machines without a shared DB	One file, one table

The three stack. max_tokens bounds a call, the ledger bounds a run, and the dashboard explains what happened yesterday. Anyone building a demo on a metered free tier benefits from all three; anyone calling a model once does not need the ledger at all.

Where the free tier actually fits

This particular project runs on MonkeyCode's free model access, which changes the ledger's unit: the cap is measured in tokens, not currency, so the numbers in the table above are directly comparable to the allowance itself. The free server option puts the demo and the ledger on the same host, which is exactly why the ephemeral-filesystem finding surfaced in a weekend rather than in week three.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The allowance quoted when the project started was on the order of ten million tokens, and the free server option was available then too. Both are availability claims rather than guarantees: free-tier terms and allowances change, so read the current project page before sizing a cap around a number from a blog post — this one included.

If the caps match the workload, the free tier is a reasonable place to run a demo of this size, and the ledger is worth keeping regardless of where the calls go.

Limitations and who should skip this

• Single-node SQLite only. Two hosts need a shared store, typically Postgres, plus the same BEGIN IMMEDIATE discipline.
• Estimates must be plausible. Set estimate well under actual usage and the cap leaks by the difference on every in-flight call; max_tokens = estimate closes that gap.
• The sweep overcharges after a crash. Conservative by design, but it means the cap can shrink without any tokens being spent.
• Tokens are not money. Any reconciliation against an invoice needs a price table this artifact does not have.
• Skip it entirely if the provider already enforces hard pre-request caps, or if the project needs a real LLM gateway with routing, retries, and central accounting. A gateway does this better; this is a weekend's version of one.

The reusable part is the shape, not the code: reserve before spending, reconcile after, and make a crash cost something. Four columns and a lock are enough to turn an allowance into a limit that a program actually respects.