Case Study: A Token Ledger and Concurrency Lock for a Free-Tier Changelog Bot

The model call is the easy half of shipping a free-tier tool; the hard half is a ledger that tells you, before you call anything, whether you can still afford to. This case study walks through one small internal job end to end: a changelog drafter that turns merged pull request titles into release notes, with a budget guard that refuses work instead of overrunning a free allotment. Everything below is runnable, and the tests never touch the network or spend a single token.

Background: a Friday task nobody wanted to own

Our repository merges roughly forty pull requests a week, and somebody had to turn those into readable release notes before the Friday deploy. The obvious fix was a scheduled job that drafts the notes, but it arrived with three constraints that shaped the whole design. First, there was no budget line, so the job had to run on free model access. Second, there was no on-call rotation, so a failed run had to be boring instead of alarming. Third, the release could not depend on it, which meant the job needed a way to say "I am out of budget" and exit cleanly.

The constraint people underestimate is not the model call itself. It is accounting: once you are spending a fixed allotment you cannot top up, every run becomes a withdrawal, and you need a record of withdrawals that survives a crash. Without that record, a retry loop, a duplicated cron entry, or a provider outage can quietly consume the month in an afternoon.

Goal and non-goals

The goal was deliberately narrow, and writing the non-goals first saved a week of scope creep.

1. Reserve budget before every model call, using a conservative estimate for the prompt and the reply.
2. Reconcile the reservation with the provider's reported usage once the call returns.
3. Refuse a run when the estimate would cross a safety line, and exit with a non-zero code the scheduler ignores.
4. Survive a crashed run without permanently locking the budget.

Explicit non-goals: no per-tenant billing, no real-time dashboards, no retry orchestration inside the budget guard, and no attempt to predict long-run spend from a single week of data. Those belong to a different tool with a different failure profile.

Where free model access and a free server actually fit

The project advertises a free tier that includes free model access and a free server option, and it states an allotment of ten million tokens for that free tier. Treat both figures as advertised terms rather than guarantees, because quotas and free-tier conditions change and you should confirm the current numbers on the project page before you design around them. What matters for this article is the shape of the constraint, not the exact number: a finite allotment, a reset period you do not control, and a model client you did not write.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server option matters because it removes the "where does the cron live" argument. What it does not remove is state: if the free server restarts and your ledger file disappears, your accounting disappears with it. Check whether your plan gives you a persistent volume, and if it does not, keep the ledger somewhere that survives a restart before you schedule a single run.

Implementation

Step 1: two tables, one writer

The ledger is SQLite, because a single job with a single writer needs transactions and not a database cluster. The budget table holds the allowance and the period key, and the runs table holds one row per attempt.

# ledger.py
import sqlite3, time

SAFETY = 0.80          # stop before the last fifth of the allotment
PERIOD = "%Y-%m"       # adjust if your terms reset on a different boundary

SCHEMA = """
CREATE TABLE IF NOT EXISTS budget (
  job       TEXT PRIMARY KEY,
  allowance INTEGER NOT NULL,
  period    TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS runs (
  id            INTEGER PRIMARY KEY AUTOINCREMENT,
  job           TEXT NOT NULL,
  started_at    REAL NOT NULL,
  finished_at   REAL,
  est_tokens    INTEGER NOT NULL,
  actual_tokens INTEGER,
  status        TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS runs_job_time ON runs (job, started_at);
"""

def connect(path):
    conn = sqlite3.connect(path, timeout=5, isolation_level=None)
    conn.execute("PRAGMA journal_mode=WAL")
    conn.executescript(SCHEMA)
    return conn

def period_key(now):
    return time.strftime(PERIOD, time.gmtime(now))

Step 2: reserve before you call anything

The reservation is where the interesting cases live, so it runs inside BEGIN IMMEDIATE. That makes two concurrent schedulers serialize, and the second one sees the first one's charge instead of racing past it.

def sweep_stale(conn, job, max_age_s=1800, now=None):
    now = now or time.time()
    conn.execute(
        "UPDATE runs SET status='aborted', finished_at=? "
        "WHERE job=? AND status='running' AND started_at < ?",
        (now, job, now - max_age_s),
    )

def reserve(conn, job, est_tokens, now=None):
    now = now or time.time()
    conn.execute("BEGIN IMMEDIATE")
    try:
        sweep_stale(conn, job, now=now)
        row = conn.execute(
            "SELECT allowance, period FROM budget WHERE job=?", (job,)
        ).fetchone()
        if row is None:
            conn.execute("ROLLBACK")
            raise RuntimeError(f"no budget row for job={job!r}")
        allowance, period = row
        if period != period_key(now):
            conn.execute("UPDATE budget SET period=? WHERE job=?", (period_key(now), job))
            spent = 0
        else:
            spent = conn.execute(
                "SELECT COALESCE(SUM(COALESCE(actual_tokens, est_tokens)), 0) "
                "FROM runs WHERE job=? AND status <> 'aborted'",
                (job,),
            ).fetchone()[0]
        if spent + est_tokens > int(allowance * SAFETY):
            conn.execute("ROLLBACK")
            return None
        run_id = conn.execute(
            "INSERT INTO runs (job, started_at, est_tokens, status) "
            "VALUES (?, ?, ?, 'running')",
            (job, now, est_tokens),
        ).lastrowid
        conn.execute("COMMIT")
        return run_id
    except BaseException:
        if conn.in_transaction:
            conn.execute("ROLLBACK")
        raise

Step 3: settle with the provider's number

When the call returns, you replace the estimate with the number the provider reports. When the call fails, you pass None, which leaves the estimate in place through COALESCE and prevents a retry storm from spending unseen budget.

def settle(conn, run_id, actual_tokens, status="ok"):
    conn.execute(
        "UPDATE runs SET finished_at=?, actual_tokens=?, status=? WHERE id=?",
        (time.time(), actual_tokens, status, run_id),
    )

Step 4: the job itself stays boring

The runner has one branch that matters: if reserve returns None, print why and exit with code 2. Do not retry, do not sleep, do not page anyone, because the next scheduled window is soon enough.

# changelog_job.py
import sys, ledger

def main(db_path, est_tokens, client, prs):
    conn = ledger.connect(db_path)
    run_id = ledger.reserve(conn, "changelog", est_tokens)
    if run_id is None:
        print("budget guard: refusing this run", file=sys.stderr)
        return 2
    try:
        text, used = client.draft(prs)
    except Exception:
        ledger.settle(conn, run_id, actual_tokens=None, status="failed")
        raise
    ledger.settle(conn, run_id, actual_tokens=used)
    print(text)
    return 0

Step 5: test the guard with a fake client

Because the guard is normal Python around a normal database, you can test the entire decision path offline. The fake client below returns a fixed usage count, so the assertions describe budget arithmetic rather than model behavior.

# test_ledger.py
import time, ledger

class FakeClient:
    def __init__(self, text="## Changes\n- nothing to report", used=137):
        self.text, self.used, self.calls = text, used, 0
    def draft(self, prs):
        self.calls += 1
        return self.text, self.used

def fresh(tmp_path, allowance=1000):
    conn = ledger.connect(str(tmp_path / "budget.db"))
    conn.execute(
        "INSERT INTO budget (job, allowance, period) VALUES (?, ?, ?)",
        ("changelog", allowance, ledger.period_key(time.time())),
    )
    return conn

def test_refuses_when_estimate_crosses_safety_line(tmp_path):
    conn = fresh(tmp_path, allowance=1000)
    assert ledger.reserve(conn, "changelog", 400) is not None   # 400 <= 800
    assert ledger.reserve(conn, "changelog", 401) is None       # 801 > 800

def test_actual_replaces_estimate(tmp_path):
    conn = fresh(tmp_path, allowance=1000)
    run_id = ledger.reserve(conn, "changelog", 400)
    ledger.settle(conn, run_id, actual_tokens=120)
    assert ledger.reserve(conn, "changelog", 680) is not None   # 120 + 680 = 800
    assert ledger.reserve(conn, "changelog", 1) is None

def test_stale_run_stops_holding_budget(tmp_path):
    conn = fresh(tmp_path, allowance=1000)
    now = time.time()
    ledger.reserve(conn, "changelog", 700, now=now - 7200)
    assert ledger.reserve(conn, "changelog", 700, now=now) is not None

def test_failed_run_keeps_the_estimate(tmp_path):
    conn = fresh(tmp_path, allowance=1000)
    run_id = ledger.reserve(conn, "changelog", 300)
    ledger.settle(conn, run_id, actual_tokens=None, status="failed")
    assert ledger.reserve(conn, "changelog", 500) is not None   # 300 + 500 = 800
    assert ledger.reserve(conn, "changelog", 1) is None

Run the suite and inspect the ledger with these commands.

python -m pytest test_ledger.py -q
sqlite3 budget.db "SELECT id, status, est_tokens, actual_tokens FROM runs ORDER BY id DESC LIMIT 5;"

Results: what the guard actually catches

I am not quoting live token counts here, because free-tier terms change and your numbers will differ from mine. What is reproducible is the behavior contract, and you can verify each row in the table below against the tests above.

Failure mode	Without the ledger	With the ledger
Duplicate cron entry	Two drafts run, both spend budget	Second reservation serializes and sees the first charge
Worker killed mid-run	Row stays running, budget blocked forever	Stale sweep marks it aborted after the age limit
Optimistic estimate	Real spend drifts above the plan	settle replaces the estimate with reported usage
Provider outage and retries	Each retry looks free and is not	Failed runs keep the estimate as their charge
New reset period	Manual bookkeeping, usually forgotten	Period key mismatch resets the spend window

The most useful output is not a number but a refusal. A run that exits with code 2 and a one-line reason on stderr is a system that knows its own limits, and that is far easier to hand to a teammate than a job that silently eats the month.

To get your own measurements, seed the budget row with your real allowance, run the job on schedule for a week, and compare est_tokens against actual_tokens in the ledger. If your estimates are consistently twenty percent low, raise the safety multiplier instead of rewriting the prompt.

sqlite3 budget.db "SELECT ROUND(100.0*SUM(COALESCE(actual_tokens,est_tokens))/10000000,2) || '% of a 10M allotment' FROM runs WHERE job='changelog' AND status<>'aborted';"

Limitations and who should not use this

This design is honest about what it cannot do, and several teams should pick something else.

• If you cannot read usage from the provider, the ledger is estimates all the way down, and you should treat the safety line as the real limit rather than a margin.
• If your free server has no persistent disk, the ledger must live off-box or be exported and committed after every run, otherwise a restart zeroes your accounting.
• If your reset boundary is not a calendar month, change PERIOD and period_key together, then add a test for the boundary you actually have.
• If you have several writers or tenants, SQLite's single-writer model will frustrate you quickly, and a small Postgres instance is the cheaper decision.
• If a missed changelog blocks a release, this guard is the wrong trade: refuse less, alert more, and keep a human escape hatch.

Lessons learned

Three things survived contact with the schedule, and none of them were about prompting. First, put the ledger in place before the prompt, because accounting you add later never quite matches the code paths that already exist. Second, fail closed on budget and fail open on output, so the release notes can be written by hand on the rare Friday the guard says no. Third, keep the model client behind one narrow method, which is what made the offline tests possible in an afternoon.

If you want to try this shape yourself, the free tier's free model access plus the free server option is enough to run exactly one job like this one, so start with the ledger, confirm the current quota terms, and write the prompt last.