48-Hour Field Notes: A Free Tier Is a Budget, Not a Benchmark

At 01:40 on a Tuesday, a build went red on a branch nobody had touched in nine days. I pasted the tail of the log into a chat box, asked what broke, and got a confident answer that pointed at line 214. The log had 187 lines. That was not a model failure so much as a missing instrument: I had no meter, no ladder, and no way to tell a plausible answer from a checkable one.

So I spent the next 48 hours building the instrument instead of the prompt. The notes below are about that instrument — a spend ledger, an escalation ladder, and a citation check — and about what a free model tier plus a free server box can and cannot carry. It is a workflow you can reproduce, not a review of anybody's model quality.

Hour 0–6: put the meter in front of the prompt

The first version of the ledger took forty lines and was worth more than every prompt tweak I made afterwards. The idea is blunt: every call charges the ledger before it returns, and the ledger refuses the call when the day's cap is already spent. Not after. Before.

# ledger.py — a spend guard, not a router
import json, os, pathlib, time

STATE = pathlib.Path(os.environ.get("LEDGER_PATH", "ledger.json"))
CAP = int(os.environ.get("TOKEN_CAP_PER_DAY", "200000"))  # example value, not a vendor number

class BudgetExceeded(Exception):
    pass

def today():
    return time.strftime("%Y-%m-%d", time.gmtime())  # UTC on purpose

def load():
    if STATE.exists():
        return json.loads(STATE.read_text())
    return {"day": today(), "spent": 0, "calls": []}

def charge(tokens, label, model):
    s = load()
    if s["day"] != today():
        s = {"day": today(), "spent": 0, "calls": []}
    if s["spent"] + tokens > CAP:
        raise BudgetExceeded(f"{s['spent']}/{CAP} spent, call needs {tokens}")
    s["spent"] += tokens
    s["calls"].append({"t": time.time(), "model": model, "tokens": tokens, "label": label})
    STATE.write_text(json.dumps(s, indent=2))
    return s

Two design choices mattered later. The reset boundary is UTC, because my laptop was not. And label is a free-text field I fill with the CI job name, so a two-day ledger reads like a log of decisions rather than a list of numbers.

Hour 6–18: the ladder is a policy, not a menu

Escalation is where most of the spend leaks out. If the cheap step answers, you stop; if it answers without evidence, you do not promote it to a stronger model, you reject it and re-run the same step with a narrower slice of log. Promoting a bad question just buys a more expensive bad answer.

What the answer contains	Ladder step	Stop condition
A predicate that exists in the log verbatim	keep it, promote nothing	file the patch
A predicate plus a guess about intent	one step up, log trimmed to that job	answer names a line
No citation at all	discard, re-run with tail -n 80	two discards in a row
Citation outside the line range	discard and log a floor failure	always

That table is the entire routing logic. It fits on an index card because it is not trying to be clever; it is trying to make the expensive step rare and the cheap step honest.

Hour 18–30: a floor check, so the ladder has a direction

The failure that started this run was a hallucinated line number, so the fix is a citation contract. Every answer must cite log lines as L<number>, and a small function decides whether the citation points at anything real.

# floor.py
import re

CITE = re.compile(r"\bL(\d+)\b")

def floor_check(answer: str, log: str) -> dict:
    lines = log.splitlines()
    cites = [int(m) for m in CITE.findall(answer)]
    out_of_range = [c for c in cites if c < 1 or c > len(lines)]
    return {"cited": cites, "out_of_range": out_of_range, "ok": bool(cites) and not out_of_range}

Three tests pin the contract, and they are cheap enough to run on every change to the prompt:

LOG = "step 1 ok\nstep 2 ok\nstep 3 FAIL: assertion x == 4\n"

def test_no_citation_is_a_failure():
    assert floor_check("the build failed", LOG)["ok"] is False

def test_out_of_range_is_a_failure():
    assert floor_check("see L99", LOG)["ok"] is False

def test_real_line_passes():
    assert floor_check("see L3", LOG)["ok"] is True

The instrumented run changed how I read answers. A rejected answer stopped feeling like a wasted call and started feeling like the ladder working — the ledger shows the charge, the floor check shows the reason, and the pair is auditable a week later.

Hour 30–42: moving the loop off the laptop

A triage loop that only runs when I remember to start it is a demo. MonkeyCode, the project whose free model access and free server option I used here, is relevant at exactly this point: the operator describes free model access (currently advertised as a ten-million-token starter pool) and a free server option, and I treat both as vendor claims dated to this writing, not as capacity guarantees. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Verify the current limits in the project console before you build a plan on them, and check the repository for the license that applies to your use.

On the box, the loop is a foreground script and the ledger is the only state:

export LEDGER_PATH=/var/tmp/ledger.json
export TOKEN_CAP_PER_DAY=200000          # your number, not mine
nohup python3 triage.py --log ./build.log > triage.out 2>&1 &

Two rules came out of this move. Scrub the log before it leaves the runner — free compute does not make a leaked token cheaper. And treat a restart as a state reset: if the box is recycled, ledger.json goes with it, so copy it out or accept that your daily cap restarts too.

Hour 42–48: what I would repeat, what I would delete

Repeat the ordering: meter, ladder, floor, then prompt. Repeat the UTC reset, which saved me a confusing morning. Repeat the index-card table, because a policy you cannot recite is a policy you will not follow at 2am.

Delete the idea that a free tier is a benchmark. It is a budget with a deadline attached, and budgets reward restraint rather than cleverness. I would also drop any prompt longer than the log slice it describes; trimming tail -n 80 did more for answer quality than three paragraphs of instructions.

Who should not use this

If your logs contain secrets you cannot scrub, do not ship them anywhere, free or paid. If you need a response-time SLA or a fixed GPU footprint, neither a free tier nor a free server is the right shape for you — rent the guarantee. And if your failures are not line-addressable — flaky tests, race conditions, timing — a citation floor check will reject good answers and teach you nothing; write a reproducer first.

If you want to run the whole loop this week without provisioning anything, the free model access and free server option are the lowest-friction way I know to start — begin with the ledger, keep the cap small, and let the floor check tell you when to climb the ladder.