Build a Token Ledger Before You Burn Through a Free Model Tier

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why this is worth reading: a free model endpoint with a large token allowance is a good place to validate a new CLI workflow, but it can burn through the allowance in a single retry loop before you notice. I built a small stateful budget guard that checks the projected cost before the call, records actual usage after the call, and refuses to touch the ledger when the endpoint sends an unexpected response. It works as a disposable first pass on a free endpoint and leaves you a clean exit when the shape changes.

MonkeyCode's outreach describes an open-source project with a free model route and a free hosted server. I do not treat either as a permanent dependency. I treat them as a test target: an endpoint I can call without a contract while I am still changing prompts, timeouts, and schemas. The tool below is independent of MonkeyCode's exact model list; it assumes only an OpenAI-style chat completion path and usage accounting in the response. Swap one function if the free server does not follow that shape.

The problem with a free allowance

Most model dashboards report aggregate usage after the fact. That is enough for casual work, but it is not enough when you wire an endpoint into a loop. I have seen two avoidable failures in my own drafts. A retry-on-timeout wrapper restarted a slow request four times before the first response arrived, multiplying total token spend. A long context buffer kept sending the same 6k-token history on every turn because I forgot to trim old messages. The dashboard showed the total drop, but not which call caused it.

A local ledger fixes that by refusing to send the request when the projected total exceeds the budget. It does not replace the provider dashboard. It makes the decision before the endpoint gets a chance to consume tokens.

The artifact

The script below does three jobs:

• load a budget and already-used amount from a JSON file
• make a conservative preflight estimate for the next call
• record the actual usage returned by the endpoint and save the ledger atomically

Preflight is deliberately rough: prompt bytes divided by four, plus the requested max response tokens, plus a 15 percent margin. That is not tokenizer-accurate for non-English text or code-heavy prompts, but it is intentionally conservative because the goal is to stop accidental waste, not to replace metering. If you need precise preflight numbers, add a local tokenizer for the model you are calling.

python -m venv .venv && source .venv/bin/activate
pip install httpx

# .env
MONKEYCODE_BASE_URL=https://your-free-server.example.com/v1
MONKEYCODE_API_KEY=your-key
MODEL_NAME=the-current-free-model
TOKEN_BUDGET=30000000
MAX_RESPONSE_TOKENS=256
TIMEOUT_S=30
LEDGER_PATH=token_ledger.json

Then source it and call:

set -a; source .env; set +a
python budgeted_call.py 'Summarize this connection error in one sentence.'

budgeted_call.py:

#!/usr/bin/env python3
import json
import os
import sys
import time
from pathlib import Path

import httpx

BASE_URL = os.getenv('MONKEYCODE_BASE_URL', '').rstrip('/')
API_KEY = os.getenv('MONKEYCODE_API_KEY', '')
MODEL = os.getenv('MODEL_NAME', '')
BUDGET = int(os.getenv('TOKEN_BUDGET', '30000000'))
LEDGER = Path(os.getenv('LEDGER_PATH', 'token_ledger.json'))
MAX_TOKENS = int(os.getenv('MAX_RESPONSE_TOKENS', '256'))
TIMEOUT_S = float(os.getenv('TIMEOUT_S', '30'))


def load_ledger():
    if LEDGER.exists():
        data = json.loads(LEDGER.read_text())
        return int(data.get('used', 0))
    return 0


def save_ledger(used):
    tmp = LEDGER.with_suffix('.tmp')
    tmp.write_text(json.dumps({'used': used, 'updated': int(time.time())}, indent=2))
    tmp.replace(LEDGER)


def preflight_estimate(prompt, max_tokens):
    prompt_tokens = len(prompt.encode('utf-8')) // 4
    return prompt_tokens + max_tokens


def run(prompt):
    if not BASE_URL or not API_KEY or not MODEL:
        sys.exit('Set MONKEYCODE_BASE_URL, MONKEYCODE_API_KEY, and MODEL_NAME first.')

    used = load_ledger()
    estimate = preflight_estimate(prompt, MAX_TOKENS)
    margin = int(estimate * 0.15)
    projected = used + estimate + margin

    if projected > BUDGET:
        sys.exit(f'Blocked: projected={projected} used={used} budget={BUDGET}. Shorten the prompt or raise the budget.')

    response = httpx.post(
        f'{BASE_URL}/chat/completions',
        headers={'Authorization': f'Bearer {API_KEY}'},
        json={
            'model': MODEL,
            'messages': [
                {'role': 'system', 'content': 'Answer concisely. Return JSON only when asked.'},
                {'role': 'user', 'content': prompt},
            ],
            'max_tokens': MAX_TOKENS,
        },
        timeout=TIMEOUT_S,
    )
    response.raise_for_status()
    payload = response.json()

    usage = payload.get('usage') or {}
    total = usage.get('total_tokens')
    if total is None:
        total = int(usage.get('prompt_tokens', 0)) + int(usage.get('completion_tokens', 0))

    if total <= 0:
        sys.exit('Endpoint returned no usable token count; ledger was not updated.')

    used += total
    save_ledger(used)

    content = payload['choices'][0]['message']['content']
    print(json.dumps({
        'text': content,
        'total_tokens': total,
        'used': used,
        'remaining': BUDGET - used,
    }, indent=2))


if __name__ == '__main__':
    run(sys.argv[1] if len(sys.argv) > 1 else 'Reply with the word pong.')

Test the guard before you trust it

Use a failure fixture that does not hit the real endpoint. The expected result is a non-zero exit and an unchanged ledger.

MONKEYCODE_BASE_URL=http://127.0.0.1:9 python budgeted_call.py 'ping'

If you want a decision table for a canary suite, keep the checks tiny:

Scenario	Expected exit	Ledger change
Missing base URL or model	non-zero	none
Unreachable endpoint or timeout	non-zero	none
Projected spend over budget	non-zero	none
Valid response with usage.total_tokens	zero	used increases
Response without a usable token count	non-zero	none

I run this once before I allow any larger script to call the endpoint. A failed run tells me which part of the integration changed instead of leaving me to guess from a balance chart.

Where the free server fits

For a solo build, the free server is most useful as a canary target, not as a permanent backend. I point this script at the free route first, keep the model name in an environment variable, and store all results in the local ledger. If the endpoint changes one day, the only change is a URL or model name. If the endpoint reports different usage fields, the script stops instead of silently undercounting.

I also set a hard mental exit: if the free endpoint is slow enough that I need a timeout above 30 seconds, it is not ready for the actual CLI. The ledger cannot fix latency; it only prevents it from getting expensive while I measure.

One important context about the 30,000,000-token figure

The reference I was given describes a free tier with a 30,000,000-token allocation and a free server option. I do not verify quota pages as part of a code article, so I keep the number as TOKEN_BUDGET in an environment variable rather than hard-coding it. Check the current page before you rely on that number; if the allocation is different today, the script does not need to change.

Who should not use this

Do not use this local ledger if you need concurrent workers sharing one budget, hard SLOs on latency, audit trails, or compliance review for private data. A single JSON file is not concurrency-safe, and a free endpoint is the wrong home for sensitive prompts. Use the ledger as a canary, not as your production accounting system.

Next iteration

If you run this against a free server, tell me which usage fields the response actually returned. That determines whether the missing-usage guard is protecting you or getting in your way: total_tokens only, split prompt_tokens and completion_tokens, or something else entirely.

If you have a MonkeyCode free server route, plug it into this script first; if not, the same budget guard works with any endpoint that returns usage.