The Quota You Can't See: A Client-Side Ledger for Free Model Tiers

Free model tiers hide their meters. You call an endpoint. You get a response. You never see the quota drain. Then a 429 arrives with no warning. This article builds a local ledger that fixes that.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The invisible meter

Most model APIs report usage. The response contains a usage object. It lists prompt tokens and completion tokens. Most clients ignore it.

Ignoring that data is a mistake. Quotas are finite. Free tiers reset on a schedule. Without records, you cannot predict the reset. You cannot plan around it.

A ledger changes the equation. Every call becomes a row. Every row has a timestamp and a token count. Aggregates reveal your burn rate.

What the ledger records

Five fields matter. The endpoint tells you which model you used. The timestamp tells you when. The token counts tell you how much. The status code tells you if it worked. The latency tells you how slow it was.

Store them in SQLite. SQLite is a single file. It needs no server. It survives process restarts.

import sqlite3
from datetime import datetime, timezone

class ModelLedger:
    def __init__(self, db_path="ledger.db"):
        self.conn = sqlite3.connect(db_path)
        self.conn.execute("""
            CREATE TABLE IF NOT EXISTS calls (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                ts TEXT NOT NULL,
                endpoint TEXT NOT NULL,
                prompt_tokens INTEGER DEFAULT 0,
                completion_tokens INTEGER DEFAULT 0,
                total_tokens INTEGER DEFAULT 0,
                status INTEGER,
                latency_ms INTEGER,
                usage_source TEXT DEFAULT 'reported'
            )
        """)
        self.conn.commit()

    def record(self, endpoint, usage, status, latency_ms, usage_source="reported"):
        prompt = usage.get("prompt_tokens", 0)
        completion = usage.get("completion_tokens", 0)
        total = usage.get("total_tokens", prompt + completion)
        self.conn.execute(
            """INSERT INTO calls
               (ts, endpoint, prompt_tokens, completion_tokens, total_tokens, status, latency_ms, usage_source)
               VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
            (datetime.now(timezone.utc).isoformat(), endpoint,
             prompt, completion, total, status, latency_ms, usage_source)
        )
        self.conn.commit()

The usage dict may be missing. Free endpoints sometimes omit it. Default to zero. Record the row anyway. A row without tokens still shows call volume.

Wrapping the client

A ledger needs a wrapper. The wrapper records before returning. It measures latency with time.monotonic(). It reads usage from the JSON response.

import time
import httpx

ledger = ModelLedger()
MODEL_URL = "https://your-model-endpoint"

def tracked_chat(payload):
    start = time.monotonic()
    response = httpx.post(MODEL_URL, json=payload, timeout=30)
    latency_ms = int((time.monotonic() - start) * 1000)
    usage = {}
    try:
        usage = response.json().get("usage", {})
    except ValueError:
        pass
    ledger.record(MODEL_URL, usage, response.status_code, latency_ms)
    return response

Every call now leaves a trail. The trail answers three questions. How many tokens today? How many requests failed? How slow was the endpoint?

Reading the burn rate

Raw rows are noise. Aggregates are signal. Run a daily summary.

SELECT date(ts) AS day,
       COUNT(*) AS calls,
       SUM(total_tokens) AS tokens,
       AVG(latency_ms) AS avg_latency
FROM calls
GROUP BY date(ts)
ORDER BY day DESC;

The output shows your burn rate. Compare it against the tier's quota. You can now predict when the 429 arrives.

Add a failure view.

SELECT status, COUNT(*) AS count
FROM calls
GROUP BY status
ORDER BY count DESC;

A high 429 count means the quota is close. A high 500 count means the endpoint is unstable. Each status tells a different story.

Run the summary daily. Save the output to a file. Diff the file week over week. Trends matter more than single days. A steady climb means your usage is growing. A flat line means the tier is stable.

Predicting the reset

Free tiers reset on a schedule. The schedule is usually daily. Query the last 24 hours.

SELECT SUM(total_tokens) AS tokens_last_24h
FROM calls
WHERE ts >= datetime('now', '-1 day');

Compare that to the tier's daily limit. If you used 80 percent in six hours, the reset is the only thing saving you. Plan heavy jobs for after the reset.

Estimating when usage is missing

Some endpoints never return usage. Your ledger then records zeros. That hides the burn rate.

Estimate instead. A local tokenizer gives a rough count. Use tiktoken for OpenAI-compatible endpoints.

import tiktoken

def estimate_tokens(text):
    enc = tiktoken.get_encoding("cl100k_base")
    return len(enc.encode(text))

This is an approximation. It is better than zero. Label estimated values in the ledger.

def tracked_chat_estimated(payload):
    start = time.monotonic()
    response = httpx.post(MODEL_URL, json=payload, timeout=30)
    latency_ms = int((time.monotonic() - start) * 1000)
    prompt_text = payload.get("prompt", "")
    prompt_tokens = estimate_tokens(prompt_text)
    completion_text = response.text
    completion_tokens = estimate_tokens(completion_text)
    usage = {
        "prompt_tokens": prompt_tokens,
        "completion_tokens": completion_tokens,
        "total_tokens": prompt_tokens + completion_tokens,
    }
    ledger.record(MODEL_URL, usage, response.status_code, latency_ms, usage_source="estimated")
    return response

The estimate is wrong. It is consistently wrong. That consistency makes trends useful.

Verifying the ledger

A ledger must prove itself. Insert synthetic rows. Query them back. Check the math.

ledger.record("test-endpoint", {"prompt_tokens": 10, "completion_tokens": 5}, 200, 120)
ledger.record("test-endpoint", {}, 429, 15)
ledger.record("test-endpoint", {"prompt_tokens": 100, "completion_tokens": 50}, 200, 300)

Run the daily summary. Expected: three calls, 165 tokens, one 429. If the numbers match, the ledger works.

Add a second check. Query the failure view. Expected: two 200s, one 429. The math is simple. The confidence is real.

Run the verification after every schema change. A ledger with a broken schema is worse than no ledger. It produces false confidence. The verification script takes seconds. Run it.

Limitations

This ledger is client-side. It sees only your calls. It cannot see other consumers of the same tier. It cannot see the provider's internal counters.

The usage field is a claim. It comes from the provider. It is usually accurate. It is not audited.

Streaming responses complicate things. The usage object arrives in the final chunk. Your wrapper must buffer it. That adds latency.

SQLite is single-writer. Concurrent processes will lock. Use a queue or a separate file per process.

Who should skip this

Teams with centralized gateways already have metrics. A local ledger duplicates that work. Use your existing observability stack instead.

Hobby projects fit this pattern. One process. One file. One source of truth. That is the sweet spot.

The ledger changes the question

Before the ledger, the question was "Why did I get a 429?" After the ledger, the question is "When will the next one arrive?" That is a better question. It has an answer.

The free model tier from MonkeyCode works with this pattern. Any HTTP endpoint does. Record the first call today. The next 429 will have a cause.

A free server option is enough to reproduce the setup.