A side project shipped with a single usage counter on the dashboard, and that counter showed one number: total tokens consumed. Eleven days after launch, the number crossed the free allowance limit, and the project went dark in the middle of a demo. The counter had been accurate the whole time, and it was completely useless because it answered only one question: how much had been spent. It could not answer the question that actually mattered, which was which feature had burned through the budget.
The failure was not a rate limit and not a provider outage. It was an accounting failure. The free allowance was generous enough for months of normal use, but a single feature with a long system prompt and an aggressive retry loop consumed most of it in under two weeks. The fix was not to buy more tokens; it was to build a metering layer that attributed every token to a feature, a user, and a time window, so that the next budget decision could be made with evidence.
This article describes a lightweight metering service that runs alongside an LLM gateway. The reference setup uses MonkeyCode's free model access, which includes a 10-million-token allowance at the time of writing, and the free server option for hosting the metering service itself. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The code is deliberately small, because a metering layer that is hard to deploy will not get deployed.
Step 1: Capture usage at the edge
The first requirement is that every LLM response records its usage fields before the data is discarded. Most providers return prompt_tokens, completion_tokens, and total_tokens in the response body, and the gateway already parses that body for other purposes. The metering middleware intercepts the parsed response and writes one row per request.
# middleware.py
import sqlite3
import time
import uuid
class UsageMeter:
def __init__(self, db_path: str):
self.conn = sqlite3.connect(db_path)
self.conn.execute("""
CREATE TABLE IF NOT EXISTS usage_events (
id TEXT PRIMARY KEY,
ts INTEGER NOT NULL,
feature TEXT NOT NULL,
user_id TEXT,
model TEXT NOT NULL,
prompt_tokens INTEGER NOT NULL,
completion_tokens INTEGER NOT NULL,
total_tokens INTEGER NOT NULL,
status INTEGER NOT NULL
)
""")
self.conn.commit()
def record(self, feature: str, user_id: str, model: str,
prompt_tokens: int, completion_tokens: int,
status: int) -> None:
self.conn.execute(
"INSERT INTO usage_events VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
(uuid.uuid4().hex, int(time.time()), feature, user_id, model,
prompt_tokens, completion_tokens,
prompt_tokens + completion_tokens, status)
)
self.conn.commit()
The schema is intentionally flat. A single table with a timestamp, a feature label, an optional user ID, and the three token counts is enough to answer most attribution questions, and it keeps the deployment footprint small enough to run on a free server tier.
Step 2: Attribute every token to a feature
The gateway must know which feature triggered each request, and the cleanest way to pass that context is through a request header. The client sends X-Feature: summarize or X-Feature: chat, and the middleware reads that header when the response arrives.
# gateway_integration.py
from middleware import UsageMeter
meter = UsageMeter("usage.db")
def handle_response(request, response):
feature = request.headers.get("X-Feature", "unknown")
user_id = request.headers.get("X-User-Id")
usage = response.json().get("usage", {})
meter.record(
feature=feature,
user_id=user_id,
model=response.json().get("model", "unknown"),
prompt_tokens=usage.get("prompt_tokens", 0),
completion_tokens=usage.get("completion_tokens", 0),
status=response.status_code,
)
return response
The header convention costs nothing and prevents a common failure mode where the gateway knows the endpoint but not the product feature behind it. A single endpoint can serve many features, and the URL alone is rarely enough to attribute cost.
Step 3: Compute burn rate and forecast exhaustion
Raw rows are not a budget; they are a log. The next step is a query that aggregates usage per day and fits a simple linear projection to the daily totals. A 7-day moving average smooths weekend dips, and the projected exhaustion date is the point where the cumulative projection crosses the allowance.
-- daily_burn.sql
SELECT
date(ts, 'unixepoch') AS day,
SUM(total_tokens) AS daily_total,
SUM(SUM(total_tokens)) OVER (ORDER BY date(ts, 'unixepoch')) AS running_total
FROM usage_events
WHERE status = 200
GROUP BY day
ORDER BY day;
# forecast.py
from datetime import datetime, timedelta
import sqlite3
def forecast_exhaustion(db_path: str, allowance: int) -> str:
conn = sqlite3.connect(db_path)
rows = conn.execute("""
SELECT date(ts, 'unixepoch'), SUM(total_tokens)
FROM usage_events WHERE status = 200
GROUP BY date(ts, 'unixepoch')
ORDER BY date(ts, 'unixepoch')
""").fetchall()
if len(rows) < 3:
return "insufficient data"
recent = rows[-7:]
daily_avg = sum(r[1] for r in recent) / len(recent)
consumed = sum(r[1] for r in rows)
remaining = allowance - consumed
if remaining <= 0:
return "already exhausted"
if daily_avg == 0:
return "no burn detected"
days_left = remaining / daily_avg
return (datetime.utcnow() + timedelta(days=days_left)).date().isoformat()
The projection is deliberately naive. A linear model over a 7-day window is honest about its assumptions, and it is far better than the alternative, which is no forecast at all. Teams that need more accuracy can swap in a seasonal decomposition later, but the linear version is enough to trigger a conversation about budget.
Step 4: Alert before the wall, not after
The forecast only helps if someone sees it. A small scheduler checks the projection once per day and sends a message when the projected exhaustion date crosses a threshold. The alert levels are 50 percent, 80 percent, and 90 percent of the allowance, and each level suggests a different action.
# alert.py
def check_alerts(db_path: str, allowance: int) -> list[str]:
conn = sqlite3.connect(db_path)
total = conn.execute(
"SELECT SUM(total_tokens) FROM usage_events WHERE status = 200"
).fetchone()[0] or 0
pct = total / allowance
alerts = []
for threshold in (0.5, 0.8, 0.9):
if pct >= threshold:
alerts.append(f"usage at {pct:.0%} of allowance")
return alerts
The alerting rule is simple, but it changes the operational posture. Instead of discovering exhaustion when a request fails, the team sees the trend weeks in advance and can decide whether to compress prompts, disable a feature, or move a workload to a paid tier.
The free server fit
The metering service itself is a natural fit for the free server option because it is low-traffic and stateless except for the SQLite file. A single instance that receives one write per LLM request and runs one forecast query per day will stay well within the resource limits of a free tier, and the SQLite file can be backed up with a daily cron job.
The deployment is a single process with one environment variable for the database path. No container orchestration, no external database, no queue. That simplicity is the point: a metering layer that requires a dedicated database cluster will never be deployed for a side project, and an un-deployed metering layer is the same as no metering layer.
Limitations and who should skip this
The linear forecast assumes a stable usage pattern, and it will be wrong after a product launch, a pricing change, or a viral post. The SQLite backend does not scale to millions of rows without maintenance, so teams processing more than a few hundred thousand requests per month should move to a real database. The metering layer also depends on the provider returning accurate usage fields, which is true for most OpenAI-compatible endpoints but not all.
Teams with a dedicated budget and a paid tier do not need this system, because their billing dashboard already provides per-feature breakdowns. Anyone running a compliance-sensitive workload should avoid storing user IDs in plain SQLite, since the file is unencrypted by default.
The free allowance was never the problem. The problem was that a single aggregate number made it impossible to see which feature was eating the budget, and by the time the number crossed the limit, the only available action was an emergency rewrite. A metering layer with attribution, a burn-rate forecast, and three alert thresholds turns that emergency into a scheduled decision. The whole system is about a hundred lines of Python, and it runs comfortably on a free server. If the budget matters, the ledger is worth the hour it takes to build.
Top comments (0)