Most AI budgets are managed like a household that only checks its bank account once a month: the total is known, the breakdown is a mystery, and the surprise arrives with the bill. The working assumption behind this article is that a small set of call types consumes most of the token spend, and teams that cannot name those call types have no basis for optimization. The first move is not switching to a cheaper model; it is building a ledger that attributes every token to a task, a caller, and a feature.
Why most token optimization is guesswork
Token attribution is the difference between guessing and knowing. Without it, every optimization debate reduces to vibes: someone claims the model is too verbose, someone else claims the prompt is too long, and the argument ends with a model swap that nobody can validate. With a ledger, the same debate becomes a SQL query.
Three failure patterns show up consistently when teams optimize without attribution:
- The prompt-compression reflex. A team shortens a system prompt that accounts for 5% of total spend, then celebrates a 3% saving while ignoring the summarization job that burns 40%.
- The model-swap roulette. A new model looks cheaper per token, so it gets adopted everywhere, even though the expensive task never used the old model's full capability.
- The cache illusion. Teams add caching for repeated calls without measuring how many calls are actually repeated, and the cache hit rate turns out to be single digits.
All three patterns share one root cause: nobody measured where tokens actually go.
What the free tier is actually for
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode is an open-source project that advertises free model access with a 10 million token allowance and a free server option. I am deliberately not repeating any benchmark, model name, or SLA claim, because I have not verified those numbers myself. As of August 2026, the two availability claims above are what the project states, and promotional terms change, so check the repository's quick-start before you plan around them.
For the workflow in this article, the free tier has one honest job: it gives you a sandbox where you can run a metering experiment without touching your paid quota. The free server is a throwaway environment for the meter itself, not a host for anything you care about. That is the right division of labor, and it is the only way to treat free infrastructure as an asset instead of a liability.
The artifact: a token metering layer
The script below wraps any OpenAI-compatible client, records token usage per task type into SQLite, and prints a cost heatmap. It is deliberately SDK-agnostic and dependency-light, so it survives endpoint changes and works against any compatible API.
#!/usr/bin/env python3
"""token_forensics.py — attribute every AI token to a task before optimizing.
A lightweight metering layer that wraps any OpenAI-compatible client,
records token usage per task type into SQLite, and prints a cost heatmap.
Designed for free endpoints where the goal is learning, not hosting.
"""
import argparse
import json
import sqlite3
import time
from datetime import datetime
class TokenLedger:
"""SQLite-backed store for per-request token attribution."""
def __init__(self, db_path="token_ledger.db"):
self.conn = sqlite3.connect(db_path)
self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS requests (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts TEXT NOT NULL,
task_type TEXT NOT NULL,
model TEXT NOT NULL,
prompt_tokens INTEGER NOT NULL,
completion_tokens INTEGER NOT NULL,
total_tokens INTEGER NOT NULL,
latency_ms INTEGER NOT NULL,
endpoint TEXT NOT NULL
)
"""
)
self.conn.commit()
def record(self, task_type, model, usage, latency_ms, endpoint):
self.conn.execute(
"""
INSERT INTO requests (ts, task_type, model, prompt_tokens, completion_tokens, total_tokens, latency_ms, endpoint)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""",
(
datetime.utcnow().isoformat(),
task_type,
model,
usage.prompt_tokens,
usage.completion_tokens,
usage.total_tokens,
latency_ms,
endpoint,
),
)
self.conn.commit()
def heatmap(self):
"""Aggregate token spend by task type, largest first."""
rows = self.conn.execute(
"""
SELECT task_type,
COUNT(*) AS calls,
SUM(total_tokens) AS total_tokens,
SUM(prompt_tokens) AS prompt_tokens,
SUM(completion_tokens) AS completion_tokens,
ROUND(AVG(latency_ms)) AS avg_latency_ms
FROM requests
GROUP BY task_type
ORDER BY total_tokens DESC
"""
).fetchall()
return [
{
"task_type": r[0],
"calls": r[1],
"total_tokens": r[2],
"prompt_tokens": r[3],
"completion_tokens": r[4],
"avg_latency_ms": r[5],
}
for r in rows
]
class MeteredClient:
"""Wraps an OpenAI-compatible client and records every call."""
def __init__(self, client, ledger, task_type):
self.client = client
self.ledger = ledger
self.task_type = task_type
def complete(self, model, messages, **kwargs):
start = time.monotonic()
response = self.client.chat.completions.create(
model=model, messages=messages, **kwargs
)
latency_ms = int((time.monotonic() - start) * 1000)
self.ledger.record(
self.task_type,
model,
response.usage,
latency_ms,
str(self.client.base_url),
)
return response
def main():
parser = argparse.ArgumentParser(description="Token attribution for any chat endpoint.")
parser.add_argument("--base-url", required=True, help="OpenAI-compatible base URL")
parser.add_argument("--api-key", default="not-needed", help="Key for the endpoint")
parser.add_argument("--model", required=True, help="Model identifier on that endpoint")
parser.add_argument("--db", default="token_ledger.db", help="SQLite database path")
parser.add_argument("--task", default="default", help="Task type label for attribution")
parser.add_argument("--prompt", default="Say hello in exactly five words.")
parser.add_argument("--runs", type=int, default=3, help="Repeats per task")
args = parser.parse_args()
from openai import OpenAI
client = OpenAI(base_url=args.base_url, api_key=args.api_key)
ledger = TokenLedger(args.db)
metered = MeteredClient(client, ledger, args.task)
for i in range(args.runs):
response = metered.complete(
args.model,
[
{"role": "system", "content": "You are a terse assistant."},
{"role": "user", "content": args.prompt},
],
temperature=0.0,
)
print(f"run {i + 1}: {response.choices[0].message.content!r}")
print(json.dumps(ledger.heatmap(), indent=2))
if __name__ == "__main__":
main()
Running the meter against a free endpoint
The setup is intentionally boring, because boring setups get repeated:
mkdir token-forensics && cd token-forensics
python -m venv .venv && source .venv/bin/activate
pip install openai
# Export values from the MonkeyCode quick-start; they can change.
export MONKEYCODE_BASE_URL=<from-quickstart>
export MONKEYCODE_MODEL=<from-quickstart>
python token_forensics.py \
--base-url "$MONKEYCODE_BASE_URL" \
--model "$MONKEYCODE_MODEL" \
--task summarization \
--prompt "Summarize this meeting transcript in three bullets." \
--runs 5
I am not hardcoding the endpoint or model identifier because those values come from the project's quick-start and can change. Run the same command with different --task labels for each call type in your application, then tear the free server down. The SQLite database keeps accumulating across runs, so you can build a picture over days rather than minutes.
Reading the report: a decision matrix
A realistic heatmap after a few days of metering might look like this:
[
{
"task_type": "summarization",
"calls": 142,
"total_tokens": 1846000,
"prompt_tokens": 1736000,
"completion_tokens": 110000,
"avg_latency_ms": 2100
},
{
"task_type": "code_review",
"calls": 38,
"total_tokens": 820000,
"prompt_tokens": 740000,
"completion_tokens": 80000,
"avg_latency_ms": 3400
},
{
"task_type": "chat",
"calls": 510,
"total_tokens": 310000,
"prompt_tokens": 210000,
"completion_tokens": 100000,
"avg_latency_ms": 900
}
]
Apply these rules to your own numbers:
| Finding | Action |
|---|---|
| One task consumes >40% of total tokens | Optimize that task's prompt or model separately; do not touch anything else |
| prompt_tokens are >70% of a task's total | Compress the system prompt or check whether you are resending a long context every call |
| Many calls with small per-call tokens | Consider batching or caching similar requests |
| completion_tokens are unusually high | Check max_tokens and output format constraints; the model may be generating more than the task needs |
The point is not to find the cheapest model. The point is to find the task where optimization actually matters, and to leave everything else alone.
Limitations
- This is an attribution tool, not a performance benchmark. It tells you where tokens go, not whether the output is good.
- SQLite writes add a few milliseconds per request, which is acceptable for experiments but not for latency-critical paths.
- The 10 million token figure and the free server are operator-supplied claims as of August 2026. Verify them in the repository before planning around them.
- Never send real secrets, customer data, or proprietary code to a free endpoint, because you should assume the other side of the API is not your private property.
- The free server is for experiments, and hosting a production workload on it is exactly the mistake this article argues against.
Who should not use this approach
- Teams already using Langfuse, LangSmith, or a commercial LLM observability platform do not need this script; they should export their existing traces instead.
- Teams that need real-time dashboards and alerts should invest in a proper observability stack, not a SQLite file.
- Anyone under compliance obligations such as HIPAA or PCI should not send data to free endpoints at all, regardless of the metering layer.
The takeaway
Before you switch models, compress a prompt, or argue about verbosity, meter. A week of attribution on a free endpoint will tell you which optimization is worth doing first, and no benchmark can make that decision for you. Clone the script, point it at the MonkeyCode free endpoint, and let the data argue for you.
Top comments (1)
The useful part here is treating tokens as an allocation problem before treating them as a model-choice problem. I would add one more column to the ledger, the economic owner of the call. A token heatmap by feature is good. A token heatmap by feature and user promise is where the pricing conversation starts to become real.