I built an LLM cost tracker on my laptop at Hyderabad airport
We had a layover in Hyderabad. Long one. Like six hours. I had my laptop and an idea I'd been sitting on.
The problem
I build stuff with LLMs. Chatbots, summarizers, agents that check if my homework makes sense. But you never know what it costs until the bill comes. You call GPT-4o, it works, you call it again, it works again, you ship it, and then your dad asks about the credit card.
I wanted cost to show up in real-time. In the terminal. Without changing my code. Two lines:
import tokenshark
tokenshark.monitor()
Every OpenAI or Anthropic call gets intercepted, cost-calculated, and logged. Like this:
[TokenShark] OpenAI gpt-4o | 1,247↑ 382↓ | $0.0604 | 340ms | trace_id: req_7a3f
[TokenShark] Anthropic claude-3-5-sonnet | 890↑ 1,204↓ | $0.0891 | 1.2s | user: alice
I called it TokenShark because sharks are cool and tokens cost money.
The catch
Hyderabad airport WiFi is that kind that "connects" but doesn't actually work. I couldn't pip install anything. No packages, no testing against real SDKs. I had an architecture doc I wrote weeks ago 459 lines of exactly how this should work. And I had weird knowledge of the OpenAI Python SDK.
So I typed. For hours.
What I built
Eight files, ~1,350 lines.
proxy.py patches the actual OpenAI and Anthropic clie
nt classes — not module-level shortcuts, the real classes. I remembered from reading the SDK source that their client is instance-based. You do client = OpenAI() and then client.chat.completions.create(...). So patching openai.chat.completions.create only catches module-level calls. Real code uses instances. I had to patch openai.resources.chat.completions.Completions.create and anthropic.resources.messages.Messages.create.
Streaming was the worst part. OpenAI streams don't include usage by default. You need stream_options={"include_usage": True} or every chunk has usage=None. TokenShark auto-injects that. But if some proxy strips it out anyway, it falls back to tiktoken estimation and marks the row as estimated in the database.
Anthropic streams differently — usage is split across message_start and message_delta events. I had to assemble the full picture from partial data.
Cached tokens: OpenAI nests them at usage.prompt_tokens_details.cached_tokens, but some older responses have a flat usage.cached_tokens. I check both. Anthropic has cache_creation_input_tokens and cache_read_input_tokens as flat attributes. Got all this from memory and a few web searches that squeezed through the spotty airport WiFi.
tracker.py has the cost calculation, SQLite schema, and database writes. Every call gets logged with provider, model, token counts, cost in dollars, latency, custom tags, whether the cost was estimated or exact, and error messages if it failed.
config.py loads tokenshark.yaml, merges with defaults, checks for accidentally-committed API keys at every nesting level (not just top-level), and resolves Slack webhooks from env vars. I caught a real bug here — my first draft used dict(_DEFAULT_CONFIG) which is a shallow copy, so mutating nested sections corrupted the module-level default. Fixed with copy.deepcopy(). Felt pretty smart about that one.
exceptions.py, __init__.py, pyproject.toml — standard stuff. I made openai and anthropic optional extras so you don't have to install both if you only use one.
Testing with fake SDKs
Couldn't test against real packages. No internet, no pip. So I built fake OpenAI and Anthropic SDKs. Faithful replicas of the real class structures with realistic streaming behavior.
My fake OpenAI sends chunks where only the final one has usage data (and only if stream_options was set). My fake Anthropic sends message_start and message_delta events with split usage. Tested sync and async clients, streaming and non-streaming, errors, Ollama routing, double monitor() calls, config merging, dangerous key detection — everything.
Hand-verified the cost math. Fake Claude call with 200 cache-read tokens, 80 cache-creation tokens, 30 output tokens at placeholder rates: (200*3 + 80*15)/1e6 + 30*0.75/1e6 = $0.0018225. Matched exactly what my code logged to SQLite.
I was sitting in a plastic airport chair at 2 AM running SQLite queries on my laptop. This is my life.
What's left
About 60% through Day 2. Still need: CLI tool, web dashboard, Slack alerts, real tests against actual SDKs, verified pricing (some are placeholders right now), more providers.
Why I'm sharing this
Not trying to be a prodigy. Just a kid who likes building things and got stuck at an airport with an idea and too much energy.
But LLM costs are invisible until they hurt. Making them visible in real-time with zero friction feels like something every developer should have.
If you've ever been surprised by an LLM bill, maybe give it a look. Or star it. Or tell me why my architecture is wrong. All good outcomes.
pip install tokenshark[openai,anthropic] # soon
Or hack on it:
git clone https://github.com/your-org/tokenshark.git
cd tokenshark
pip install -e ".[openai,anthropic]"
May your tokens be few and your insights be many.
Top comments (0)