I'm 13, Building a CLI Tool for LLM Cost Tracking, and Shipping It in 10 Days

#cli #llm #python #showdev

The problem

A few weeks ago I read a Reddit comment that stuck with me:

"I lost my house — and, subsequently, my marriage — due to a token cost miscalculation."

42 upvotes. Not because it's rare — because everyone who's shipped an LLM feature has their own smaller version of that fear. You add an OpenAI call to your app, ship it, and then you're checking your billing dashboard every few hours hoping nothing spiked while you weren't looking.

I went looking for a tool that just... shows you the cost. In real time. In your terminal. Without an account, without a dashboard, without fifteen minutes of setup.

It didn't exist. So I'm building it.

What TokenShark does

import tokenshark
tokenshark.monitor()

That's the entire integration. One import. Your existing OpenAI or Anthropic calls now log their cost, latency, and token usage locally — no data leaves your machine, no account required.

tokenshark dashboard

Shows you a live terminal view of every call, sorted by cost, with a running total for the day.

# tokenshark.yaml
daily_budget: 5.00
slack_webhook: "your-webhook-here"

Set a budget. Get a Slack alert before you blow through it. Optionally raise a hard stop.

Why I'm building this specifically

I did a lot of research before writing any code — competitive analysis across Helicone, LangSmith, Langfuse, Opik, Promptfoo. Here's what I found:

Every single one of them is dashboard-first, not terminal-first. Setting up Helicone means routing your traffic through their proxy. LangSmith needs an account, an API key, environment variables, SDK wrapping. Langfuse self-hosted needs Docker and S3 configuration. None of them are "one import and you're done."

LangSmith has a confirmed pricing bug. They show cached tokens at full price when Anthropic actually charges 10% of the normal rate for cache reads. Multiple developers have hit this. TokenShark calculates cache pricing correctly — cache creation at 25% of base input, cache reads at 10% — verified against official docs.

Nobody does per-request attribution well. The most repeated complaint I found in research across Reddit and GitHub issues: developers can see total cost, but can't tell which specific request, feature, or user caused a spike. TokenShark supports optional metadata tagging so you can break costs down by whatever matters to you.
*Building from Oman.

The build

I'm running this as a public 10-day sprint. Architecture decided in advance, one clear goal per day, shipping to PyPI by day 7, launching on Show HN by day 9.

Some of the interesting technical decisions so far:

Class-level patching, not module-level. My first architecture draft patched openai.chat.completions.create directly. Turns out that only catches one calling pattern — most real code does client = OpenAI(); client.chat.completions.create(), which the module-level patch silently misses. Caught this before writing the actual interceptor by testing against a fake SDK I built specifically to verify behavior offline.
Sync and async support from day one. Wasn't in my original spec, but async is common enough in production LLM code that skipping it would have made the tool feel incomplete on launch.
Never logging prompt content. Only metadata — token counts, cost, latency, optional tags you provide. This is a deliberate privacy decision, not a limitation. TokenShark should never be a compliance risk for anyone using it.

What's next

Days 3-6 are cost verification, the dashboard, Slack alerts, and metadata tagging. Day 7 is PyPI. Day 9 is Show HN.

I'll be posting here and on Twitter as it progresses — the wins and the bugs both. If you've hit the "surprise LLM bill" problem yourself, I'd genuinely like to hear about it. It's exactly the kind of thing that's shaping what TokenShark becomes before v0.1 even ships.