I'm 13, Building a CLI Tool for LLM Cost Tracking, and Shipping It in 10 Days

The problem

A few weeks ago I read a Reddit comment that stuck with me:

"I lost my house — and, subsequently, my marriage — due to a token cost miscalculation."

42 upvotes. Not because it's rare — because everyone who's shipped an LLM feature has their own smaller version of that fear. You add an OpenAI call to your app, ship it, and then you're checking your billing dashboard every few hours hoping nothing spiked while you weren't looking.

I went looking for a tool that just... shows you the cost. In real time. In your terminal. Without an account, without a dashboard, without fifteen minutes of setup.

It didn't exist. So I'm building it.

What TokenShark does

import tokenshark
tokenshark.monitor()

That's the entire integration. One import. Your existing OpenAI or Anthropic calls now log their cost, latency, and token usage locally — no data leaves your machine, no account required.

tokenshark dashboard

Shows you a live terminal view of every call, sorted by cost, with a running total for the day.

# tokenshark.yaml
daily_budget: 5.00
slack_webhook: "your-webhook-here"

Set a budget. Get a Slack alert before you blow through it. Optionally raise a hard stop.

Why I'm building this specifically

I did a lot of research before writing any code — competitive analysis across Helicone, LangSmith, Langfuse, Opik, Promptfoo. Here's what I found:

Every single one of them is dashboard-first, not terminal-first. Setting up Helicone means routing your traffic through their proxy. LangSmith needs an account, an API key, environment variables, SDK wrapping. Langfuse self-hosted needs Docker and S3 configuration. None of them are "one import and you're done."

LangSmith has a confirmed pricing bug. They show cached tokens at full price when Anthropic actually charges 10% of the normal rate for cache reads. Multiple developers have hit this. TokenShark calculates cache pricing correctly — cache creation at 25% of base input, cache reads at 10% — verified against official docs.

Nobody does per-request attribution well. The most repeated complaint I found in research across Reddit and GitHub issues: developers can see total cost, but can't tell which specific request, feature, or user caused a spike. TokenShark supports optional metadata tagging so you can break costs down by whatever matters to you.
*Building from Oman.

The build

I'm running this as a public 10-day sprint. Architecture decided in advance, one clear goal per day, shipping to PyPI by day 7, launching on Show HN by day 9.

Some of the interesting technical decisions so far:

• Class-level patching, not module-level. My first architecture draft patched openai.chat.completions.create directly. Turns out that only catches one calling pattern — most real code does client = OpenAI(); client.chat.completions.create(), which the module-level patch silently misses. Caught this before writing the actual interceptor by testing against a fake SDK I built specifically to verify behavior offline.

• Sync and async support from day one. Wasn't in my original spec, but async is common enough in production LLM code that skipping it would have made the tool feel incomplete on launch.

• Never logging prompt content. Only metadata — token counts, cost, latency, optional tags you provide. This is a deliberate privacy decision, not a limitation. TokenShark should never be a compliance risk for anyone using it.

What's next

Days 3-6 are cost verification, the dashboard, Slack alerts, and metadata tagging. Day 7 is PyPI. Day 9 is Show HN.

I'll be posting here and on Twitter as it progresses — the wins and the bugs both. If you've hit the "surprise LLM bill" problem yourself, I'd genuinely like to hear about it. It's exactly the kind of thing that's shaping what TokenShark becomes before v0.1 even ships.

---

• Follow along: [https://github.com/Sovyte/TokenShark-v.01]

Comments

[Bhavin Sheth]

Love the privacy-first approach. Seeing costs right in the terminal feels much more natural than opening another dashboard every few minutes. Looking forward to seeing how this evolves.

[Sovyte]

Thank you so much for your support! im currently working on other projects and the landing page for tokenshark so the progress is a bit stalled, but i think by 19th a full launch is expected.

[Mike Czerwinski]

The class-level-patching catch is the kind of detail that separates someone who actually tested against real usage patterns from someone who read the API docs, module-level patching on openai.chat.completions.create silently misses the client = OpenAI() instantiation pattern that most real code actually uses, and you caught it before shipping by building a fake SDK to verify offline. That's a good instinct at any age. The LangSmith cached-token pricing bug you cite (full price instead of the 10% cache-read rate) is a genuinely useful finding if it's accurate, worth double-checking against current docs before you lean on it publicly since pricing details change and get corrected quietly. Never logging prompt content as a deliberate privacy decision rather than a limitation is the right default for a tool like this. Good luck with the sprint, following along for the cost-verification and metadata-tagging days.

[Sovyte]

Good catch, I'll re-verify the LangSmith cache pricing claim against their current docs before it goes in the README, I really appreciate you noticing that. And yea, the fake SDK approach was specifically because I didn't trust myself to reason through the real API behavior without actually testing it

[Mike Czerwinski]

Not trusting yourself to reason through the real API behavior without testing it is exactly the right instinct, most people twice your age ship on the docs and find out from a support ticket. One thing worth folding into the re-verify pass: pricing pages lag actual billing behavior more often than they lag the other way, so if you can pull a real response header or invoice line that shows the cache-read rate applied, that's a stronger receipt than the docs either way, docs corrected quietly is exactly the failure mode you're already worried about.

[Sovyte]

Sorry for the late reply I've been mid-sprint and then off-grid for a family trip, just catching up on this now.
This is a genuinely good point and I hadn't thought about it that way. I was planning to re-verify against the pricing page, but you're right that the page could just as easily be stale in the other direction — corrected quietly without an announcement, which is exactly the failure mode I'm trying to avoid by citing it.
Going to try pulling an actual API response header or checking if I can find an invoice line showing the cache-read rate applied, rather than trusting the docs either way. If I can get a real receipt like that, I'll cite it directly instead of the pricing page. Appreciate you pushing on this, it's the difference between a claim I can actually stand behind and one I'm just hoping is still true.

[Yevhenii]

It's interesting to see more tools focusing on making AI infrastructure lighter rather than adding more layers.

We've spent a lot of time optimizing context usage on the inference side, but cost visibility is another piece that's often missing during development.

[Sovyte]

"Simplicity is the ultimate sophistication."— Leonardo da Vinci