DEV Community

NOTEGG
NOTEGG

Posted on

Find where your LLM API bill leaks—without uploading your logs

#llm #python #api #opensource

An LLM API bill tells you how much you spent. Choosing what to change takes more investigation.

Are repeated requests worth reusing? Are prompts actually hitting the provider’s cache? Are failed or truncated calls still adding to the bill?

We built Linklbee Scan, an open-source Python tool that reads existing usage logs on your machine and generates an HTML report to help answer those questions.

No API key, no model calls, and no log uploads. It uses the Python standard library and runs on Python 3.9+.

Try it without preparing a log

Install the scanner and run the built-in demo:

pip install linklbee-scan
linklbee-scan --demo --lang en
Enter fullscreen mode Exit fullscreen mode

The demo uses a synthetic example log. It lets you inspect the report format before using your own records; its figures are not customer savings results.

To analyze your own log:

linklbee-scan calls.jsonl --lang en --out report.html
Enter fullscreen mode Exit fullscreen mode

You can also pass a folder instead of a single file.

Check what the scanner is reading

Before interpreting a report, check how your fields are mapped:

linklbee-scan calls.jsonl --peek --lang en
Enter fullscreen mode Exit fullscreen mode

This reads the first 200 records and prints the field mapping without writing a report.

That matters because different logging systems capture different information. Token counts can tell you about spending, but they cannot answer every question about repeated requests or shared prompt prefixes.

What to look for in the report

Depending on the fields available, the scanner checks patterns including:

  • Output’s share of the bill.
  • Cache hit rate and prompts below the provider’s cache minimum.
  • Shared prefixes across calls.
  • Identical calls resent within seconds.
  • Failed calls that were billed.
  • Responses cut off at the output limit.
  • Potential output savings when revising an earlier document instead of rewriting it.

Each check is marked as found, none, or needs more data. When a required field is missing, the report names it rather than guessing.

Turn a finding into an experiment

A finding gives you something to investigate. For example:

  • Repeated identical calls: check whether reuse is valid for that workload and its freshness requirements.
  • Low cache usage: inspect prompt length and prefix consistency.
  • Cut-off responses: investigate output limits and any follow-up requests.
  • Rewritten documents: evaluate whether updating only changed sections preserves the required result.

Then compare cost, quality, and latency on representative tasks. Estimated savings in a report and savings measured after an application change are different things.

Supported logs and privacy

The scanner reads JSONL, JSON arrays, CSV/TSV, .log, and .gz files. It includes mappings for common provider responses and exports from tools such as Langfuse, LangSmith, Helicone, and LiteLLM.

Processing stays on your machine. The generated report contains aggregate figures rather than prompt or answer text.

The project is MIT-licensed and published by NOTEGG, the team building Linklbee.

Source and usage instructions:

https://github.com/linklbee/scan
Linklbee: https://linklbee.com/

If you try it, we’d especially like to know: does the report help you choose a concrete next step? Missing log formats, unclear findings, and incorrect field mappings are useful feedback too.

Top comments (0)