DEV Community

wartzar-bee
wartzar-bee

Posted on

You can see your cloud bill. Can you see what your AI agent's context costs?

#ai #llm #opensource #productivity

You can see your cloud bill. You can see your CI minutes tick down. But the fastest-growing line item in an AI-agent app is the one number you can't see: how many tokens your agent's context is worth on every single model call.

It's invisible because it never shows up as one big charge. It's a few thousand tokens, resent hundreds of times a day, quietly compounding. By the time it's a real number on the invoice, it's baked into every request you make.

Here's how to read it in ten seconds — no account, no logs, no config:

npx @wartzar-bee/tokenscope scan .
Enter fullscreen mode Exit fullscreen mode

Point it at the directory that holds your agent's prompts, tools, and configs. It prints the token footprint and the files responsible:

tokenscope scan — src
Estimated token footprint: 8,454 tokens across 6 files
(estimate ≈ 4 chars/token — a tokenizer-free proxy for relative comparison, not a billing figure)

Top files by estimated tokens:
      2991  share.mjs
      2134  scan.mjs
      1092  core.mjs
       835  report.mjs
       718  pricing.mjs
       684  benchmark.mjs
Enter fullscreen mode Exit fullscreen mode

That's the whole point: the top file is usually a system prompt, a tool schema, or a wall of few-shot examples that someone added "just to be safe." Now you can see which one, and how much it weighs, before it's part of every call.

Why this is the number that bites

Most agent frameworks resend the accumulated context on each step — memory, history, tool definitions, the lot. So a prompt that's 2,000 tokens heavier isn't a one-time cost; it's 2,000 tokens × every call × every user. The per-call log looks fine. The monthly bill does not. (I watched an agent on a timer burn 136M tokens overnight doing almost nothing — same root cause.)

tokenscope gives you a single, reproducible footprint number so a "let's just add this to the prompt" PR stops being invisible.

Three ways to use it

1. Ad-hoc, right now — measure any repo before you ship a prompt change:

npx @wartzar-bee/tokenscope scan ./agent
Enter fullscreen mode Exit fullscreen mode

2. As a local cost gate — fail your own commit if the footprint blows a budget, so a runaway prompt never leaves your machine:

# .git/hooks/pre-push  (chmod +x)
npx @wartzar-bee/tokenscope scan . --max-total 50000 || {
  echo "Context footprint over budget — trim it before pushing."; exit 1;
}
Enter fullscreen mode Exit fullscreen mode

Using the pre-commit framework? It's a four-line entry — no hook scripting.

3. On every PR, in CI — the same check as a GitHub Action that comments the token-cost delta on the responsible files and (optionally) blocks the merge:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    mode: warn   # report-only until you trust it; switch to block later
Enter fullscreen mode Exit fullscreen mode

That's ci-guardrail — tokenscope wired into your pipeline.

Get it

It's free, open-source, and tokenizer-free — an estimate for relative comparison, not a billing oracle, so you can run it on any codebase without wiring up a provider SDK. If it saves you one "why is the bill up 40%?" afternoon, it did its job.

Top comments (0)