DEV Community

Christo
Christo

Posted on

Your LLM CLI's cost estimate is wrong in both directions - here's the field that fixes it

#typescript #cli #opensource #ai

Most BYOK CLIs print a cost line after each call. Nearly all of them compute it the same way: input_tokens * $in + output_tokens * $out, from a hardcoded rate table. Mine did too. That number is wrong, and it took measuring three real runs to see how wrong.

I recomputed against xAI's own billed figure on three recorded GrokScope runs:

Run Tokens (in/out) Actually billed Token-math estimate Error
ask 29,320 / 2,280 $0.082301 $0.072320 14% low
compare 48,189 / 2,656 $0.119872 $0.112314 7% low
trending 83,391 / 2,402 $0.152848 $0.181194 19% HIGH

Three failure modes, and the interesting part is that they don't all push the same way:

  1. Server-side tools are billed separately from tokens. Grok's x_search costs $5 per 1,000 calls; a single GrokScope query makes 6-12 of them. Token math can't see that spend at all, so it under-reports.
  2. Cached input is 85% cheaper. grok-4.5 bills cached input at $0.30/M against $2.00/M cold. The trending run had 51,968 cached tokens - that discount was larger than its tool spend, so the same formula over-reported by 19%.
  3. There's a tier cliff. grok-4.5 is $2/$6 per M under 200k prompt tokens and $4/$12 at or above. One hardcoded pair can't express that, so a long-context call silently reports half its real price.

Errors 1 and 2 point in opposite directions, which is what makes this nasty: you can't correct for it with a fudge factor, and on any given run you don't know which way you're wrong.

The fix was sitting in the response the whole time. xAI's Responses API returns usage.cost_in_usd_ticks alongside the token counts - documented as "the actual amount billed, after all applicable discounts (including prompt caching reductions) have been applied, and inclusive of all token costs and server-side tool invocation costs." 1 USD = 10^10 ticks.

// we were doing this...
const usage = { inputTokens: raw.input_tokens, outputTokens: raw.output_tokens };
// ...while raw.cost_in_usd_ticks sat right there in the same object
Enter fullscreen mode Exit fullscreen mode

GrokScope v1.4.0 prefers it everywhere:

$ grokscope ask "bun vs node in 2026"
...
70,821 tokens - $0.1975 billed        # exact - no tilde, no hedge
Enter fullscreen mode Exit fullscreen mode

Three implementation notes that might save you an afternoon:

  • Don't drop the estimate. Proxies, older cached responses and offline mocks won't have the field. Keep the rate table as a labelled fallback and mark which one you printed - GrokScope's --json emits costUsd plus costExact: true|false.
  • Round to 8 decimals, not 6. The docs' own example is 158,500 ticks = $0.00001585. At 6 decimals a cheap call rounds to zero.
  • Validate hard, coerce never. Accept only a finite non-negative number. A null, a string, or a negative must fall back cleanly - Number(null) is 0, and a $0.00 cost line that's actually a parse failure is worse than no cost line.

Nice side effect: if you point the CLI at a model with no entry in the rate table, the old code printed no dollar figure at all. The exact field doesn't care what the model is, so it prints one now.

MIT, BYOK, and the whole pipeline runs offline against a doc-accurate mock - 120 e2e checks, no API key needed to contribute.

https://github.com/Booyaka101/grokscope

Top comments (0)