Imagine you hire a researcher to answer a question.
They hand you a neat one-page report and say, "That took me 10 minutes."
What they don't tell you:
- They made two wrong turns.
- They rewrote the summary because the first draft was in the wrong format.
- They actually spent 20 minutes, but only billed you for the parts that worked.
That is how most AI agents report their cost today.
The attempts nobody counts
An agent step rarely succeeds on the first call. The model answers in prose when you asked for JSON, so the step asks again. A reviewer rejects the answer, so the review runs again.
Your provider bills every one of those calls. Most tracking only keeps the call whose answer you ended up using. The failed attempts are spent, billed, and missing from your own numbers.
Here is one small job, two steps, with every call on the receipt:
step attempt result cost
extract 1 failed: prose instead of JSON $0.005
extract 2 ok $0.006
review 1 failed: answer rejected $0.027
review 2 ok $0.028
──────
what you paid $0.066
what success-only shows $0.034
The dashboard says $0.034. The provider charges $0.066. In this run, 48% of the spend is invisible, and nothing about the answer tells you it happened.
The numbers are illustrative, but the shape is not. Any retry that happens below the layer that records cost is a retry you pay for twice and see once.
Put the retry where the bill can see it
spendgraph makes one design choice about this: the retry wraps the whole recorded call, not just the model request inside it.
retry inside the call retry around the call
┌ recorded call ──────────┐ ┌ recorded call ┐ attempt 1 $0.005
│ model ✗ │ └───────────────┘
│ model ✓ │ ┌ recorded call ┐ attempt 2 $0.006
└─────────────────────────┘ └───────────────┘
one row: $0.006 two rows: $0.011
On the left, the retry is smaller and cheaper to build, and your accounting is wrong. On the right, every attempt is its own priced record, so the total adds up.
Two other choices follow from it:
- Only retry what a retry can fix. A reply in the wrong shape may parse on the next sample, so it is retried. A prompt that never rendered your question will not fix itself, so it fails once and stops, instead of billing you twice for the same mistake.
- Costs are integers. Every cost is stored in micro-dollars (µ$1 = $0.000001). Floats drift after a few thousand additions of a tenth of a cent; integers give the same monthly total however they were summed.
In code
A stage is one prompt, one schema, and one priced reply. You get the answer as soon as it is ready, and the price when it resolves:
import { runStage } from "@spendgraph/stage";
const outcome = await runStage(prompts, llm, "classify-email", SCHEMA, {
question: "Is this email spam or clean?",
}, {
attempts: 3,
emit: (event) => {
if (event.type === "stage:failed") {
console.warn(`attempt ${event.attempt} failed: ${event.error.message}`);
}
},
});
console.log(outcome.data);
const micros = await outcome.pricing;
A few things worth knowing:
-
promptsis your prompt store andllmis your model client. The running a stage page shows how to set both up. -
pricingis a promise, and the answer does not wait for it. A billing lookup has no business on the path of a user waiting for a reply. -
pricingcan resolve toundefinedwhen no price could be found. That is deliberate: an unknown price is not the same as a zero one, and it should not look like one. - Each failed attempt fires
stage:failedas it happens, and shows up as its own priced record, so the extra spend is on the bill and not only in your logs.
If you only want spend tracking on the client you already use, without stages, the SDK wraps it in one line:
import OpenAI from "openai";
import { SpendGraph } from "@spendgraph/sdk";
const meter = new SpendGraph({
apiKey: process.env.SPENDGRAPH_API_KEY,
baseUrl: process.env.SPENDGRAPH_BASE_URL,
});
const openai = meter.wrap(new OpenAI());
Use openai exactly as before. Each call's usage is read off the reply and reported in the background, and the meter never throws into your code.
The bottom line
If a retry happens where your cost tracking cannot see it, your cost tracking is wrong, and wrong in the direction that makes things look cheaper than they are.
Count every attempt. Look at which steps fail most. The prompt that fails half the time is usually a cheaper fix than a cheaper model.
Top comments (4)
@fnlog0 The attempt-level receipt is a useful complement to the execution traces I'm working on in agent-inspect. I'd preserve usage, price revision, and pricing status separately so repricing an old unpriced record doesn't erase the original evidence. One boundary question: does the OpenAI wrapper observe SDK-internal retries, or only the final response? Making that coverage explicit would keep “one recorded call” from quietly becoming “one provider attempt.”
Thanks, both fair.
Pricing: an event we can't price is stored at $0 today. We already plan to fill in the real cost once a price exists. Your point improves that: keep the original record and add the new price beside it, instead of erasing it.
Retries: the OpenAI wrapper only sees the final response, so the OpenAI client's built-in retries (maxRetries) are invisible to it. For wrap(), one recorded call means one completed call. runStage is the path that records every attempt. I'll make that explicit in the article.
How does agent-inspect draw that line?
The retry-wraps-the-whole-call choice is the right one, and the diagram makes it obvious why: anything that keeps a retry inside the recorded call is choosing to make its own accounting lie.
One thing I'm curious about: pricing resolving to undefined when no price could be found is a good instinct on its own, but what happens to that record once it reaches a dashboard or a monthly total? If an undefined price just gets skipped over in aggregation, you've reintroduced the exact blind spot this whole design exists to close, just moved one layer up, from provider billing to your own rollup. Does spendgraph surface unknown-cost records as their own bucket somewhere, or is that left to whoever builds on top of it?
Good question, and you've put your finger on a real gap.
In the stage runner, a ledger with even one unpriced stage reports its total as null, not as a smaller number, because a partial sum isn't the total. It also gives the known cost, labelled as a floor, and a count of stages still unpriced.
On the hosted side it's weaker than that. An event we can't price is accepted and stored at $0, with its token counts kept. The ingest response names the unmatched model ids right away, the SDK logs each one once, and the dashboard marks those rows "unpriced". The Overview warns you, with something like "3 of 20 requests priced at $0, their spend is missing from the totals below". So it isn't hidden, but it isn't a separate bucket either: it's a $0 inside the sum, with a warning on top.
Your question made me check the case where a price turns up later. Right now the warning clears, but the old events stay at $0, so the total ends up too low with nothing saying so. That's exactly the blind spot you described. It's on our list to fix: store an unknown cost as unknown, not zero, and re-price old events once a price exists.
More on the reasoning: spendgraph.locusgraph.com/blog/unp...