Stop Paying for Invisible Retries: How to Measure What an AI Agent Really Costs

Imagine you hire a researcher to answer a question.

They hand you a neat one-page report and say, "That took me 10 minutes."

What they don't tell you:

• They made two wrong turns.
• They rewrote the summary because the first draft was in the wrong format.
• They actually spent 20 minutes, but only billed you for the parts that worked.

That is how most AI agents report their cost today.

---

The attempts nobody counts

An agent step rarely succeeds on the first call. The model answers in prose when you asked for JSON, so the step asks again. A reviewer rejects the answer, so the review runs again.

Your provider bills every one of those calls. Most tracking only keeps the call whose answer you ended up using. The failed attempts are spent, billed, and missing from your own numbers.

Here is one small job, two steps, with every call on the receipt:

step       attempt   result                         cost
extract    1         failed: prose instead of JSON  $0.005
extract    2         ok                             $0.006
review     1         failed: answer rejected        $0.027
review     2         ok                             $0.028
                                                    ──────
                                   what you paid    $0.066
                         what success-only shows    $0.034

The dashboard says $0.034. The provider charges $0.066. In this run, 48% of the spend is invisible, and nothing about the answer tells you it happened.

The numbers are illustrative, but the shape is not. Any retry that happens below the layer that records cost is a retry you pay for twice and see once.

---

Put the retry where the bill can see it

spendgraph makes one design choice about this: the retry wraps the whole recorded call, not just the model request inside it.

retry inside the call              retry around the call

┌ recorded call ──────────┐        ┌ recorded call ┐  attempt 1  $0.005
│ model  ✗                │        └───────────────┘
│ model  ✓                │        ┌ recorded call ┐  attempt 2  $0.006
└─────────────────────────┘        └───────────────┘
one row: $0.006                    two rows: $0.011

On the left, the retry is smaller and cheaper to build, and your accounting is wrong. On the right, every attempt is its own priced record, so the total adds up.

Two other choices follow from it:

• Only retry what a retry can fix. A reply in the wrong shape may parse on the next sample, so it is retried. A prompt that never rendered your question will not fix itself, so it fails once and stops, instead of billing you twice for the same mistake.
• Costs are integers. Every cost is stored in micro-dollars (µ$1 = $0.000001). Floats drift after a few thousand additions of a tenth of a cent; integers give the same monthly total however they were summed.

---

In code

A stage is one prompt, one schema, and one priced reply. You get the answer as soon as it is ready, and the price when it resolves:

import { runStage } from "@spendgraph/stage";

const outcome = await runStage(prompts, llm, "classify-email", SCHEMA, {
  question: "Is this email spam or clean?",
}, {
  attempts: 3,
  emit: (event) => {
    if (event.type === "stage:failed") {
      console.warn(`attempt ${event.attempt} failed: ${event.error.message}`);
    }
  },
});

console.log(outcome.data);

const micros = await outcome.pricing;

A few things worth knowing:

• prompts is your prompt store and llm is your model client. The running a stage page shows how to set both up.
• pricing is a promise, and the answer does not wait for it. A billing lookup has no business on the path of a user waiting for a reply.
• pricing can resolve to undefined when no price could be found. That is deliberate: an unknown price is not the same as a zero one, and it should not look like one.
• Each failed attempt fires stage:failed as it happens, and shows up as its own priced record, so the extra spend is on the bill and not only in your logs.

If you only want spend tracking on the client you already use, without stages, the SDK wraps it in one line:

import OpenAI from "openai";
import { SpendGraph } from "@spendgraph/sdk";

const meter = new SpendGraph({
  apiKey: process.env.SPENDGRAPH_API_KEY,
  baseUrl: process.env.SPENDGRAPH_BASE_URL,
});
const openai = meter.wrap(new OpenAI());

Use openai exactly as before. Each call's usage is read off the reply and reported in the background, and the meter never throws into your code.

---

The bottom line

If a retry happens where your cost tracking cannot see it, your cost tracking is wrong, and wrong in the direction that makes things look cheaper than they are.

Count every attempt. Look at which steps fail most. The prompt that fails half the time is usually a cheaper fix than a cheaper model.