A few weeks back I watched an agentic pipeline burn a month of budget in three days. The instinct everyone reached for was the obvious one: "the fr...
For further actions, you may consider blocking this person and/or reporting abuse
"Cost in an agent isn't a number, it's a tree" is exactly right, and the
(step, tool, model)tuple is the unit that finally makes the bill legible. We went through the same arc — the reflex is "switch to a cheaper model," and it's almost always the wrong lever, because the waste isn't the price per token, it's the count of unnecessary frontier calls no one attributed.Two things I'd add from doing this in anger. First, retries are the sneakiest line item — a step that silently retries twice triples its cost and doesn't show up unless you log attempt number, and agent frameworks love to hide retries inside the tool layer. Second, once you have per-step attribution, the interesting metric isn't total cost, it's cost-per-successful-outcome, because a "cheap" step that fails and forces a re-plan upstream is more expensive than the expensive step it replaced. The logging you describe is genuinely the whole trick, but the payoff is being able to ask "which step has the worst cost-per-success," not just "which step is priciest." Did tagging change which steps you actually cut, versus what you'd have guessed?
The retries-inside-the-tool-layer point is the one most teams miss, because their observability stops at the tool boundary — the retry never shows up as a distinct call. And cost-per-successful-outcome is the right reframe: the router's job becomes "cheapest path that still succeeds," not just "cheapest path." On your question — tagging changed what we cut, because it exposed steps that looked cheap per-token but were net-negative once re-plan cost counted. Did you keep any step that got cheaper on paper but failed the success bar?
The (step, tool, model) tuple is doing a second job you have not cashed in yet, and it is the one I would care about more than the bill.
Everything here treats a retry as a spend line. That holds as long as the step is a read. The moment a step has an effect — writes a row, sends a message, moves money — the retry loop quietly re-running a hard call five times because the parser rejected a shape is not expensive. It is five effects. The cost dashboard is the cheapest detector anyone has for that, and nobody reads it as one, because the number it prints is dollars and the thing it found is a duplicate.
Payments has one rule here and it is older than any of this: the unit you retry is the unit you make idempotent. Same unit, same key. Your step boundary is already the right boundary for the failover decision, so it is already the right boundary for the idempotency key — you just have to admit the step has a class as well as a cost. Effectful steps retry against a key. Pure steps retry freely. Right now both are "a step" in the tuple.
Failover sharpens it. Routing a single step to a fallback at the step boundary is right, and it means the fallback runs against whatever the first model already did. If the first call timed out after the tool committed, the fallback is a second commit with a different model's name in the log. Not a more expensive run — a wrong one. Per-step attribution is what makes it visible at all: same step id, two models, one of which you would never have guessed was in the trajectory.
Genuine question since you have the data: when you tagged every token, did a retry show up under the same step id or as a new one? Whether the schema collapses retries into the step is the difference between "this step cost 5x" and "this step ran 5 times", and only one of those sends you looking for duplicates.
This is the sharpest extension and the one I'd want to build on. The (step, tool, model) tuple quietly assumes read-only steps — the moment a step mutates state, cost attribution and retry semantics split apart. A retry of a write isn't a duplicated cost, it's a duplicated action. The routing layer should know whether a step is idempotent before allowing a blind retry; otherwise "cheaper path" can mean "double side effect." Have you found a clean way to tag effect-vs-read at the tuple level?
“Swap the expensive model” is usually the wrong first diagnosis.
Once cost is attributed to the (step, tool, model) tuple, the bill often lives in repeated cheap steps and tool chatter that never had a success gate — not in the one frontier call everyone blames. Cost in an agent is a tree; a single sticker price is a guess.
The check I’d put next to task success: tokens (or $) per complete success, broken by step. If a cheaper route wins on spend but loses on completion, that is not a savings — it is a different, worse contract.
Exactly — the frontier call is the visible scapegoat; the real leak is the long tail of cheap steps, each firing with no success gate. Model-swap treats the symptom. The fix is attaching a success gate to the cheap steps so the router can collapse the chatter instead of echoing it. That only works if the gate is the tool's own signal, not just downstream task outcome. Do you gate tool-call retries on the tool's success signal, or on the final task result?
The “cost is a tree” framing makes a lot of sense.
I think there’s a similar observability problem with agent context too. We usually know what context was available, but not always which pieces actually influenced a decision or which assumptions became stale along the way.
Per-step attribution for cost feels useful; I wonder if long-running agents eventually need something similar for context.
The context problem is the same tuple wearing a different mask. We log what context was retrieved, but rarely what was attended — so "available context" and "used context" diverge silently. The defensible move is to attribute context cost to the step that requested it and measure outcome when it's pruned, the same way we gate a cheap tool call. Do you trace attention back to the retrieval step, or treat the prompt as one undifferentiated blob?
Per step attribution is the difference between a cost story and a guess. I would add one guardrail here, record quality outcomes beside tokens so a cheaper route is judged by task success rather than spend alone. That makes failover policy measurable and keeps the router from quietly optimizing for low cost at the expense of the hard step.
The guardrail is what turns failover from a reflex into a policy — without outcome data the router silently optimizes the easy 80% and starves the hard 20%. Recording quality beside tokens is the missing column most cost dashboards don't have. Do you gate failover on a per-step success threshold, or a rolling task-level one? Per-step catches a bad route faster, but task-level is less twitchy on noisy single steps.
The retry point is very real. I’ve seen a cheap step become the expensive one just by retrying downstream work. Tracking attempts separately from the step cost makes that obvious fast.
Yes — the "attempts" dimension is what flattens the illusion that a step is cheap. Plot cost per (step, attempt) and the 3rd retry of a $0.001 step that triggers a $2 downstream re-run stops hiding. The trap is letting attempts ride on the caller's retry loop instead of the router's budget. Do you cap attempts per step, or let the router decide when an attempt budget is exhausted and route the next one differently?