DEV Community

Cover image for The Unit Economics of an AI Agent Feature, Measured in TypeScript
Gabriel Anhaia
Gabriel Anhaia

Posted on

The Unit Economics of an AI Agent Feature, Measured in TypeScript

#typescript #agents #ai #node

The first cost question is always "what does a call cost". It is the wrong
question. A cheap call that fails, gets retried, and ends with the user giving
up and opening a ticket costs more than an expensive call that works.

The number that matters is cost per resolved task. Getting to it needs
attribution at the run level, and the levers that move it are not the ones
people reach for first.

Attribute cost to the run, not the call 

export class Budget {
  private spentUsd = 0;
  readonly calls: CallCost[] = [];

  constructor(readonly capUsd: number, private onExceed: () => never) {}

  record(u: Usage, model: string, phase: Phase) {
    const usd = price(model, u);
    this.spentUsd += usd;
    this.calls.push({ model, phase, usd, in: u.input, out: u.output,
                      cachedIn: u.cacheRead ?? 0 });
    if (this.spentUsd > this.capUsd) this.onExceed();
  }

  get spent() { return this.spentUsd; }
}
Enter fullscreen mode Exit fullscreen mode

phase is the field that makes this useful — plan, retrieve, act,
summarise. Without it you know a run cost 14 cents and nothing about where
it went. With it, the first report usually shows something surprising, and it
is usually the same thing: the summarise phase resending the entire history.

Count cached input separately. It is typically an order of magnitude cheaper
than uncached, and mixing them makes the numbers meaningless.

export function price(model: string, u: Usage): number {
  const p = PRICES[model];
  return (u.input - (u.cacheRead ?? 0)) / 1e6 * p.in
       + (u.cacheRead ?? 0)             / 1e6 * p.cacheRead
       + (u.cacheWrite ?? 0)            / 1e6 * p.cacheWrite
       +  u.output                      / 1e6 * p.out;
}
Enter fullscreen mode Exit fullscreen mode

Prices from a config file, not constants in code. They change, and a stale
constant makes every historical number quietly wrong.

Cost per resolved task 

const resolved = runs.filter((r) =>
  r.outcome === "complete" && !r.followedByTicket && !r.userDiscarded);

const costPerResolved =
  runs.reduce((s, r) => s + r.costUsd, 0) / resolved.length;
Enter fullscreen mode Exit fullscreen mode

The numerator is all runs, including the ones that failed. Those are real
spend. The denominator is only the ones that actually finished the user's job.

A feature with a 60% resolution rate costs 1.67× its apparent per-run cost.
That gap is where the improvement usually is, and improving resolution beats
shaving tokens almost every time.

Total spend across all runs divided by the runs that actually<br> resolved.

Where the money actually goes

Three distributions, in the order they usually surprise people.

Input dwarfs output. A twenty-turn agent run resends its history on every
turn. Output tokens are a rounding error next to that.

Cost is long-tailed. p50 might be 3 cents and p99 two euros. The mean is
useless; the p99 is the one that decides whether the feature is viable at
scale.

metrics.histogram("run.cost_usd", costUsd, { intent, model });
metrics.histogram("run.turns", turns, { intent });
Enter fullscreen mode Exit fullscreen mode

One intent dominates. Segment by what the user was trying to do and you
typically find one intent responsible for most of the spend — often one that
is also the least valuable.

SELECT intent, count(*), avg(cost_usd), percentile_cont(0.95)
       WITHIN GROUP (ORDER BY cost_usd)
FROM run_records
WHERE started_at > now() - interval '7 days'
GROUP BY intent ORDER BY sum(cost_usd) DESC;
Enter fullscreen mode Exit fullscreen mode

That query has redirected more roadmaps than any dashboard I have built.

Four levers, in order of effect

1. Prompt caching. Biggest single win, smallest change. A stable prefix —
system prompt, tool schemas, few-shot examples — cached across turns. The
requirement is that the prefix is byte-identical, which means no timestamps
and no per-request interpolation near the top.

// breaks the cache on every request
system: `You are a support agent. Current time: ${new Date().toISOString()}`;

// stable prefix, volatile content moved into the message
system: SUPPORT_PROMPT;
messages: [{ role: "user", content: `Current time: ${now}\n\n${input}` }];
Enter fullscreen mode Exit fullscreen mode

Compaction fights caching, which is a real tension: compact rarely and hard
rather than trimming continuously.

2. Fewer turns. Each turn resends the window, so turn count multiplies
input cost. Better tool descriptions and tools that return what the agent
needs in one call remove turns. A tool that requires three calls to answer one
question costs three turns forever.

3. Model routing by phase. Not "use the cheap model" — route. A small
model for classification and extraction, the capable one for the reasoning
that actually needs it.

const model = phase === "classify" || phase === "extract"
  ? CHEAP_MODEL
  : MAIN_MODEL;
Enter fullscreen mode Exit fullscreen mode

Measure quality per phase before and after. Routing the wrong phase costs more
than it saves, because a bad classification sends the whole run down the wrong
branch.

4. Not calling the model. The cheapest run is the one that never happens.
Cache identical requests; short-circuit intents a lookup can answer.

const hit = await semanticCache.get(input, { threshold: 0.97 });
if (hit) { metrics.increment("cache.hit"); return hit.output; }
Enter fullscreen mode Exit fullscreen mode

A high threshold — 0.97, not 0.85. Serving a near-miss from cache is a quality
bug that looks like a cost win on the dashboard.

Four cost levers ordered by effect, with caching first and avoiding the call<br> last.

Cost is a product decision, so report it that way 

export type UnitEconomics = {
  costPerRun: { p50: number; p95: number };
  costPerResolved: number;
  resolutionRate: number;
  costPerUserPerMonth: number;
  grossMarginPct: number;          // against the plan price
};
Enter fullscreen mode Exit fullscreen mode

costPerUserPerMonth against plan price is the number that decides whether a
flat-rate plan survives. A €20/month plan with a €14 median AI cost and a p95
of €60 has a pricing problem that no amount of prompt tuning will fix.

Ship this before the feature is fully rolled out. Discovering it at 100% is
how features get switched off.

The measurement that pays for itself

Log the four version fields — model, prompt, toolset, index — on every run,
and cost becomes a comparable series. Then a prompt change that saves 30% is
provable, and a provider change that costs 20% more is visible the same week
rather than at the next invoice.

Without those fields you have a monthly bill and a theory.

If this was useful

AI That Ships covers cost as an
engineering discipline — per-run budgets, phase attribution, caching that
survives real prompts, routing, and the unit economics that decide whether the
feature can scale.

AI That Ships — Taking AI Features to Production

The full series is at
xgabriel.com/ai-in-typescript.

Top comments (0)