DEV Community

Cover image for The most expensive part of my agent stack wasn’t tokens — it was the invoice roulette
Lars Winstand
Lars Winstand

Posted on Originally published at standardcompute.com

The most expensive part of my agent stack wasn’t tokens — it was the invoice roulette

I hit a weird point with my agent stack a few months ago.

Not when prompts were bad.
Not when tools were failing.
Not when n8n was half-broken.

It happened after everything started working.

The flows were stable. The AI Agent node in n8n was calling tools correctly. OpenClaw was keeping assistants alive across Slack and Discord. Background jobs were running. Users were happy.

Then I opened the invoice.

And I had that very specific engineering feeling: I understood every moving part individually, but I still could not predict the total cost.

That was the moment I stopped thinking about AI cost as “price per token” and started thinking about billing volatility as an architecture problem.

If you run agents 24/7, especially across n8n, Make, Zapier, OpenClaw, or custom workflows, this matters more than most pricing pages admit.

The trap: tokens are not the real unit of cost

At prototype stage, token math feels clean.

You estimate:

  • average prompt size
  • average response size
  • expected request volume
  • maybe a little buffer

That works for chat demos.

It breaks for automations.

An agent workflow is not one request. It is a chain reaction.

A single inbound event can trigger:

  • one LLM call to interpret the task
  • one or more tool calls
  • another LLM call to summarize tool output
  • retries after a timeout
  • fallback to another provider after a rate limit
  • an error workflow that replays part of the execution

What looked like “1 event = 1 model call” turns into “1 event = 7 billable things plus side effects.”

That is why production cost gets weird fast.

A simple example from n8n

Here is the kind of flow that looks cheap on a whiteboard and expensive in production:

  1. Webhook receives a support ticket
  2. n8n AI Agent classifies urgency
  3. Agent calls CRM tool
  4. Agent calls search or knowledge base tool
  5. Agent drafts a reply
  6. Timeout happens on step 3
  7. Execution retries
  8. Draft step runs again

Now the pricing problem is not prompt length.

It is workflow behavior.

If you are tracking retries in n8n, you already know this pattern exists. Retry metadata like execution.retryOf is a reminder that retries are normal in automation, not rare edge cases.

The bill changes even when prompts don’t

This is the part that made me stop trusting per-token forecasts.

People describe usage-based pricing as transparent. In practice, for agent systems, it often isn’t.

Why?

Because your bill is shaped by more than token volume:

  • request bursts
  • rate-limit backoffs
  • queueing
  • fallback chains
  • cache hit rate
  • background jobs
  • sync vs batch routing
  • tool fan-out

Two months can have roughly the same user demand and still produce different costs.

Not because prompts changed.
Because execution shape changed.

Maybe traffic got burstier.
Maybe a nightly summarization job collided with interactive traffic.
Maybe one provider throttled and your fallback path activated more often.
Maybe your cache hit rate dropped because prompt prefixes drifted.

That is invoice roulette.

Background jobs are where pricing models go to die

The least honest part of many AI cost estimates is that they ignore background work.

But real agent systems are full of it:

  • nightly classifiers
  • memory refresh jobs
  • ticket summarization
  • thread cleanup
  • webhook-triggered enrichments
  • multi-channel assistants sitting idle-but-not-really-idle

An OpenClaw assistant connected to Slack, Discord, Telegram, WhatsApp, and Teams may look quiet from the outside.

It is still maintaining context, reacting to events, preserving sessions, and sometimes running scheduled tasks.

That means cost keeps accumulating even when no human is actively chatting.

The vendor pricing features are useful — and also kind of a confession

This was the part that surprised me most.

Every major model vendor now has pricing features designed to reduce cost:

  • batch APIs
  • prompt caching
  • context caching
  • separate rate-limit pools
  • grounding quotas

These features are real and useful.

They also quietly admit the same thing: raw synchronous per-request billing is a bad fit for a lot of production automation.

OpenAI Batch: good feature, loud signal

OpenAI Batch is a solid option for offline work.

What you get:

  • 50% lower cost than synchronous API usage
  • separate, higher-rate-limit capacity
  • completion within 24 hours

That is great for:

  • bulk classification
  • enrichment
  • nightly summarization
  • offline evaluation jobs

Example shape:

# pseudo-workflow
# 1. collect jobs during the day
# 2. ship them to batch overnight
# 3. read results later
Enter fullscreen mode Exit fullscreen mode

The pricing win is real.

But the architectural implication matters more: now cost depends on execution mode, not just token count.

You are no longer asking “how many tokens did I send?”
You are asking “which jobs can tolerate delay, and did I route them correctly?”

That is a workflow design problem.

Anthropic prompt caching: powerful, but easy to overestimate

Anthropic caching looks fantastic on paper.

Typical pricing structure for Claude tiers includes:

  • base input pricing
  • separate cache write pricing
  • much cheaper cache hit pricing
  • output pricing

That can be a huge win if your prompt prefixes are stable and requests arrive inside the cache window.

Example:

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-4-1",
    max_tokens=1024,
    system="You are a support triage assistant.",
    messages=[
        {"role": "user", "content": "Classify this ticket and suggest next action."}
    ],
    cache_control={"type": "ephemeral"},
)
Enter fullscreen mode Exit fullscreen mode

The problem is forecasting.

Your savings now depend on:

  • whether prefixes actually stay stable
  • whether requests land inside the cache lifetime
  • whether your agent architecture reuses context consistently
  • whether tool outputs keep mutating the prompt shape

Teams often assume caching will save a lot, then discover their traffic pattern is too messy.

For always-on agents, this is one of the easiest optimizations to model optimistically and realize pessimistically.

Google Gemini Batch and caching: same story, more variables

Google Gemini has the same pattern.

There are real discounts for batch execution. There is context caching. There can also be extra costs around storage duration and grounding.

Example batch call shape:

from google import genai

client = genai.Client()

job = client.batches.create(
    model="gemini-2.5-flash",
    src=[
        {
            "contents": [
                {
                    "role": "user",
                    "parts": [{"text": "Summarize this ticket in one sentence."}]
                }
            ]
        }
    ],
    config={"display_name": "ticket-summary-batch"},
)
Enter fullscreen mode Exit fullscreen mode

Again, nothing is wrong with this.

But now your cost model depends on:

  • sync vs batch routing
  • cache duration
  • storage time
  • grounding frequency
  • request shape over time

That is not simple token accounting anymore.

Which cost-saving feature is actually worth using?

My opinion, after dealing with this in production:

Option Best use case
OpenAI Batch Best for boring offline jobs like nightly classification, summarization, and enrichment
Anthropic prompt caching Best when prompts are highly stable and traffic repeatedly hits the same prefixes
Google Gemini Batch + caching Best when you can cleanly separate async work and actually manage caching/grounding behavior

My practical ranking:

  1. Batch APIs are the easiest clear win
  2. Prompt caching is useful but easier to overestimate
  3. Complex multi-provider fallback systems create the worst forecasting problems

If you run an always-on assistant across Slack, Discord, Telegram, Teams, or WhatsApp, with memory, tools, webhooks, and scheduled jobs, the hardest part is not finding a low token price.

It is explaining next month’s bill before next month happens.

The real problem is invoice volatility

This is the shift that changed how I think about AI infrastructure.

The expensive part is not always the model.
Sometimes it is the unpredictability.

That matters because unpredictable spend changes engineering behavior.

Teams start doing weird things when they cannot trust the bill:

  • throttling useful features too early
  • avoiding background automation that would actually help users
  • over-optimizing prompts instead of fixing workflow design
  • delaying launches because finance wants tighter cost bounds

That is why predictable pricing becomes attractive long before raw per-token pricing becomes objectively expensive.

What I do now instead of naive token forecasting

I still care about model pricing.
I just do forecasting differently.

I model execution paths, not average prompts.

That means I count:

  • retries per workflow
  • average tool fan-out
  • fallback frequency
  • sync vs batch split
  • cache hit assumptions
  • background job frequency
  • burst behavior under load

A rough spreadsheet is still useful, but only if it reflects system behavior.

A better way to think about cost

Bad forecast:

monthly_cost = avg_prompt_tokens * avg_response_tokens * requests * token_price
Enter fullscreen mode Exit fullscreen mode

Better forecast:

monthly_cost =
  interactive_requests * avg_interactive_execution_path
+ batch_requests * avg_batch_execution_path
+ background_jobs * avg_background_execution_path
+ retry_overhead
+ fallback_overhead
+ cache_miss_penalty
Enter fullscreen mode Exit fullscreen mode

That is much closer to reality for agent systems.

Observability matters more when pricing gets harder to reason about

If your workflows are complex, tracing becomes non-negotiable.

For example, if you are debugging model behavior with LangSmith and OpenAI-compatible tooling:

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="<your-langsmith-api-key>"
export OPENAI_API_KEY="<your-openai-api-key>"
Enter fullscreen mode Exit fullscreen mode

That will not reduce the bill by itself.

But it helps answer the question that always shows up too late:

Why did this workflow call the model four times?

Without tracing, cost debugging turns into archaeology.

When usage-based pricing is still totally fine

I do not think every team should abandon per-token billing.

It is still a good fit when:

  1. Traffic is low-volume
  2. Workflows are simple
  3. Prompts are stable
  4. Retries are rare
  5. Background jobs are limited
  6. You can actually use batch or caching reliably

In that world, usage-based pricing can absolutely be cheaper.

But once agents run continuously, touch tools, operate across channels, and keep doing work while you sleep, predictability starts to matter more than benchmark token price.

Where flat-rate compute starts making more sense

This is exactly why products like Standard Compute exist.

If your stack already speaks the OpenAI API, swapping endpoints is much easier than rebuilding your workflows around five different pricing tricks.

The appeal is simple:

  • flat monthly pricing
  • no per-token billing
  • works with OpenAI-compatible SDKs and HTTP clients
  • better fit for always-on agents and automations
  • less time spent playing pricing Tetris across GPT, Claude, and Grok

That does not remove the need for good architecture.

You still need:

  • tracing
  • guardrails
  • sane retry policies
  • separation between real-time and batch work
  • good tool design

But it removes one category of chaos: surprise invoices caused by workflow behavior you did not model perfectly.

For teams running n8n, Make, Zapier, OpenClaw, or custom agents, that tradeoff is often worth more than squeezing out the cheapest theoretical token path.

My takeaway

The clean mental model is wrong.

AI automation cost is not just model quality multiplied by token count.

It is the sum of:

  • retries
  • rate limits
  • fallback chains
  • cache windows
  • batch queues
  • grounding requests
  • tool loops
  • background jobs
  • all the tiny workflow decisions that compound at scale

If your biggest monthly question is no longer “which model is cheapest per token?” but “why can’t I predict this invoice at all?”

That is not a finance problem.

That is architecture.

And once you see it that way, a lot of pricing decisions start looking very different.

Top comments (0)