Most of what you pay for when an agent works is the same text, sent again.
Every time an agent thinks, the whole prompt goes back to the model: its instructions, its tool definitions, the conversation so far. On a long task that repeated part is the majority of the bill. Providers know this and sell a discount on it called prompt caching — but the discount only applies if the start of the prompt is byte-for-byte identical to last time. Change one character near the top and you pay full price for everything after it.
That sounds like a small implementation detail. It is the single biggest lever on what an agent costs to run, and most agent frameworks lose it by accident.
What breaks the cache
The cache is a prefix match. Anything that varies at the top of the prompt invalidates everything below it. The usual culprits:
- A
current time: 14:32:07line in the system prompt. - A to-do list or task counter that gets rewritten each turn.
- Tool definitions serialised from a hash map, so their order changes between runs.
- Re-stamping old messages with a new timestamp when you replay history.
- Trimming one old message per turn once the context gets long, so the prefix shifts every single turn.
None of these look like a cost decision when you write them. All of them are.
The heartbeat case
The clearest example I know is in OpenClaw's own issue tracker: an open report that heartbeats put the current time into the cached start of the prompt, so each heartbeat pays to write the cache again instead of reading it. A heartbeat is supposed to be the cheapest thing an agent does — wake up, check nothing needs attention, go back to sleep. If the clock sits in the cached prefix, every one of those wake-ups is a full-price prompt write.
The fix is not clever. It is discipline about what goes at the top.
Four rules that keep the prefix stable
1. The system prompt is byte-identical every turn. Persona, instructions, skills — no clock, no counters, no "you have 3 tasks remaining". If the agent needs to know the time, it calls a tool and the answer lands in the conversation, below the cached part.
2. History only grows at the end. Old messages are replayed exactly as they were, stamped with the time they arrived, never re-stamped. Appending is free; rewriting is not.
3. Summarise once, don't trim every turn. When a conversation gets long, compact it once and then let it grow at the end again. Dropping one old message per turn means the prefix changes on every call, which means you never get a cache hit at all.
4. Waiting costs nothing. An agent blocked on a colleague, a timer or a build should make zero model calls until something arrives. This is the one people miss, because a polling loop feels harmless — it is a full prompt every few seconds.
What it's worth
Take an agent whose instructions, tools and history add up to 30,000 tokens, making 20 model calls on a task.
Sent uncached, that is 600,000 input tokens at full price.
At typical cached-input pricing, where a cache read costs roughly a tenth of normal input, the first call writes the cache and the other 19 read it. That is about 85% less on the repeated part — and the repeated part is most of the bill.
The same arithmetic applies to specialists. A sub-agent that starts with its own short brief instead of inheriting the parent's whole conversation is cheaper on every call it makes, and it can run on a smaller model without losing anything, because it only has one job.
The uncomfortable part
Caching is also why the subscription question keeps coming up. In April 2026 Anthropic stopped letting Claude Pro and Max subscriptions cover third-party agent tools, citing its terms of service and load on its systems; in May it let them back in on a separate monthly credit. If you are running an agent on a flat-rate subscription, your cost model depends on a policy someone else can change, and the caching behaviour underneath it is not yours to control.
If you are building this yourself, the four rules above are most of the work, and they are worth more than any model upgrade.
I work on AgentParley, a hosted platform for crews of AI agents. We build every prompt so the repeated part stays byte-identical, which is where the numbers above come from. If you want the longer version, including how it compares to self-hosting, there is a full comparison here.
Top comments (0)