Cut LLM Costs: Token Budgets, Caching and Cheaper Models

An AI automation that runs once an hour costs almost nothing. Then someone pastes in a long document, a retry loop goes wrong, or the volume triples, and the bill grows without anyone noticing.

The good news is that most of the cost of a language model comes from a few things you can control. This guide explains how you're charged, then builds a small set of tools: a cost calculator, a daily spending limit, a cache for repeated questions, and a way to find the cheapest model that still does the job. It also covers Anthropic's prompt caching, with the numbers worked out.

The code uses only what Python already includes, apart from the optional Claude example.

How you're charged

Language model APIs charge by the token, a small piece of text, roughly a short word or part of one. You pay separately for what you send (input) and what the model writes (output), and the output costs more.

Here are the prices for some of Anthropic's models, per million tokens, as listed on Anthropic's pricing page in October 2026:

Model	Input	Output	Cache read
Claude Haiku 5.5	$0.10	$0.50	$0.01
Claude Haiku 4.5	$1	$5	$0.10
Claude Sonnet 5.5	$2	$10	$0.10
Claude Opus 5.5	$4	$20	$0.20
Claude Fable 5.1	$10	$50	$0.25

Prices change often, so check the pricing page before you rely on these. A few other facts from that page matter:

• Output costs five times as much as input for every model in the table.
• Haiku 5.5 charges more for very long prompts. Prompts over 100,000 tokens cost $0.50 input and $2.50 output per million.
• The Batch API gives 50% off both input and output, for requests that don't need an immediate answer.
• Newer models can use more tokens. Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text, so the same prompt can cost more tokens on a newer model. Count tokens with the model you plan to use.
• Tools add tokens. Tool definitions are sent with every request, and there's a fixed overhead too. On Sonnet 5.5, that overhead is 286 tokens before your own definitions.

Where the money goes

In small automations, the cost usually comes from a handful of places:

• Long instructions repeated on every call. A 6,000-token set of rules, sent 200 times a day.
• Big inputs. A whole document where one section would do.
• Long conversations. Each turn resends the history.
• Chatty outputs. Output tokens cost five times as much as input tokens.
• Loops and retries. An agent that keeps asking for tools, or a retry with no limit.
• The wrong model. A top-tier model doing work a small one handles fine.
• Asking the same question again and again.

Before reaching for a model at all, it's worth asking whether you need one. A rule or a regular expression is free, and it never has a bad day. I compare rule-based and AI-based automation in AI agents: how they work vs RPA.

Six ways to spend less

Lever	What it does	Watch out for
A smaller model	Haiku 5.5 costs 20 times less than Sonnet 5.5 per input token	Quality may drop, so test it
Shorter prompts and outputs	Fewer tokens in and out	Cutting useful context
Prompt caching	Reused instructions cost a fraction of the price	Only pays if the prompt is reused in time
Skipping the call	A saved answer costs nothing	Stale answers
Batching	50% off when an answer can wait	Slower, and caching is best-effort
Limits	A ceiling on what any mistake can cost	Stopping legitimate work

Step 1: Know what you spend

You can't cut what you can't see. Create costs.py. It turns the usage numbers from a model response into dollars, and keeps a running total for the day.

"""Work out what language model calls cost, and stop before a budget is spent."""
import json
from datetime import date
from pathlib import Path

# Dollars per million tokens, from Anthropic's pricing page in October 2026. Prices change, so check the page.
# "cache_read" is the fraction of the input price charged for tokens read from the prompt cache.
PRICES = {
    "claude-sonnet-5-5": {"input": 2.00, "output": 10.00, "cache_read": 0.05},
    "claude-opus-5-5": {"input": 4.00, "output": 20.00, "cache_read": 0.05},
    "claude-haiku-4-5-20251001": {"input": 1.00, "output": 5.00, "cache_read": 0.10},
    # Confirm the exact model ID on the models page. Prompts over 100,000 tokens cost more on this model.
    "claude-haiku-5-5": {"input": 0.10, "output": 0.50, "cache_read": 0.10, "long": {"input": 0.50, "output": 2.50}},
}
CACHE_WRITE = {"5m": 1.25, "1h": 2.0}  # multiples of the input price


def field(thing, name):
    value = thing.get(name) if isinstance(thing, dict) else getattr(thing, name, None)
    return value or 0


def cost_of(model, usage, batch=False):
    """Dollars for one response, worked out from its usage numbers."""
    price = PRICES[model]
    fresh, read, output = field(usage, "input_tokens"), field(usage, "cache_read_input_tokens"), field(usage, "output_tokens")
    written = field(usage, "cache_creation_input_tokens")
    split = usage.get("cache_creation") if isinstance(usage, dict) else getattr(usage, "cache_creation", None)
    written_1h = field(split, "ephemeral_1h_input_tokens") if split else 0
    written_5m = written - written_1h  # without a breakdown, assume the shorter cache

    rates = price["long"] if "long" in price and fresh + read + written > 100_000 else price
    per_token = rates["input"] / 1_000_000
    dollars = (fresh + written_5m * CACHE_WRITE["5m"] + written_1h * CACHE_WRITE["1h"] + read * price["cache_read"]) * per_token
    dollars += output * rates["output"] / 1_000_000
    return dollars * (0.5 if batch else 1)


class BudgetExceeded(Exception):
    pass


class Spend:
    """Keeps a running total for each day in a small file, and refuses new calls once the daily limit is reached."""

    def __init__(self, daily_limit, path="spend.json", today=date.today):
        self.limit, self.path, self.today = daily_limit, Path(path), today

    def _load(self):
        try:
            return json.loads(self.path.read_text())
        except FileNotFoundError:
            return {}
        except ValueError:  # a damaged file must not quietly reset the total to zero
            raise BudgetExceeded(f"{self.path} is damaged, so spending can't be checked")

    def total(self):
        return self._load().get(str(self.today()), {}).get("dollars", 0.0)

    def check(self):
        if self.total() >= self.limit:
            raise BudgetExceeded(f"today's limit of ${self.limit:.2f} has been reached (${self.total():.2f} spent)")

    def record(self, model, usage, batch=False):
        data = self._load()
        day = data.setdefault(str(self.today()), {"dollars": 0.0, "calls": 0})
        day["dollars"] += cost_of(model, usage, batch)
        day["calls"] += 1
        temporary = self.path.with_suffix(".tmp")
        temporary.write_text(json.dumps(data, indent=2))
        temporary.replace(self.path)  # a crash can't leave a half-written file

A few things to notice:

• PRICES is the table from above, in code. Update it when prices change.
• cost_of() reads the usage numbers that come back with every response: fresh input, cache writes, cache reads and output. It follows the documented rules: a 5-minute cache write costs 1.25 times the input price, a 1-hour write costs 2 times, and a cache read costs a fraction of it.
• Spend keeps a daily total in a small file and refuses new calls once your limit is reached. If the file gets damaged, it stops calls instead of quietly starting from zero, because a budget that resets by accident isn't a budget.

To check the calculator against Anthropic's own worked example, save this as example_usage.py:

from costs import cost_of

# Anthropic's own worked example for Claude Opus 5.5: 50,000 input and 15,000 output tokens
print(f"No caching:        ${cost_of('claude-opus-5-5', {'input_tokens': 50_000, 'output_tokens': 15_000}):.3f}")
# Same request, with 40,000 of the input tokens read from the prompt cache
print(f"With cache reads:  ${cost_of('claude-opus-5-5', {'input_tokens': 10_000, 'cache_read_input_tokens': 40_000, 'output_tokens': 15_000}):.3f}")

It prints:

No caching:        $0.500
With cache reads:  $0.348

Those match the figures on Anthropic's pricing page: $0.20 for input and $0.30 for output without caching, and $0.04 plus $0.008 plus $0.30 with 40,000 tokens read from the cache.

Step 2: Put limits in front of every call

Create guarded_model.py. It wraps your model call with four protections.

import hashlib
import json
import time
from pathlib import Path

MAX_INPUT_TOKENS = 20_000   # refuse to send a bigger prompt than this
MAX_OUTPUT_TOKENS = 500     # a hard cap on how much the model may write


def rough_tokens(text):
    """A cautious estimate. For an exact count, use your provider's token counting."""
    return len(text) // 3


class GuardedModel:
    """Wraps a model call with a size limit, a spending limit, and a cache for repeated questions."""

    def __init__(self, send, spend, cache_dir=".reply-cache", max_age=86400):
        self.send, self.spend, self.dir, self.max_age = send, spend, Path(cache_dir), max_age
        self.calls = self.cache_hits = 0

    def _key(self, model, system, prompt, max_tokens):
        return hashlib.sha256(json.dumps([model, system, prompt, max_tokens]).encode()).hexdigest()

    def ask(self, model, system, prompt, max_tokens=MAX_OUTPUT_TOKENS, use_cache=True):
        if rough_tokens(system + prompt) > MAX_INPUT_TOKENS:
            raise ValueError(f"that prompt is over {MAX_INPUT_TOKENS} tokens, so it wasn't sent")
        max_tokens = min(max_tokens, MAX_OUTPUT_TOKENS)
        file = self.dir / (self._key(model, system, prompt, max_tokens) + ".json")
        if use_cache:
            try:
                saved = json.loads(file.read_text())
                if time.time() - saved["saved"] < self.max_age:
                    self.cache_hits += 1
                    return saved["text"]  # the same question as before, so there's nothing to pay for
            except (OSError, ValueError, KeyError):
                pass
        self.spend.check()  # raises BudgetExceeded before anything is sent
        text, usage = self.send(model, system, prompt, max_tokens)
        self.calls += 1
        self.spend.record(model, usage)
        if use_cache:
            self.dir.mkdir(exist_ok=True)
            file.write_text(json.dumps({"saved": time.time(), "text": text}))
        return text

• An input limit. A prompt over about 20,000 tokens is refused before it's sent. The check uses a cautious estimate of one token per three characters. For exact counts, use your provider's token counting.
• An output limit. Whatever max_tokens the caller asks for, the model can never write more than 500 tokens.
• A daily budget. Once the day's limit is reached, new calls are refused before anything is sent.
• A memory for repeated questions. An identical question gets the saved answer for free, even after the budget runs out.

Any change in the model, instructions, question or output limit counts as a new question, so a saved answer is only reused when it really is the same request.

A saved answer is only right if the same answer is still acceptable. That's true for something like "classify this email" and false for "what's the latest on this order?". Saved answers also sit in a folder on disk, so don't use this for private data unless that folder is protected. And pass use_cache=False when you want a fresh answer.

I use the same idea in my own tools. My Repurpose Bot has a per-run limit on how many posts it handles, so a bad day can't produce a surprise bill, and you bring your own AI key, so you can see usage on your provider's page. A limit that works without anyone watching it is the best kind.

Step 3: Use prompt caching

If every call starts with the same long instructions, you're paying full price to send them each time. Prompt caching lets the provider remember them.

Here's a send function for Claude, using Anthropic's Python library (pip install anthropic). It plugs into GuardedModel above:

import anthropic

client = anthropic.Anthropic()  # reads the key from the ANTHROPIC_API_KEY environment variable


def send(model, system, prompt, max_tokens):
    response = client.messages.create(
        model=model, max_tokens=max_tokens,
        system=[{"type": "text", "text": system, "cache_control": {"type": "ephemeral"}}],  # cache the fixed instructions
        messages=[{"role": "user", "content": prompt}])
    return response.content[0].text, response.usage

The cache_control marker on the system block tells Claude to cache everything up to and including it. Anthropic's documentation gives these rules:

• Put the fixed content first, and the marker at the end of it. Caching reads a prefix that is identical across requests. Put the marker on the last block that doesn't change, which here is the system prompt.
• Don't put changing content before the marker. A timestamp or a user's message ahead of the cached part changes the prefix every time, so you pay to write the cache on every request and never get a read.
• Automatic caching suits conversations, not this. There's also a single top-level cache_control setting that caches up to the last block. It's meant for growing multi-turn conversations. With a fixed prefix and a changing final message, it would place the cache point on the part that changes.
• There's a minimum length. For Sonnet 5.5, Opus 5.5 and Haiku 5.5, the prompt must be at least 512 tokens to be cached. For Haiku 4.5 it's 4,096. A shorter prompt is processed without caching, and there's no error.
• The cache lasts 5 minutes, and each use refreshes it at no extra cost. A 1-hour cache is available too, with "ttl": "1h" in the marker, at 2 times the input price to write.
• Check that it works. The usage in each response shows cache_creation_input_tokens (written) and cache_read_input_tokens (read). If both are zero, nothing was cached.

Does it pay off? Here's the arithmetic from cost_of() for a 6,000-token set of instructions on Sonnet 5.5, with a 300-token question and a 200-token answer. Save this as savings.py:

from costs import cost_of

MODEL = "claude-sonnet-5-5"
PREFIX, QUESTION, ANSWER = 6000, 300, 200  # tokens: the fixed instructions, the changing question, the reply


def day(calls, usage_for):
    return sum(cost_of(MODEL, usage_for(i)) for i in range(calls))


no_cache = lambda i: {"input_tokens": PREFIX + QUESTION, "output_tokens": ANSWER}
cached = lambda i: ({"input_tokens": QUESTION, "cache_creation_input_tokens": PREFIX, "output_tokens": ANSWER} if i == 0 else
                    {"input_tokens": QUESTION, "cache_read_input_tokens": PREFIX, "output_tokens": ANSWER})
every_call_writes = lambda i: {"input_tokens": QUESTION, "cache_creation_input_tokens": PREFIX, "output_tokens": ANSWER}
one_hour_cache = lambda i: ({"input_tokens": QUESTION, "cache_creation_input_tokens": PREFIX, "cache_creation": {"ephemeral_1h_input_tokens": PREFIX}, "output_tokens": ANSWER} if i == 0 else
                            {"input_tokens": QUESTION, "cache_read_input_tokens": PREFIX, "output_tokens": ANSWER})

print(f"A {PREFIX}-token set of instructions, reused on every call ({MODEL})")
print(f"  200 calls a day, no caching:                      ${day(200, no_cache):.2f}")
print(f"  200 calls a day, 5-minute cache, calls close together: ${day(200, cached):.2f}")
print(f"  48 calls a day (every 30 minutes), no caching:    ${day(48, no_cache):.2f}")
print(f"  48 calls a day, 5-minute cache that always expires: ${day(48, every_call_writes):.2f}")
print(f"  48 calls a day, 1-hour cache:                     ${day(48, one_hour_cache):.2f}")

It prints:

A 6000-token set of instructions, reused on every call (claude-sonnet-5-5)
  200 calls a day, no caching:                      $2.92
  200 calls a day, 5-minute cache, calls close together: $0.65
  48 calls a day (every 30 minutes), no caching:    $0.70
  48 calls a day, 5-minute cache that always expires: $0.84
  48 calls a day, 1-hour cache:                     $0.18

Read the middle of that output closely. With 200 calls a day, close together, caching cuts the cost by about three quarters. But with a call every 30 minutes, the 5-minute cache has always expired by the next call, so every call pays the higher write price, and caching makes things 20% worse. Switching to the 1-hour cache turns that into a saving of about three quarters.

The rule: caching helps when the next call arrives before the cache expires. Look at how often your calls happen, and pick the cache length to match.

Step 4: Use the cheapest model that passes

The biggest saving is often the simplest: use a smaller model. Haiku 5.5 costs a twentieth of Sonnet 5.5 per input token. The question is whether it's good enough.

You find out by testing, with a set of example cases and a check for each. Below is a small version that compares models. Save it as pick_model.py:

import math


def wilson_low(passed, runs, z=1.96):
    """The low end of a range the true pass rate probably lies in."""
    p = passed / runs
    centre = (p + z * z / (2 * runs)) / (1 + z * z / runs)
    half = z * math.sqrt(p * (1 - p) / runs + z * z / (4 * runs * runs)) / (1 + z * z / runs)
    return max(0.0, centre - half), min(1.0, centre + half)


def pick_model(models, cases, runs=20, minimum=0.85):
    """Find the cheapest model that is clearly good enough.

    models: {model id: ask(prompt) -> (text, usage)}    cases: [(prompt, check)], where check(text) is True for a good answer
    """
    from costs import cost_of

    rows = []
    for model, ask in models.items():
        passed, dollars, total = 0, 0.0, 0
        for prompt, check in cases:
            for _ in range(runs):
                text, usage = ask(prompt)
                passed += bool(check(text))
                dollars += cost_of(model, usage)
                total += 1
        low, high = wilson_low(passed, total)
        rows.append({"model": model, "rate": passed / total, "low": low, "high": high, "cost": dollars / total, "ok": low >= minimum})
    print(f"{'model':<28}{'pass rate':<24}{'cost per call':<15}good enough?")
    for r in rows:
        spread = f"{r['rate']:.0%} ({r['low']:.0%} to {r['high']:.0%})"
        print(f"{r['model']:<28}{spread:<24}${r['cost']:<14.4f}{'yes' if r['ok'] else 'no'}")
    good = [r for r in rows if r["ok"]]
    best = min(good, key=lambda r: r["cost"]) if good else None
    print(f"\nCheapest model that is clearly good enough (at least {minimum:.0%}): {best['model'] if best else 'none of them'}")
    return best["model"] if best else None

It runs each case several times on each model, works out the pass rate with a range, and the cost per call. Then it picks the cheapest model whose pass rate is clearly good enough. A model counts only if the low end of its range is above your minimum, so a lucky streak can't win.

To try it without spending anything, here's a demo with pretend models. Save it as choose.py:

import random

from pick_model import pick_model


def stand_in(slip, seed):
    """A pretend model: it answers correctly most of the time, and slips at a made-up rate."""
    rng = random.Random(seed)
    usage = {"input_tokens": 1200, "output_tokens": 150}  # a typical small request
    return lambda prompt: (("ORDER A-1042" if rng.random() > slip else "sorry, not sure"), usage)


models = {"claude-haiku-5-5": stand_in(0.25, 1),             # the slip rates here are invented for the demo
          "claude-sonnet-5-5": stand_in(0.05, 2),
          "claude-opus-5-5": stand_in(0.02, 3)}
cases = [(f"Which order is customer {n} asking about?", lambda text: "A-1042" in text) for n in range(5)]

pick_model(models, cases, runs=20, minimum=0.85)

The slip rates are invented, so don't read anything into which model wins here. Use real models and real cases for that. The output:

model                       pass rate               cost per call  good enough?
claude-haiku-5-5            76% (67% to 83%)        $0.0002        no
claude-sonnet-5-5           97% (92% to 99%)        $0.0039        yes
claude-opus-5-5             99% (95% to 100%)       $0.0078        yes

Cheapest model that is clearly good enough (at least 85%): claude-sonnet-5-5

The cheapest model fails the test, since its range tops out at 83%. The most expensive passes but costs twice as much as the middle one, which is just as good. The choice is the middle model.

Repeat this when a new model comes out, and when you change your prompt. Prices and quality both move.

More ways to save

Batch what can wait. If nobody is waiting for the answer, such as a nightly report or a backlog of emails to classify, the Batch API costs half as much. It's asynchronous, so results arrive later. Batch and caching discounts can be combined, though cache hits in batches are best-effort because requests run in any order.

Try the cheap model first. Let a small model handle the easy cases and escalate the hard ones to a bigger model. Your test cases show you where the line is.

Trim what you send. Send the relevant section, not the whole document. Summarize old conversation history instead of resending it all.

Ask for short answers. Output costs five times as much as input. A reply that's "one word and a reason" is much cheaper than a paragraph.

Cache your tool definitions. Tool definitions are resent with every request, and they can be cached like the system prompt. Put the cache marker on the last tool.

Limit loops. An agent that calls tools in a loop sends the growing conversation each time. A step limit caps the damage.

Watch the bill

Limits stop disasters, and monitoring shows slow leaks.

• Log the cost of every call, or at least the day's total. Spend already keeps one.
• Look at your provider's usage page now and then. Anthropic's Console tracks usage.
• Get alerted when the day's spending is far above normal, or when the budget is hit. I explain how to set up alerts like this in how to monitor your automations when something breaks.
• Watch the cache numbers. If cache_read_input_tokens is always zero, caching isn't working.

What can go wrong

Caching silently fails. The most common reasons are a prompt below the minimum length, a changing value before the marker, or calls too far apart. The usage fields tell you.

A saved answer is out of date. Use short lifetimes for anything that changes, and skip the cache for anything that must be fresh.

The budget stops real work. A limit that's too low stops legitimate calls. Set it above your normal day, and alert when you're near it.

Two programs share one budget file. Spend is simple and isn't built for many processes writing at once. For that, use your provider's own limits or a proper database.

A cheaper model quietly gets worse. That's why you test before switching, and again after.

The prices in your code go stale. Put the date next to the table, and check it now and then.

The model's tokens aren't your estimate. The estimate in the guard is deliberately rough. Newer models can use about 30% more tokens for the same text.

Hidden multipliers. If you turn on extended reasoning, the extra tokens the model produces add to the output you pay for. Check the usage numbers when you change settings.

Quick answers

How much does a typical call cost?
It depends on the model and the size. In the demo above, a request of 1,200 input tokens and 150 output tokens costs about $0.0002 on Haiku 5.5, $0.004 on Sonnet 5.5 and $0.008 on Opus 5.5.

Is a smaller model always worse?
Not for every task. Classification, extraction and routing are often handled well by small models. Test your own cases.

Does caching change the answers?
No. Anthropic's documentation says prompt caching has no effect on output generation, only on cost and speed.

Is prompt caching the same as saving answers?
No. Prompt caching saves the provider reprocessing your instructions, and you still get a fresh answer each time. Saving answers skips the call entirely.

What about other providers?
The same ideas apply, though prices and caching rules differ. My Bluesky automation bot guide uses Google's Gemini, and the same levers work there: smaller models, shorter prompts, limits and repeat-question caching.

Should I worry about a few dollars?
Not at small scale. The point is to be sure that a bug or a spike can't turn a few dollars into a few hundred.

The short version

Know what each call costs, put a ceiling on what any mistake can cost, reuse what you can, and use the smallest model that passes your tests.

Start with the limits, because they protect you while you work on everything else.

You can find more practical automation guides on Procwire.