Spring AI Token Usage: Measure Cost Before You Pick a Model — LLM Cost Control 1/4

Julia Denysova — Thu, 30 Jul 2026 15:26:31 +0000

Cutting LLM costs in Spring AI starts with two choices: which model answers a request, and what defaults your ChatClient adds to every one it sends.

Neither is worth changing until you can see where the tokens go. That is why this article starts with measurement.

This is Part 1 of four, and it covers the first three of ten cost drivers. Driver #0 tells you where the money actually goes; #1 and #2 are the two decisions that shape every request your application sends. The remaining seven attach to what you build here.

A note on the numbers: where a price ratio matters for the argument (input vs. output, cache read vs. write), this series quotes real July 2026 list prices with a link. All other examples use a flat rate of $1 per million input tokens, so you can redo the calculation with your own provider's price sheet. You should do that, because these prices change every few months.

Driver #0 — Spring AI observability measurement: you cannot cut what you cannot see

Provider invoices and usage dashboards usually show your spending by model and by token type — input, output, and cached. That is useful, but it is not enough. The numbers cannot tell you which feature, client, or advisor inside your application was responsible for that usage.

Spring AI integrates with Spring Boot's Micrometer-based observability to fill this gap. Its core AI components automatically emit that data. ChatModel, EmbeddingModel, and ImageModel implementations (support varies by provider) publish model-level observations, including token usage where available. ChatClient (including advisors) and VectorStore report execution observations and traces rather than token usage metrics.

Each metric includes built-in tags, such as the model name and token type. These tags separate models and providers, but not callers: every request to the same model carries the same tag values, so they cannot tell two features apart on their own. Spring AI marks tags as low- or high-cardinality: low-cardinality tags go on both metrics and traces, high-cardinality tags go on traces only. To attribute usage to a specific feature or client, add your own low-cardinality tags. Keep those values few and stable, not one per user or request. For more on how tags work, and how to keep their number under control, see Micrometer's documentation on tags and naming.

With this data in place, you can answer the questions the later drivers raise. Is history growth (Part 2) really increasing costs? Watch prompt token usage over time. Did the tool-search advisor (Part 3) reduce the amount of context sent to the model? Compare prompt token usage before and after enabling it. Is a reasoning model becoming more expensive than expected (Part 2)? Output token trends can reveal unexpectedly long responses and rising generation costs.

Two more steps make this setup safer, not just more informative. First, create an alert for sudden increases in token usage or request volume. A bug that puts an LLM call inside a loop can burn through your budget fast, and an alert catches it within minutes instead of at the end of the month. Second, configure provider-side spending controls where available, such as budgets, quotas, or usage limits. This is a final safeguard that no application framework can enforce.

Treat observability as step zero. Set it up before optimising anything else, so every later change has a measurable before-and-after comparison.

Driver #1 — Model choice: stop paying flagship prices for mini-model work

The fastest way to overspend on an LLM application is to send every request to the most capable model you can find. Flagship and budget models on the same provider's price sheet can differ tenfold in price per token, or more. But a large share of typical workloads — classification, extraction, routing, short summaries — passes the quality bar on the smaller model already. Sending those requests to a flagship model increases cost without improving the outcome.

Price is only one factor, though. Models also differ in what they can do: some accept images or audio, some are built for tool calling or strict structured output, some reason step by step while others just answer directly. The rule is the same either way: pick the smallest model that covers what the task actually needs. Do not pay for image support on a text-only pipeline, and do not pay for a reasoning model just to reformat JSON. Requirements change over time, so treat your model choice as temporary, not final. Providers release cheaper and better models every few months, and a task that needed the flagship model yesterday often works fine on the mini model next quarter.

This only pays off if switching models is easy. Spring AI's answer is that the model is configuration, not architecture. ChatClient does not depend on any one provider or model: your service code does not need to know which model answers, portable options like maxTokens and temperature carry over between models, and provider-specific settings stay isolated in one options object. Spring AI 2.0 makes this explicit: default options set on a ChatClient act as a partial "delta" over the model's start-up configuration. The framework signals this same idea in its own defaults: the OpenAI integration in 2.0 defaults to gpt-5-mini, not the flagship model. A good pattern is one client per cost tier:

@Configuration
public class ChatClients {

    @Bean
    @Primary
    ChatClient cheapClient(ChatClient.Builder builder) {
        return builder
              .defaultOptions(ChatOptions.builder()
                  .model("gpt-5-mini"))
              .build();
    }

    @Bean
    ChatClient flagshipClient(ChatClient.Builder builder) {
        return builder
              .defaultOptions(ChatOptions.builder()
                  .model("gpt-5"))
              .build();
    }
}

A ticket classifier feature could inject @Qualifier("cheapClient") ChatClient, while a contract-analysis feature injects the flagship one. Two details matter here. First, build clients from the auto-configured builder, not from a raw ChatClient.builder(chatModel). The auto-configured builder already has observability wired in, and the token metrics from Driver #0 depend on that. Second, this example covers two tiers on one provider. If you mix vendors in the same application, create a separate ChatClient from each auto-configured ChatModel instead. Either way, the size of the change stays small: swapping models is a property change, and swapping vendors just adds a starter dependency plus properties. No code changes, no prompt rewrites, no new SDK to learn.

Driver #2 — One shared client: every request carries every default

A ChatClient default is not free. Everything attached to the client — system prompt, tools, memory advisor — is included in every request that client sends. One shared "do-everything" client means a simple 10-token call (e.g. a classifier feature) carries the same 2,000-token system prompt, tool schemas, and conversation history as your most complex feature. If the classifier runs 50,000 times a month, that is 100 million input tokens of extra content it never actually needs — $100 a month at the example rate, for nothing.

Spring AI's unit of cost control is the client, not the whole application. ChatClient.Builder is auto-configured as a prototype bean, so each task can build its own client with only the defaults that task needs: .defaultSystem() for the prompt, .defaultOptions() for model and token settings, .defaultAdvisors() and .defaultTools() for the rest. Every later driver in this series — memory windows, RAG advisors, tool search — is attached at exactly this level. This is why building per-task clients should come before the drivers that follow.

@Configuration
class TaskClients {

    @Bean
    ChatClient classifierClient(ChatClient.Builder builder) {
        return builder
                .defaultSystem("Classify the ticket. Reply with one word: BILLING, TECH, or OTHER.")
                .defaultOptions(ChatOptions.builder()
                        .model("gpt-5-mini")
                        .maxTokens(5)
                        .temperature(0.0))
                .build();
    }

    @Bean
    ChatClient supportChatClient(ChatClient.Builder builder, ChatMemory memory) {
        return builder
                .defaultSystem(new ClassPathResource("prompts/support.st"))
                .defaultOptions(ChatOptions.builder()
                        .model("gpt-5")
                        .maxTokens(800))
                .defaultAdvisors(MessageChatMemoryAdvisor.builder(memory).build())
                .build();
    }
}

Note the Spring AI 2.0 change: .defaultOptions() and the per-call .options() now take a ChatOptions.Builder, not a fully built instance. This builder is merged with the model's defaults before the first advisor runs, and only the fields you explicitly set override them. That merge lets you override only the options you need without replacing the model's configured defaults:

classifierClient.prompt(text)
    .options(ChatOptions.builder().maxTokens(1)) // override for a yes/no case
    .call()
    .content();

The result: each request pays only for the defaults its own task actually declared. That is a requirement for every control that follows.

What's next

Drivers #1 and #2 decide which model answers a request and which client sends it; Driver #0 shows you what those decisions cost. You now have one client per task, each carrying only the defaults that task needs, and the token metrics to prove it.

None of that limits the size of a single request. A reasoning model can bill more hidden thinking than visible answer. A support conversation resends its whole history on every turn. The same system prompt goes out thousands of times a day at full price, unless you structure it so the provider can cache it.

Part 2 covers those three. Each one is a limit or an advisor you attach to the clients you have just built.

Part 2 arrives in August 2026.

How to Reduce LLM Costs in Spring AI 2.0: 10 Practical Controls

Julia Denysova — Thu, 30 Jul 2026 15:26:02 +0000

Spring AI's defaults are built for a fast start; they do not guarantee a low monthly cost. Shipping an LLM feature is easy — making it cost-efficient is not. This series shows the spots where money leaks, along with the control that closes each one.

Spring AI 2.0 reached GA on 12 June 2026. It needs Spring Boot 4, moves the tool-calling loop out of the ChatModel, adds tool search, and extends structured outputs. Tool search and the extended structured-output controls point in the same direction: they determine how many tokens your application sends and receives.

The bill grows quietly. A chatbot with a 2,000-token system prompt, run 100,000 times a month, sends 200 million tokens of the same text. At an example rate of $1 per million input tokens, that is $200 a month — before a single user message. Then add conversation history, which is sent in full on every turn. Add retrieved RAG documents and the JSON schema of every registered tool. The input side can grow 10× with no change in traffic at all. Output tokens cost several times more per token than input tokens, and reasoning models bill their hidden "thinking" as output too.

The provider sets the prices. The framework gives you controls that can reduce the number of tokens you pay for.

This series works through ten cost drivers, numbered #0 to #9. Each one is a place where tokens repeat or grow without anyone deciding they should, and each comes with the Spring AI control that cuts it. They are spread across four parts. Part 1 is live. Parts 2 to 4 follow in August 2026.

Part 1 — Token Usage: Measure Cost Before You Pick a Model (Drivers #0–#2)

Provider dashboards show what you spent, but not which feature spent it. Spring AI's observability closes that gap, and from there you can match each model to its task and stop features from carrying defaults they never needed.

Part 2 — Prompt Caching and Chat Memory: Where the Tokens Go (Drivers #3–#5)

This part covers limiting response length, bounding how much conversation history you resend, and structuring prompts so that provider caching actually works.

Part 3 — RAG and Tool Calling: Paying for Context You Don't Use (Drivers #6–#7)

Retrieved documents and tool definitions are added to every request, whether the model needs them or not. This part covers retrieving less, cleaning up what was retrieved, and sending tool schemas only when they are relevant.

Part 4 — Retries and Embeddings: Failed Answers and Full Re-indexes (Drivers #8–#9)

A failed response is billed in full, and retrying it resends the entire context. This part covers preventing malformed output at the provider level, then the indexing side: embedding model choice, vector size, batching, and re-embedding only what has changed.

Start with Part 1. Every later driver asks you to trade something — context, memory, response length — and per-feature metrics are what tell you which trade is worth making. Part 1 also covers the cheapest change available: sending routine work to a smaller model, which usually needs no more than a property change.

DEV Community: Julia Denysova