Spring AI's defaults are built for a fast start; they do not guarantee a low monthly cost. Shipping an LLM feature is easy — making it cost-efficient is not. This series shows the spots where money leaks, along with the control that closes each one.
Spring AI 2.0 reached GA on 12 June 2026. It needs Spring Boot 4, moves the tool-calling loop out of the ChatModel, adds tool search, and extends structured outputs. Tool search and the extended structured-output controls point in the same direction: they determine how many tokens your application sends and receives.
The bill grows quietly. A chatbot with a 2,000-token system prompt, run 100,000 times a month, sends 200 million tokens of the same text. At an example rate of $1 per million input tokens, that is $200 a month — before a single user message. Then add conversation history, which is sent in full on every turn. Add retrieved RAG documents and the JSON schema of every registered tool. The input side can grow 10× with no change in traffic at all. Output tokens cost several times more per token than input tokens, and reasoning models bill their hidden "thinking" as output too.
The provider sets the prices. The framework gives you controls that can reduce the number of tokens you pay for.
This series works through ten cost drivers, numbered #0 to #9. Each one is a place where tokens repeat or grow without anyone deciding they should, and each comes with the Spring AI control that cuts it. They are spread across four parts. Part 1 is live. Parts 2 to 4 follow in August 2026.
Part 1 — Token Usage: Measure Cost Before You Pick a Model (Drivers #0–#2)
Provider dashboards show what you spent, but not which feature spent it. Spring AI's observability closes that gap, and from there you can match each model to its task and stop features from carrying defaults they never needed.
Part 2 — Prompt Caching and Chat Memory: Where the Tokens Go (Drivers #3–#5)
This part covers limiting response length, bounding how much conversation history you resend, and structuring prompts so that provider caching actually works.
Part 3 — RAG and Tool Calling: Paying for Context You Don't Use (Drivers #6–#7)
Retrieved documents and tool definitions are added to every request, whether the model needs them or not. This part covers retrieving less, cleaning up what was retrieved, and sending tool schemas only when they are relevant.
Part 4 — Retries and Embeddings: Failed Answers and Full Re-indexes (Drivers #8–#9)
A failed response is billed in full, and retrying it resends the entire context. This part covers preventing malformed output at the provider level, then the indexing side: embedding model choice, vector size, batching, and re-embedding only what has changed.
Start with Part 1. Every later driver asks you to trade something — context, memory, response length — and per-feature metrics are what tell you which trade is worth making. Part 1 also covers the cheapest change available: sending routine work to a smaller model, which usually needs no more than a property change.
Top comments (0)