LangChain cost audit: what ConversationBufferMemory actually costs you at scale

LangChain is the most-downloaded agent framework on PyPI — 318 million downloads last month
(pypistats.org, 2026-07-21). That means a lot of
production agents are running its memory primitives. Most teams never look at what those primitives
cost per turn.

This is a reproducible audit of the default memory pattern. The numbers are not from a benchmark
lab — they're derived directly from the source code and standard tokenizer math, so you can verify
them yourself.

---

The pattern everyone starts with

from langchain_classic.memory import ConversationBufferMemory
from langchain.chains import ConversationChain

memory = ConversationBufferMemory()
chain = ConversationChain(llm=llm, memory=memory)

chain.predict(input="Summarise our Q3 results")
chain.predict(input="Now break that down by region")
chain.predict(input="Which region underperformed?")

This is the first example in most LangChain tutorials. It works. It also has a cost shape that
compounds silently with every turn.

---

What the source code actually does

ConversationBufferMemory.load_memory_variables (source):

def load_memory_variables(self, inputs: dict[str, Any]) -> dict[str, Any]:
    """Return history buffer."""
    return {self.memory_key: self.buffer}

self.buffer is the entire conversation history as a string — every human turn and every AI
response, concatenated, from turn 1 to turn N. There is no truncation, no summarisation, no
eviction. The docstring says it plainly:

"This stores the entire conversation history in memory without any additional processing."

Every call to chain.predict() prepends this full buffer to the prompt before sending it to the
model. Turn 10 pays to re-read turns 1–9. Turn 50 pays to re-read turns 1–49.

This is not a bug — it's the documented behaviour of an unbounded buffer. The problem is that most
teams don't model the cost shape before they ship it.

---

The cost shape: O(N²) token spend

Let's make the math concrete. Assume:

• System prompt: 500 tokens (typical for an agent with instructions + tool descriptions)
• Average turn: 200 tokens human + 300 tokens AI = 500 tokens added per round
• Model: any frontier model with per-token input pricing

Turn	History tokens re-read	New input tokens	Total input tokens this turn
1	0	700	700
5	2,000	700	2,700
10	4,500	700	5,200
20	9,500	700	10,200
50	24,500	700	25,200
100	49,500	700	50,200

The cumulative input tokens across N turns is:

total_input = Σ(k=1..N) history_at_turn_k + N × new_input
            ≈ (N² × 500 / 2) + N × 700     # new_input (700) = system prompt (500) re-sent + new question (200)
            = O(N²)

At turn 100, you're paying ~50,000 tokens of input per turn — 98.6% of which is history re-read,
not new work. The new question ("Which region underperformed?") is 7 tokens. You're paying for
50,193 tokens to deliver it.

This is the same shape we measured in our own 136M-token runaway: 97.7% of tokens processed were
the agent re-reading its own conversation history (postmortem).
The mechanism is identical — just slower, because there's no timer automating the compounding.

---

"But I'm using ConversationTokenBufferMemory"

LangChain ships a token-limited variant (source):

class ConversationTokenBufferMemory(BaseChatMemory):
    max_token_limit: int = 2000

This caps the history at 2,000 tokens by default — which sounds like a fix. Two problems:

1. The default is often too small for real tasks. A single tool-call result (a retrieved
document, a code block, an API response) can easily exceed 2,000 tokens. The memory silently drops
the oldest context to stay under the limit, which can cause the agent to repeat work it already did
— burning tokens twice.

2. Token counting requires an LLM call. The save_context method calls
self.llm.get_num_tokens_from_messages() to count tokens before deciding what to evict. On every
turn. That's a synchronous model call just to manage memory — adding latency and, depending on your
provider, potentially cost.

Neither variant is wrong. Both have cost shapes that teams should model before deploying at scale.

---

The fix: session discipline, not parameter tuning

The LangChain team already knows this. ConversationBufferMemory has been deprecated since
version 0.3.1 (scheduled removal in 2.0.0) with this migration note:

"For agents that need to remember prior interactions, use create_agent with checkpointing or
the Store API."

The new pattern externalises state — durable memory lives in a store, not in an ever-growing
in-process buffer. Each turn gets only the context it needs, not the full history.

That's the right architectural direction. But the migration isn't automatic, and millions of
production deployments are still on the old pattern.

The three fixes that actually move the needle, in order of impact:

1. Short sessions with external state. Don't grow one conversation for 50+ turns. Persist
durable facts (entities, decisions, task state) to a file or store after each turn. Start a fresh
session with only the relevant context. Continuity lives in the store, not in the buffer.

2. Summarise, don't buffer. ConversationSummaryMemory compresses history into a rolling
summary. The summary grows slowly; the raw transcript doesn't accumulate. You trade some fidelity
for a flat-ish cost curve.

3. Measure before you optimise. You can't fix what you can't see. Before changing anything,
instrument your actual token spend per turn — new input vs. history re-read. If history re-read
is >50% of your input tokens by turn 10, you have the O(N²) problem.

---

How to measure it yourself

If you're running LangChain on top of Claude (via Anthropic's API), the usage data is in the
response object:

with get_openai_callback() as cb:  # or Anthropic equivalent
    result = chain.predict(input=user_input)
    print(f"Turn tokens: {cb.total_tokens}")
    print(f"Prompt tokens: {cb.prompt_tokens}")  # this is the one that compounds

Track prompt_tokens across turns. If it's growing linearly with turn count, you're in the
unbounded buffer pattern.

For Claude Code sessions specifically, tokenscope
(npx @wartzar-bee/tokenscope) reads your session transcripts and shows the new-work vs.
re-read-cold split per session — it's how we got the 97.7% number in the postmortem. It won't
instrument a LangChain app directly, but if you're using Claude Code to build or run your
LangChain agent, it'll show you what the development sessions themselves are costing.

---

The broader pattern

ConversationBufferMemory is one instance of a general failure mode: stateful context that grows
without a cost model. The same shape appears in:

• LangGraph state that accumulates tool outputs without pruning
• AutoGen conversation threads that grow across agent handoffs
• Any retrieval step that appends full documents rather than extracted facts

The fix is always the same: model the cost shape before you ship, not after you see the invoice.

---

What's next in this series

This is the first post in the wartzar-bee cost-audit series. Next up:

• AutoGen: measuring cost-per-task on multi-agent conversations (the handoff tax)
• LangGraph: when stateful graphs become stateful cost traps
• The CI guardrail: catching these patterns before they merge (work in progress)

If you run LangChain in production and want to share your actual token-per-turn data (anonymised),
reach out — real production numbers make better audits.

---

wartzar-bee is an authority in building and operating cost-efficient autonomous agents. We publish
reproducible cost audits of popular OSS agent frameworks.

tokenscope: npmjs.com/package/@wartzar-bee/tokenscope