How Much Can Compact Memory Save in Agentic AI Systems?

Agentic systems need memory.

A customer-service agent must remember what it promised a customer. An NPC must preserve relationships and past events. An enterprise assistant must recover decisions made days or months earlier.

The most direct solution is to send the conversation history back to the language model.

But there is a problem:

The history grows, and the cost of loading it grows with it.

Larger context windows let a model process more information, but they do not make all that information relevant to the next decision.

This problem motivated ASM-CM — Aletheion Compact Memory Model.

ASM-CM investigates a different strategy:

growing local history
        ↓
compact associative memory
        ↓
selective retrieval
        ↓
minimum authorized context
        ↓
local or remote LLM

Instead of sending the entire past, the system retrieves only the memories that are relevant to the current interaction.

This article estimates how much that approach could save.

---

The basic formula

Let:

• H be the number of tokens in the complete history;
• R be the number of tokens retrieved by the memory system;
• C be the number of calls per day;
• P be the price per million input tokens.

The approximate monthly saving is:

saving =
30 × C × (H - R) / 1,000,000 × P

This calculation covers only the input tokens that are no longer sent.

We must still subtract:

• local infrastructure;
• electricity;
• persistent storage;
• operations;
• maintenance of the memory system.

Output-token cost also remains unchanged unless the LLM runs locally.

---

Scenario 1: one person using an agent every day

Imagine one person interacting with an agent 50 times per day.

On average:

Complete history:        20,000 tokens
Retrieved context:        2,000 tokens
Avoided tokens:          18,000 per call
Daily calls:                 50

In one month:

18,000 × 50 × 30
= 27,000,000 avoided tokens

Using current OpenAI API input prices as a reference:

Model	Price per 1M input tokens	Monthly saving
GPT-5.6 Luna	$1.00	$27.00
GPT-5.6 Terra	$2.50	$67.50
GPT-5.6 Sol	$5.00	$135.00

Reference: OpenAI API models and pricing

For one person, the saving is not revolutionary. It becomes more significant when the number of users and agents increases.

---

Scenario 2: a company with 100 users

Now consider:

Users:                          100
Calls per user per day:          50
Total calls per day:          5,000
Average history:             20,000 tokens
Retrieved context:            2,000 tokens

Monthly avoided input:

5,000 × 18,000 × 30
= 2.7 billion tokens

Approximate saving:

Model	Monthly saving
GPT-5.6 Luna	$2,700
GPT-5.6 Terra	$6,750
GPT-5.6 Sol	$13,500

At this scale, memory stops being only a product feature.

It becomes an infrastructure decision.

---

Scenario 3: a platform with long-running agents

Consider a platform processing:

10,000 calls per day
32,000 history tokens per call
 2,000 tokens retrieved by ASM-CM

That is a reduction of 30,000 input tokens per call.

In one month:

10,000 × 30,000 × 30
= 9 billion avoided tokens

Potential saving:

Model	Monthly saving
GPT-5.6 Luna	$9,000
GPT-5.6 Terra	$22,500
GPT-5.6 Sol	$45,000

These figures assume uncached input and do not yet subtract the cost of the local memory infrastructure.

---

What about prompt caching?

This comparison must address an important objection:

LLM providers discount cached input tokens.

For the reference models, cached input currently costs approximately one tenth of regular input:

Model	Regular input	Cached input
Luna	$1.00	$0.10
Terra	$2.50	$0.25
Sol	$5.00	$0.50

Reference: OpenAI model comparison

If every removed history token would otherwise receive a perfect cache hit, the saving would also be approximately ten times smaller.

For the enterprise scenario with 2.7 billion avoided tokens:

Model	Without caching	Ideal cached history
Luna	$2,700	$270
Terra	$6,750	$675
Sol	$13,500	$1,350

Real deployments will usually sit between these extremes.

Static prompt sections may be cached, while new events, tool results, and retrieved memories continue to change.

A serious comparison must therefore measure:

• regular input tokens;
• cached input tokens;
• cache-hit rate;
• cache expiration;
• retrieval quality;
• total cost per interaction.

---

The second saving: KV cache

When the LLM runs locally, another cost becomes important: the Transformer's KV cache.

During generation, a Transformer generally retains keys and values associated with previous tokens.

The exact size depends on the architecture, but we can construct an illustrative example using:

32 layers
8 KV heads
head dimension 128
BF16

The approximate storage per token would be:

2 × 32 × 8 × 128 × 2 bytes
= 131,072 bytes
= 128 KiB per token

At 32K tokens:

128 KiB × 32,768
≈ 4 GiB per stream

If a memory layer reduces the context sent to the LLM from 32K to 2K:

Active context	Estimated KV cache
32K	4 GiB
2K	256 MiB
Reduction	3.75 GiB

This estimate does not apply to every Transformer. Models using grouped-query attention, quantization, or other optimizations will have different numbers.

The underlying property remains relevant:

Reducing active context can also reduce the memory required by the KV cache.

---

What we measured with ASM-CM

In the current experimental protocol, ASM-CM achieved:

• 100% MQAR retrieval at 32K;
• approval across three seeds;
• approximately 140 KiB of retained state per stream;
• approximately 363.66 MiB of peak VRAM for the evaluated component;
• 15 out of 15 memory-pilot cases passed;
• retrieval after as many as 10,000 distractors per character;
• confirmation with three independently trained checkpoints;
• one hour of timed endurance with a real process restart;
• snapshot, restoration, and hash verification.

The scope of these measurements matters:

The approximately 363 MiB figure belongs to the ASM-CM component under the measured protocol.

A complete application also needs memory for:

• the LLM;
• the database;
• the user interface;
• the runtime;
• other services.

---

Do one thousand agents require one thousand models?

Not necessarily.

The model can be shared while each agent retains an independent state.

At approximately 140 KiB per stream:

1,000 × 140 KiB
≈ 137 MiB

Adding the measured shared component:

ASM-CM component:       approximately 366 MiB
One thousand states:    approximately 137 MiB
Approximate total:      approximately 503 MiB

This is a state-storage projection, not a benchmark of one thousand concurrent agents.

We still need to measure:

• concurrency;
• batching;
• latency;
• contention;
• aggregate throughput;
• persistence;
• production resource consumption.

---

Privacy also has economic value

There is another saving that does not appear directly on an API invoice.

If the complete history is repeatedly sent to an external LLM, the organization must manage:

• governance;
• retention;
• contracts;
• auditing;
• data minimization;
• exposure of sensitive information;
• access policies.

A local memory architecture enables a different flow:

private local data
        ↓
ASM-CM
        ↓
associative retrieval
        ↓
authorization
        ↓
minimum context
        ↓
LLM

This does not make the system automatically secure.

It still requires:

• encryption;
• authentication;
• access control;
• tenant isolation;
• auditing;
• verifiable deletion;
• protection against memory poisoning.

But it introduces an important architectural distinction:

The model producing the answer does not need to receive the organization's complete memory.

---

Where should the saving be largest?

ASM-CM should be more economically interesting when:

• the history is long;
• there are many interactions;
• only a small part of the past is relevant;
• every user or agent needs isolated memory;
• data must remain local;
• the external LLM is expensive;
• KV cache limits local inference;
• agents continue running for days or months.

The saving should be smaller when:

• conversations are short;
• there are few calls;
• history receives an almost perfect cache-hit rate;
• the external model is very inexpensive;
• nearly the entire history must be retrieved;
• local infrastructure costs more than the eliminated tokens.

---

The benchmark we still need

To turn this projection into a validated commercial claim, we need to compare:

Complete history
vs.
Prompt caching
vs.
Summarization
vs.
Vector RAG
vs.
ASM-CM

The benchmark must measure:

• regular input tokens;
• cached tokens;
• output tokens;
• answer quality;
• recall;
• latency;
• VRAM;
• energy;
• cost per thousand interactions;
• retrieval failures;
• security and isolation.

Being less expensive is not enough.

The retrieved memory must remain correct.

---

Conclusion

In an illustrative enterprise scenario with:

• 5,000 calls per day;
• an average history of 20K tokens;
• a retrieved context of 2K tokens;

a memory layer such as ASM-CM could avoid approximately 2.7 billion input tokens per month.

Depending on the model and cache utilization, that could represent hundreds or thousands of dollars per month.

At larger platform scale, the potential saving could reach tens of thousands of dollars per month.

But the most important value may not be purely financial.

It is the possibility of building agents that:

• remember for longer;
• retrieve only what is necessary;
• keep private data local;
• disclose less information;
• do not have to reload their entire past before every decision.

Perhaps better agents do not need to remember everything at the same time.

Perhaps they need to remember the right thing at the right moment.

---

ASM-CM is an experimental AletheionAGI project.

We are selecting partners for persistent-memory pilots in agents, games, and
local systems.

Contact: [e-mail]