DEV Community: Sentra

Your embeddings forget exactly like a human brain does

Taran Singhania — Tue, 01 Sep 2026 07:44:16 +0000

If you have built agent memory on a vector store, you have probably watched recall quietly degrade as the store grows, and assumed you needed better embeddings or a bigger index.

We spent several months measuring that decay, and the result was not what we expected: LLM memory systems forget with the same mathematics as human memory, reproducing numbers from some of the most replicated experiments in clinical psychology. No tuning required to get there.

The dimensional lie

Start with the finding everything else follows from. Take an embedding model that advertises 384 or 1,024 dimensions and measure where the variance actually lives:

import numpy as np

# X: (n_samples, n_dims) matrix of embeddings from any pretrained model
X = X - X.mean(axis=0)
eigenvalues = np.linalg.svd(X, compute_uv=False) ** 2
p = eigenvalues / eigenvalues.sum()

# participation ratio: how many dimensions are doing real work
effective_dims = 1.0 / np.sum(p ** 2)
print(f"nominal: {X.shape[1]}, effective: {effective_dims:.1f}")

Run that on a model advertising 384 to 1,024 dimensions and you get an effective dimensionality around 16. Learned representations concentrate their variance into roughly 3 to 10% of their nominal dimensions.

That is not a defect of one model. It is a property of learned representations, and it is why compression works at all. It is also why memory built on those representations behaves like a crowded room rather than a filing cabinet: with 16 effective dimensions, every new memory lands close to existing ones, and closeness is interference.

Forgetting is competition, not decay

The Ebbinghaus forgetting curve is usually taught as memory fading over time. Our measurements say the mechanism is different: memories compete, and competition looks like decay.

The test is simple. Measure the forgetting exponent normally, then remove the competing memories and measure again:

with competitors present     ->  power-law forgetting, exponent ~ human curve
competitors removed          ->  exponent drops ~50x

Fifty-fold. Time barely matters; neighbours matter enormously. Which means the practical lever on agent memory recall is not retention windows or TTLs, it is reducing how many near-identical items compete for the same region of embedding space.

If your retrieval quality fell off a cliff after you tripled the corpus, this is why. You did not lose information, you added competitors.

False memories, for free

The part that genuinely surprised us. The classic false-memory experiment (a lure word that was never presented gets "recalled" because it is semantically central to the list) reproduces on raw cosine similarity over unmodified pre-trained embeddings:

Measurement	Rate
Human false-memory rate, classic studies	~0.55
Raw cosine similarity, no tuning	0.583

Zero parameter fitting. Nobody engineered this. Semantic similarity alone manufactures confident recollections of things that were never stored, at approximately the human rate.

Read that back as an engineering statement: a vector store will hand your agent a plausible fact that was never written down, and it will look exactly like a real retrieval. No confidence score distinguishes them, because from the geometry's point of view there is nothing to distinguish.

What this means if you are building agent memory

Four consequences, in the order they will bite you:

Similarity is not truth. Nearest-neighbour search returns what is close, and closeness is a proxy for relevance that degrades as the corpus grows. It was never a proxy for correctness.
Scaling the index makes recall worse, not better. More documents means more competitors in a 16-dimensional space. This is the opposite of the intuition that a bigger memory is a better memory.
Recency heuristics are patches, not fixes. Sorting by timestamp helps because it breaks ties, not because the system understands that something was superseded.
You cannot fix this with a better embedding model. The concentration of variance is a property of learned representations generally. A model with more nominal dimensions still concentrates them.

The architectural conclusion we drew, and the reason we build what we build: if similarity cannot tell you what is true, the system has to record truth explicitly, at write time, with structure that geometry does not provide.

{
  "statement": "Acme's latency fix slipped to Q3",
  "valid_from": "2026-04-03",
  "valid_to": null,
  "supersedes": "fact_8812",
  "source": "meeting:2026-04-03#turn-58",
  "visible_to": ["role:account-team"]
}

Three fields there do work that no embedding can do. valid_from and valid_to make time explicit rather than inferred. supersedes records that a previous belief was replaced, so the old one can be retired instead of competing forever. source makes the claim checkable.

None of that is a better vector. It is a different data model, and it exists precisely because the geometry has the failure modes above.

Try it on your own store

The participation-ratio snippet above runs on any embedding matrix in about three lines. If your effective dimensionality comes back in the teens while you are paying for 1,024, you now know why your recall curve looks like a psychology textbook.

Full methodology, the compression results behind the 3 to 10% figure, and the rest of the experiments are in the original writeup. If you want the practical version, we wrote up why embeddings alone are not memory and what breaks when retrieval is treated as memory.

This research came out of building Sentra, a company brain for teams and AI agents. We went looking for a compression result and found a psychology paper instead.

Your agent bill is a context problem, not a budget problem

Taran Singhania — Mon, 24 Aug 2026 04:39:32 +0000

Every new primitive eventually becomes a bill. Cloud taught us that with compute, storage, egress and GPU hours. Tokens are next, and the first team to hit the wall in public was Uber: their CTO reportedly said the company had exhausted its AI budget months into 2026, largely from coding-agent usage.

The instinct is to treat this as a budgeting problem. Set limits, build dashboards, make teams justify usage. Some of that is necessary and none of it touches the cause.

Here is the actual cause, and you can compute it yourself.

The arithmetic nobody runs before the invoice

Anthropic's published rates, as of August 2026 (per million tokens):

Model	Input	Output
Claude Fable 5	$10	$50
Claude Opus 5	$5	$25
Claude Sonnet 5	$2	$10
Claude Haiku 4.5	$1	$5

Now a fleet. Five agents, 200 model calls each per day, 20,000 input tokens and 1,500 output tokens per call:

agents, calls_per_day, days = 5, 200, 30
in_tok, out_tok = 20_000, 1_500

requests = agents * calls_per_day * days        # 30,000
input_mtok  = requests * in_tok  / 1_000_000    # 600 MTok
output_mtok = requests * out_tok / 1_000_000    # 45 MTok

# Opus 5: $5 in / $25 out
monthly = input_mtok * 5 + output_mtok * 25      # 3000 + 1125
print(monthly)                                   # 4125.0

$4,125 a month, and 73% of it is input. You are not paying for what the model writes. You are paying for what you keep re-telling it.

That ratio is the whole story. Most cost work targets the wrong 27%.

Four levers, ranked by what they actually do

1. Prompt caching: lowers the price, not the volume

Anthropic bills cache reads at 0.1x the base input rate, with writes at 1.25x for a five-minute window or 2x for one hour. So a stable prefix gets 90% cheaper to resend.

cacheable = 0.40          # share of input that is a stable prefix
cached   = input_mtok * cacheable * 5 * 0.1     # 120 MTok at 0.1x
uncached = input_mtok * (1 - cacheable) * 5     # 480 MTok at full
print(cached + uncached + output_mtok * 25)     #  ~3765.0

Saves about 9% here. Real, free to adopt, and it has one property worth internalising: caching is indifferent to whether the content is true. A policy that changed last month caches exactly as happily as one that changed this morning. It is a price lever, never a correctness lever.

2. Model routing: the biggest single line item

Same fleet, same context, Haiku 4.5 instead of Opus 5:

# Haiku 4.5: $1 in / $5 out
print(input_mtok * 1 + output_mtok * 5)          # 825.0

$4,125 to $825. Most teams resist this longest and it is usually the largest available reduction. Route by difficulty, not by habit.

3. Batch processing: half price for anything asynchronous

Both major providers discount batch APIs by 50% on input and output. If a workload can tolerate latency, this is a config change worth exactly half the bill on that traffic.

4. Send less context: the only lever that compounds

The first three lower the price of the tokens you send. This one lowers how many you need to send, which is the only approach that keeps working as your corpus grows.

Retrieval payloads grow with document count, because more documents match. Compiled facts do not:

// retrieval: 4 documents, ~7,000 tokens, 2 of them contradictory
{ "results": [
  {"source":"gdrive","title":"Acme QBR v3","chunk":"...2400 tokens..."},
  {"source":"gdrive","title":"Acme QBR v2","chunk":"...2200 tokens..."},
  {"source":"slack","channel":"#eng-acme","chunk":"...1800 tokens..."},
  {"source":"crm","record":"Opportunity","chunk":"...900 tokens..."}
]}

// resolved facts: ~200 tokens, with validity and provenance
{ "facts": [
  {"statement":"Acme latency fix slipped to Q3",
   "valid_from":"2026-04-03",
   "supersedes":"fact_8812 (Q2 commitment)",
   "source":"meeting:2026-04-03#turn-58"}
]}

On Terminal-Bench 2.1 we measured this directly: an agent given a task-scoped memory layer used 41.2% fewer tokens at 72.6% lower model cost, while accuracy rose from 83.37% to 88.31% mean reward across 445 trials. Our own evaluation, so read the methodology rather than trusting the number, and note the shape: cost down and accuracy up is what you expect when the mechanism is less-but-better context rather than a cleverer model.

The cost reduction exceeds the token reduction because fewer retries and shorter runs compound with smaller payloads.

What to do this week, in order

Split the bill into four buckets: system prompt, retrieved context, conversation history, output. One day of work, and it redirects everything after it. Most teams find one bucket dominates and it is rarely the one they assumed.
Delete dead weight from the system prompt. It is billed on every request, forever.
Turn on caching for stable prefixes. Move every variable token to the end of the prompt, or your hit rate will be near zero for reasons that look mysterious.
Cap conversation history. Resending a full transcript each turn grows cost quadratically across a long session.
Route easy work to a smaller model.
Then look at whether your agents are re-deriving the same context repeatedly. If they are, that is a knowledge-layer problem and no amount of caching fixes it.

You can model your own numbers with our agent token cost calculator, which applies the published rates and both reduction paths to your fleet shape.

The uncomfortable framing

The phrase going around is tokenmaxxing: maximise usage, burn tokens, trust that value follows. The critique writes itself, token budgets measure input rather than output.

But tokenmaxxing is not stupid. It is the first rational response to genuinely useful AI. If an engineer ships faster with a coding agent, they will run the coding agent, and telling them to run it less is a bad trade dressed as discipline.

The better question is what each token is spent on. An agent that re-reads your repository every session is not doing more work than one that remembers it. It is doing the same work more expensively, and slightly worse, because the context it rebuilds is noisier than the context it could have kept.

Contextmaxxing beats tokenmaxxing. Spend on relevance, not volume.

I work on Sentra, a company brain that resolves cross-system facts once and serves them to agents over MCP, which is the fourth lever above. The longer argument, minus the arithmetic, is in the original essay. If you want every published figure in this category with sources, we maintain a statistics page.

MCP moves context. It doesn't create it.

Taran Singhania — Fri, 21 Aug 2026 15:05:29 +0000

The first time you wire Claude to a real tool and watch it work, it feels like the future showed up early. It reads a file, inspects a database, searches a repo, opens a ticket, and stitches an answer out of systems that used to need ten browser tabs.

Two hours later you ask it something that spans two of those tools, and it confidently tells you something that stopped being true in March.

That gap is not a model problem, and more connectors will not close it. MCP moves context. It does not create context.

What MCP actually solves

The Model Context Protocol is an open standard for connecting AI applications to external systems. The official docs use the analogy of a USB-C port for AI applications, and that is the right mental model: one protocol instead of a pile of bespoke integrations with slightly different auth, schemas and failure modes.

Adding a remote MCP server to Claude Code is one line:

claude mcp add --transport http sentra https://api.sentra.app/mcp/

Or declaratively, for a client that reads a config file:

{
  "mcpServers": {
    "sentra": {
      "url": "https://api.sentra.app/mcp/",
      "headers": {
        "Authorization": "Bearer sk_sentra_YOUR_KEY"
      }
    }
  }
}

That is genuinely infrastructure, and it deserves the hype. Tool access stopped being every vendor's custom problem.

For what I would call level-one questions, this is enough:

"What is the status of ticket ENG-4471?"     -> one system, one lookup
"Summarize the last call with Acme."          -> one transcript
"When does the Acme contract renew?"          -> one CRM field

Where it stops

Now try a question that lives between tools:

"Is Acme at risk of churning, and why?"

There is no system that holds the answer. The pieces are scattered:

Where the evidence lives	What it says
CRM	Renewal in 47 days, stage unchanged for 6 weeks
Gong transcript, 12 Mar	Champion sounded frustrated about latency
Support	Two escalations, second one never followed up
Slack #eng-acme	An engineer explains the real blocker
Roadmap doc	The fix slipped a quarter
Meeting, 3 Apr	An exec promised a date that no longer holds

MCP will happily fetch all six. It will not tell the agent that these six form one evolving risk, that the promised date is stale, or that the support escalation contradicts the CRM stage.

So the agent gets artifacts. It does not get state.

The connector landscape, honestly

If you are choosing how to give agents access to Slack, Jira, email and tickets, these are the real options, and they solve different problems:

Approach	Good at	Structurally cannot
MCP servers	One integration serving many agents; growing ecosystem of community and vendor servers	Decide what any of the data means, or whether it is current
Enterprise search (Glean and similar)	Broad connector coverage, permissions mirrored from each source	Return facts rather than documents; it hands back ranked artifacts
iPaaS (Zapier, Workato, n8n)	Hundreds of prebuilt connectors, fastest path to moving data	Answer questions; these are built to trigger actions, so agents get events, not understanding
Per-agent memory (Mem0, Zep, Letta)	Durable recall for one agent in one application	Share what one agent learned with the other agents in your company
Organizational memory layer	Resolve cross-system facts with time, provenance and permissions	Be lightweight; this is real infrastructure and you feel it

The dividing line that matters: connectors move content, a memory layer resolves it. When two systems disagree, a connector faithfully delivers both answers and leaves the agent to guess.

Query time versus write time

The obvious objection: let the agent figure it out at query time. Give it every tool, let it search, let it reason.

You can get far that way, and Anthropic's own context-engineering guidance describes just-in-time strategies where agents load information at runtime through references. Every serious agent system will use some version of it.

But at query time the agent is reconstructing the world after the fact. On every single request it has to:

1. identify which systems might hold the answer
2. search each one
3. resolve identities  (is "Acme Inc" the same as "Acme Corporation"?)
4. compare timestamps
5. detect which artifacts went stale
6. reconcile contradictions
7. infer permissions
8. and *then* answer the actual question

Steps 1 through 7 are rediscovery. Your agent pays for them in tokens and latency, on every request, forever, and it re-derives them from residue rather than from the moment the change happened.

Because here is the thing: some meaning is only cheap to capture when the work happens.

A customer promise is clearest the moment it is made
A decision reversal is clearest when the meeting ends
A contradiction is clearest when a new commitment conflicts with an old one
A handoff is clearest when ownership actually moves

Write-time memory records the state change as work arrives, with provenance and validity, instead of waiting for an agent to ask the perfect question three weeks later.

The distinction in one line: query time retrieves artifacts, write time captures state transitions. Companies do not run on artifacts. They run on changes.

What that looks like in a response

Query-time retrieval returns documents, and your prompt absorbs all of them:

{
  "results": [
    { "source": "gdrive", "title": "Acme QBR deck v3", "chunk": "...2400 tokens..." },
    { "source": "gdrive", "title": "Acme QBR deck v2", "chunk": "...2200 tokens..." },
    { "source": "slack",  "channel": "#eng-acme",     "chunk": "...1800 tokens..." },
    { "source": "crm",    "record": "Opportunity",     "chunk": "...900 tokens..." }
  ]
}

Four results, roughly 7,000 tokens, two of which contradict each other, and nothing marking which is current.

A write-time layer answers with resolved facts instead:

{
  "facts": [
    {
      "statement": "Acme's latency fix slipped to Q3",
      "valid_from": "2026-04-03",
      "supersedes": "fact_8812 (Q2 commitment, 2026-03-12)",
      "source": "meeting:2026-04-03#turn-58",
      "visible_to": ["role:account-team"]
    },
    {
      "statement": "Acme champion raised latency as a renewal blocker",
      "valid_from": "2026-03-12",
      "source": "call:acme-qbr#turn-22",
      "confidence": "high"
    }
  ]
}

Two facts, a few hundred tokens, each with provenance, a validity window, and an explicit record of what it replaced. The superseded Q2 commitment is not returned alongside its replacement, which is the whole point.

Does this actually pay for itself?

Some numbers from our own evaluation, so treat them as vendor-run and check the methodology rather than taking my word: on Terminal-Bench 2.1, an agent given a task-scoped memory layer reached 88.31% mean reward against an 83.37% published baseline across 445 trials, with 72.6% lower model cost and 41.2% fewer tokens. Per-task trial data is published at our Terminal-Bench writeup.

Accuracy up and cost down at the same time is the signature you expect when the mechanism is less but better context, rather than a smarter model.

The decision rule

Use MCP. It is the right wire, and the ecosystem is going to keep getting better.

Just be clear about what it is: MCP is the finger pointing at the moon. It gets your agent to the systems where work happens. Whether the agent understands what it finds there is a separate layer, and if you skip it, every agent you deploy will rediscover your company from scratch, expensively, and sometimes wrongly.

If your agents answer single-system questions, connectors are enough. The moment two systems can disagree about the same fact, you need something that decides which one is true.

I work on Sentra, where we build that layer as a company brain: cross-system facts with time, provenance and role-scoped access, served to agents over MCP. If you want the longer argument without the code, the original essay is here. If you are weighing connector approaches specifically, we keep an honest comparison of the options above.