Every guide to giving an AI agent memory starts the same way. Pick an embedding model, stand up a vector database, chunk your data, tune your retrieval. I assumed I would do exactly that. Then, before I wrote a single line of it, I looked hard at the problem I was actually trying to solve, and I could not find the part that needed any of it.
This is the reasoning that led me to build a memory API for agents with no vector database, no embeddings, and no model in the loop. I am also going to tell you exactly when that decision is wrong, because it often is, and you should know which side of the line you are on before you copy anyone's architecture, including mine.
The question nobody was asking
Here is what agents actually kept failing at. Not "find me something similar in meaning to this." They failed at "remember the specific thing I was told last session."
Take a coding agent, the case most people reading this have felt. The agent that helped you on Monday has no idea on Tuesday that you already decided to use Postgres, that the deploy step runs through a specific script, that the client's name is spelled a particular way. Every session starts from zero. You re-explain the same context every morning.
That is not a search problem. There is nothing fuzzy about it. You know the exact thing you want back and you know what to call it. It is a key, a value, and the ability to read it later, unchanged.
Vector search answers a different question: "what are the things most similar in meaning to this query." That is a genuinely hard and genuinely useful capability. It is also not what "remember that we chose Postgres for this project" needs. That needs store under a name, get it back by that name, later, reliably.
Two kinds of memory that get blurred together
Once I separated them, the whole design fell out.
There is semantic memory: retrieval over a large body of text where you do not know the exact item you want, only roughly what it is about. "What did the design doc say about rate limits?" You want the relevant passage even if you cannot name it. This is what embeddings and vector databases are for, and they are very good at it.
Then there is named memory: facts and state you can point at. "The user prefers metric units." "This project deploys on Fridays." "The last invoice number was 1043." You are not searching by meaning. You stored something specific and you want it back by name.
A lot of what people call "agent memory" is the second kind wearing the first kind's clothes. The reflex is to reach for semantic search because that is what the tutorials show, when the actual need is a reliable place to put named facts and read them back.
When a vector database is the right call
This is the important part, and it is the reason I can write the rest of this honestly.
If your problem is retrieval over an unstructured corpus, you want embeddings. If an agent needs to answer questions about a hundred documents it has never been told how to index, if recall has to be fuzzy, if "find me the relevant thing" is the whole job, then a vector database is the correct tool and a key-value store is the wrong one. Do not let a simplicity pitch talk you out of the right architecture. If that is your shape, close this tab and go set up your embeddings. You will be happier.
The mistake is not using vector search. The mistake is defaulting to it for a problem that was never semantic to begin with.
What you get back by not adding one
Deciding my problem was named memory, not semantic memory, took an entire category of machinery off the table. No embedding step on every write. No index to tune. No re-embedding when you change models. No debugging why a retrieval that should have been obvious ranked third. No second piece of infrastructure to run and pay for.
What is left is boring in the best way. You name a key, you store a value, you read it back exactly as you left it. It has a TTL if you want facts to expire on their own. You can list what an agent knows and do a literal text search over it. That is close to the entire surface area.
And there is a point about trust hiding in that simplicity. This is memory. It is the thing your agent believes about the world. I would rather that be a store I can inspect completely and reason about with certainty than a similarity ranking I have to interrogate. For this specific job, the boring inspectable version is not a compromise. It is a feature.
The turn: where this beats your own Postgres table
The honest objection at this point is: fine, if it is just keys and values, why not a table in the Postgres I already run? For a single agent, you should. That is not a business, that is a CREATE TABLE.
It changes the moment you have more than one agent that need to read each other's writes. A planner hands off to an executor. A research agent leaves findings for a writer agent. Now you are not storing memory, you are sharing state across processes, and you are suddenly building the boring hard parts yourself: namespacing so agents do not clobber each other, scoping so the right agents see the right memory, a permission model, key management. That is the part worth not writing again. Shared memory across agents, handed to you, is the actual reason to reach for a service here rather than a column in a table you already have.
Durability and TTLs are commodity. Shared state with the access model already solved is not.
What it deliberately is not, and what is next
It does not do semantic search, and it will not pretend to. It does not read your documents and decide what matters. You direct what gets remembered. If you want meaning-based retrieval over a corpus, this is the wrong layer and I will tell you so.
The one thing I know it is missing, and I only know because someone pushed me on it publicly, is a lifecycle state on memories: a way to mark a fact as retired rather than deleted, with a link to what replaced it, so an agent can tell which memories are still load-bearing and which are just history. That is a temporal problem, not a semantic one, which is exactly why it belongs in a store like this without dragging in embeddings. It is the next real thing I am building.
If your agent memory need is genuinely semantic, use a vector database. If it is named facts and shared state, you may have been reaching for far more machinery than the problem asked for. That was the whole realization, and I built the tool I wished I had found instead of the one every tutorial pointed me at.
I would rather be told where this reasoning breaks than agreed with, so if you see the hole, say so.
Top comments (7)
The split between semantic memory (what's similar in meaning) and named memory (what am I storing under this key) is exactly the right cut, and most agent memory discussions conflate them because every tutorial starts with the same embedding setup. A third type worth separating is temporal-structured memory: facts that need a valid_until boundary or a supersedes check — things like "the last invoice number was 1043" where retrieval isn't fuzzy, but it's also not a pure get-by-key because the value may have been updated and you want the current one, not the original. That's still not a semantic search problem, but it's also not a flat KV store — it's closer to an append-only assertion log with a supersedes edge and a current() view. The reason this matters in practice is that agents mixing write discipline (each new fact explicitly supersedes the old one) with named retrieval get the reliability you describe, but without the silent-staleness bug where outdated facts sit next to their replacements and the agent reads whichever one was returned first.
This is the sharpest version of the point anyone has made, and "append-only assertion log with a supersedes edge and a current() view" is close to a spec. You have basically resolved the open question I was sitting on in another thread here: how do you add versioning without the flat store quietly becoming a heavy one. Your answer, that the discipline lives in the write (each fact explicitly supersedes the prior) rather than in a fat value type, is the right place to put it. The store stays conceptually simple, and current() is the only new read semantics you actually need.
The invoice example is the perfect wedge because it is genuinely none of the three easy buckets: not semantic, not a plain get-by-key, and not a full mutable row you overwrite in place (which is exactly how you lose the audit trail). Append plus supersedes keeps the history without turning retrieval fuzzy.
The one place I see a real tradeoff is what current() does when two live assertions exist for the same key and neither supersedes the other, concurrent writers, or a client that forgot the supersedes link. A flat KV store makes that impossible by construction (last write wins, silently), which is worse for correctness but simpler. An assertion log surfaces the conflict, which is more correct but now the agent, or the API, has to have an answer for it. I think that is a good trade, but it is the part that stops it from being free.
You are the third person on this post to independently land on temporal or versioned memory, which has moved it from a someday item to the thing I am designing next. This comment is going straight into those notes. Thank you for the rigor.
The concurrent-write conflict is exactly the right pressure point, and I think current() needs to surface it rather than silently pick a winner. When two live assertions share a key with no supersedes link between them, the store should return both and let the caller decide, or at minimum tag the result as conflicted. The alternative, letting last-write-wins happen implicitly inside current(), recreates the exact flat-KV hazard you were trying to escape, just one layer deeper.
This is the right call, and it is the point that settles it for me. Silent last-write-wins inside current() is the flat-KV hazard reborn one layer deeper, and worse than the original because the user thought they had escaped it. If current() ever has to choose between two live unlinked assertions, it has already failed. The part you have nailed is that you cannot fully prevent this at write time either. Two genuinely concurrent writers can both see nothing to supersede and both commit, and now you have two live assertions with no link. So surfacing the conflict at read is not the fallback, it is required. current() has to be able to say "this key is contested" rather than quietly pick. Returning both, or at minimum flagging the read as conflicted, is where I am landing too. The honest limit to publish alongside it is exactly this: the store guarantees you never silently read a stale fact, and in the rare true-conflict case it tells you instead of guessing. Still design notes, not shipped. But this thread is where the semantics are actually getting decided, so thank you for staying in it.🙏
I like this split between named memory and semantic memory. The part I’d add is versioning: once a named fact becomes shared state between agents, “who wrote this, when, and under which project/session?” starts mattering almost as much as the value itself. Otherwise the key-value store stays simple, but debugging stale memory turns into archaeology.
"Debugging stale memory turns into archaeology" is exactly the failure mode, and you have named the part I underweighted in the post. I framed the gap as lifecycle state (is this fact still live or retired), but you are right that provenance is the other half: once a fact is shared state, who wrote it, when, and under which project or session becomes almost as load-bearing as the value.
You are actually the second person to land on versioning independently, so I am now treating it as the real next thing rather than a nice-to-have. The open question I am sitting with is how much to add without quietly turning the simple store into the thing it was trying not to be. It is a temporal and authorship concern, not a semantic one, so it fits without dragging in embeddings, but there is a real line between useful provenance and a full version history per key.
Genuinely useful push. This is going in the notes for the lifecycle work.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.