Always-on memory agent vs RAG: when to drop your vector database (2026)
Summary. In mid-July 2026, Shubham Saboo, a senior AI product manager at Google, published an open-source project called the Always-On Memory Agent on Google's official GitHub under a permissive MIT license. It stores structured memories in SQLite, re-reads and consolidates them every 30 minutes, and answers questions by reading that memory directly. The repository's pitch is blunt: "No vector database. No embeddings. Just an LLM that reads, thinks, and writes structured memory." It runs on Gemini 3.1 Flash-Lite, the model Google shipped on March 3, 2026 at $0.25 per 1 million input tokens and $1.50 per 1 million output tokens. That price is what makes an always-running memory loop affordable at all. For teams paying $50 to $700 a month for a managed vector database like Pinecone, the design raises a fair question: can you delete the retrieval stack? Sometimes. This piece gives the cost math, the point where the approach breaks, and a decision rule you can apply this quarter.
The short answer up front: the no-vector-database design is a strong fit for bounded, low-to-medium memory agents where the whole store fits comfortably in a model's context. It stops being cheap, and stops being fast, once the memory grows past a few hundred thousand tokens, because both the background consolidation and every query re-read the store. RAG with a vector index inverts that trade. It costs more to stand up and operate, and it stays roughly flat as the corpus grows. Choose by the size and growth rate of what the agent has to remember, not by which architecture reads better on social media.
What Google actually shipped
The Always-On Memory Agent is a reference implementation, not a product. It was built with Google's Agent Development Kit (ADK), the agent framework Google introduced in spring 2025, and it runs on Gemini 3.1 Flash-Lite. According to the repository and reporting by VentureBeat, the system runs continuously, ingests files or API input, and keeps its state in a plain SQLite database. It handles text, images, audio, video and PDF input, and it ships with a local HTTP API and a Streamlit dashboard so you can watch what it stores.
Inside, three specialist subagents split the work:
An ingest agent uses Gemini's multimodal input to turn each new item into a structured record: a short summary, the entities and topics it mentions, and an importance score. That record lands in a memories table.
A consolidation agent runs on a timer, every 30 minutes by default. It reviews memories that have not yet been consolidated, looks for connections between them, and writes back a synthesized summary, one key insight, and the links it found. Saboo compares this to sleep: the agent builds new understanding while idle, with no prompt driving it.
A query agent answers questions. It reads the memories and the consolidation insights, synthesizes a response, and cites the memory IDs it used as sources, which gives you a basic audit trail for where an answer came from.
Shubham Saboo, a senior AI product manager at Google, frames the design in the repository with a deliberately provocative line: "No vector database. No embeddings. Just an LLM that reads, thinks, and writes structured memory." The point is not that retrieval disappears. The point is that the model does the organizing and the recall, instead of an embedding pipeline and a similarity index.
VentureBeat was careful to bound the claim, and so should any buyer. The repository is a memory layer built with specialist subagents and persistent storage. It is not a shared memory framework that several independent agents write into at once, and the published materials do not include production compliance controls such as deterministic policy boundaries, retention guarantees or formal audit workflows. Read it as an engineering template, not a finished enterprise platform.
How RAG works, and what it costs to run
Retrieval-augmented generation solves the same problem a different way. You take your documents, split them into chunks, and run each chunk through an embedding model to get a vector. You store those vectors in a vector database. At query time you embed the question, search for the nearest vectors, and pass the top matches to the model as context. The model only ever sees a small, relevant slice of the corpus.
That design has four moving parts a team has to build and keep running: an embedding pipeline, vector storage, indexing logic, and synchronization to keep the index current as source data changes. None of it is exotic in 2026, but all of it is real operational surface. The trade you get for that surface is scale. Because search returns only the top matches, query cost and latency stay roughly flat whether the store holds one million or one hundred million vectors.
The running cost depends heavily on which vector store you pick. As tracked across 2026 pricing comparisons, PostgreSQL with the pgvector extension is effectively free beyond the database instance you already pay for, landing near $45 a month at ten million vectors on a managed instance, and $50 to $180 a month for the underlying Postgres. Pinecone's serverless tier starts around a $50 monthly minimum, sits near $70 a month at ten million vectors, and climbs past $700 a month at one hundred million vectors, billed across storage, read units and write units. One widely cited comparison found pgvector roughly 75% cheaper than Pinecone for teams already on Postgres below the ten-to-fifty-million-vector range, with Pinecone earning its price once you need a fully managed store at hundreds of millions of vectors and consistently low latency.
Retrieval by reasoning vs retrieval by similarity
The cleanest way to see the difference is to name what each design does at query time. RAG retrieves by similarity: it finds the chunks whose vectors sit closest to the question. The Always-On Memory Agent retrieves by reasoning: it hands the memory to a model and asks it to find and synthesize the relevant parts. Those are genuinely different cost curves and failure modes.
| Dimension | Always-On Memory Agent | RAG with a vector database |
|---|---|---|
| How data is stored | Structured records and summaries in SQLite | Chunks plus embeddings in a vector index |
| How retrieval works | A model reads and reasons over memory | Nearest-vector similarity search |
| Infrastructure to run | One model, one SQLite file, a timer | Embedding pipeline, vector store, indexing, sync |
| Query cost driver | Tokens re-read per query, grows with store | Search over the index, roughly flat with size |
| Scaling behavior | Degrades as memory outgrows the context window | Stable into hundreds of millions of vectors |
| Best fit | Bounded, low-to-medium memory agents | Large or fast-growing corpora |
Neither column is strictly better. The memory agent removes infrastructure and adds token cost that scales with memory size. RAG adds infrastructure and holds cost flat. The right answer depends on how much the agent has to remember and how fast that grows.
The cost math
Gemini 3.1 Flash-Lite is the reason the always-on loop is even worth discussing. Google prices it at $0.25 per 1 million input tokens and $1.50 per 1 million output tokens, and positions it for high-volume workloads, claiming it runs 2.5 times faster than Gemini 2.5 Flash on time to first token with a 45% increase in output speed. On Google's published benchmarks it posts an Elo of 1432 on Arena.ai, 86.9% on GPQA Diamond and 76.8% on MMMU Pro. For a background service that re-reads memory around the clock, cheap and fast input tokens are the whole ballgame.
Here is a worked illustration, using the published rate and one stated assumption so you can swap in your own numbers. Consolidation runs 48 times a day. If the store holds 50,000 tokens and each cycle re-reads it, that is 2.4 million input tokens a day, about $0.60 a day or roughly $18 a month for consolidation input alone, before you count query traffic. Now grow the store tenfold to 500,000 tokens: the same cadence becomes 24 million input tokens a day, near $6 a day or about $180 a month, and every user query also has to read a larger store. The lesson is not a specific dollar figure. It is the shape of the curve. Token cost for this design rises with the size of what you remember, on a fixed 30-minute clock.
| Setup | What you pay for | Indicative monthly cost, 2026 |
|---|---|---|
| Always-On Memory Agent, small store | Flash-Lite tokens for consolidation and queries | Tens of dollars at ~50k tokens |
| Always-On Memory Agent, large store | Same, but re-reading a bigger store each cycle | Rises with store size, ~$180+ input at ~500k tokens |
| pgvector on Postgres | The database instance you already run | ~$45 at 10M vectors |
| Pinecone serverless | Storage, read units, write units | ~$70 at 10M vectors, $700+ at 100M |
| Managed memory service (Zep) | Credit-based subscription | From ~$25, scales with usage |
Put the two curves side by side and the crossover is obvious. Below a few hundred thousand tokens of memory, the no-database design is cheaper and far simpler, because you are paying cents to tens of dollars in tokens and running no extra infrastructure. Above that, a vector index that keeps query cost flat starts to win, and it keeps winning as the corpus grows.
Where the no-vector-database design breaks
The sharpest public critique of the launch went straight to this point. As reported by VentureBeat, a developer posting as Iffy challenged the "no embeddings" framing, arguing that the system still has to chunk, index and retrieve structured memory, and that it may work well for small-context agents but break down once memory stores become much larger. That is the technically important read. Removing a vector database does not remove retrieval design. It moves the complexity from an index into the model's context window and the consolidation logic.
Three limits follow from that, and all of them are size-driven. First, cost: as shown above, tokens re-read per cycle and per query scale with the store, so a memory that grows without bound grows the bill without bound. Second, latency and quality: once the memory no longer fits the context comfortably, you are back to selecting what to load, which is the exact retrieval problem RAG already solves, only now hand-rolled. Third, recall precision: similarity search is deterministic and tunable, while asking a model to find the right memory is neither, and it can miss or invent under load.
There is a second class of risk that has nothing to do with size. A background process that rewrites memory on its own schedule can drift. One developer reacting to the launch argued the main cost of always-on agents is not tokens but "drift and loops," where the agent's self-updated memory slowly diverges from reality. For a weekend prototype that is a curiosity. For a support agent that quotes policy to customers, it is a defect.
The wider memory landscape
The Always-On Memory Agent is not the only way to give an agent memory in 2026, and a fair decision compares it to the managed options too. The dedicated memory frameworks have matured into a real category. Mem0 is the adoption leader, with more than 90,000 developers building on it and a $24 million Series A behind it, and it is the pragmatic pick for personalization. Zep leads on benchmark accuracy, reporting 63.8% on LongMemEval, and uses a temporal context graph for time-aware recall, with credit-based pricing from about $25 a month. Letta, formerly MemGPT, is an agent runtime rather than a drop-in library, with an operating-system-inspired design where core memory stays in-context like RAM and the agent manages its own paging.
| Option | Retrieval method | When it fits |
|---|---|---|
| Always-On Memory Agent | A model reads and consolidates memory | Bounded memory, minimal infrastructure, full control |
| pgvector on Postgres | Similarity search inside your database | You already run Postgres, under ~10-50M vectors |
| Pinecone or Weaviate | Managed vector search | Large, fast-growing corpora needing low latency |
| Mem0 | Managed memory API with extraction | Fast personalization across a user base |
| Zep | Temporal knowledge graph | Time-aware reasoning over changing facts |
| Letta | In-context core memory plus paging | Long-running autonomous agents |
The honest framing is that Saboo's repository is a template you own end to end, while Mem0, Zep and Letta are services or runtimes you adopt. If you want zero external dependencies and a memory you can read in a SQLite browser, the open template wins. If you want someone else to own recall quality and lifecycle, the managed frameworks earn their fee.
Governance is the real enterprise question
The loudest reactions to the launch were not about speed or price. They were about control. A developer named Franck Abe called the combination of Google ADK and 24/7 memory consolidation "brilliant leaps for continuous agent autonomy," then warned that an agent "dreaming" and cross-pollinating memories in the background without deterministic boundaries becomes "a compliance nightmare." That captures the enterprise problem precisely. As soon as memory stops being session-bound, you have to answer who can write it, what gets merged, how long it is retained, when it is deleted, and how anyone audits what the agent learned over time.
This is where the reference implementation is explicitly incomplete, and where a buyer should not pretend otherwise. The published materials do not lay out deterministic policy boundaries, retention guarantees, segregation rules or formal audit workflows. Those are not optional in a regulated setting. If an agent's persistent memory holds personal data, you inherit obligations under regimes like the EU's data rules and, in India, the Digital Personal Data Protection Act 2023, including the ability to delete a person's data on request. A memory that silently consolidates and rewrites itself makes "delete everything about this user" a harder promise to keep than it looks. Solving that is engineering work you add on top of the template, not a feature you get for free.
When to drop the vector database
Use a plain test. Estimate the size of what the agent must remember at steady state, and how fast it grows.
Drop the vector database when the memory is bounded and small to medium: a personal assistant, a single project's context, an internal copilot for one team, a prototype you want running this week. Here the whole store fits in context, token cost is trivial, and the operational simplicity of one model and one SQLite file is a real advantage. You also keep full control and a memory you can inspect directly.
Keep the vector database when the corpus is large or grows quickly, when you need predictable low latency at scale, when recall has to be precise and tunable, or when compliance demands strict, auditable retrieval boundaries. A knowledge base spanning hundreds of thousands of documents, a customer-facing search system, or anything under formal audit belongs on a real index. The infrastructure you pay for buys you flat cost and controllable behavior as you scale.
And consider a hybrid, because most production systems land there. Use the always-on consolidation pattern for the agent's working memory, the recent, bounded context it reasons over turn to turn, and keep a vector store for the large reference corpus. That gives you cheap, simple short-term memory and scalable, auditable long-term retrieval, which is closer to how the strongest agent systems are actually built in 2026.
India-specific considerations
For Indian product teams and global capability centres, two factors sharpen the choice. The first is cost discipline. A design that runs on a $0.25-per-million-token model and a single SQLite file is attractive when budgets are tight and a managed vector store priced in dollars is a recurring line item. For an early-stage product or an internal tool, the open template can take you a long way before you pay for infrastructure. The second is the Digital Personal Data Protection Act 2023. If persistent agent memory holds personal data of Indian users, consent, purpose limitation and deletion rights apply, and a self-consolidating memory makes those rights harder to honour. Design the retention and deletion path before you ship, not after a request arrives. Teams that treat memory as regulated data from day one avoid an expensive retrofit later.
FAQ
What is Google's Always-On Memory Agent?
It is an open-source reference implementation, published by Google senior AI product manager Shubham Saboo in mid-July 2026 under an MIT license. It stores structured memories in SQLite, consolidates them every 30 minutes using Gemini 3.1 Flash-Lite, and answers queries by reading that memory directly, without a vector database or embeddings.
Does it really use no vector database or embeddings?
Yes, by design it stores structured records in SQLite and lets the model read and reason over them instead of running similarity search. Critics note this does not remove retrieval design, it moves it into the context window and consolidation logic, which works for bounded memory but strains once the store grows large.
Is the memory agent cheaper than RAG?
For small to medium memory, usually yes, because you run one model and one SQLite file with no extra infrastructure. At larger sizes the cost inverts: consolidation and queries re-read the whole store on a 30-minute clock, so token cost climbs, while a vector index like pgvector near $45 a month keeps query cost roughly flat.
When should I keep my vector database?
Keep it when the corpus is large or fast-growing, when you need predictable low latency at scale, when recall must be precise and tunable, or when compliance requires auditable retrieval boundaries. A search system over hundreds of thousands of documents belongs on a real index such as Pinecone, pgvector or Weaviate rather than a self-consolidating memory.
How much does Gemini 3.1 Flash-Lite cost?
Google prices Gemini 3.1 Flash-Lite at $0.25 per 1 million input tokens and $1.50 per 1 million output tokens, as of its March 3, 2026 release. Google states it runs 2.5 times faster than Gemini 2.5 Flash on time to first token, which is what makes an around-the-clock memory loop economically viable.
What are the main risks of always-on memory?
Two stand out. Size-driven cost and latency, because re-reading a growing store gets slower and more expensive, and governance drift, where a background process that rewrites memory can diverge from reality. One developer summarized the real cost as "drift and loops" rather than tokens, which matters most for customer-facing agents.
How does it compare to Mem0, Zep and Letta?
Those are managed frameworks or runtimes you adopt, while Saboo's repository is a template you own end to end. Mem0 leads on adoption with over 90,000 developers, Zep leads on accuracy at 63.8% on LongMemEval, and Letta is an agent runtime with operating-system-inspired memory paging for long-running autonomous agents.
Can I use both a memory agent and RAG together?
Yes, and most production systems do. Use the always-on consolidation pattern for the agent's bounded working memory, and keep a vector store for the large reference corpus. That combination gives you cheap, simple short-term memory alongside scalable, auditable long-term retrieval, which reflects how strong agent systems are built in 2026.
How eCorpIT can help
eCorpIT builds and governs production AI agents for teams that need memory to be both useful and auditable. We size the memory workload first, then pick the architecture that fits, whether that is an always-on consolidation loop, a vector store, a managed framework, or a hybrid of the three, and we build the retention, deletion and audit paths that regulated data demands. If you are deciding how your agent should remember, talk to our senior engineering team through our enterprise AI agent development service or our RAG knowledge assistant service, and reach us any time at /contact-us/. For the broader picture, see our guide to enterprise AI agents in production, our pgvector versus dedicated vector database decision, and when a large context can replace retrieval entirely in long context versus RAG on cost and latency.
References
- Google PM open-sources Always On Memory Agent, ditching vector databases for LLM-driven persistent memory (VentureBeat)
- Google Cloud's Always-On Memory Agent replaces RAG and embeddings with continuous LLM consolidation on Gemini 3.1 Flash-Lite (MarkTechPost)
- Gemini 3.1 Flash-Lite intelligence, performance and price analysis (Artificial Analysis)
- Gemini 3.1 Flash-Lite API pricing, $0.25/$1.50 per 1M tokens (devtk.ai)
- How much does a vector database cost? A 2026 pricing comparison (Mixpeek)
- Vector database pricing 2026: Pinecone vs pgvector vs Weaviate (Spendark)
- pgvector vs Pinecone 2026: which vector DB should you actually use? (Kunal Ganglani)
- Best AI agent memory frameworks in 2026: compared and ranked (Atlan)
- Best AI agent memory systems in 2026: 8 frameworks compared (Vectorize)
- State of AI agent memory 2026 (Mem0)
- Google PM open-sources Always On Memory Agent (Hacker News discussion)
Last updated: July 29, 2026.
Top comments (0)