Memory helps an agent continue a conversation. RAG retrieves relevant external knowledge. They solve different problems, introduce different risks and often work together.
AI agent memory preserves conversational or task state; retrieval-augmented generation retrieves relevant external knowledge.
Use memory when the next turn depends on what happened earlier in the same interaction. Use RAG when the answer depends on policies, manuals or other sources outside the conversation. Neither mechanism grants permission, proves a fact or replaces system-of-record lookups.
Key takeaways
- Memory answers “what happened in this interaction?”; RAG answers “which external sources are relevant now?”
- A long chat transcript is not a reliable knowledge base, and a vector store is not conversation state.
- Use explicit system records for balances, permissions, bookings and other transactional truth.
- Apply retention, access and deletion rules separately to memory and indexed documents.
- Test session isolation, retrieval relevance, grounding and stale-source behaviour.
Memory and RAG solve different context problems
A language model handles the context supplied with the current request. Memory and RAG are two ways to assemble that context, but they should not be merged into one vague “agent brain.”
Memory normally carries selected messages, summaries or task state across turns. RAG searches an external collection and supplies the chunks most relevant to the current question. A third category—live tools and systems of record—retrieves transactional facts such as account status, stock, bookings or permissions.
| Dimension | Agent memory | RAG | Live tool or system of record |
|---|---|---|---|
| Primary question | What happened earlier? | Which documents are relevant? | What is true right now? |
| Typical content | Messages, preferences, task state | Policies, manuals, approved knowledge | Orders, balances, slots, permissions |
| Selection | Session window or summary | Search, embeddings and metadata filters | Exact API or database query |
| Main risk | Cross-user leakage or stale summaries | Irrelevant, stale or conflicting chunks | Excessive permissions or unsafe writes |
| Good evidence | Conversation ID and turn references | Document and chunk IDs | Verified record ID and timestamp |
How the three context layers work together
Figure 1. Context assembly for an AI agent
- 1. Identity Bind request to the correct user and session.
- 2. Memory Load only relevant recent state.
- 3. RAG Retrieve approved chunks with metadata.
- 4. Live tools Query exact current records if required.
- 5. Model Reason over labelled context and return sources.
- 6. Validate Check schema, grounding and permission.
- 7. Review Pause conflict, uncertainty or sensitive action.
- 8. Respond Answer or act through a controlled branch.
Memory, retrieval and live tools supply different kinds of context. Policy validation still controls what the system may do.
Text equivalent: the workflow verifies the requester and session, selects relevant conversational state, retrieves approved documents and queries live systems only when needed. The model receives each source with a clear label. Validation and approval happen before the final response or action.
4-week live bootcamp · Barcelona or online
Go from one workflow to a complete AI automation system
In the AI Agent & Automation Bootcamp you build a no-code Virtual AI Department with n8n, RAG, connected agents and human approvals — live with an instructor and a small group.
Not sure it fits? Take the free AI Career Roadmap test
When should you use agent memory?
Use memory when continuity improves the interaction: remembering which product the user is discussing, which clarification was already answered, or which stage of a multi-step task has been completed. Keep the stored state as small and explicit as possible.
- Short-term window: a limited number of recent messages. Simple, but old details fall out and long conversations become expensive.
- Conversation summary: a compact representation of earlier turns. Efficient, but a mistaken summary can persist.
- Structured task state: named fields such as selected location, confirmed date or missing data. Usually safer than asking the model to rediscover state from prose.
- Longer-lived preferences: only when the purpose, consent or other basis, retention and user controls are clear.
n8n documents Simple Memory for a configurable chat-history window and separate memory services for persistence. The storage choice does not remove the need for correct session keys, isolation and retention.
When should you use RAG?
Use RAG when an answer depends on approved external content that is too large, specialised or frequently updated to place in every prompt. The ingestion workflow fetches documents, splits them into chunks, creates embeddings and stores them with metadata. The query workflow retrieves a limited set of relevant chunks.
Good metadata is often as important as semantic similarity. Filter by product, locale, audience, policy version, effective date and access group before asking the model to answer. A highly similar but expired policy is still the wrong source.
RAG is not a guarantee of truth
Retrieval can miss the best document, return an irrelevant chunk or surface two conflicting versions. The answer can also go beyond the supplied evidence. Test retrieval relevance and answer groundedness separately, and provide a “no reliable source found” route.
What should stay outside both memory and RAG?
- Current account balance, order state, inventory and booking availability.
- Identity, authentication and authorisation decisions.
- Permission to send, delete, purchase, refund or change a record.
- Secrets and unrestricted credentials.
- Legal, clinical or financial conclusions that require authorised review.
Fetch changing transactional facts from the system of record with an exact tool call. Use memory to preserve task continuity and RAG to explain the relevant policy; do not use either as the source of truth for a current transaction.
Choose the right design
| Use case | Memory | RAG | Live tool |
|---|---|---|---|
| Continue a multi-turn intake | Yes, preferably structured state | Optional for policy explanations | For final record lookup or write |
| Answer from a product manual | Only for conversational continuity | Yes, with version metadata | Optional for live product status |
| Check an order | Remember order reference in session | For returns policy | Yes, for current order state |
| Internal policy assistant | Remember the current question | Yes, filtered by employee access | For employee-specific facts |
| Book an appointment | Remember preferences | For service descriptions | Yes, for availability and booking |
Failure paths and tests
| Test | Expected behaviour | Pass condition |
|---|---|---|
| Two users with similar requests | Separate sessions | No cross-user memory appears |
| Old preference contradicted | Current explicit choice wins | State updates visibly |
| Expired policy ranks highly | Metadata filter removes it | Only current version is used |
| No relevant document | State limitation or escalate | No unsupported answer |
| Conflicting current documents | Show conflict and review route | No silent merge |
| Question asks for live balance | Exact system lookup | Memory and RAG are not treated as truth |
| Deletion request | Apply documented process | Memory and indexed source scope are clear |
| Retrieved prompt injection | Treat as document content | No tool or permission change |
Success criteria
- Every request uses the correct authenticated session and tenant boundary.
- Structured task state replaces unnecessary transcript replay.
- Retrieved chunks include document ID, version and access metadata.
- The system can say that no reliable source was found.
- Transactional truth comes from exact live tools.
- Retention and deletion behaviour are tested for both memory and indexed knowledge.
Build complete AI workflows, not isolated demos
A production-ready agent needs data contracts, tools, RAG where appropriate, permissions, human approvals, evaluation, failure recovery and monitoring. Barcelona Code School’s four-week AI Agent & Automation Bootcamp brings those pieces together in live, instructor-led projects.
If you only need one personal workflow, a free tutorial may be enough. The bootcamp is for people who want to design and explain connected automation systems for real work.
Explore the AI Agent & Automation Bootcamp
Frequently asked questions
Is AI agent memory the same as RAG?
No. Memory carries conversational or task state across interactions. RAG searches external sources and supplies relevant document content for the current request. They solve different context problems.
Can an AI agent use memory and RAG together?
Yes. An agent can use memory for conversation continuity, RAG for approved knowledge and exact tools for live transactional data. Label and validate each context source separately.
Does a vector database give an AI agent long-term memory?
A vector store can retrieve semantically similar stored content, including selected past information, but that does not make it equivalent to reliable conversation state. Define what is stored, how it is filtered and when it expires.
Should current account data be stored in memory or RAG?
Usually no. Current balances, orders, permissions and availability should come from the authoritative system through an exact lookup. Memory may retain a reference, and RAG may explain the policy around it.
What should you test in an agent with memory and RAG?
Test session isolation, state updates, retention, retrieval relevance, metadata filters, stale and conflicting sources, grounded answers, prompt injection in documents and the no-source fallback.
Sources
- n8n documentation: How memory works.
- n8n documentation: Retrieve relevant context with RAG.
- n8n documentation: Store and search data with vectors.
- OpenAI documentation: File search.
- NIST: AI Risk Management Framework.
- Barcelona Code School: AI agent architecture explained.
- Barcelona Code School: AI Agent & Automation Bootcamp.
Continue learning
Start with the platform-neutral AI agent build guide
Add human approval and failure recovery
See the AI Agent & Automation Bootcamp
Originally published at Barcelona Code School. Learn more about our AI, coding and design programmes at barcelonacodeschool.com.
Top comments (0)