Usually, when writing code using AI such as Claude or Gemini, we may use a .md file to store the responsibilities, instructions, and activities that need to be performed by the Agent. When we create a module using this .md file, the Agent can use the instructions and context provided in the file throughout the conversation.
Likewise, when the user asks a query and wants to save information as a profile, the system can extract the context/info from the user's entered details and store it as user-specific memory.
User enters:
E.g.
I am Ravi from Chennai
Here, the Agent can extract the name and place and save them in the user profile.
Next, the user asks about:
List Top 10 Attractions near me
The Agent already knows that the user is from:
Chennai
by retrieving it from the profile. So, it will list the Top 10 attractions near Chennai.
What do we remember?
Three layers: profile, episodes, procedures
How do we learn it?
An LLM extracts facts the user states, never guesses.
Where is it kept?
A Store, one folder per user.
How is it used?
The system prompt is rebuilt on every call.
Three layers are:
Profile - Who the user is now, stored as one small, structured record and recalled on every call.
E.g. name, city, language, level, allergies
Episode - What happened, stored as many small notes and recalled only when relevant.
E.g. trips planned, bugs fixed, complaints
Procedure - How to treat this user, stored as a few rules and recalled always, but edited rarely.
E.g. keep answers under 5 lines
Personalized layers can be stored as JSON-serialized objects, such as dictionaries, lists, or strings, in RAM, a pickle file, or persistent storage. When the app loads next time, deserialize the object and store it as a predefined structure.
RAG as an Agent Tool
RAG Pipeline
Documents will be chunked, embedded, and stored in a vector database. When a user query comes in, the system searches the database, gets the relevant context, and generates the answer using the retrieved context.
The question here is: When do we need to call RAG? Do we need to call it every time?
E.g.
Ask a query to the Agent:
Hi
There is no need to refer to RAG for this kind of question because retrieval can add unnecessary latency and cost.
Remedy
RAG can be exposed as a tool that the Agent can call. Whenever a question comes in, the Agent decides whether it needs to call the RAG tool based on the question. If RAG is needed, the Agent calls the RAG tool and uses the retrieved context to generate the answer. Otherwise, the Agent can answer directly using the LLM.
Top comments (0)