RAG - Multivector Retrievel, Multi Hop, Conversational RAG

Multi-Vector Retrieval

In a typical RAG pipeline, every chunk is converted into an embedding and stored in the vector database.

In Multi-Vector Retrieval, instead of creating a single embedding for a chunk, multiple embeddings are created to represent different aspects of the original chunk.

Each embedding captures a different perspective of the same content. The metadata of each embedding contains a reference to the original chunk.

When a user query is received, the retriever searches across all these embeddings. Since each embedding represents a different aspect of the content, the chances of retrieving more relevant information are higher.

This approach generally provides better retrieval performance.

However, it is a costly approach because:

• Multiple embeddings are generated for every chunk.
• More storage is required in the vector database.
• More tokens are consumed during embedding generation.

If no other optimization technique is able to improve the RAG performance, Multi-Vector Retrieval can be considered as a final optimization step.

---

Multi-Hop

Multi-Hop Retrieval is used when the answer cannot be obtained from a single piece of context. Instead, the LLM has to retrieve multiple related contexts and connect them to generate the final answer.

Example

Suppose the knowledge base contains the following information:

1. Biryani contains spices.
2. The spices used in biryani are cardamom, cinnamon, cloves, etc.

Now suppose the user asks:

"What spices need to be added to biryani?"

The LLM first retrieves the information:

"Biryani contains spices."

It then hops to the next related context:

"The spices used in biryani are cardamom, cinnamon, cloves, etc."

Finally, it combines both pieces of information to generate the complete answer.

In this process, the LLM hops from one retrieved context to another until it gathers enough information to answer the user's query.

Each retrieved context should have a meaningful relationship with the next one so that the LLM can follow the chain of information and produce the correct response.

Conversational RAG

A user may not restrict themselves to asking only one query. They may ask a sequence of queries. Here, we are going to look at how to build Conversational RAG.

Traditional Flow

1. User Query

2. Retriever

3. Related Documents

4. LLM

5. Answer

A user may ask questions that are indirect or related to the previous question. So, we need to implement Query Transformation and Expansion to get the related queries and their responses.

Therefore, we need to have memory to store the previous conversation history.

Conversational RAG will be:

1. Conversation History + Current Question

2. Intent

3. Query Transformation and Expansion

4. Retriever

5. Relevant Context

6. LLM

7. Context-Aware Answer

Here, we are going to see how we can store conversation history in memory.

We can use persistent memory such as PostgreSQL or SQLite, or short-term memory using Redis, Valkey, Memcached, etc. We can also use a pickle file to store the history.

If we are using a file, make sure that the application we are using is single-threaded.

To store the history, we can use the session ID as the primary key.

Always store the entire history in long-term memory. However, we cannot store the entire history in short-term memory.

The purpose of storing history in short-term memory is to reduce latency.

To solve this problem, we can store the entire history in long-term memory and summarize the conversation history and store the summary in short-term memory whenever needed.