RAG Pipeline Optimization Techniques
The following techniques are commonly used to optimize a RAG pipeline:
- Semantic Caching
- Query Transformation and Expansion
- Context Compression
- Parent Retrieval
- Multi-Vector Retrieval
These techniques are used to improve the quality of the responses generated by the LLM.
Parent Retrieval
Today, we are going to focus on Parent Retrieval.
A chunk is a small portion of a document or paragraph, typically consisting of 500–1000 words. When these chunks are converted into embeddings and stored in a vector database, semantically related chunks are positioned close to one another.
When a user submits a query, the system retrieves the chunks that are closest to the query. However, not all the retrieved chunks may be the most relevant. Sometimes, we may miss other chunks that provide better context for the user's query.
Example
Suppose we have the following paragraphs:
Paragraph 1 (P1)
- L1
- L2
- L3
- L4
Each line is stored as an individual chunk:
- P1C1
- P1C2
- P1C3
- P1C4
Paragraph 2 (P2)
- L1
- L2
- L3
Each line is also stored as individual chunks:
- P2C1
- P2C2
- P2C3
Suppose the expected answer to the user query is P1C2.
The vector database retrieves:
- P1C2
- P2C3
Although P1C2 is correctly retrieved, the related chunks P1C1, P1C3, and P1C4 may provide much better context than P2C3. Since these chunks are not retrieved, we may lose important context that could improve the final LLM response.
If we store the document paragraph-wise instead of using smaller chunks, we may retrieve unnecessary context, which increases token consumption.
Solution – Parent Retrieval
Suppose Paragraph 1 is divided into four chunks:
- P1C1
- P1C2
- P1C3
- P1C4
Whenever a specific chunk is retrieved from the vector database, it also contains a reference to its parent paragraph, which is stored in the chunk's metadata.
For example, if P1C2 is retrieved, the metadata also contains a reference to Paragraph 1.
Using this reference, the retriever can also fetch the remaining chunks belonging to the same parent paragraph:
- P1C1
- P1C3
- P1C4
This ensures that we do not miss important context that is closely related to the user's query.
Example
Suppose the user query is:
"Create a dictionary"
The retrieved chunks are:
- P1C2
- P2C2
- P2C3
Instead of sending only these chunks to the LLM, Parent Retrieval also includes the remaining chunks from the same parent paragraph:
- P1C1
- P1C3
- P1C4
Providing this additional context enables the LLM to generate a more accurate response.
When Should Parent Retrieval Be Used?
Parent Retrieval is useful when we feel that important context is missing from the retrieved chunks and we want to include additional context that is closely related to the query.
The parent paragraph is stored similarly to other chunks. However, it is not used directly for vector search. Instead, its reference is stored in each chunk's metadata, allowing the retriever to fetch the parent context whenever required.
Top comments (0)