Multi-Agent RAG application with QA based on Internet Search and Proprietary Docs

The key benefits of Retrieval Augmented Generation (RAG) is being able to answer user queries based on a company's proprietary documents. In this post, I describe an application that I built using LangFlow and AstraDB (vector db) that allows someone to ask a question about machine learning, and receive an answer based on both generally available information and also information as it applies to a company's proprietary information.

For the proprietary information, I used documents based on CtiPath's Enterprise-Ready Machine Learning Operations (eMLOps) solution.

For this application, there are two agents:

An Orchestrator Agent - this agent is responsible for coordinating the information from an internet search tool and the CtiPathDocs Agent which is in tool mode.
CtiPathDocs Agent - this agent accesses information about CtiPath's eMLOps solution from a vector db.

The vector db (AstraDB) has been loaded previously using this simple flow:

Vector DB Load Data Flow

Once the data is loaded, the main flow is built using the two agents described above as shown here:

Multi-Agent RAG Application Flow

The prompts are very important in this application. The CtiPathDocs agent is instructed to search the vector db to determine how CtiPath's eMLOps solution applies to the user's query.

CtiPathDocs Agent Prompt

Meanwhile the Orchestrator Agent is allowed to search the internet via a Google Search API tool or search the vector db using the CtiPathDocs agent. General information should be gathered from the internet, while the RAG agent should be used to determine how CtiPath's eMLOps solution applies to the query.

Orchestrator Agent Prompt

Notice that the Orchestrator Agent is allowed to iterate through internet searches and RAG searches until it answers the query and determines how CtiPath's eMLOps solution applies.

When the Orchestrator Agent answers a general question about machine learning ("What is machine learning monitoring?") it provides both a general answer, and also explains how it relates to CtiPath's eMLOps solution.

Orchestrator Agent Output

This simple multi-agent RAG application shows the power of Gen AI to delivery specific information to a customer, even when they are seeking general information.