DEV Community

Cover image for πŸ€– Building RAG chatbot with NeMo Guardrails !πŸ’‘
Sanjay πŸ₯·
Sanjay πŸ₯·

Posted on • Updated on

πŸ€– Building RAG chatbot with NeMo Guardrails !πŸ’‘

Discover how the power of NeMo Guardrails ensures safer and more controlled conversations chatbot !

πŸ› οΈ Tools Overview:

  1. NeMo Guardrails πŸ›‘οΈ
  2. LangChain πŸ”—
  3. Gradio πŸŽ›οΈ
  4. Chroma DB πŸ—ƒοΈ

βš”οΈ NeMo Guardrails

Checkout more on why NeMo Guardrails ?

Here are some key components a config folder should be containing:

  • Config: Houses essential details such as models and high-level flow instructions.
  • Rails: Define dialogue flows using colang, including welcoming messages.
  • Actions: Specify actions for different dialogue scenarios, such as blocking certain words.

Explore more on configuring Nemo Guardrails here.


πŸ”— LangChain with NeMo Guardrails

LangChain gives developers a framework to construct LLM‑powered apps easily.

Integrate LangChain with Nemo Guardrails effortlessly:

# Initialization
llm = ChatOpenAI(openai_api_key=openai_api_key, model_name="gpt-3.5-turbo-16k")

# Loading configs
config = RailsConfig.from_path("config")
app = LLMRails(config=config, llm=llm)

# Generate response
response = await app.generate_async(messages="Here's our message")
Enter fullscreen mode Exit fullscreen mode

πŸ›’ Vector Database with NeMo Guardrails

NeMo Guardrails support knowledge base only in markdown format.

Is there any other way ? Yes, using vector database.

Lets use Chroma DB to:

  1. Create a Vector Database: Ingest documents of any format to build a robust knowledge base.
  2. Perform Similarity Search: Connect to Croma DB and retrieve relevant chunks for user messages.

Now you can generate response with message and relevant_chunks

relevant_chunks = vector_search(message)
response = await app.generate_async(messages=[{"role": "context", "content": {"relevant_chunks": relevant_chunks}}, {"role": "user", "content": message}])
Enter fullscreen mode Exit fullscreen mode

πŸš€ Demo

AWS Guard Chatbot

Note: It has only minimal guards added from NeMo for demo,
So its possible to prompt injection 
Enter fullscreen mode Exit fullscreen mode
Without Guardrails
Without Guardrails
With Guardrails
With Guardrails

πŸ’» Explore the Source Code

Curious to explore further? Head over to the GitHub repository for the complete source code!

Top comments (0)