DEV Community

Ajmal Hasan
Ajmal Hasan

Posted on

2 1 1

GenAI: Building RAG Systems with LangChain

In the age of Generative AI, Retrieval-Augmented Generation (RAG) has emerged as a powerful approach for building intelligent, context-aware applications. RAG combines the strengths of large language models (LLMs) with efficient document retrieval techniques to answer queries based on specific data. In this blog, we explore how to implement a RAG pipeline using LangChain, GPT-4o, Ollama, Groq etc.

Github Repo ->


Image description

Key Features of the RAG Pipeline

  1. Data Retrieval: Fetch data from web sources, local files, or APIs using LangChain’s loaders.
  2. Document Processing: Break down documents into smaller chunks for efficient retrieval using text splitters, enabling better indexing and faster search results.
  3. Vector Embeddings: Represent document chunks as high-dimensional vectors using OpenAI embeddings or other embedding techniques for flexible integration.
  4. Query Processing: Retrieve the most relevant document chunks and use LLMs (like GPT-4o or similar models) to generate accurate, context-based answers.
  5. Interactive UI: A seamless user interface built with Streamlit for document uploads, querying, and result visualization.
  6. Model Integration: The pipeline supports both cloud-based and local models, ensuring adaptability based on project needs.

Tools and Libraries Used

This implementation relies on a range of powerful libraries and tools:

  • langchain_openai: For OpenAI embeddings and integrations.
  • langchain_core: Core utilities for building LangChain workflows.
  • python-dotenv: To manage API keys and environment variables securely.
  • streamlit: For creating an interactive user interface.
  • langchain_community: Community-contributed tools, including document loaders.
  • langserve: For deploying the pipeline as a service.
  • fastapi: To build a robust API for the RAG application.
  • uvicorn: To serve the FastAPI application.
  • sse_starlette: For handling server-sent events.
  • bs4 and beautifulsoup4: For web scraping and extracting data from HTML content.
  • pypdf and PyPDF2: For processing and extracting data from PDF files.
  • chromadb and faiss-cpu: For managing vector stores and efficient similarity search.
  • groq: For integrating with GPT-4o.
  • cassio: Tools for enhanced vector operations.
  • wikipedia and arxiv: For loading data from online sources.
  • langchainhub: For accessing pre-built tools and components.
  • sentence_transformers: For creating high-quality vector embeddings.
  • langchain-objectbox: For managing vector embeddings with ObjectBox.
  • langchain: The backbone of the RAG pipeline, handling document retrieval and LLM integration.

How It Works

  1. Setting Up the Environment:

    • Use environment management tools to securely load API keys and configure settings for both cloud-based and local models.
  2. Data Loading:

    • Load data from multiple sources, including online documents, local directories, or PDFs.
  3. Document Splitting:

    • Split large documents into smaller, manageable chunks to ensure faster retrieval and better accuracy during searches.
  4. Vector Embeddings with ObjectBox:

    • Convert document chunks into numerical vectors for similarity-based searches.
    • Use ObjectBox or other vector databases to store embeddings, enabling high-speed data retrieval.
  5. Query Handling:

    • Combine document retrieval with context-aware response generation to answer queries with precision and clarity.

Local vs Paid LLMs

When implementing an RAG pipeline, choosing between local and paid LLMs depends on project needs and constraints. Here's a quick comparison:

Feature Local LLMs Paid LLMs (e.g., OpenAI GPT)
Data Privacy High – Data stays on local machines. Moderate – Data sent to external APIs.
Cost One-time infrastructure setup. Recurring API usage costs.
Performance Dependent on local hardware. Scalable and optimized by providers.
Flexibility Fully customizable. Limited to API functionality.
Ease of Use Requires setup and maintenance. Ready-to-use with minimal setup.
Offline Capability Yes. No – Requires internet connection.

For projects requiring high privacy or offline functionality, local LLMs are ideal. For scalable, maintenance-free implementations, paid LLMs are often the better choice.


Why RAG Matters

RAG empowers applications to:

  • Provide accurate and context-aware responses based on user-specific data.
  • Handle large datasets efficiently with advanced retrieval mechanisms.
  • Combine retrieval and generation seamlessly, enhancing the capabilities of LLMs.
  • Support flexible deployment options for diverse project needs.

GitHub Repository

You can explore the complete implementation in this GitHub repository. It includes all the documentation needed to build your own RAG-powered application.



Core Concepts of LangChain

1. LLMs (Large Language Models)

  • LangChain provides a simple interface to interact with LLMs like OpenAI's GPT, Anthropic, Hugging Face models, etc.
  • You can use models for:
    • Text generation.
    • Summarization.
    • Classification.
  • Example:

     from langchain.llms import OpenAI
    
     llm = OpenAI(model="text-davinci-003")
     response = llm("What is LangChain?")
     print(response)
    

2. Chains

Chains combine multiple steps to perform a specific task, like taking user input, processing it with an LLM, and returning the output.

  • Types of Chains:

    • LLMChain: Single input-output step.
    • SequentialChain: Combine multiple chains in sequence.
    • TransformChain: Modify input/output between chains.
  • Example: SequentialChain

     from langchain.chains import SimpleSequentialChain
     from langchain.prompts import PromptTemplate
     from langchain.llms import OpenAI
    
     prompt1 = PromptTemplate(input_variables=["input"], template="Summarize: {input}")
     prompt2 = PromptTemplate(input_variables=["summary"], template="Explain in detail: {summary}")
    
     llm = OpenAI(model="text-davinci-003")
     chain1 = LLMChain(llm=llm, prompt=prompt1)
     chain2 = LLMChain(llm=llm, prompt=prompt2)
    
     sequential_chain = SimpleSequentialChain(chains=[chain1, chain2])
     output = sequential_chain.run("LangChain simplifies LLM usage.")
     print(output)
    

3. Agents

Agents allow dynamic decision-making by the AI model. They determine which tools or APIs to call based on the input.

  • Types of Agents:

    • Zero-shot agents: Rely purely on the LLM's reasoning.
    • React-based agents: Combine reasoning and acting.
    • Custom agents: Define your own logic and tools.
  • Example:

     from langchain.agents import initialize_agent, Tool
     from langchain.llms import OpenAI
    
     tools = [
         Tool(name="Calculator", func=lambda x: eval(x), description="Useful for math calculations")
     ]
    
     llm = OpenAI(model="text-davinci-003")
     agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)
    
     result = agent.run("What is 5 plus 7?")
     print(result)
    

4. Memory

Memory allows an AI model to retain information between interactions.

  • Types of Memory:

    • Buffer Memory: Keeps a running log of conversations.
    • Entity Memory: Stores facts about specific entities.
    • Custom Memory: Create custom implementations based on use cases.
  • Example:

     from langchain.chains import ConversationChain
     from langchain.memory import ConversationBufferMemory
     from langchain.llms import OpenAI
    
     memory = ConversationBufferMemory()
     conversation = ConversationChain(llm=OpenAI(model="text-davinci-003"), memory=memory)
    
     conversation.run("What is LangChain?")
     conversation.run("Can you explain its memory system?")
    

5. Prompts

  • LangChain provides tools for managing prompt templates to guide LLMs effectively.
  • Prompt Template Components:
    • Input Variables.
    • Template structure.
  • Example:

     from langchain.prompts import PromptTemplate
    
     prompt = PromptTemplate(
         input_variables=["topic"],
         template="Explain {topic} in simple terms."
     )
    
     print(prompt.format(topic="machine learning"))
    

6. Tools

Tools in LangChain extend functionality by integrating external APIs, databases, or other utilities.

  • Common Tools:
    • Search Engines (Google, Bing).
    • Python REPL (for calculations).
    • Custom APIs.
  • Example:

     from langchain.agents import Tool
    
     def search_tool(query):
         return f"Searching for: {query}"
    
     tool = Tool(name="WebSearch", func=search_tool, description="Search the web")
    

7. Document Loaders

Document loaders enable processing external data (PDFs, CSVs, etc.) into text for the LLM.

  • Common Loaders:
    • PDFLoader.
    • CSVLoader.
    • WebPageLoader.
  • Example:

     from langchain.document_loaders import PDFLoader
    
     loader = PDFLoader("sample.pdf")
     documents = loader.load()
     print(documents[0].page_content)
    

8. Retrievers

  • Retrievers fetch relevant information from a knowledge base or document store.
  • Used in question-answering systems or chatbots.
  • Example:

     from langchain.vectorstores import FAISS
     from langchain.embeddings import OpenAIEmbeddings
    
     vectorstore = FAISS.load_local("vector_store", OpenAIEmbeddings())
     retriever = vectorstore.as_retriever()
     response = retriever.get_relevant_documents("What is LangChain?")
     print(response)
    

9. VectorStores

  • LangChain integrates vector databases like FAISS, Pinecone, Weaviate, etc.
  • These allow searching through embeddings for context-aware responses.
  • Use Cases: Q&A, semantic search.

10. Callbacks

  • LangChain supports callback hooks to log, debug, or extend functionality at runtime.
  • Example:

     from langchain.callbacks import StdOutCallbackHandler
    
     handler = StdOutCallbackHandler()
     llm = OpenAI(model="text-davinci-003", callbacks=[handler])
     llm("Tell me about LangChain.")
    

11. Evaluation

  • LangChain provides tools for automated evaluation of prompts, models, and workflows.
  • Use these to analyze the quality of responses in tasks like summarization, QA, etc.

12. Integration with APIs

LangChain can work with external APIs to fetch real-time data or perform tasks:

  • Integration with OpenAI, HuggingFace, or custom APIs.
  • Example: Integrating Google Search, SQL databases, or REST APIs.

13. Custom Extensions

  • You can define custom prompts, chains, tools, or memory modules.
  • This makes LangChain highly adaptable to specific use cases.

14. Deployment

  • LangChain applications can be deployed as:
    • APIs.
    • Streamlit web apps.
    • CLI tools.
  • Use frameworks like FastAPI, Flask, or other deployment platforms.

This demonstration highlights the immense potential of combining LangChain with LLMs and vector databases. Whether you're building chatbots, knowledge assistants, or research tools, RAG provides a solid foundation for delivering robust, data-driven results.

Billboard image

Use Playwright to test. Use Playwright to monitor.

Join Vercel, CrowdStrike, and thousands of other teams that run end-to-end monitors on Checkly's programmable monitoring platform.

Get started now!

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay