Bernard K

Posted on Apr 16

How I Built a Document Q&A Bot Using LangChain, FAISS, and Docker

#langchain #ai #python #docker

Setting up a Document Q&A Bot using LangChain, FAISS, and Docker has been quite the expedition into aligning advanced technology with practical constraints. Being based in Kenya with its infrastructure quirks and tight budgets, I needed something effective yet lightweight enough for environments with intermittent connectivity. Here's how I went about it and what I learned along the way.

Why I Chose LangChain, FAISS, and Docker

Having worked with LangChain on previous AI automation projects, I knew it could process natural language effectively. The challenge was integrating it with a reliable search component,FAISS (Facebook AI Similarity Search),to manage document indexing and retrieval efficiently. Docker was the logical choice for maintaining environment consistency across different machines, especially when using budget hardware.

Building the Core: LangChain and FAISS

Integrating LangChain with FAISS allowed me to create a document Q&A bot that searches through large amounts of text to provide relevant answers. Here's a breakdown of how I implemented it:

Setting Up LangChain

Start by setting up LangChain. If you're unfamiliar with this framework, it simplifies the creation of pipeline structures for text processing. Here's a snippet to show how I initialized it:

from langchain import Pipeline

# Initialize your langchain pipeline
pipeline = Pipeline(steps=[
    # Here you would define steps like data processing, machine learning, etc.
])

This straightforward setup prepares for processing natural language queries.

Implementing FAISS for Search

FAISS was transformative for the search functionality. It's fast and handles high-dimensional data well.

import faiss
import numpy as np

# Create data to index
data = np.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype='float32')

# Create an index
index = faiss.IndexFlatL2(data.shape[1])

# Add data to the index
index.add(data)

# To search for the nearest neighbor
distances, indices = index.search(np.array([[1.0, 2.0]], dtype='float32'), 1)
print(indices)  # Output: array([[0]])

I found FAISS particularly efficient when dealing with large datasets, which I anticipate as the data grows.

Packaging with Docker

Using Docker helped manage dependencies and environment setups, crucial when deploying on systems with different specs.

Dockerfile for Environment Consistency

Here's a snippet of the Dockerfile that I used:

# Use an official Python runtime as a parent image
FROM python:3.9

# Set the working directory in the container
WORKDIR /usr/src/app

# Copy the current directory contents into the container at /usr/src/app
COPY . .

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Run langchain script when the container launches
CMD ["python", "langchain_script.py"]

This setup prevented dependency issues on different machines and made deploying updates consistently easy.

Challenges of Connectivity and Budget Constraints

In a region with spotty internet, building a system that handled offline queries was essential. I implemented local caching of common queries and some preprocessing techniques to ensure the app didn't rely on constant internet access.

Results: What Worked and What Didn't

Deploying this setup reduced our search query time from around 2 seconds to under 200 milliseconds. But there were challenges. Budget hardware meant I had to limit the number of threads FAISS could use to avoid maxing out CPU resources.

Offline capabilities were mixed. Local caching worked for common phrases, but less frequent queries still struggled without internet access. This balance is something I continually tweak.

What's Next?

I'm exploring ways to include more efficient data preprocessing steps into the LangChain pipeline to further reduce internet dependency. Additionally, I'm optimizing FAISS to handle larger datasets on limited hardware.

LangChain, FAISS, and Docker together brought significant improvements in managing large-scale document Q&A tasks despite constraints in emerging markets. It's a process of constant iteration and adaptation to our environment's realities.

DEV Community