Building a Customer Support AI Chatbot from a Knowledge Base

Building a Customer Support AI Chatbot from a Knowledge Base

To build a customer support chatbot that answers questions using your company's proprietary knowledge base, use a Retrieval-Augmented Generation (RAG) architecture. This approach retrieves relevant documents from your database first, then feeds them to a Large Language Model (LLM) as context to generate an accurate, grounded response.

Architecture Overview
The system consists of three main components:

Knowledge Base: Your raw support documents, FAQs, or markdown files.
Vector Database: A database (like ChromaDB or Pinecone) that stores vector embeddings of your document chunks to enable semantic search.
LLM (OpenAI GPT): The generative model that synthesizes the retrieved context into a natural, conversational support response.

Step 1: Environment Setup
Install the required Python libraries. We will use chromadb as our vector database and openai for embeddings and text generation.

pip install openai chromadb

Step 2: Prepare and Index the Knowledge Base
This script reads your support documents, generates vector embeddings using OpenAI's text-embedding-3-small model, and stores them in a local ChromaDB collection.

import chromadb
from openai import OpenAI

Initialize clients

client = OpenAI(api_key="YOUR_OPENAI_API_KEY")
chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection(name="kb_collection")

Sample knowledge base data

knowledge_base = [
{"id": "kb_01", "text": "To request a refund, navigate to Settings > Billing and click 'Request Refund'. Refunds take 5-10 business days to process."},
{"id": "kb_02", "text": "Our support hours are Monday through Friday, 9:00 AM to 5:00 PM EST. We are closed on weekends and major holidays."},
{"id": "kb_03", "text": "The Basic plan costs $19/month. The Pro plan costs $49/month and includes API access and priority support."}
]

def get_embedding(text):
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding

Index documents

for doc in knowledge_base:
embedding = get_embedding(doc["text"])
collection.add(
embeddings=[embedding],
documents=[doc["text"]],
ids=[doc["id"]]
)
print("Knowledge base indexed successfully.")

Step 3: Build the Retrieval and Generation Pipeline
This function processes user queries, retrieves the most relevant document chunk from the vector database, and uses GPT-4o-mini to draft the final response based strictly on that context.

def query_chatbot(user_query):
# 1. Generate embedding for the user query
query_vector = get_embedding(user_query)

# 2. Query the vector database for the closest match
results = collection.query(
    query_embeddings=[query_vector],
    n_results=1
)

# Extract the retrieved context
if results['documents'] and len(results['documents'][0]) > 0:
    context = results['documents'][0][0]
else:
    context = "No relevant information found."

# 3. Construct the prompt with system instructions and retrieved context
system_prompt = (
    "You are a precise customer support assistant. Answer the user's question using ONLY the provided context. "
    "If the context does not contain the answer, say 'I am sorry, but I do not have that information in my knowledge base.' "
    "Do not make up or assume any facts.\n\n"
    f"Context:\n{context}"
)

# 4. Generate response from the LLM
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_query}
    ],
    temperature=0.0  # Kept at 0.0 to minimize creative hallucinations
)

return response.choices[0].message.content

Example Execution

user_question = "How long does a refund take?"
answer = query_chatbot(user_question)
print(f"Q: {user_question}\nA: {answer}")

Limitations and Production Considerations

Hallucinations: Setting LLM temperature to 0.0 and using strict system prompts minimizes, but does not completely eliminate, the risk of the model generating incorrect information.
Context Window Limits: If your documents are large, you must implement a text-splitting strategy (e.g., recursive character splitting) to chunk documents into manageable sizes (e.g., 500-character segments) before indexing.
Data Privacy: Sending queries and document chunks to OpenAI transmits data to external servers. If you handle highly sensitive or regulated data, consider self-hosting an open-source model (like Llama 3) and running embeddings locally.

Need this done fast? order it on Kwork.