This guide is written for developers with basic Python knowledge. It walks you through building a privately deployable personal knowledge base system step by step — upload documents, auto-vectorize, and get intelligent Q&A — all in under 200 lines of code.
Background: Why You Need a Knowledge Base That "Talks"
Have you ever run into these situations:
- You've bookmarked hundreds of technical articles but can never find them when you need them
- Internal company documents keep piling up, and onboarding relies on word of mouth
- Your own notes are scattered everywhere, and searching by keywords is pure luck The problem with traditional search is that it matches keywords rather than semantics. When you ask "how to optimize slow queries," it doesn't realize this is the same thing as "SQL performance tuning." The LLM-era solution is RAG (Retrieval-Augmented Generation): first use vector similarity to find the most relevant content, then let the LLM answer questions based on that content. The results far surpass keyword search and remain fully controllable — hallucination issues are dramatically reduced. In this article, we'll build exactly that system.
Overall Architecture
Document Input (PDF / TXT / MD)
v
Text Chunking
v
Vectorization (Embedding API)
v
Store in Vector DB (ChromaDB)
v
User Query -> Retrieve relevant chunks -> Assemble Prompt -> LLM generates answer
Tech stack:
- Vectorization & Q&A model: call text-embedding-3-small + gpt-4o via WRouter
- Vector database: ChromaDB (locally deployed, zero config)
- Document parsing: LangChain Document Loaders
Environment Setup
bash
pip install openai chromadb langchain langchain-community tiktoken pypdf
WRouter is compatible with the OpenAI SDK — just replace the base_url and you're done; no other code changes are needed:
python
from openai import OpenAI
client = OpenAI(api_key="Your WRouter API Key", base_url="https://www.wrouter.ai/v1")
WRouter supports mainstream models like GPT-4o, Claude 3.5, and Gemini, all managed with a single Key — no need to register separate accounts with each provider. Sign up at: www.wrouter.ai
Step 1: Document Loading and Chunking
python
from langchain_community.document_loaders import PyPDFLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
import os
def load_documents(file_path: str):
"""Supports PDF and plain text"""
ext = os.path.splitext(file_path)[-1].lower()
if ext == ".pdf":
loader = PyPDFLoader(file_path)
else:
loader = TextLoader(file_path, encoding="utf-8")
return loader.load()
def split_documents(docs, chunk_size=500, chunk_overlap=50):
splitter = RecursiveCharacterTextSplitter(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
separators=["\n\n", "\n", "。", "！", "？", " ", ""]
)
return splitter.split_documents(docs)
How to choose chunk_size?
- Too small (< 200 chars): insufficient context, the LLM can't answer complete questions
- Too large (> 1000 chars): retrieval accuracy drops, and more tokens are consumed
- Recommended 400–600 chars, keep ~10% overlap to prevent semantic breaks
Step 2: Vectorize and Store in ChromaDB
python
import chromadb
from openai import OpenAI
client = OpenAI(api_key="Your WRouter API Key", base_url="https://www.wrouter.ai/v1")
# Initialize ChromaDB (data persists locally)
chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection(
name="knowledge_base",
metadata={"hnsw:space": "cosine"} # use cosine similarity
)
def embed_texts(texts: list[str]) -> list[list[float]]:
"""Batch vectorize, up to 2048 items per call"""
response = client.embeddings.create(
model="text-embedding-3-small",
input=texts
)
return [item.embedding for item in response.data]
def index_documents(chunks):
"""Vectorize document chunks and store them in the database"""
batch_size = 100 # process 100 items per batch
for i in range(0, len(chunks), batch_size):
batch = chunks[i:i + batch_size]
texts = [chunk.page_content for chunk in batch]
metadatas = [chunk.metadata for chunk in batch]
ids = [f"chunk_{i + j}" for j in range(len(batch))]
embeddings = embed_texts(texts)
collection.add(
embeddings=embeddings,
documents=texts,
metadatas=metadatas,
ids=ids
)
print(f"Processed {min(i + batch_size, len(chunks))}/{len(chunks)} chunks")
# Usage example
docs = load_documents("your_document.pdf")
chunks = split_documents(docs)
index_documents(chunks)
print(f"Indexing complete, {len(chunks)} document chunks total")
Step 3: Retrieval + Q&A
python
def retrieve(query: str, top_k: int = 5) -> list[str]:
"""Retrieve the document chunks most relevant to the question"""
query_embedding = embed_texts([query])[0]
results = collection.query(
query_embeddings=[query_embedding],
n_results=top_k,
include=["documents", "metadatas", "distances"]
)
# Filter out results with too-low similarity (cosine distance > 0.5 means weak relevance)
docs_with_scores = zip(
results["documents"][0],
results["distances"][0]
)
return [doc for doc, dist in docs_with_scores if dist < 0.5]
def answer(query: str) -> str:
"""Main RAG Q&A function"""
# 1. Retrieve relevant documents
relevant_chunks = retrieve(query)
if not relevant_chunks:
return "Sorry, no relevant content was found in the knowledge base for this question."
# 2. Assemble context
context = "\n\n---\n\n".join(relevant_chunks)
# 3. Construct the prompt
system_prompt = """You are an assistant that answers questions based on a knowledge base. Please strictly answer based on the context provided below, and do not fabricate information. If the context does not contain enough information, clearly tell the user."""
user_prompt = f"""Context: {context}
Question: {query}
Please answer based on the above content."""
# 4. Call the LLM
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
],
temperature=0.3 # low temperature recommended for knowledge-base Q&A
)
return response.choices[0].message.content
Step 4: Add a Simple Command-Line Interface
python
def main():
print("📚 Personal knowledge base ready. Enter a question to query (type quit to exit)\n")
while True:
query = input("You: ").strip()
if query.lower() in ("quit", "exit", "q"):
break
if not query:
continue
print("\nAI: ", end="", flush=True)
response = answer(query)
print(response)
print()
if __name__ == "__main__":
# Build the index on first run:
# docs = load_documents("your_document.pdf")
# chunks = split_documents(docs)
# index_documents(chunks)
main()
Demo
Using a 50-page technical document as an example, indexing takes about 8 seconds, and vectorization cost is about ¥0.02 (using WRouter to call text-embedding-3-small).
Q&A example:
You: What performance optimization approaches are mentioned in this document?
AI: Based on the document content, the following performance optimization approaches are mentioned:
1. Database layer: it is recommended to build composite indexes on high-frequency query fields and enable query caching...
2. Application layer: use asynchronous processing to reduce main-thread blocking...
3. Network layer: enable HTTP/2 and configure a sensible CDN strategy...
(Source: page 23, page 31)
Advanced Directions
- Multi-document management: create a separate Collection for each document, supporting cross-library retrieval
- Hybrid retrieval: combine vector retrieval + BM25 keyword retrieval to complement each other's strengths
- Conversation memory: maintain multi-turn conversation history to support follow-up questions
- Switch to stronger models: swap gpt-4o for claude-3-5-sonnet to handle complex reasoning tasks — with WRouter it's a one-line change
Summary
The core code of the entire system is fewer than 150 lines, yet it delivers:
- Automatic document parsing and chunking
- Semantic vectorization with persistent storage
- A closed loop of similarity retrieval + LLM Q&A For API access, we recommend API Gateway like WRouter（www.wrouter.ai）、Openrouter（www.openrouter.ai）. It's compatible with the OpenAI SDK and supports unified calls to models like GPT-4o, Claude, and Gemini — ideal for individual developers and small teams to get started quickly, without registering separate accounts or managing individual Keys for each model.
Top comments (0)