I built SageBox to find out whether I could create a useful AI search and chat system without sending documents or questions to a cloud AI provider.
SageBox runs entirely on a Beelink SER9 MAX mini PC with 64 GB of memory. It searches private document collections and answers questions using retrieval-augmented generation.
How SageBox Works
Documents are divided into smaller chunks and converted into vector embeddings. PostgreSQL stores the document content and embeddings, while the pgvector extension provides vector similarity search.
When a user submits a question, an n8n workflow creates an embedding for it and searches the database for relevant passages. Those passages are added to a prompt and sent to a local AI model, which generates an answer based on the retrieved information.
Improving Retrieval Quality
Basic vector similarity does not always provide enough context for an accurate answer. SageBox uses hybrid retrieval that combines vector similarity with lexical search.
The system can also retrieve chunks located near the strongest result. This is useful when a relevant explanation extends across multiple sections of a document.
Each knowledge base has its own configuration, including the AI model, system prompt, retrieval strategy, similarity threshold, result limits, and temperature.
Building the User Interface
I created a custom WordPress interface that allows users to choose a knowledge base, enter a question, and receive an answer through a chat-style page.
The WordPress plugin sends the request to a production n8n webhook. The workflow handles authorization, loads the knowledge-base configuration, performs retrieval, builds the prompt, calls the local model, and returns the completed answer.
Why Local AI Matters
A local system gives an organization greater control over its documents, embeddings, database, and AI processing. It can be useful when information is confidential, proprietary, or unsuitable for external AI services.
Building SageBox required work across local AI models, workflow automation, PostgreSQL, pgvector, SQL retrieval, prompt design, Ubuntu Server, and WordPress development.
As I transition from technical writing into AI consulting, SageBox demonstrates the kind of practical and private AI system I can build for organizations that want to search and use their own information securely.
Read the complete article:
https://aitransformer.online/how-i-built-sagebox-my-local-ai-search-and-chat-rag-system/
Top comments (0)