Build an Enterprise Knowledge Base with Open WebUI + APIVAI (RAG)

Build an enterprise knowledge base with Open WebUI + APIVAI

You can stand up a private, self-hosted "chat with our documents" knowledge base without writing a
RAG pipeline from scratch. Open WebUI has built-in document upload and
retrieval; APIVAI supplies the language model (Claude or GPT) over one OpenAI-compatible
connection, at a fraction of official price. Your documents stay on your own server; only the
model calls go out.

This guide covers the setup and the design choices that matter.

Why this stack

• Self-hosted UI + storage — documents and chat history live on infrastructure you control.
• No custom RAG code — Open WebUI handles upload, chunking, embedding, and retrieval.
• One model connection — APIVAI gives you Claude and GPT models behind a single key.
• Cost control — pay-as-you-go at a fraction of list; crypto/USDT/Alipay accepted.

1. Run Open WebUI and connect APIVAI

Start Open WebUI (Docker), then add APIVAI as an OpenAI connection (Admin Panel → Settings →
Connections → OpenAI API):

• API Base URL: https://api.apivai.com/v1
• API Key: your APIVAI key

Confirm models load in the chat model selector. (See the dedicated "Connect Open WebUI to APIVAI"
guide for screenshots and troubleshooting.)

2. Add your knowledge base

In Open WebUI:

1. Go to Workspace → Knowledge (or Documents).
2. Create a collection and upload your files (PDF, Markdown, DOCX, TXT).
3. Open WebUI chunks and embeds them automatically.
4. In a chat, reference the collection with # (e.g. #company-handbook) so the model answers from those documents.

3. Choose the answering model

• Claude Sonnet is a strong default for knowledge-base Q&A: large context, faithful summarization, careful answers over long source material.
• GPT-5.5 is great when answers should be fast and conversational, or multilingual.

Set the model per chat or as the workspace default. Because it's OpenAI-compatible, switching is
just picking a different model name.

4. Make answers trustworthy

• Cite sources: instruct the model (system prompt) to quote or reference the document section it used, and to say "not in the documents" when it isn't.
• Scope retrieval: keep collections focused (one per domain: HR, product, support) so retrieval stays relevant.
• Access control: use Open WebUI's user/group permissions so teams only see their collections.
• Keep it fresh: re-upload documents when they change; remove stale ones.

5. Embeddings note

Open WebUI can compute embeddings locally (default) or via an external embeddings endpoint. APIVAI
focuses on the chat/completions side (Claude and GPT), so keep Open WebUI's built-in/local
embeddings for retrieval and use APIVAI for the answering model — a clean, low-cost split.

Example use cases

• Internal HR/IT helpdesk over policy docs.
• Product/support team answering from manuals and past tickets.
• Sales enablement over spec sheets and pricing.

FAQ

Where do my documents live? On your Open WebUI server — they are not sent to APIVAI; only the
chat prompt (with retrieved snippets) is.

Which model for a knowledge base? Claude Sonnet for faithful long-context answers; GPT-5.5 for
fast multilingual replies.

Do I need to build embeddings/RAG myself? No — Open WebUI handles it; APIVAI is just the
answering model.

Get started

Run Open WebUI, connect APIVAI (https://api.apivai.com/v1 + key), upload your docs, and pick a
model. Examples: APIVAI examples repo.