Building a Grounded Enterprise Knowledge Base: RAG with Solon AI

Every team eventually hits the same wall with LLMs: the model is confident, fluent, and completely wrong about your internal policies, your product docs, or last quarter's numbers. It was never trained on your data, so it hallucinates plausible-sounding answers. Retrieval-Augmented Generation (RAG) is the standard fix — you retrieve the relevant facts from your own corpus first, then let the model answer grounded in that context.

The problem is that most RAG stacks feel like a pile of loosely-related libraries stitched together: one thing to parse PDFs, another to chunk text, a vector database SDK, an embedding client, and a prompt-assembly layer you write by hand. Solon AI (v4.0.5) collapses that whole pipeline into a small, coherent set of interfaces. This post walks through building a real enterprise knowledge base — ingesting company docs, storing them in pgvector, and answering questions grounded in the retrieved content — using only the built-in APIs.

The pipeline in one picture

A production RAG system has two distinct phases:

1. Ingestion (offline): load raw documents → split into chunks → embed → store in a vector repository.
2. Query (online): embed the question → search for relevant chunks → augment the prompt → let the model answer.

Solon AI models each stage with a dedicated abstraction, and they compose cleanly. Let's build both phases.

Phase 1: Ingesting documents

Enterprise knowledge lives in messy formats — Markdown wikis, HTML pages, PDFs, Word docs, spreadsheets. Solon AI ships a DocumentLoader per format as separate Maven modules (solon-ai-load-markdown, solon-ai-load-html, solon-ai-load-pdf, solon-ai-load-word, solon-ai-load-excel, solon-ai-load-ppt). Each loader turns raw bytes into a List<Document>.

A Document is a simple, fluent value object:

Document doc = new Document("The refund window is 30 days from purchase.")
        .title("Refund Policy")
        .url("https://wiki.acme.com/policies/refund")
        .metadata("department", "support")
        .metadata("version", 3);

The metadata map is the workhorse for enterprise scenarios — you'll use it later to filter by department, product line, or access level.

Loading and splitting

Raw documents are usually too large to embed as a single vector, so they must be chunked. Solon AI provides a SplitterPipeline that chains multiple DocumentSplitter stages. A common, robust combination is: split on structural boundaries first (RegexTextSplitter), then enforce a hard token ceiling (TokenSizeTextSplitter) so no chunk exceeds the embedding model's context window.

import org.noear.solon.ai.rag.Document;
import org.noear.solon.ai.rag.DocumentLoader;
import org.noear.solon.ai.rag.loader.MarkdownLoader;
import org.noear.solon.ai.rag.loader.HtmlSimpleLoader;
import org.noear.solon.ai.rag.splitter.SplitterPipeline;
import org.noear.solon.ai.rag.splitter.RegexTextSplitter;
import org.noear.solon.ai.rag.splitter.TokenSizeTextSplitter;

public List<Document> loadAndSplit(String url) throws IOException {
    String text = HttpUtils.http(url).get();

    DocumentLoader loader;
    if (text.contains("</html>")) {
        loader = new HtmlSimpleLoader(text.getBytes(StandardCharsets.UTF_8));
    } else {
        loader = new MarkdownLoader(text.getBytes(StandardCharsets.UTF_8));
    }

    // Split structurally, then cap each chunk at 500 tokens.
    return new SplitterPipeline()
            .next(new RegexTextSplitter())
            .next(new TokenSizeTextSplitter(500))
            .split(loader.load());
}

TokenSizeTextSplitter is not a naive character-count chopper. Internally it tokenizes with jtokkit (the CL100K_BASE encoding), and when it reaches the chunk boundary it backs up to the last sentence-ending punctuation (., ?, !, or newline) so chunks don't get cut mid-sentence. The default constructor targets 500 tokens per chunk, which is a sensible starting point for most embedding models.

Phase 2: Storing in pgvector

For the vector store, this example uses PostgreSQL with the pgvector extension — a pragmatic choice for teams that already run Postgres and don't want to operate a separate vector database. Solon AI supports many backends (Redis, Elasticsearch, Milvus, Qdrant, Chroma, Weaviate, MariaDB, MySQL, and more), and they all implement the same RepositoryStorable interface, so swapping backends later is a one-line change.

You build a PgVectorRepository from two things: an EmbeddingModel and a JDBC DataSource.

import org.noear.solon.ai.embedding.EmbeddingModel;
import org.noear.solon.ai.rag.repository.PgVectorRepository;
import org.noear.solon.ai.rag.repository.pgvector.MetadataField;

EmbeddingModel embeddingModel = EmbeddingModel.of("http://127.0.0.1:11434/api/embed")
        .provider("ollama")
        .model("bge-m3")
        .build();

// Promote hot metadata keys to real indexed columns.
List<MetadataField> metadataFields = new ArrayList<>();
metadataFields.add(MetadataField.text("department"));
metadataFields.add(MetadataField.text("title"));
metadataFields.add(MetadataField.numeric("version"));

PgVectorRepository repository = PgVectorRepository.builder(embeddingModel, dataSource)
        .tableName("acme_knowledge")
        .metadataFields(metadataFields)
        .build();

The build() call does the DDL heavy-lifting for you. On first initialization it runs CREATE EXTENSION IF NOT EXISTS vector, creates the table with an embedding VECTOR(n) column sized to your embedding model's dimensions(), stores the full metadata map as JSONB, and creates an ivfflat cosine-distance index. The metadataFields you declare get promoted to dedicated typed columns (TEXT / NUMERIC / JSONB) so you can filter on them efficiently instead of digging through JSON on every query.

Saving is then trivial — the repository handles batching, embedding, and upserts internally:

List<Document> docs = loadAndSplit("https://wiki.acme.com/policies/refund.md");
repository.save(docs);

Under the hood save partitions the documents according to the embedding model's batchSize(), calls the embedding model per batch, and inserts with ON CONFLICT (id) DO UPDATE — so re-ingesting an updated document overwrites the old vector rather than duplicating it. For large corpora, there's an overload that reports progress:

repository.save(docs, (done, total) ->
        System.out.printf("Embedded batch %d / %d%n", done, total));

And an async variant, asyncSave(docs, progressCallback), that returns a CompletableFuture<Void> if you want to kick ingestion off the request thread.

Phase 3: Querying with grounding

Now the online path. The simplest retrieval is a bare query string, which uses sensible defaults (top 4 results, similarity threshold 0.4):

List<Document> hits = repository.search("How long is the refund window?");

For real applications you'll want control, and that's what QueryCondition gives you. This is where enterprise metadata filtering pays off — you can restrict retrieval to a specific department while still ranking by semantic similarity:

import org.noear.solon.ai.rag.util.QueryCondition;

QueryCondition condition = new QueryCondition("How long is the refund window?")
        .limit(5)
        .similarityThreshold(0.5)
        .filterExpression("department == 'support' AND version >= 2");

List<Document> hits = repository.search(condition);

The filterExpression is parsed by Solon's built-in expression engine (SnEL) and pushed down to the store — for pgvector it becomes a SQL WHERE clause against those promoted metadata columns, so the filter runs in the database, not in your JVM after the fact. Each returned Document carries a getScore() reflecting its similarity to the query, so you can inspect or threshold results yourself.

Assembling the grounded prompt

Retrieval only gets you the facts; you still have to feed them to the model. Repository has a convenience method, promptAugment, that does the search and packs the results into a user message:

import org.noear.solon.ai.chat.ChatModel;
import org.noear.solon.ai.chat.message.ChatMessage;

ChatModel chatModel = ChatModel.of("http://127.0.0.1:11434/api/chat")
        .provider("ollama")
        .model("qwen2.5")
        .build();

ChatMessage grounded = repository.promptAugment("How long is the refund window?");

String answer = chatModel.prompt(grounded).call().getMessage().getContent();

promptAugment builds on ChatMessage.ofUserAugment, which formats the original question together with the current timestamp and the retrieved references into a single user message. The model now answers from the supplied context instead of its training-time guesses.

Going further: Agentic RAG

The pattern above is passive retrieval — you decide what to search for, once, before the model runs. But some questions need the model to decide what to look up, and to look up several things. Solon AI supports this with RepositoryTool, which wraps a repository as a callable tool the model can invoke on its own:

import org.noear.solon.ai.rag.RepositoryTool;

ChatModel agent = ChatModel.of("http://127.0.0.1:11434/api/chat")
        .provider("ollama")
        .model("qwen2.5")
        .defaultToolAdd(new RepositoryTool(repository))
        .build();

// The model can now call `repository_query` itself, with multiple search
// terms, whenever it decides it needs background knowledge.
String answer = agent.prompt("Compare our refund and exchange policies, and note any version differences.")
        .call().getMessage().getContent();

RepositoryTool exposes a repository_query tool that accepts a list of search terms (up to five) plus a topK, runs each search, and formats the merged results back to the model. This shifts the system from "passive retrieval" to "active knowledge-seeking" — the model can break a complex question into several targeted lookups. If you have a reranking model, you can pass it as a second constructor argument (new RepositoryTool(repository, rerankingModel)) to reorder hits by relevance before they reach the model.

Why this composes well

The thing worth noticing is that every stage is an interface, and swapping an implementation never touches the rest of your code:

• Switch from pgvector to Redis or Milvus? Change the Repository construction line. save, search, and promptAugment are identical because they're defined on RepositoryStorable / Repository.
• Add PDF ingestion? Add the solon-ai-load-pdf dependency and swap the loader. The splitter and repository don't care where the Document came from.
• Move from passive to agentic retrieval? Wrap the same repository in a RepositoryTool.

That uniformity is the real payoff. RAG stops being a bespoke integration project and becomes a matter of picking implementations off a shelf, all speaking the same small vocabulary of Document, Repository, and QueryCondition.

If you want to explore the full set of loaders, splitters, and vector-store backends, the source and docs live at solon.noear.org.