Every growing company hits the same wall eventually. Someone asks a question in Slack that’s already been answered in a PDF from eight months ago, or buried in a page of documentation nobody remembers writing. The information exists. It’s just scattered across enough files that finding it takes longer than re-deriving the answer from scratch.
The usual fix is a paid knowledge base tool with per-seat pricing that scales badly the moment a team grows past a handful of people. There’s a cheaper path that gets less attention: build your own retrieval system on infrastructure you already control, using automation software instead of a subscription product.
Why keyword search stops working
A basic search bar handles small document collections fine. Ten files, no problem. A hundred files, and it starts missing things constantly, not because the answer isn’t there, but because the words don’t match. Someone writes “customer churn” in a report and a colleague searches for “client retention” six months later and gets nothing, even though the paragraph they need is sitting three documents down.
Chat tools built on top of a general-purpose language model don’t fully solve this either. Ask one a question about your internal process and it will either admit it has no idea or, worse, generate something plausible-sounding that has nothing to do with how your company actually operates. Neither failure mode is acceptable when someone is relying on the answer to do their job correctly.
This is the gap that retrieval-augmented generation, usually shortened to RAG, is built to close. Instead of asking a model to answer from memory, you first search a database of your own content for the passages most relevant to the question, then hand those passages to the model along with the question itself. The model still writes the answer, but it’s writing from material it was just shown rather than guessing.
Turning that idea into a working system
Making RAG useful requires more than picking a vector database and calling it done. Content has to be pulled in from wherever it lives, cleaned up, split into pieces small enough to search accurately, converted into vectors, and stored somewhere that can retrieve the right pieces fast when a question comes in. Then there’s the chat side: taking a message, searching for context, remembering what was said earlier in the conversation, and assembling all of that into a single prompt before a model ever sees it.
That’s a lot of moving parts, and it’s exactly the kind of multi-step process that automation platforms like n8n are built to coordinate. Rather than writing custom backend code to glue every service together, n8n handles the orchestration: receiving uploads through webhooks, routing content through processing steps, calling external APIs in sequence, and triggering a ready-made chat interface without anyone having to build a front end.
A well-designed version of this splits the work into separate, independent pipelines rather than one sprawling workflow. One pipeline handles PDFs. Another pulls in content from web URLs, stripping out navigation, footers, and scripts so only the actual page content survives. A third runs the chat agent itself. Keeping them separate means new documents can be indexed at any time without touching or restarting the conversation logic, and each pipeline can be debugged or extended on its own.
Text that clears a basic quality check (enough usable content, not an empty or corrupted extraction) gets broken into overlapping chunks rather than one long block. The overlap matters more than it sounds like it would: without it, a sentence that spans a chunk boundary loses context on one side or the other, and retrieval quality suffers in ways that are hard to notice until you’re staring at a wrong answer wondering why.
Each chunk then gets converted into a vector, a numerical representation of its meaning, using Google’s Gemini embedding model. Those vectors are stored in Qdrant, a vector database built specifically for similarity search at scale, alongside metadata that ties each chunk back to its source document.
Answering a question runs the same pipeline in reverse. The incoming message gets embedded with the same model used during ingestion, Qdrant searches for the closest matching chunks, and a filter discards anything below a similarity threshold so the model isn’t handed irrelevant context just to fill space. Recent conversation history gets pulled in too, so a follow-up question that only makes sense in context of the previous message still gets answered correctly. All of it, instructions, history, retrieved chunks, and the actual question, gets combined into one prompt before Gemini’s chat model generates the response, running at a low temperature so it stays close to the source material rather than improvising.
What actually breaks in practice
A few details separate a demo from something a team can rely on daily. If nothing in the vector database clears the similarity threshold, the system should say so plainly rather than letting the model fall back on general knowledge and produce a confident-sounding guess. A wrong answer that sounds right is worse than an honest “not found in the knowledge base.”
Scanned documents are another common trap. Without OCR built into the pipeline, a photographed or scanned PDF produces no extractable text at all, and it gets rejected rather than silently ingested as an empty, useless entry. That’s a fixable gap, but it’s one worth knowing about before uploading a folder full of old scanned contracts and wondering why none of them show up in answers.
Access to internal systems during content fetching sometimes requires relaxing certificate validation to reach servers running self-signed certificates. That’s a reasonable shortcut while testing against internal documentation, but it’s a genuine security tradeoff that deserves a second look before anything customer-facing goes live.
Where this fits
None of this requires exotic infrastructure. It’s a handful of well-understood components (an automation tool, an embedding model, a vector database, and a document store for history) arranged so that each piece does one job well. IT Path Solutions has built and run implementations along these exact lines, connecting the ingestion and retrieval logic behind building an n8n AI chat agent with RAG into something a team can actually put in front of real documents.
The version that ships as a starting template is intentionally scoped for a single user or account, with no delete function yet and no multi-tenant support. Those are reasonable boundaries for a first version, not flaws, and they mark exactly where a team would extend the system with role-based access, hybrid search that combines keyword and vector matching, or a review step before answers go out. The core idea holds regardless of how far it’s extended: stop making people search. Let them ask.
Top comments (0)