No citation, no claim: building a RAG backend that refuses to hallucinate"

Yasantha Hettiarachchi — Sat, 12 Sep 2026 13:13:25 +0000

A few weeks ago, an AI assistant gave me a confident, well-written answer about one of my own PDFs. It was completely wrong. Nothing in the document said what the model claimed — it had simply filled the gap with something plausible.

That's the quiet failure mode of retrieval-augmented generation (RAG) done casually. It looks grounded because you've stuffed some document chunks into the prompt. But nothing actually forces the model to use them. For a demo, that's fine. For anything you'd put in front of a real user, it isn't.

So I built DocuQuery — a RAG backend with a single rule baked into its core: no citation, no claim. Every answer has to point back to the specific part of the source document it came from, or the system says it doesn't know. It's open source (github.com/yasantha/DocuQuery), and here's how it works and the decisions that actually mattered.

## The core idea

The pipeline is the standard RAG shape:

Ingest & chunk → embed → store in pgvector → retrieve top-k → generate WITH citations → verify grounding

The part most tutorials skip is the last two steps. Retrieving relevant chunks is easy. Making the model speak only from them — and proving that it did — is the actual engineering.

Grounding: no citation, no claim

Two things enforce it.

First, before generation: if the top retrieved chunks fall below a similarity threshold, the system short-circuits and returns "not found" rather than sending weak context to the model. A surprising amount of hallucination is really a retrieval problem in disguise — the search quietly came back with nothing useful, and the model papered over the gap.

Second, at generation: the model is instructed to answer strictly from the supplied chunks and to return the specific chunk IDs it used — not just prose. Every returned ID is then validated against the retrieved set, so the model can't cite something that wasn't actually there. If it can't support the answer from the chunks, it's instructed to say so, which maps to a "not found" response.

The response isn't just text — it carries its receipts:

{
  "answer": "The renewal notice period is 60 days before expiry.",
  "citations": [
    { "document": "vendor-agreement.pdf", "page": 7, "score": 0.89 }
  ],
  "cost": { "embedding": 0.0001, "generation": 0.0021, "cached": false }
}

If a user can't see where an answer came from, they can't trust it. Making citations a first-class field — not an afterthought in the prose — changes how much you can rely on the whole system.

Why PostgreSQL + pgvector

I deliberately didn't reach for a dedicated vector database. For a huge class of applications, pgvector inside PostgreSQL is more than enough — and it's one less piece of infrastructure to run, secure, and back up. Embeddings live in the same database as everything else, with an IVFFlat index for fast nearest-neighbour search. If you already run Postgres, you already have a vector store.

Cloud or fully local — your choice

Vendor lock-in at the model layer is a real risk, so provider choice is a config switch, not a rewrite. DocuQuery runs embeddings and generation through Claude, Gemini, or Ollama (fully local). The same code path gives you a "best quality" cloud preset and a "nothing leaves this machine" local preset. For anyone working with sensitive documents, that local option matters a lot.

Cost, as a first-class concern

RAG gets expensive quietly — every query is an embedding call plus a generation call, and it adds up fast. Two things help:

Semantic caching: if a new question is close enough to one already answered, serve the cached result instead of paying for it again. Each cache entry records which document it was built from, so when a document is re-ingested, the entries tied to it are evicted — no stale answers surviving an update (with a TTL as a backstop).
A cost dashboard: every response records what it cost and whether it was cache-served, so I can see the cloud-vs-local split and actual savings rather than guessing.

You optimise what you measure.

Testing against a real vector store

RAG has a lot of moving parts, so the tests spin up a real pgvector-enabled PostgreSQL via Testcontainers — not a mock. Retrieval and grounding get validated against actual vector search, which is the only reliable way to catch the subtle bugs (wrong distance metric, off-by-one chunking) that fake-backed unit tests sail straight past.

The stack

Java 25 · Spring Boot 3.5 · PostgreSQL + pgvector · Claude / Gemini / Ollama (pluggable) · React + Vite · JUnit 5 + Testcontainers · Docker Compose.

What I'd tell anyone starting a RAG project

The interesting engineering isn't the LLM call — that's a few lines now. It's everything around it: keeping answers honest, making retrieval good enough that the right passages actually surface, controlling cost, and making the whole thing testable. Get those right, and RAG stops being a demo and starts being something you'd ship.

The code is open source — have a look, break it, tell me what you'd do differently:
👉 github.com/yasantha/DocuQuery

I'd genuinely value hearing how others are handling two things in particular: citation quality and cost control in production RAG. How are you approaching grounding in your own builds? Let me know in the comments.

DEV Community: Yasantha Hettiarachchi