DEV Community: VinayKumar KsheeraSagar

[Boost]

VinayKumar KsheeraSagar — Mon, 21 Sep 2026 13:52:55 +0000

VinayKumar KsheeraSagar

Sep 18

Why AI Agents Need Verifiable Evidence: Building an MCP-Native Retrieval Engine with PostgreSQL

#ai #mcp #python #opensource

3 min read

Hybrid retrieval in one Postgres query: RRF over tsvector + pgvector

VinayKumar KsheeraSagar — Mon, 21 Sep 2026 12:15:58 +0000

Dense vector search is great until your agent asks for parseAuthHeader and gets back three chunks about "authentication token handling" — semantically close, functionally useless. Same story with file paths, error codes, and compliance clause numbers. These are lexical needles, and embeddings blur them.

This isn't a niche complaint. XERJ has been picking up steam on the strength of "stop making agents grep," and Volcengine's OpenViking has ~38k stars for treating agent context as structured, addressable storage rather than a vector dump. Both are good. Both are also new infrastructure you now operate. XERJ in particular already does hybrid BM25 + kNN with RRF — if you're greenfield and happy to run a dedicated engine, genuinely go look at it.

I had a constraint they don't solve for: the evidence had to live in the same transaction as the data it describes, in a database my team already backs up and already knows how to restore at 3am.

The usual fix is to bolt on BM25 from a dedicated search service, then fuse results in application code. That means a second stateful cluster: its own backups, its own failure modes, and no transactional guarantee that your index agrees with your source of truth.

I wanted to know how far Postgres 16 + pgvector could get on its own. Turns out: all the way.

One engine, two retrieval paths

Knowledge Fabric runs full-text search over tsvector and dense search over an HNSW index in the same database, then fuses the two ranked lists with Reciprocal Rank Fusion:

**score = 1 / (60 + rank_lexical) + 1 / (60 + rank_vector)**

RRF only needs ranks, not scores, so you skip the entire problem of normalizing BM25 against cosine similarity. A chunk that places top-3 on both paths wins. A chunk that's #1 lexically and invisible semantically still surfaces — which is exactly what you want when the query is a function name.

One query. One backup. One consistency model.

Evidence you can verify, not just text you hope is right

This part matters more than it sounds. In an agentic setup, retrieved text isn't just context — it's the authorization premise for a state-changing tool call. If the agent reads a policy chunk and then executes a deploy, something needs to prove that chunk wasn't tampered with.

Every chunk gets a deterministic SHA-256 hash and a composite provenance digest, canonicalized per RFC 8785 so byte-level serialization differences don't produce different hashes for identical content. A downstream policy layer can then verify the agent acted on authentic evidence before approving execution.

It's an MCP server

retrieve_evidence, get_document, explain_retrieval — bounded tools over stdio and HTTP via FastMCP. Works with Claude Code, Cursor, or your own harness. explain_retrieval exists because "why did it return that?" is a question you will ask roughly forty times in week one.

Docker Compose quickstart, benchmarks, and the full implementation: https://github.com/sagarv48/knowledge-fabric

If you've tuned RRF in production — did you keep k at 60, or did you find your corpus wanted something different? I'm curious whether the default holds up on codebases with heavy identifier repetition.

Why AI Agents Need Verifiable Evidence: Building an MCP-Native Retrieval Engine with PostgreSQL

VinayKumar KsheeraSagar — Fri, 18 Sep 2026 04:59:50 +0000

Why standard RAG fails agentic workflows, and how we built Knowledge Fabric using PostgreSQL, pgvector, and FastMCP.

Most Retrieval-Augmented Generation (RAG) setups treat context retrieval as a simple black box: prompt in, cosine similarity ranking, top-k text dump out.

In conversational chatbots, a slightly noisy chunk is usually harmless. But in agentic architectures—where models make tool calls, plan execution steps, and trigger real-world actions—retrieval without provenance is a major vulnerability. If an agent cannot verify where an excerpt originated, what version of a document it reflects, or whether the text was modified, policy checks fail and downstream execution becomes unreliable.

To address this, we built Knowledge Fabric: an open-source, vendor-neutral evidence retrieval engine designed natively for the Model Context Protocol (MCP) and built on PostgreSQL with pgvector.

The Problem: Naive RAG vs. Agentic Evidence

When an agent needs context to answer an enterprise query or decide on an action, standard vector stores present three common issues:

Exact-Term Blind Spots: Pure dense vector search often misses exact identifiers (e.g., invoice IDs, function signatures, error codes).
Missing Citations & Provenance: Agents receive text strings without cryptographic hashing, source revision IDs, or bounding offsets.
Bloated Context Windows: Shoveling sprawling context into prompts increases latency and token costs while confusing agent decision boundaries.

Architectural Design: The Three-Tier Fabric

Rather than building an all-in-one monolith that handles planning, retrieval, and write executions simultaneously, we separate concerns into three distinct roles:

Knowledge Fabric (The Librarian): Collects, indexes, searches, and provides verifiable evidence packages with clear citations.
Intent Fabric (The Planner & Referee): Translates user intent and evidence into policy-validated plans.
Enterprise Adapters (The Controlled Doors): Read from external systems and execute approved write mutations.

Workflow:

Storage Engine: Standard Infrastructure First

We deliberately avoided introducing custom daemons or proprietary vector databases. Knowledge Fabric is built directly on PostgreSQL + pgvector:

Hybrid Search with Reciprocal Rank Fusion (RRF): Fuses PostgreSQL BM25-style lexical search (tsvector) with semantic vector embeddings (pgvector) to ensure exact keyword recall alongside conceptual relevance.
Evidence Packaging: Every retrieved passage includes document URI, offset boundaries, chunk hash, and retrieval explanation metadata.
Native FastMCP Interface: Exposes retrieval directly over stdio and HTTP endpoints for Claude Code, Cursor, Codex, and custom agent harnesses.

FastMCP Tool Surface

Knowledge Fabric exposes five core MCP tools:

retrieve_evidence: Performs hybrid search and returns bounded, cited evidence packages.
get_document: Fetches full document content and metadata by ID or URI.
list_sources: Inspects currently indexed collections and document trees.
explain_retrieval: Returns the scoring breakdown (lexical vs. vector RRF ranks) for an evidence query.
health: Confirms database connectivity and vector index status.

3-Minute Quickstart

You can test Knowledge Fabric locally using Docker Compose:

Clone and launch PostgreSQL with pgvector:

git clone https://github.com/sagarv48/knowledge-fabric.git
cd knowledge-fabric
docker compose up -d

Install dependencies & initialize the database:

python -m venv .venv
source .venv/bin/activate
pip install -e .
python -m knowledge_fabric.cli init-db

Run the FastMCP Server: python -m knowledge_fabric.mcp.server

You can now add the server to your Claude Desktop or Cursor MCP configuration:

{
  "mcpServers": {
    "knowledge-fabric": {
      "command": "python",
      "args": ["-m", "knowledge_fabric.mcp.server"],
      "env": {
        "DATABASE_URL": "postgresql://postgres:postgres@localhost:5432/knowledge_fabric"
      }
    }
  }
}

What's Next & Getting Involved

Knowledge Fabric is early and actively developed. We are currently refining local embedding pipelines (Ollama integration) and chunking strategies.

GitHub Repository: https://github.com/sagarv48/knowledge-fabric
Roadmap & Issues: Check out our open issues for connector ideas and hybrid search tuning.

If you are building with MCP or experimenting with agent architectures, I would love to hear how you currently tackle citation and evidence verification in your workflows!