DEV Community: Datta sai krishna Naidu

Local-first multi-hop RAG: Chroma + an entity graph, zero tokens per query

Datta sai krishna Naidu — Wed, 05 Aug 2026 01:46:35 +0000

Chroma's sweet spot is local-first: embed your documents, query them
on your own machine, no infrastructure ceremony. But local-first RAG
hits a wall on multi-hop questions:

"Which university did the founder of the company that acquired Polar
Metrics study at?"

The answering passage shares almost no vocabulary with the question.
It's connected to it — through an acquisition, a founder, a biography
— and similarity search can't follow connections. The usual fix is to
put an LLM in the retrieval loop to decompose the question, which
breaks exactly what makes local-first attractive: now every query
costs tokens, takes seconds, and returns something different each run.

hubmesh takes the other road:
keep retrieval as pure math. At index time it builds an entity–document
graph from your corpus with spaCy NER (no LLM, no tokens). At query
time it runs Personalized PageRank from the question's entities over
that graph and fuses the result with Chroma's cosine scores. The whole
query path is numpy and scipy: ~100ms on a 30K-doc corpus, offline,
and bit-identical across runs — three properties an LLM-in-the-loop
retriever cannot offer at any price.

Setup

pip install "hubmesh[chroma,kg]"
python -m spacy download en_core_web_sm

Index

from hubmesh import Planner
from hubmesh.adapters import ChromaStore
from hubmesh.kg import build_entity_kg
import spacy

embed = ...  # your embedding callable

store = ChromaStore.from_documents(docs)                       # ephemeral
# store = ChromaStore.from_documents(docs, persist_directory="./chroma")
# store = ChromaStore.from_documents(docs, host="localhost", port=8000)

nlp = spacy.load("en_core_web_sm")
kg = build_entity_kg(store.get_many(store.all_ids()), nlp=nlp)
planner = Planner(store=store, kg=kg, nlp=nlp, embed=embed)

Query

result = planner.retrieve("Which university did the founder of the "
                          "company that acquired Polar Metrics study at?",
                          top_k=10)
print([s.doc.id for s in result.sources][:3])
for path in result.reasoning:      # the graph route, not a rationalization
    print(" -> ".join(path.node_ids))

Why it works: intersection, not just proximity

The scoring composite has three parts — cosine relevance, PPR
diffusion, and a convergence term that scores each document by the
geometric mean of diffusion from every question entity separately. A
multi-hop answer sits at the intersection of the question's anchors;
pooled similarity computes a union. Intersections are where bridge
documents live. (The formula lineage is a network-topology paper,
NNSI, ICOMP'25 — same idea, different graph.)

Numbers

Full HotpotQA dev (7,405 questions): 75.2% supporting-fact recall@10
vs 69.3% naive cosine over identical embeddings. MuSiQue 2/3/4-hop:
+6.0/+3.2/+5.0 points. Honest trade: recall@2 dips 0.75 pts under the
convergence term (flag documented to disable for top-2 workloads).
Harness and raw JSONs ship in the repo.

The agent story

pip install "hubmesh[mcp]" adds an MCP server (in the official MCP
Registry as io.github.DemigodDSK/hubmesh) with agent-steerable
retrieval: pass seed_entities to aim the next hop at what you just
read, exclude_docs to explore new ground. Local Chroma + local graph

your agent's reasoning: multi-hop RAG where the only LLM in the system is the one you already run.

Repo: github.com/DemigodDSK/hubmesh · MIT · numbers reproducible via benchmarks/

Multi-hop questions break vector search. Here is a graph layer for Qdrant that fixes them.

Datta sai krishna Naidu — Wed, 05 Aug 2026 01:45:45 +0000

Qdrant answers "which vectors are near this query?" in milliseconds at
billion scale. But there's a class of questions where nearness is the
wrong criterion entirely:

"Where did the founder of the company that acquired Slack study?"

The passage that answers this talks about Marc Benioff and USC. It
never mentions Slack. Cosine similarity — any similarity — ranks it
low, because the answer doesn't look like the question. It's
connected to the question, three entity-hops away: Slack → acquired
by Salesforce → founded by Benioff → studied at USC. That's a topology
problem, and no amount of ANN speed solves a topology problem.

hubmesh is a small MIT
library that adds the topology layer on top of your existing Qdrant
collection. Qdrant keeps doing what it's best at (first-pass ANN);
hubmesh builds an entity–document graph at index time and, at query
time, diffuses Personalized PageRank from the question's entities over
that graph, fusing graph reachability with your cosine scores. No LLM
is involved at query time — retrieval is one sparse matrix iteration,
deterministic, roughly 100ms on a 30K-document corpus.

Setup

pip install "hubmesh[qdrant,kg]"
python -m spacy download en_core_web_sm

Index: your Qdrant collection + an entity graph

from hubmesh import Planner
from hubmesh.adapters import QdrantStore
from hubmesh.kg import build_entity_kg
import spacy

embed = ...  # your embedding callable: text -> np.ndarray

# any of: in-memory, on-disk, or your running Qdrant server
store = QdrantStore.from_documents(docs, url="http://localhost:6333")

# entity-document graph via spaCy NER — zero LLM tokens to build
nlp = spacy.load("en_core_web_sm")
kg = build_entity_kg(store.get_many(store.all_ids()), nlp=nlp)

planner = Planner(store=store, kg=kg, nlp=nlp, embed=embed)

Query: diffusion + similarity, fused

result = planner.retrieve(
    "Where did the founder of the company that acquired Slack study?",
    top_k=10,
)
for path in result.reasoning:
    print(f"{path.score:.3f}  " + " -> ".join(path.node_ids))
# 0.031  ent:slack -> doc:acquisition -> ent:salesforce -> doc:benioff_bio

That reasoning field is not a post-hoc explanation — it's the actual
graph route that surfaced each document, which means your RAG pipeline
can show why a passage was retrieved.

What the scoring actually is

Each candidate document gets a composite of three signals, normalized
and combined (the formula descends from a network-topology paper —
NNSI, ICOMP'25 — where the same lesson appeared: no single centrality
metric identifies important nodes, but a weighted composite does):

Relevance — cosine against your Qdrant vectors (geometry)
Structure — Personalized PageRank mass diffused from the question's entities (topology)
Convergence — the geometric mean of per-entity diffusion, so a document must be reachable from every anchor in the question, not just flooded with score from one

Numbers (harness and raw JSONs in the repo)

Benchmark	recall@10 vs naive cosine, same embeddings
HotpotQA full dev (7,405 q)	75.2% vs 69.3% (+5.9 pts)
MuSiQue 2/3/4-hop	+6.0 / +3.2 / +5.0 pts
vs HippoRAG-style PPR-only, same graph	+29.8 pts

Disclosed trade-off: the convergence term buys depth recall with
top-rank precision — recall@2 is 0.75 pts below naive on full dev. If
you retrieve with top_k=2, disable it (use_convergence=False).
Everything above reproduces with the scripts in benchmarks/.

Agents can steer it

retrieve accepts seed_entities and exclude_docs, so an agent can
iterate: retrieve, read, then aim hop two at the entity it just
discovered. There's an MCP server included (hubmesh-mcp, listed in
the official MCP Registry) — the repo contains a field report of
Perplexity driving a 3-hop chain through it, tool call by tool call.

When NOT to use this

Single-hop corpora where similarity already wins; corpus-wide summary
questions ("what are the main themes?") — that's community-summary
territory (Microsoft GraphRAG's use case), a different query class.
hubmesh is for multi-hop factual retrieval, and it deliberately keeps
Qdrant as the geometry engine underneath.

Repo: github.com/DemigodDSK/hubmesh · PyPI: pip install hubmesh · MIT