Local-first multi-hop RAG: Chroma + an entity graph, zero tokens per query

#python #rag #ai #opensource

Chroma's sweet spot is local-first: embed your documents, query them
on your own machine, no infrastructure ceremony. But local-first RAG
hits a wall on multi-hop questions:

"Which university did the founder of the company that acquired Polar
Metrics study at?"

The answering passage shares almost no vocabulary with the question.
It's connected to it — through an acquisition, a founder, a biography
— and similarity search can't follow connections. The usual fix is to
put an LLM in the retrieval loop to decompose the question, which
breaks exactly what makes local-first attractive: now every query
costs tokens, takes seconds, and returns something different each run.

hubmesh takes the other road:
keep retrieval as pure math. At index time it builds an entity–document
graph from your corpus with spaCy NER (no LLM, no tokens). At query
time it runs Personalized PageRank from the question's entities over
that graph and fuses the result with Chroma's cosine scores. The whole
query path is numpy and scipy: ~100ms on a 30K-doc corpus, offline,
and bit-identical across runs — three properties an LLM-in-the-loop
retriever cannot offer at any price.

Setup

pip install "hubmesh[chroma,kg]"
python -m spacy download en_core_web_sm

Index

from hubmesh import Planner
from hubmesh.adapters import ChromaStore
from hubmesh.kg import build_entity_kg
import spacy

embed = ...  # your embedding callable

store = ChromaStore.from_documents(docs)                       # ephemeral
# store = ChromaStore.from_documents(docs, persist_directory="./chroma")
# store = ChromaStore.from_documents(docs, host="localhost", port=8000)

nlp = spacy.load("en_core_web_sm")
kg = build_entity_kg(store.get_many(store.all_ids()), nlp=nlp)
planner = Planner(store=store, kg=kg, nlp=nlp, embed=embed)

Query

result = planner.retrieve("Which university did the founder of the "
                          "company that acquired Polar Metrics study at?",
                          top_k=10)
print([s.doc.id for s in result.sources][:3])
for path in result.reasoning:      # the graph route, not a rationalization
    print(" -> ".join(path.node_ids))

Why it works: intersection, not just proximity

The scoring composite has three parts — cosine relevance, PPR
diffusion, and a convergence term that scores each document by the
geometric mean of diffusion from every question entity separately. A
multi-hop answer sits at the intersection of the question's anchors;
pooled similarity computes a union. Intersections are where bridge
documents live. (The formula lineage is a network-topology paper,
NNSI, ICOMP'25 — same idea, different graph.)

Numbers

Full HotpotQA dev (7,405 questions): 75.2% supporting-fact recall@10
vs 69.3% naive cosine over identical embeddings. MuSiQue 2/3/4-hop:
+6.0/+3.2/+5.0 points. Honest trade: recall@2 dips 0.75 pts under the
convergence term (flag documented to disable for top-2 workloads).
Harness and raw JSONs ship in the repo.

The agent story

pip install "hubmesh[mcp]" adds an MCP server (in the official MCP
Registry as io.github.DemigodDSK/hubmesh) with agent-steerable
retrieval: pass seed_entities to aim the next hop at what you just
read, exclude_docs to explore new ground. Local Chroma + local graph