For multi-hop questions in enterprise RAG you've got two options, and they break in different places. Stay on plain vector retrieval and it works until the answer depends on a middle fact that doesn't sound like the question. Add a knowledge graph and you can follow that fact, but now you're paying for extraction and entity resolution, and the graph only returns what was stored.
I'd start with the first and make the second earn its place with a test.
Where vectors break
Cosine similarity scores topical closeness. That's why vector search is fine for a single-document lookup like a return policy. It can't follow a link from one fragment to the next, the way a foreign key ties two rows together.
Take three facts in separate chunks: CHIP-402 is assembled into Engine-B, Engine-B is installed in Titan-EV, Titan-EV is committed under contract LON-FLEET-26. Ask about the chip's impact on contract penalties and top-k retrieval may return the first and third and miss the middle one. That's a failure mode to test for, not something I've measured. Without that link the model has to bridge the gap itself, and it can invent relationships between entities that are all real.
What the graph changes
The hybrid design I propose keeps vector search. Ingestion also extracts entities and relationships into a property graph. A dependency question finds the root entity, runs a bounded traversal, and the path goes into the prompt with the relevant clauses. The model explains a path it was given. It doesn't infer links from fragments.
It isn't free. Extraction runs over the whole corpus. "Bavaria AutoPart GmbH" and "BAP Logistics" have to resolve to one entity, without gluing together things that are separate. Dense graphs need hard hop limits, 2 or 3, say. And extraction errors or stale records end up in the path.
Before you graph anything
- Build a test set of about 20 questions where the answer needs facts from two or more documents, and score it next to your single-fact questions. Single-fact fine and multi-hop low points at retrieval structure. That's a heuristic, not proof.
- Try query decomposition or iterative retrieval: ask for the chip's vehicles, then those vehicles' contracts. Some multi-hop cases get fixed with no graph at all.
- If that's not enough, pilot a bounded graph in one high-risk domain with dense dependencies. Don't attempt an enterprise-wide graph in one step.
The full article has the diagrams, the sources and the complete checklist: https://abivarma.medium.com/the-polaroid-trap-why-vector-search-fails-at-multi-hop-enterprise-reasoning-507dfeb19664
Top comments (0)