Vector search is fast. It's simple to set up. And for document search, it keeps giving you the wrong answer.
We ran into this repeatedly while building at Brainfish. Agents querying contracts, policy documents, technical specs — documents where structure and context matter. The queries looked reasonable. The retrieved chunks looked plausible. But the answers were wrong.
Not hallucinated-wrong. Wrong-context wrong. The right words, from the wrong place in the document.
That pushed us to build something different: Hierarchical Reasoning Retrieval (HRR).
The Problem With Flat Retrieval
Standard RAG works like this:
- Split the document into chunks
- Embed each chunk as a vector
- At query time, find the top-k chunks by cosine similarity
- Pass them to the LLM
The flaw is step 3. Similarity search finds text that sounds like your query. It doesn't find text that answers your query.
A contract's termination clause isn't semantically similar to "what are the exit conditions?" — but it's the answer. A drug interaction warning buried in section 7.3.2 doesn't look like your question — but it's exactly what the agent needs to surface.
Flat chunking destroys the structure that makes the answer findable. By the time the LLM sees the input, the document hierarchy is gone.
Documents Have Structure. Use It.
Every real document has hierarchy. Contracts have parts, sections, clauses, sub-clauses. Technical specs have chapters, sections, notes, examples. Policies have rules, exceptions, references.
That hierarchy exists precisely because humans use it to navigate. When a lawyer searches a contract they've never seen before, they don't read it linearly — they scan the table of contents, go to the relevant section, and drill into the specific clause.
HRR gives agents the same capability.
How HRR Works
Ingestion
When a document is ingested into ReasonDB, we build a tree from its structure. Headings, sections, and subsections become nodes. Leaf nodes contain the actual content.
Then we summarize bottom-up: leaf summaries roll up into section summaries, section summaries roll up into chapter summaries, all the way to the root. Every parent node knows what its children contain.
Retrieval
At query time, the agent doesn't search embeddings. It traverses the tree:
- Read the root summary — which top-level branches are relevant to this query?
- Traverse into relevant branches — read section summaries
- Drill into leaf nodes where the answer actually lives
- Return the exact passage with its full path and a confidence score
The agent reasons its way to the answer the same way a domain expert would — starting at the summary level and drilling down.
What you get back
- The precise answer, not a ranked list of chunks
- The full context of where it sits in the document
- A traceable reasoning path (which branches were explored, which were skipped)
- A confidence score based on structural fit, not similarity score
Before and After
Here's a concrete example. Query: "What are the late payment penalties?"
Vector search result:
"Payments are due within 30 days of invoice date. The company reserves the right to suspend services for non-payment."
Semantically related. Not the answer.
HRR result:
"Section 4.3.2 — Late Payment: Invoices unpaid after 30 days accrue interest at 1.5% per month. After 60 days, the vendor may suspend delivery and refer the account to collections."
Exactly the clause. Full context. Reasoning path shows it traversed: Contract Root → Financial Terms → Payment Conditions → Late Payment.
The Technical Stack
ReasonDB is written in Rust. HRR is implemented on top of:
- redb — embedded ACID-compliant storage for the document tree
- tantivy — BM25 full-text search for candidate pre-filtering
- tokio — async parallel beam search across tree branches
- rig-core — multi-provider LLM abstraction (OpenAI, Anthropic, Gemini, and more)
The query language is RQL — SQL-like syntax with a REASON clause:
SELECT * FROM contracts
REASON 'What are the late payment penalties?'
LIMIT 5;
You can combine it with filters and keyword search in a single query:
SELECT * FROM contracts
WHERE tags CONTAINS ANY ('nda')
SEARCH 'termination'
REASON 'What are the exit conditions and notice periods?'
LIMIT 5;
Open Source
We built HRR to solve a problem we kept running into. We open sourced it because document intelligence is a problem the whole community is working on, and we think the next generation of document databases should be built in the open.
If you're building agents that query documents — legal, compliance, support, research — we'd love for you to try it, break it, and tell us what's missing.
brainfish-ai
/
reasondb
The first database built to let AI agents think their way to the right answer using structural reasoning, rather than guessing based on vector similarity.
AI-Native Document Intelligence
The database that understands your documents.
Built for AI agents that need to reason, not just retrieve
Docs • Quick Start • API Reference
What is ReasonDB?
ReasonDB is an AI-native document database built in Rust, designed to go beyond simple retrieval. While traditional databases and vector stores treat documents as data to be indexed, ReasonDB treats them as knowledge to be understood - preserving document structure, enabling LLM-guided traversal, and extracting precise answers with full context.
ReasonDB introduces Hierarchical Reasoning Retrieval (HRR), a fundamentally new architecture where the LLM doesn't just consume retrieved content - it actively navigates your document structure to find exactly what it needs, like a human expert scanning summaries, drilling into relevant sections, and synthesizing answers.
ReasonDB is not another vector database.…
Ships as a single binary. Docker image available. Swagger UI included. No infrastructure required to get started.
Built by the team at Brainfish.
Top comments (0)