DEV Community

Ayush
Ayush

Posted on Originally published at 136.112.138.59.nip.io

Why Vector RAG Fails in Codebases: Building an Architectural Knowledge Graph with FalkorDB

#ai #falkordb #webdev #architecture

🚨 The Nightmare Scenario: "What Must I Know Before I Touch This?"

Picture this: You ask an autonomous AI coding agent—or a newly onboarded engineer—to refactor a payment retry loop in your core billing service.

The agent searches your documentation and codebase using standard Vector RAG. It finds an Architectural Decision Record (ADR) titled "Linear Retry Policy with Fixed Backoff", follows the instructions, writes the code, and opens a pull request.

Two hours later, production crashes. Why?

Because three months ago, that exact linear retry policy caused a cascading thundering herd outage and was reverted in PR #42. A subsequent ADR—"Exponential Backoff with Full Jitter"—explicitly superseded it. But because the obsolete ADR matched the query embedding with high cosine similarity, the vector database happily served obsolete guidance.

Semantic similarity is completely time-blind.

When architecture evolves, flat vector embeddings and relational databases break down:

  1. Time-Blind Retrieval: Flat vector indexes have no concept of supersession edges or temporal validity windows.
  2. Multi-Hop Blind Spots: Computing the transitive blast radius of touching a single module requires traversing multi-hop dependency graphs. Relational SQL struggles with recursive CTE performance, and vector DBs can't even express it.
  3. Siloed Institutional Memory: Why code was written, who reviewed it, what broke, and what ADR governs it are scattered across git commits, GitHub PR discussions, CODEOWNERS files, and issue trackers.

To solve this for the FalkorDB Hackathon (Tracks 1 & 3), we built Argon (Ar): an evidence-backed architectural briefing engine grounded in the FalkorDB knowledge graph.

🎬 Watch the Demo & Live App

Want to see it in action first?

  • 🌐 Live Web App: https://136.112.138.59.nip.io/
  • 💻 Interactive Workspace: Switch to synthetic-demo and try searching "Refactor billing/retry.ts" or "I want to work on the auth layer".

⚡ The Core Philosophy: "No Graph, No Answer"

Argon rejects the idea that an LLM should invent or summarize architecture without rigorous, auditable grounding.

We established a strict Fail-Closed Verification Engine:

  • Every architectural claim output by Argon must cite a verified FalkorDB graph node #id.
  • Uncited or hallucinated claims are automatically stripped during a validation pass.
  • If FalkorDB is unreachable or zero verified citations exist, the system triggers a hard refusal card: "No graph, no answer."

No ungrounded guessing. No hallucinated policies.

🏛 How FalkorDB Models the Living Codebase

Instead of treating code as flat text documents, Argon mines repository commit histories, GitHub PRs, reviews, issues, and markdown ADRs into a connected openCypher property graph in FalkorDB.

(:Person)-[:AUTHORED]->(:PullRequest)-[:TOUCHES]->(:File)-[:IN_COMPONENT]->(:Component)
(:Person)-[:REVIEWED]->(:PullRequest)-[:REVERTS]->(:PullRequest)
(:Person|:Team)-[:OWNS]->(:Component)
(:Decision)-[:GOVERNS]->(:Component)
(:Decision)-[:SUPERSEDES]->(:Decision)
(:Decision)-[:EXTRACTED_FROM]->(:PullRequest)-[:CLOSES]->(:Issue)
(:Component)-[:DEPENDS_ON]->(:Component)
Enter fullscreen mode Exit fullscreen mode

Here is why FalkorDB is the ideal engine for this challenge:

1. Temporal Validity Windows (valid_from <= $asOf < valid_to)

Every decision node in FalkorDB holds validity timestamps. When Argon retrieves active architectural constraints, it evaluates:

MATCH (c:Component {name: $componentName})<-[:GOVERNS]-(d:Decision)
WHERE d.valid_from <= $asOf AND (d.valid_to IS NULL OR d.valid_to > $asOf)
RETURN d.id, d.title, d.status
ORDER BY d.valid_from DESC
Enter fullscreen mode Exit fullscreen mode

If an ADR was superseded, its valid_to is populated with the superseding commit's timestamp. Obsolete decisions simply vanish from active queries.

2. Transitive Blast Radius in 2ms

When a file is modified, how far does the ripple effect spread? With FalkorDB's openCypher engine, variable-length graph traversal computes downstream blast radii effortlessly:

MATCH (f:File {path: $filePath})-[:IN_COMPONENT]->(c:Component)
OPTIONAL MATCH (c)<-[:DEPENDS_ON*1..3]-(downstream:Component)
RETURN c.name AS root, collect(DISTINCT downstream.name) AS blast_radius
Enter fullscreen mode Exit fullscreen mode

In relational databases, this requires expensive recursive CTEs. In FalkorDB, it executes on CPU in ~2 milliseconds.

3. Real-Time Risk Ranking via In-Engine PageRank

Which components are the most fragile structural bottlenecks in your monorepo? Rather than pulling data into an offline Python script, Argon calls FalkorDB’s native graph algorithms directly:

CALL algo.pageRank('Component', 'DEPENDS_ON') 
YIELD node, score 
RETURN node.name, score 
ORDER BY score DESC
Enter fullscreen mode Exit fullscreen mode

4. Hybrid Vector Search + Instant Multi-Hop Expansion

Need to find historical precedents? Argon embeds the query with Matryoshka embeddings (512-d) and queries FalkorDB's native vector index (db.idx.vector.queryNodes), then immediately joins the matching decision with its governing component and the GitHub PR that created it:

CALL db.idx.vector.queryNodes('Decision', 'embedding', 5, $queryVec) 
YIELD node AS decision, score
MATCH (decision)-[:GOVERNS]->(c:Component)
OPTIONAL MATCH (decision)-[:EXTRACTED_FROM]->(pr:PullRequest)-[:CLOSES]->(i:Issue)
RETURN decision.title, c.name, pr.number, i.title, score
Enter fullscreen mode Exit fullscreen mode

📊 Benchmark Results (EVAL-01): The Proof

We benchmarked Argon against Naive Vector RAG and a Raw LLM Baseline across a curated golden fixture dataset of 24 real-world and synthetic historical git revert cases:

Metric Naive Vector RAG Raw LLM (Gemma 4) Argon (FalkorDB) Delta vs Vector RAG
Recall@1 4.2% 16.7% 62.5% +58.3%
Recall@3 8.3% 33.3% 95.8% +87.5%
Recall@5 29.2% 41.7% 100.0% +70.8%
MRR 0.162 0.257 0.788 +0.626
Superseded Hallucinations 12.5% 54.2% 0.0% (Clean) -12.5%
Avg Query Latency 123ms Cached 2ms ~60x faster

By enforcing graph constraints, Argon achieved 100% Recall@5 and 0.0% superseded decision leakage, compared to a staggering 54.2% hallucination rate when prompting a raw LLM without a graph.

🔌 Model Context Protocol (MCP): Bringing Graph Power to Cursor & Claude Code

Argon's queries are not locked inside our web interface. We packaged 13 graph tools into a production Model Context Protocol (MCP) server.

You can plug Argon directly into Claude Code or Cursor by adding this to your .mcp.json:

{
  "mcpServers": {
    "argon": {
      "command": "pnpm",
      "args": ["--silent", "--filter", "@argon/mcp", "start"],
      "env": {
        "FALKORDB_URL": "falkor://localhost:6379",
        "ARGON_WORKSPACE_ID": "synthetic-demo"
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Now, before your coding agent writes a single line of code, you can prompt:

"Use Argon: what is the blast radius of modifying billing/retry.ts, what ADRs govern it, and who are the CODEOWNERS?"

The agent retrieves an explainable tool trace and a deterministic risk verdict (PROCEED, REVIEW, or HOLD) computed strictly from FalkorDB.

🎯 Key Takeaway

Language models don't fail in complex codebases because they lack intelligence—they fail because flat text and vector embeddings provide incomplete, time-blind context.

By grounding agentic workflows in a high-performance graph database like FalkorDB, we can replace fuzzy hallucinations with mathematical traversals, temporal accuracy, and verifiable citations.

Try out the live deployment and let us know what you think:
👉 https://136.112.138.59.nip.io/

Built for the WeMakeDevs x FalkorDB Hackathon 2026.

Top comments (1)

Collapse
 
alexcodebytes profile image
Oleksandr •

"No Graph, No Answer" is the right principle — most "AI architecture assistant" demos fail because they confidently summarize a superseded ADR and nobody catches it until production. The fail-closed pattern is what makes this auditable.

One thing I'd push on: the supersession graph is only as good as the ADR hygiene of the repo. If the team doesn't write "supersedes" edges explicitly, your SUPERSEDES traversal finds nothing and Argon happily returns the stale ADR with a valid citation — technically "no hallucination" but practically the same failure.

Curious — how do you detect supersession that isn't explicitly labeled? Do you infer from PR revert edges (PR #42 reverts PR #17 implies PR #17's decision is dead), or does it require manual ADR authorship?