Stop Chunking Your Relationships: Why We Paired a Knowledge Graph with a Vector DB

Shekhar Kadyan — Fri, 03 Jul 2026 07:06:18 +0000

If you have spent any time building AI agents for enterprise use cases this year, you have inevitably hit the "RAG Wall."

The foundation models (Claude 3.5, GPT-4o) are incredible at reasoning, but they are fundamentally stateless. To fix this, the industry default has been flat semantic RAG: we dump all our corporate data into a chunker, embed it into a Vector DB, and run a cosine similarity search when the user asks a question.

It works flawlessly for finding a specific PDF. It fails catastrophically when an agent needs to understand why a decision was made across multiple systems.

The Problem: Vector Search Destroys Lineage

Real enterprise data is messy precisely because it is relational. A decision often starts as a Slack thread, gets formalized in a Jira ticket, and ends up as a modified clause in a SharePoint contract.

When you blindly slice those documents into 500-token chunks for a Vector DB, you completely strip out the edges and linkages. You turn a cohesive chronological timeline into isolated paragraphs. When the AI agent asks, "What is the status of the Acme Corp billing dispute?", standard RAG just dumps three unrelated text chunks into the prompt and leaves the model to hallucinate the timeline.

The Architecture: Vector DB + Knowledge Graph

To solve this corporate amnesia, we realized the ingestion pipeline needed to change before the AI ever saw a prompt. Instead of a flat vector store, we built an event-streaming context layer that routes data into a dual-store architecture.

Here is the high-level flow we use in PipesHub:

Event Streaming (Kafka): We continuously ingest unstructured data from silos (Slack, GitHub, Salesforce, Drive).
Entity Extraction: Before storing the data, an extraction layer identifies the entities (Users, Companies, Tickets, Pull Requests) and the relationships between them.
Dual Routing:

The raw text chunks go to the Vector DB (for broad semantic context).
The extracted relationships go to a Knowledge Graph (for hard lineage and temporal tracking).

Now, the backend infrastructure actually maps how your tools connect instead of just doing fuzzy keyword matching.

Exposing the Architecture via MCP

Having a Knowledge Graph paired with a Vector DB is great, but forcing an AI agent to write custom Cypher queries or build bespoke API connectors to access it is incredibly brittle.

This is where the Model Context Protocol (MCP) comes in.

We expose our paired database architecture as an MCP Server. When an orchestration framework (like LangGraph or the OpenAI Agents SDK) needs to answer a complex query, it acts as an MCP Client. Our server dynamically exposes the dual-store data as standard MCP Tools, letting the agent decide how to fetch its memory:

Tool A (query_graph_relations): The agent hits the graph to find the exact lineage between a Slack approval and a Jira ticket.
Tool B (semantic_search): The agent hits the Vector DB to grab the actual text of the contract.

The Shift to Headless Context

We open-sourced PipesHub because we believe the future of AI isn't another empty chat interface. The long-term winner will be the invisible, headless data substrate that runs in the background, maintaining a persistent, permission-aware memory graph that any agent can plug into via MCP.

Stop blindly dumping chunked documents into your prompts. Start mapping your entities. Your token costs will drop, and your hallucinations will practically disappear.

If you are fighting the RAG Wall or building MCP servers, I'd love to hear how you are handling entity extraction. You can check out how we implemented the graph routing in our repo here: https://github.com/pipeshub-ai/pipeshub-ai

DEV Community: Shekhar Kadyan

Stop Chunking Your Relationships: Why We Paired a Knowledge Graph with a Vector DB