If you have spent any time building AI agents for enterprise use cases this year, you have inevitably hit the "RAG Wall."
The foundation models (Claude 3.5, GPT-4o) are incredible at reasoning, but they are fundamentally stateless. To fix this, the industry default has been flat semantic RAG: we dump all our corporate data into a chunker, embed it into a Vector DB, and run a cosine similarity search when the user asks a question.
It works flawlessly for finding a specific PDF. It fails catastrophically when an agent needs to understand why a decision was made across multiple systems.
The Problem: Vector Search Destroys Lineage
Real enterprise data is messy precisely because it is relational. A decision often starts as a Slack thread, gets formalized in a Jira ticket, and ends up as a modified clause in a SharePoint contract.
When you blindly slice those documents into 500-token chunks for a Vector DB, you completely strip out the edges and linkages. You turn a cohesive chronological timeline into isolated paragraphs. When the AI agent asks, "What is the status of the Acme Corp billing dispute?", standard RAG just dumps three unrelated text chunks into the prompt and leaves the model to hallucinate the timeline.
The Architecture: Vector DB + Knowledge Graph
To solve this corporate amnesia, we realized the ingestion pipeline needed to change before the AI ever saw a prompt. Instead of a flat vector store, we built an event-streaming context layer that routes data into a dual-store architecture.
Here is the high-level flow we use in PipesHub:
Event Streaming (Kafka): We continuously ingest unstructured data from silos (Slack, GitHub, Salesforce, Drive).
Entity Extraction: Before storing the data, an extraction layer identifies the entities (Users, Companies, Tickets, Pull Requests) and the relationships between them.
Dual Routing:
The raw text chunks go to the Vector DB (for broad semantic context).
The extracted relationships go to a Knowledge Graph (for hard lineage and temporal tracking).
Now, the backend infrastructure actually maps how your tools connect instead of just doing fuzzy keyword matching.
Exposing the Architecture via MCP
Having a Knowledge Graph paired with a Vector DB is great, but forcing an AI agent to write custom Cypher queries or build bespoke API connectors to access it is incredibly brittle.
This is where the Model Context Protocol (MCP) comes in.
We expose our paired database architecture as an MCP Server. But here is the critical architectural decision: we don't make the LLM choose which database to query.
Many early MCP implementations expose separate tools (e.g., a query_graph tool and a search_vector tool) and let the agent decide which one to use. In production, this introduces a massive point of failure—the LLM frequently guesses wrong, adding latency and returning hallucinations.
Instead, our MCP server (pipeshub-ai/mcp-server) abstracts the databases entirely. We expose high-level tools to the agent (like pipeshub_search and pipeshub_directory).
When an orchestration framework needs context, it simply passes the user's question to the pipeshub_search tool. Our backend microservice takes over deterministically. Under the hood, it simultaneously runs GraphDB graph traversals, Qdrant vector similarity, and Page Ranking to map the entity lineage and retrieve the semantic text.
The Shift to Headless Context
We open-sourced PipesHub because we believe the future of AI isn't another empty chat interface. The long-term winner will be the invisible, headless data substrate that runs in the background, maintaining a persistent, permission-aware memory graph that any agent can plug into via MCP.
Stop blindly dumping chunked documents into your prompts. Start mapping your entities. Your token costs will drop, and your hallucinations will practically disappear.
If you are fighting the RAG Wall or building MCP servers, I'd love to hear how you are handling entity extraction. You can check out how we implemented the graph routing in our repo here: https://github.com/pipeshub-ai/pipeshub-ai
Top comments (4)
The lineage point lands hard. Flat semantic RAG is great at "find me the document" and quietly terrible at "reconstruct why this decision happened," because chunking is literally an edge-deletion step — you throw away the relationships and then ask the model to hallucinate them back from three unrelated paragraphs. Splitting graph-for-lineage from vector-for-text is the right decomposition.
Where I'd push: entity extraction is your load-bearing wall, and enterprise data is exactly where entity resolution gets ugly — "Acme Corp" vs "Acme Inc" vs a Salesforce ID, the same Jira ticket referenced three ways. How are you handling resolution/dedup at ingest, and do you version the graph so a re-extraction doesn't silently rewrite history an agent already cited last week? Exposing both stores as distinct MCP tools (
query_graph_relationsvssemantic_search) is smart — but do you let the model pick between them, or route deterministically on query shape? Letting the agent choose adds a decision hop that can go wrong; deterministic routing is more boring and, in my experience, more debuggable when a retrieval comes back wrong.This is a fantastic push. You are hitting the exact load-bearing walls that make or break this architecture in production. We had to tackle these directly in the PipesHub codebase.
1. Entity Resolution (Why we don't merge across silos)
You correctly pointed out that "Acme Corp" vs "Acme Inc" is where pipelines die. Our solution is that we intentionally do not attempt cross-system entity resolution at ingestion. We found that trying to algorithmically merge a Salesforce ID, a Jira tag, and a Slack channel into a single "Master Acme" graph node creates a brittle nightmare of false positives.
Instead, we enforce strict intra-system deduplication. Every connector forces data into a dual-node pattern in GraphDB mapped strictly to its source externalRecordId. "Acme Corp" (Jira) and "Acme Inc" (Salesforce) remain distinct, isolated nodes. We push the cross-system resolution to the query layer: our unified backend uses Qdrant's vector similarity alongside GraphDB's PageRank to retrieve both nodes for a given query, and we rely on the LLM to synthesize the connection in the final context window. We happily trade global graph purity for ingestion reliability.
2. Handling State and Updates
Silently serving stale data is fatal for RAG. Instead of trying to maintain an append-only historical graph, we handle state via continuous delta syncs. Our connectors maintain "Sync Points" (delta links and timestamps) via Kafka event streaming. When a record is updated in the source system, the Data Entities Processor deterministically overwrites the specific node in graphDB and Qdrant. The AI always cites the absolute current state of the source-of-truth.
3. MCP Tool Routing (You caught a simplification in the post)
Your instinct here is spot on, and I actually oversimplified that routing step in the article for the sake of explaining the dual-store concept. You are 100% right that letting the LLM arbitrarily decide whether to hit query_graph_relations or semantic_search introduces a massive point of failure and a decision hop that goes wrong.
In reality, our MCP server abstracts that choice away entirely. We don't expose the databases as distinct tools to the agent. Instead, our MCP server exposes a single, unified Query microservice. That backend service deterministically handles combining the GrahDB graph traversals, Qdrant vector similarity, and Page Ranking under the hood. The agent just asks a question, and our deterministic pipeline guarantees a unified result. As you said boring is absolutely better here—when a retrieval fails, we know exactly where to look in the trace.
Are you currently building an internal tool to handle this, or evaluating existing GraphRAG frameworks?
New to to RAG. Curious how PipesHub compares to other KG grounding systems, Microsoft’s GraphRAG, for instance? Thanks for the detailed post!
Microsoft's GraphRAG is a pretty useful, but it solves a completely different problem than PipesHub. The core difference comes down to how the graph is built and what data it targets.
Microsoft GraphRAG: Probabilistic Extraction
GraphRAG is designed to make sense of massive corpuses of unstructured text (like thousands of PDFs). It feeds chunks of text to an LLM to extract entities, infer relationships, and build a topology from scratch. Because it relies on the model to guess how things connect based on semantic proximity, the relationships are probabilistic. It also burns significant LLM generation tokens during indexing just to map the graph.
PipesHub: Deterministic Mapping
PipesHub is built specifically for structured enterprise SaaS silos (Jira, Slack, GitHub). We do not use LLMs to guess how your data connects. Instead, our backend continuously ingests events via Kafka and maps the native, hard-coded links already present in the source API payloads (like a specific Slack thread linking directly to a Jira ticket).
Here is how that plays out under the hood:
No LLM Generation Tax for Graph Building: The graph topology is built via deterministic backend code. You only pay standard, cheap embedding tokens to populate the vector database (Qdrant). For unstructured docs like PDFs, we default to fast parsing libraries (like Docling or pdfplumber), only routing to a multimodal LLM if it’s a scanned visual that requires visual OCR and reasoning.
Absolute Lineage: The graph never hallucinates an edge. The relationship between a Slack message, a Jira ticket, and a GitHub PR is a mathematical certainty derived straight from the API data.
Approximately 30% Token Savings on Retrieval: Because the backend does the relationship synthesis off-prompt, we hand the agent a surgically precise context block instead of a raw API dump. This routinely cuts agent workflow token costs by about 30%.
In short: Use GraphRAG if you need an agent to read massive, unlinked unstructured document dumps and infer conceptual entities. Use PipesHub if you need an agent to track cross-platform workflows, code, and decisions with absolute certainty and a tiny prompt footprint.