What the retrieval layer of a Graph RAG system actually looks like

The first version retrieved everything. Every node reachable from the seed entity, up to four hops. The context window hit 8,000 tokens on a basic query about a mid-size Korean manufacturer. The model started confusing entities.

I pulled it back to 50 nodes, which is where it still is. I don't love that number. It's a hard cutoff that causes real problems for large corporate groups — Samsung Electronics has hundreds of subsidiaries and a query about their supply chain exposure hits the limit before returning a complete picture. But 50 is where the model stops confusing things and starts giving useful answers, and I haven't found a smarter expansion strategy that changes that tradeoff.

The system is a Graph RAG layer over a 50 million-row supply chain knowledge graph. The entity resolution took about two weeks to get right. The retrieval layer itself took three days. Here's what the retrieval actually does.

Every retrieval starts from seed entities. For a query like "what is Samsung SDI's exposure to US tariffs on Korean steel," the first step is which nodes in the graph correspond to "Samsung SDI" and "US tariffs." I run this through an entity lookup that checks normalized names and aliases:

def identify_seed_entities(query_text: str, session) -> list[str]:
    entities = extract_entities(query_text)
    results = []
    for entity_mention in entities:
        candidates = session.run("""
            MATCH (e:Entity)
            WHERE e.name_normalized CONTAINS $mention_normalized
               OR any(alias IN e.aliases WHERE alias CONTAINS $mention_normalized)
            RETURN e.id, e.name, e.canonical_type
            ORDER BY e.mention_count DESC
            LIMIT 3
        """, mention_normalized=normalize(entity_mention)).data()
        if candidates:
            results.append(candidates[0]["e.id"])
    return results

mention_count ranks entities by how many source documents they appear in. This is a rough importance proxy. For "Samsung SDI" it works fine. For more ambiguous mentions it sometimes picks the wrong candidate, especially for smaller companies that share name fragments. I haven't fixed that properly.

From the seed nodes, the subgraph expansion follows supply chain relationship types and cuts off at three hops. Four hops started returning nodes that were technically connected but not relevant to any reasonable interpretation of the query. A steel tariff affecting a Korean parts supplier affecting a construction equipment company is three hops and stays useful. The fourth hop typically reached conglomerates that own everything and added noise.

result = session.run("""
    MATCH path = (seed:Entity {id: $seed_id})-[r:SUPPLIES_TO|SOURCES_FROM|OWNS|CONTROLS*1..3]-(connected)
    WITH connected, length(path) AS hops
    WHERE hops <= 3
    RETURN connected.id AS entity_id, connected.name AS entity_name, hops
    ORDER BY hops ASC, connected.document_count DESC
    LIMIT 50
""", seed_id=seed_id)

After getting the subgraph, I fetch the source document excerpts that were asserted through those entities. These are the actual facts — the sentences from filings, news articles, procurement documents that describe the relationships:

result = session.run("""
    MATCH (e:Entity)-[r:ASSERTED_IN]->(d:Document)
    WHERE e.id IN $entity_ids
    RETURN e.name AS entity_name,
           d.excerpt AS fact_text,
           d.source AS source,
           d.published_at AS published_at,
           r.assertion_confidence AS confidence
    ORDER BY r.assertion_confidence DESC, d.published_at DESC
    LIMIT 100
""", entity_ids=entity_ids)

The hundred-fact fetch gets trimmed to forty before going to the model. The trim is TF-IDF ranking against the query terms, which is coarse but seems to work. The bigger problem is that facts from entities reached through low-confidence ER merges get the same treatment as facts from directly verified sources. I flag anything below 0.80 confidence with [uncertain] in the context, but that's a manual addition after discovering the model was treating all facts equally regardless of how confident the underlying merge was.

Packaging the final context is straightforward: facts sorted by confidence, each labeled with source and date. Uncertain ones get the flag. The model prompt says to treat flagged facts as preliminary.

What takes the most time isn't any of this. It's the cases where the query involves an entity the graph doesn't have at resolution quality. The graph might have "Samsung SDI" resolved, but a smaller battery cell supplier mentioned in a procurement document might exist as a raw string node that never got merged with anything. The retrieval finds it, returns facts, and the model reasons off it as if it were a resolved entity. I currently have no way to flag "this entity wasn't fully resolved" at retrieval time. That's the next thing I need to add.

---

I build er-api, a multilingual entity resolution service for Korean, Japanese, Chinese, and English corporate data. More at hannune.ai.