LLM Knowledge Graphs: A Practical Guide

LLM knowledge graphs convert unstructured text into structured, queryable representations. Instead of retrieving semantically similar chunks, you extract entities, relationships, and attributes into a graph that supports precise traversal, reasoning, and auditability. For teams building agentic systems or advanced RAG pipelines, a knowledge graph provides explicit fact storage that complements vector embeddings.

Pipeline Overview

A production graph pipeline has four stages: ingestion, extraction, normalization, and storage. Ingestion covers PDFs, HTML, and raw text. Extraction uses an LLM to identify entities and relations. Normalization deduplicates entities and resolves references. Storage persists the graph in a database that supports traversal queries.

Structured Extraction with LLMs

The core step is prompting an LLM to return structured data. JSON mode and function calling make this reproducible. Oxlo.ai supports both features across its chat models, including Qwen 3 32B for multilingual documents and DeepSeek R1 671B for complex reasoning over technical schemas.

Because Oxlo.ai is fully OpenAI SDK compatible, you can drop the following script into an existing Python project by changing only the base URL and model name.

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ["OXLO_API_KEY"]
)

document = """
Tesla, Inc. is an American electric vehicle company headquartered in Austin, Texas.
It was founded in 2003 by Martin Eberhard and Marc Tarpenning. Elon Musk led the
Series A investment round in 2004 and later became CEO.
"""

completion = client.chat.completions.create(
    model="qwen3-32b",
    messages=[
        {
            "role": "system",
            "content": (
                "You are an entity extraction system. "
                "Extract organizations, people, locations, and events. "
                "Return valid JSON with keys: entities, relationships."
            )
        },
        {
            "role": "user",
            "content": f"Extract from the following text:\n\n{document}"
        }
    ],
    response_format={"type": "json_object"},
    temperature=0.1
)

data = json.loads(completion.choices[0].message.content)
print(json.dumps(data, indent=2))

For stricter schema enforcement, you can switch to function calling and define the expected entity types as tool definitions. Models such as Llama 3.3 70B and Kimi K2.6 on Oxlo.ai handle multi-turn tool use reliably.

Storage and Schema Design

Property graph databases like Neo4j, Memgraph, or Amazon Neptune are common choices. RDF stores such as Apache Jena handle ontology-heavy use cases. Schema design should balance specificity and flexibility. Start with core node labels and relationship types, then expand based on query patterns.

A minimal schema for corporate research might include node labels for Person, Organization, and Location, plus relationships such as FOUNDED, HEADQUARTERED_IN, and LED_BY. Keep relationship names directional and consistent so that traversal queries remain simple.

GraphRAG and Retrieval

GraphRAG combines vector similarity with graph traversal. After embedding a question, you retrieve seed nodes, then expand across relationships to collect multi-hop context. This reduces the semantic drift that pure vector search suffers on complex, multi-fact questions. Community summaries generated over graph clusters can also provide global context.

Implementation typically involves two indices: a vector index for initial retrieval and a graph index for structured expansion. When a user asks, "Which investors later became CEOs of companies they funded?", a vector search alone struggles, but a graph traversal from Investor to Company to CEO returns an exact path.

Agentic Memory and Tool Use

Agents benefit from graphs as long-term memory. An agent can write new facts after each turn, query existing knowledge before acting, and verify claims against stored relationships. With Oxlo.ai, models like GLM 5 and Kimi K2.6 support long-horizon agentic tasks and tool use, so you can implement graph updates as function calls that the model invokes autonomously.

For example, an agent might expose a add_relationship(source, target, relation) tool. After extracting a new fact from a user message, the model calls the tool, and the graph is updated in real time. On subsequent turns, the agent queries the graph to ground its responses.

Inference Cost and Context Windows

Extraction workloads are inherently long-context. A single request may contain a full PDF, extensive few-shot examples, and a large JSON schema. Under token-based billing, this becomes expensive quickly. Oxlo.ai uses flat per-request pricing, so the cost does not scale with input length. This makes it practical to pass entire documents or codebases in a single prompt for extraction, or to run iterative agentic loops that read and write large graph contexts.

Models like DeepSeek V4 Flash and Kimi K2.6 offer context windows up to 131K or 1M tokens, which you can use without token anxiety. See Oxlo.ai pricing for plan details.

Conclusion

LLM knowledge graphs give your systems structured memory and verifiable reasoning paths. By combining robust extraction models, a well-designed storage layer, and an inference backend that handles long contexts economically, you can deploy pipelines that scale from prototype to production. Oxlo.ai provides the model variety, JSON mode, and request-based pricing that make these workloads feasible.