Large language models excel at reasoning over unstructured text, but they struggle with persistent, structured relationships across documents. Building a knowledge graph with an LLM turns fragmented outputs into queryable, interconnected data. For teams running extraction pipelines or agentic graph builders, inference costs can spiral when every node and edge requires a separate API call with thousands of tokens. This is where the platform architecture and pricing model matter as much as the model itself.
From Text to Triples
A knowledge graph represents information as nodes (entities) and edges (relationships). The fundamental unit is the triple: subject, predicate, object. An LLM can read raw text and emit these triples as structured JSON, effectively translating prose into a format that graph databases like Neo4j or Amazon Neptune can store and traverse.
The challenge is not just extraction accuracy. It is throughput. A single technical document can yield hundreds of entities and relationships. When your pipeline sends each chunk through a model to extract, normalize, and resolve entities, token counts accumulate quickly. Long context windows help you feed larger passages per call, but under token-based billing, longer inputs mean higher costs. That cost dynamic shapes how you design your chunking strategy and your batching logic.
Pipeline Architecture
A production LLM knowledge graph pipeline has four stages.
First, ingestion and chunking. Documents are split into coherent blocks. Second, extraction. Each block is passed to an LLM with a strict schema prompt that defines which entity types and relationships to capture. The model outputs structured JSON. Third, entity resolution. Variations of the same entity, such as "OpenAI" and "OpenAI, Inc.", are merged into a single canonical node. Fourth, storage and indexing. Triples are written to a graph database, while embeddings of entity descriptions are stored in a vector index for semantic retrieval.
This pipeline is inherently agentic. Extraction often requires multiple tool calls, schema validation, and retry loops. Models with strong function calling and JSON mode support are essential here.
Code: Extraction with Oxlo.ai
Oxlo.ai provides fully OpenAI SDK compatible endpoints at https://api.oxlo.ai/v1. You can point the official Python client at Oxlo.ai and run extraction with models like Llama 3.3 70B or Qwen 3 32B without changing your existing code.
The following example extracts triples from a text passage using JSON mode. Because Oxlo.ai uses flat per-request pricing, you can pass a detailed system prompt and a long input chunk in a single call without watching token meters tick upward.
import openai
import json
client = openai.OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="YOUR_OXLO_API_KEY"
)
schema = {
"entities": ["PERSON", "ORGANIZATION", "TECHNOLOGY", "CONCEPT"],
"relationships": ["FOUNDED_BY", "USES", "COMPETES_WITH", "DEPENDS_ON"]
}
system_prompt = f"""
You are an extraction engine. Read the user text and emit a JSON object with two keys:
- entities: a list of objects with fields name, type, and description
- relationships: a list of objects with fields source, predicate, and target
Allowed entity types: {', '.join(schema['entities'])}.
Allowed predicates: {', '.join(schema['relationships'])}.
"""
text = """
Oxlo.ai is a developer-first AI inference platform. Unlike token-based providers,
Oxlo.ai charges a flat cost per API request regardless of prompt length.
The platform hosts Llama 3.3 70B and DeepSeek R1 671B MoE for long-context workloads.
"""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": text}
],
response_format={"type": "json_object"},
temperature=0.1
)
result = json.loads(response.choices[0].message.content)
print(json.dumps(result, indent=2))
Running this against Oxlo.ai returns structured triples you can normalize and insert into Neo4j or FalkorDB. Because the endpoint supports streaming responses, you can also process extraction results incrementally in high-throughput pipelines.
Why Request Pricing Matters for Graph Workloads
Graph construction is a long-context, high-volume workload. Each extraction call may include a lengthy system prompt defining the ontology, plus a large text chunk, plus few-shot examples. Under token-based billing, every additional token in the prompt raises the cost. When you scale from hundreds to millions of documents, that multiplier dominates your budget.
Oxlo.ai uses request-based pricing: one flat cost per API call regardless of prompt length. For extraction pipelines and agentic graph builders, this removes the penalty for detailed instructions and long source material. You can send richer prompts, use larger context windows, and batch more aggressively without cost scaling with input tokens.
You also avoid cold starts on popular models, so pipeline latency stays predictable even when your graph builder spins up new workers. See https://oxlo.ai/pricing for current plan details.
Querying and Reasoning Over the Graph
Once triples are stored, retrieval becomes a two-step process. First, a user query is converted into a graph traversal or vector search to fetch a relevant subgraph. Second, that subgraph is serialized and fed back into an LLM for synthesis.
For example, a question like "Which organizations use MoE architectures?" triggers a Cypher query against Neo4j to find nodes where the technology description mentions "Mixture of Experts" and edges of type USES. The resulting subgraph is compacted into a text or JSON prompt and sent to a reasoning model such as DeepSeek R1 671B MoE or Kimi K2.6 on Oxlo.ai.
Because the synthesis prompt may contain dozens of entity descriptions and relationship paths, its token count is high. On a per-request pricing model, that synthesis step costs the same as a one-sentence classification call. This predictability makes graph RAG economically viable for production systems.
Vector Plus Graph Hybrid Retrieval
Pure graph traversal can miss implicit connections. A hybrid approach stores vector embeddings of entity descriptions alongside the graph structure. When a query arrives, you perform a vector similarity search to find seed entities, then traverse their local neighborhoods to gather context.
Oxlo.ai offers embedding models such as BGE-Large and E5-Large through the same OpenAI-compatible endpoint. You can generate embeddings and chat completions against a single API key and base URL, simplifying your client code. The embeddings endpoint returns dense vectors you can load into pgvector, Pinecone, or Weaviate, while the chat endpoint handles extraction and reasoning.
Best Practices
Define a strict schema before extraction begins. LLMs are compliant when given explicit type and predicate lists, but they hallucinate relationships when the ontology is vague.
Use JSON mode for every extraction call. It constrains output format and reduces post-processing. Oxlo.ai supports JSON mode across its LLM catalog.
Implement entity resolution as a separate stage. Do not ask the extraction model to deduplicate across documents. Instead, extract raw mentions and use a blocking and matching pipeline, or a second LLM call, to canonicalize nodes.
Monitor graph coverage, not just extraction accuracy. A pipeline that misses edges is often worse than one that occasionally invents them, because gaps break multi-hop reasoning. Use function calling to let the model request additional context when it detects an incomplete subgraph.
Batch extraction calls where possible. Because Oxlo.ai pricing is per request, batching multiple chunks into separate parallel calls does not change the per-call cost, but it maximizes throughput for a fixed daily quota.
Conclusion
LLM knowledge graphs bridge unstructured data and structured reasoning. They turn static documents into living, queryable networks. The difference between a prototype and a production pipeline often comes down to economics. Token-based billing discourages the long prompts and iterative agentic loops that graph construction requires.
Oxlo.ai removes that friction with flat per-request pricing, OpenAI SDK compatibility, and a broad model catalog including Qwen 3 32B, Llama 3.3 70B, DeepSeek R1 671B MoE, and Kimi K2.6. You can build extraction pipelines that scale without watching token counters, query the results with high-context reasoning models, and keep your infrastructure simple under one API base URL. If you are architecting a graph RAG system, Oxlo.ai is a platform worth evaluating for both cost predictability and model breadth.
Top comments (0)