Large language models excel at pattern matching and fluent generation, yet they remain stateless and prone to confabulating facts outside their training distribution. Knowledge graphs ground inference in explicit, queryable structure, but building and querying them traditionally requires brittle pipelines. Combining the two gives you natural language interfaces over curated, relational data, with the LLM handling interpretation and the graph enforcing correctness. The result is a system that can answer multi-hop questions, trace provenance, and update its understanding of the world without a full retraining cycle.
Architecture: From Unstructured Text to Queryable Graph
A production LLM plus knowledge graph stack usually has four stages. First, an LLM extracts entities and relations from raw documents. Second, a graph database persists those triples under a schema you control. Third, a retrieval layer resolves user questions into subgraphs via Cypher, Gremlin, or vector similarity over node embeddings. Fourth, a generation LLM synthesizes an answer from the retrieved context. Each stage can be tuned independently, so you can swap models or expand the schema without rebuilding the entire pipeline.
Entity and Relation Extraction
The extraction stage is where an LLM transforms unstructured text into structured triples. JSON mode and low temperature settings keep the output deterministic. On Oxlo.ai, you can run this with any general-purpose or reasoning model through the standard OpenAI SDK. For multilingual document collections, Qwen 3 32B is a strong candidate. For deep reasoning over technical specs, DeepSeek R1 671B MoE or GLM 5 provides the necessary precision.
import os
import json
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ["OXLO_API_KEY"]
)
text = "Apple Inc. was founded by Steve Jobs and Steve Wozniak in Cupertino."
response = client.chat.completions.create(
model="llama-3.3-70b", # verify exact model identifier in your Oxlo.ai dashboard
messages=[{
"role": "user",
"content": (
"Extract entities and relations from the text below. "
"Return valid JSON with keys 'entities' and 'relations'.\n\n" + text
)
}],
response_format={"type": "json_object"},
temperature=0.1
)
structured = json.loads(response.choices[0].message.content)
print(structured)The returned JSON can be validated against a JSON Schema before it ever touches the graph database, which prevents malformed insertions.
Graph Construction and Storage
Once you have triples, insert them into a graph store such as Neo4j. A simple Cypher merge guards against duplicates while preserving provenance.
MERGE (a:Organization {name: "Apple Inc."})
MERGE (b:Person {name: "Steve Jobs"})
MERGE (c:Location {name: "Cupertino"})
MERGE (b)-[:FOUNDED]->(a)
MERGE (a)-[:HEADQUARTERED_IN]->(c)For large-scale ingestion, batch the merges and index on node labels and relationship types. The graph now serves as a single source of truth that outlives any single model version.
Retrieval and Subgraph Context
Natural language questions rarely map one-to-one to graph queries. You can bridge the gap by asking an LLM to generate Cypher from a schema description, or by using vector search over node embeddings to find entry points and then expanding to neighboring nodes. Hybrid retrieval tends to work best: embeddings find the relevant entities, and graph traversal collects the multi-hop context needed for the answer.
schema_desc = """
Nodes: Organization(name), Person(name), Location(name)
Edges: FOUNDED(Person->Organization), HEADQUARTERED_IN(Organization->Location)
"""
question = "Where is the company founded by Steve Jobs headquartered?"
cypher_prompt = f"""Given this schema:
{schema_desc}
Write a Cypher query to answer the question.
Return only the query.
Question: {question}"""
response = client.chat.completions.create(
model="deepseek-r1-671b", # DeepSeek R1 671B MoE for reasoning-heavy translation
messages=[{"role": "user", "content": cypher_prompt}],
temperature=0.0
)
query = response.choices[0].message.content.strip()
# Execute query against Neo4j to retrieve subgraph
subgraph = neo4j_session.run(query).data()Augmented Generation with Oxlo.ai
After retrieval, the serialized subgraph becomes part of the prompt context. This is where context length and cost structure matter. Subgraphs can grow quickly: a multi-hop expansion around a central entity can return dozens of nodes and edges, consuming thousands of tokens in JSON or natural language serialization.
On token-based providers, longer context directly increases inference cost. Oxlo.ai uses flat per-request pricing, so a large subgraph does not inflate the price of the generation call. That makes graph-augmented workloads significantly more predictable, especially for agentic pipelines that may iterate across multiple retrieval and reasoning steps. See the Oxlo.ai pricing page for current plan details.
context_block = json.dumps(subgraph, indent=2)
final = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "Answer using only the provided graph context. Cite node names."},
{"role": "user", "content": f"Context:\n{context_block}\n\nQuestion: {question}"}
],
temperature=0.2,
stream=True
)
for chunk in final:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")Streaming responses and JSON mode are available across the Oxlo.ai model catalog, so you can mix streaming generation with structured extraction in the same pipeline without switching clients.
Where Oxlo.ai Fits in the Stack
Oxlo.ai is not merely a drop-in inference backend. The platform offers 45-plus models across seven categories, including reasoning specialists such as DeepSeek R1 671B MoE, GLM 5, and Kimi K2.6, as well as general-purpose workhorses like Llama 3.3 70B and Qwen 3 32B. Because the API is fully OpenAI SDK compatible, you can point existing extraction and RAG code to https://api.oxlo.ai/v1 without rewriting clients.
The flat per-request pricing model is particularly relevant here. Knowledge graph pipelines often require long-context prompts that bundle schema definitions, few-shot examples, and large subgraph serializations. Under token-based pricing, those prompts incur a premium on every call. On Oxlo.ai, the cost remains constant per request, which can yield substantial savings for agentic and long-context workloads. There are no cold starts on popular models, so latency stays consistent even when traffic spikes.
Practical Considerations
Schema evolution. Decide early whether to enforce a rigid ontology or allow the LLM to propose new entity types dynamically. Rigid schemas are easier to validate but require maintenance. Dynamic schemas adapt to new documents but can pollute the graph with inconsistent labels.
Evaluation. Measure extraction accuracy with precision and recall against a labeled holdout set. For end-to-end QA, compare graph-augmented answers against a baseline retrieval system using answer correctness and citation recall. Tools such as Ragas or a custom judge LLM on Oxlo.ai can automate this.
Updates and deletion. Unlike vector databases that only append, graph databases support updates and deletions. When source documents change, trace which triples they contributed and issue targeted MERGE or DELETE statements to keep the graph synchronized.
Conclusion
Integrating large language models with knowledge graphs gives you the fluency of modern generative AI plus the precision of structured, queryable data. The architecture is straightforward: extract, store, retrieve, and generate. Oxlo.ai provides the inference layer for every stage, with a broad model catalog, OpenAI SDK compatibility, and flat per-request pricing that keeps long-context graph workloads affordable. If you are building agentic systems that reason over complex domains, this combination is worth evaluating today.
Top comments (0)