Integrating LLM with Knowledge Graph

#aiinfrastructure #oxlo #ai

Large language models handle unstructured text well, but they struggle with precise, multi-hop relational reasoning and often hallucinate facts. Knowledge graphs store entities and relationships in a traversable structure, yet they require specialized query languages like Cypher or SPARQL that are not user-friendly. Combining an LLM with a knowledge graph gives you a natural language interface over verified, structured data. In this guide, we will build a simple but robust integration pipeline using Oxlo.ai as the inference layer.

Why Combine LLMs and Knowledge Graphs

Retrieval-Augmented Generation, or RAG, over vector databases often returns isolated text chunks that lack relational context. A knowledge graph preserves explicit connections, so you can answer questions like "Which suppliers of Company X are based in countries sanctioned by Policy Y?" by traversing edges rather than hoping the answer appears in a single chunk. The LLM's role is to bridge human language and the graph: extracting entities, generating queries, interpreting result sets, and repairing failed queries.

Architecture Overview

A typical LLM-plus-graph stack has three stages. First, an extraction stage parses raw documents into nodes and edges. Second, a graph database such as Neo4j or Amazon Neptune stores and indexes the structure. Third, an inference stage uses an LLM to translate user questions into formal queries and to summarize graph responses into readable answers. Oxlo.ai powers the inference stage, offering fully OpenAI SDK compatible APIs with no cold starts.

Extracting Entities and Relations

You can use a capable general-purpose model to perform structured extraction. Because source documents can be long, sending the full text in a single prompt is desirable. Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. Unlike token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, your cost does not scale with input length.

from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

EXTRACTION_PROMPT = """
You are an information extraction engine.
Read the text below and output a JSON list of triples.
Each triple must have keys: subject, relation, object.

Text:
{text}
"""

text = "Apple Inc. was founded by Steve Jobs. Apple Inc. is headquartered in Cupertino."

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "Extract structured triples as JSON."},
        {"role": "user", "content": EXTRACTION_PROMPT.format(text=text)}
    ],
    response_format={"type": "json_object"}
)

triples = json.loads(response.choices[0].message.content)
print(triples)

Generating Cypher Queries with an LLM

Once the graph is populated, users will ask questions in plain English. The LLM can generate Cypher queries when given the graph schema. Schema descriptions can become lengthy for rich domains, which again favors Oxlo.ai's flat per-request pricing.

SCHEMA = """
Node types:
- (:Company {name: string, founded: int})
- (:Person {name: string})
- (:City {name: string})

Relationships:
- (:Company)-[:FOUNDED_BY]->(:Person)
- (:Company)-[:HEADQUARTERED_IN]->(:City)
"""

USER_QUESTION = "Who founded the company headquartered in Cupertino?"

query_prompt = f"""
You are a Cypher expert.
Given the schema below, write a Cypher query that answers the user's question.
Return only the query, no explanation.

Schema:
{SCHEMA}

Question: {USER_QUESTION}
"""

response = client.chat.completions.create(
    model="qwen-3-32b",
    messages=[{"role": "user", "content": query_prompt}]
)

cypher_query = response.choices[0].message.content.strip()
print(cypher_query)

Executing and Summarizing Results

After running the Cypher query against your graph database, you often get raw JSON or tabular data that is hard to read. You can pipe the results back to the LLM for a final natural language summary.

results = [
    {"p.name": "Steve Jobs"}
]

summarize_prompt = f"""
A graph database returned the following results for the question: {USER_QUESTION}

Results:
{json.dumps(results, indent=2)}

Write a concise, accurate answer in natural language.
"""

response = client.chat.completions.create(
    model="deepseek-r1-671b",
    messages=[{"role": "user", "content": summarize_prompt}]
)

answer = response.choices[0].message.content.strip()
print(answer)

DEV Community