Entity disambiguation turns ambiguous strings like "Apple" or "Washington" into specific knowledge-base entries using surrounding context. We will build a lightweight Python resolver that detects surface forms, retrieves candidates from a local index, and uses an LLM to pick the correct entity. I run this on Oxlo.ai because flat per-request pricing keeps costs stable even when candidate descriptions make the prompt long, and the OpenAI-compatible client drops in without extra dependencies.
What you'll need
- Python 3.10+
- The
openaiSDK:pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Set up the Oxlo.ai client
I always start with a smoke test. Point the OpenAI SDK at Oxlo.ai and verify the connection returns a response.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "user", "content": "Say OK"},
],
)
print(response.choices[0].message.content)Step 2: Build a candidate entity index
A production system might query Wikidata or an internal graph. Here we keep a hard-coded dictionary mapping lowercase surface forms to candidate lists. Each candidate carries an ID, canonical name, type, and description.
CANDIDATE_INDEX = {
"apple": [
{"id": "Q312", "name": "Apple Inc.", "type": "ORG", "description": "Technology company headquartered in Cupertino, California"},
{"id": "Q89", "name": "Apple", "type": "PLANT", "description": "Edible fruit of the apple tree"}
],
"washington": [
{"id": "Q1223", "name": "Washington", "type": "GPE", "description": "State in the Pacific Northwest of the United States"},
{"id": "Q9834", "name": "George Washington", "type": "PERSON", "description": "First president of the United States"}
],
"python": [
{"id": "Q28865", "name": "Python", "type": "TECH", "description": "General-purpose programming language"},
{"id": "Q245462", "name": "Python", "type": "ANIMAL", "description": "Family of nonvenomous snakes"}
]
}Step 3: Define the disambiguation system prompt
The prompt is the contract. It tells the model to return strict JSON with one entry per ambiguous mention, including confidence and a one-sentence rationale.
SYSTEM_PROMPT = """You are an entity disambiguation engine. Your task is to map ambiguous mentions in a user-provided text to the correct candidate entity from a supplied list.
For each mention, output an object with:
- mention: the exact text as it appeared in the input
- entity_id: the id of the chosen candidate
- entity_name: the canonical name of the chosen candidate
- confidence: HIGH, MEDIUM, or LOW
- reasoning: one sentence explaining why the candidate fits the context
Return ONLY a JSON object with a single key "entities" containing the list of results. If a mention has no suitable candidate, set entity_id to null."""Step 4: Detect mentions and query the LLM
We scan the input text for any surface form in our index, build a structured prompt with the candidates, and ask the model to resolve them. I use llama-3.3-70b here because it follows JSON instructions reliably and runs without cold starts on Oxlo.ai.
import json
import re
def detect_mentions(text, index):
found = []
text_lower = text.lower()
for surface in index:
pattern = r'\b' + re.escape(surface) + r'\b'
if re.search(pattern, text_lower):
found.append(surface)
return found
def resolve(text, index, client):
mentions = detect_mentions(text, index)
if not mentions:
return {"entities": []}
candidate_block = ""
for m in mentions:
candidate_block += f"\nMention '{m}':\n"
for c in index[m]:
candidate_block += f" - {c['id']} ({c['type']}): {c['name']} - {c['description']}\n"
user_message = f"""Text: {text}
Candidates:{candidate_block}
Return the JSON disambiguation results."""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
raw = response.choices[0].message.content.strip()
if raw.startswith("
```"):
raw = raw.split("```
")[1].strip()
if raw.startswith("json"):
raw = raw[4:].strip()
return json.loads(raw)Step 5: Connect the pipeline
Now we wire the pieces together. I added a small helper to pretty-print results so console output stays readable during development.
def disambiguate(text, index, client):
result = resolve(text, index, client)
print(json.dumps(result, indent=2))
return resultRun it
Pass a sentence containing multiple ambiguous entities. The model receives the full candidate descriptions and resolves each based on context.
if __name__ == "__main__":
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
text = "Apple is opening a new engineering office in Washington, and Python developers are invited to the launch event."
disambiguate(text, CANDIDATE_INDEX, client)Example output:
{
"entities": [
{
"mention": "Apple",
"entity_id": "Q312",
"entity_name": "Apple Inc.",
"confidence": "HIGH",
"reasoning": "Opening an engineering office indicates a technology organization."
},
{
"mention": "Washington",
"entity_id": "Q1223",
"entity_name": "Washington",
"confidence": "HIGH",
"reasoning": "An office location refers to the U.S. state."
},
{
"mention": "Python",
"entity_id": "Q28865",
"entity_name": "Python",
"confidence": "HIGH",
"reasoning": "Developers attending a technical event refers to the programming language."
}
]
}Wrap up and next steps
This pipeline gives you a working baseline. Two concrete ways to push it further: replace the hard-coded index with a vector search over an embedding store so you can scale to thousands of candidates, or add a caching layer in front of the Oxlo.ai client to avoid re-resolving frequent mentions.
Top comments (0)