Named Entity Recognition with an LLM lets you extract people, organizations, locations, and custom labels from unstructured text without maintaining spaCy pipelines or annotation teams. In this guide, I will walk through a minimal Python service that calls an Oxlo.ai model to identify entities and return them as structured JSON. The final script is something I have used to bootstrap extraction on messy, real-world documents.
What you'll need
- Python 3.10 or newer
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai
Step 1: Initialize the Oxlo.ai client
I start by creating a client pointing at Oxlo.ai's OpenAI-compatible endpoint. This single object handles authentication and requests.
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="YOUR_OXLO_API_KEY"
)
# Quick connectivity check
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "Say OK"}],
max_tokens=10
)
print(response.choices[0].message.content)
Step 2: Write the NER system prompt
The system prompt is the contract. I tell the model exactly which entity types to extract and the JSON schema it must follow. This keeps output deterministic and easy to parse.
SYSTEM_PROMPT = """You are a precise Named Entity Recognition system.
Extract entities from the user text and return a single JSON object.
Do not include markdown formatting, explanations, or text outside the JSON.
Entity types:
- PERSON
- ORGANIZATION
- LOCATION
- DATE
- MONEY
JSON schema:
{
"entities": [
{
"text": "exact text span",
"label": "one of the entity types above",
"start": 0,
"end": 10
}
]
}
If no entities are found, return {"entities": []}.
"""
Step 3: Build the extraction function
I wrap the API call in a small function that accepts raw text and returns a Python dictionary. I parse the response content as JSON and handle the occasional malformed response gracefully.
import json
def extract_entities(text: str) -> dict:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
temperature=0.1,
max_tokens=1024
)
raw = response.choices[0].message.content.strip()
# Remove accidental markdown fences if the model produces them
if raw.startswith("
```"):
raw = raw.split("\n", 1)[1].rsplit("```
", 1)[0].strip()
try:
return json.loads(raw)
except json.JSONDecodeError:
return {"entities": [], "_raw": raw}
Step 4: Validate against a schema
Before I push results downstream, I verify that every returned object contains the required keys. This catches drift without adding heavy dependencies.
def validate_entities(result: dict, source_text: str) -> dict:
validated = {"entities": []}
for ent in result.get("entities", []):
if all(k in ent for k in ("text", "label", "start", "end")):
# Sanity check: start and end must be inside text bounds
if 0 <= ent["start"] < ent["end"] <= len(source_text):
validated["entities"].append(ent)
return validated
Step 5: Batch process a list of documents
For production use, I usually process multiple snippets in a loop. I collect validated results into a single list for export or database insertion.
def process_documents(documents: list[str]) -> list[dict]:
outputs = []
for doc in documents:
raw_result = extract_entities(doc)
clean = validate_entities(raw_result, doc)
outputs.append({
"source": doc[:80] + "..." if len(doc) > 80 else doc,
"entities": clean["entities"]
})
return outputs
Run it
Here is a small main block that exercises the pipeline on a few messy strings. I run this directly to verify offsets and labels before wiring it into an ETL job.
if __name__ == "__main__":
docs = [
"Apple Inc. is planning to open a new office in Berlin by March 2025, investing $50 million.",
"Dr. Sarah Chen presented her findings at Stanford University last Tuesday.",
"No entities here, just a random string of thoughts."
]
results = process_documents(docs)
for r in results:
print(json.dumps(r, indent=2))
Example output:
{
"source": "Apple Inc. is planning to open a new office in Berlin by March 2025, investing $50 million.",
"entities": [
{"text": "Apple Inc.", "label": "ORGANIZATION", "start": 0, "end": 10},
{"text": "Berlin", "label": "LOCATION", "start": 51, "end": 57},
{"text": "March 2025", "label": "DATE", "start": 61, "end": 71},
{"text": "$50 million", "label": "MONEY", "start": 85, "end": 96}
]
}
{
"source": "Dr. Sarah Chen presented her findings at Stanford University last Tuesday.",
"entities": [
{"text": "Sarah Chen", "label": "PERSON", "start": 4, "end": 14},
{"text": "Stanford University", "label": "ORGANIZATION", "start": 43, "end": 62}
]
}
{
"source": "No entities here, just a random string of thoughts.",
"entities": []
}
Wrap-up
You now have a lightweight NER service backed by an Oxlo.ai model. A good next step is to benchmark this pipeline against a labeled dataset in your domain and swap in a specialized model like Qwen 3 32B if you need multilingual coverage. You could also wire the extraction function into an async queue to handle larger volumes without blocking. For cost predictability on long documents, remember that Oxlo.ai uses per-request pricing, so your bill stays flat regardless of input length. Check the details at https://oxlo.ai/pricing.
Top comments (0)