Building a Named Entity Recognition Tool with LLM: A Step-by-Step Guide

#learnai #oxlo #ai

We are building a lightweight Named Entity Recognition pipeline that uses an LLM to label people, organizations, locations, and dates in unstructured text. It runs through Oxlo.ai's OpenAI-compatible API, so you pay a flat rate per request instead of scaling with token count, which keeps costs predictable even when you feed it long articles. I will walk through the exact code I wrote to ship this.

What you'll need

Python 3.10 or higher
An Oxlo.ai API key from https://portal.oxlo.ai
The OpenAI SDK installed: pip install openai

Step 1: Define the schema and system prompt

The model needs exact instructions and a strict output format. I define a system prompt that lists the four entity types we want and forces a JSON array response. Keeping the labels consistent makes downstream filtering easier.

SYSTEM_PROMPT = """You are a Named Entity Recognition system.
Analyze the user text and extract entities.
Return only a JSON object with this exact structure:
{
  "entities": [
    {"type": "PERSON", "text": "..."},
    {"type": "ORG", "text": "..."},
    {"type": "LOCATION", "text": "..."},
    {"type": "DATE", "text": "..."}
  ]
}
Rules:
- Use exact words from the text.
- Do not guess. If no entities exist, return an empty list.
- Output only the JSON object, no markdown and no explanation."""

Step 2: Set up the Oxlo.ai client

Oxlo.ai exposes an OpenAI-compatible endpoint, so the only difference from a standard OpenAI setup is the base URL and model name. I use llama-3.3-70b because it follows structured instructions reliably.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"  # from https://portal.oxlo.ai
)

Step 3: Build the extraction function

I wrap the API call in a small function that sends the text and parses the JSON response. Oxlo.ai supports JSON mode, so I set response_format to lock the output structure.

import json

def extract_entities(text: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

Step 4: Add post-processing validation

LLMs occasionally hallucinate entity spans. I add a simple guard that drops any entity whose text does not appear in the original input. This keeps the pipeline honest without adding latency.

def validate_entities(result: dict, source_text: str):
    cleaned = []
    for ent in result.get("entities", []):
        span = ent.get("text", "")
        if span and span in source_text:
            cleaned.append(ent)
    return {"entities": cleaned}

Step 5: Run a batch over a document

Real documents usually contain multiple paragraphs. I split the input on double newlines and run each chunk through the extractor, collecting results into a single list.

def process_document(doc: str):
    paragraphs = [p.strip() for p in doc.split("\n\n") if p.strip()]
    all_entities = []
    for para in paragraphs:
        raw = extract_entities(para)
        validated = validate_entities(raw, para)
        all_entities.extend(validated["entities"])
    return {"entities": all_entities}

Run it

Here is a concrete test block with a short news snippet. When I run this locally against Oxlo.ai, it returns structured entities in under a second.

if __name__ == "__main__":
    sample = """Apple Inc. is planning to open a new office in Austin next March.
Tim Cook announced the expansion during a press conference in Cupertino on January 10."""

    output = process_document(sample)
    print(json.dumps(output, indent=2))

Expected output:

{
  "entities": [
    {"type": "ORG", "text": "Apple Inc."},
    {"type": "LOCATION", "text": "Austin"},
    {"type": "DATE", "text": "next March"},
    {"type": "PERSON", "text": "Tim Cook"},
    {"type": "LOCATION", "text": "Cupertino"},
    {"type": "DATE", "text": "January 10"}
  ]
}

Next steps

Wire this into a FastAPI endpoint so other services can POST text and receive annotated JSON. If you need to process entire contracts or books in a single shot, switch the model to kimi-k2.6 on Oxlo.ai. Its 131K context window lets you run NER over long documents without chunking, and the flat per-request pricing means the cost stays the same even when the input grows. See https://oxlo.ai/pricing for plan details.

DEV Community