Building a Named Entity Recognition Tool with LLM

Named Entity Recognition remains one of the most common preprocessing steps in production NLP pipelines. Extracting names, dates, locations, and custom categories from unstructured text historically required spaCy, fine-tuned BERT variants, or brittle regex. Large language models can perform zero-shot NER with better accuracy and far less engineering, but token-based inference costs scale directly with document length. For long reports, legal contracts, or agentic workflows that scan hundreds of pages, those costs compound quickly. Oxlo.ai removes that constraint with flat, request-based pricing, making LLM-powered NER practical at any input length.

Why LLMs for NER

Traditional NER systems need retraining whenever your schema changes. An LLM adapts from a prompt. It handles nested entities, implicit references, and multilingual text without a custom model fork. With structured output modes, you can enforce valid JSON schemas so the model returns clean, machine-readable annotations instead of free text.

Modern inference platforms also support function calling and multi-turn conversations, which means you can build agentic pipelines that extract entities, verify them against a knowledge base, and refine results in a single session.

Designing the Extraction Schema

Start by defining exactly what you want extracted. A strict schema reduces hallucination and makes downstream parsing trivial. Below is a minimal example for a news monitoring pipeline.

{
  "entities": [
    {
      "text": "string",
      "label": "PERSON | ORG | LOC | DATE | PRODUCT | EVENT",
      "start": 0,
      "end": 0
    }
  ]
}

Keep labels concise and mutually exclusive. If you need custom types such as REGULATION_ID or CHEMICAL_COMPOUND, add them to the enum and include a one-line definition in the system prompt.

Selecting a Model on Oxlo.ai

Oxlo.ai hosts more than 45 open-source and proprietary models that work as drop-in replacements for the OpenAI SDK. For NER, the best choice depends on your text and latency requirements.

• Llama 3.3 70B is a reliable general-purpose flagship. It follows structured output instructions tightly and works well for English-centric extraction.
• Qwen 3 32B shines when your documents are multilingual or when you need agent workflows that combine NER with reasoning across languages.
• DeepSeek R1 671B MoE and DeepSeek V4 Flash are useful when entities are implied rather than explicit. The extra reasoning capacity helps disambiguate pronouns and indirect references.
• Kimi K2.6 supports 131K contexts and advanced reasoning, so you can pass entire chapters or long transcripts without aggressive chunking.

All of these models are available through a single OpenAI-compatible endpoint with no cold starts, so you can switch models by changing one string in your client configuration.

Building the Pipeline

Because Oxlo.ai is fully OpenAI SDK compatible, you can reuse existing Python tooling. Point your client to https://api.oxlo.ai/v1, set the model alias, and request JSON mode.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("OXLO_API_KEY"),
    base_url="https://api.oxlo.ai/v1"
)

SYSTEM_PROMPT = """You are a precise NER engine.
Extract all entities matching the schema.
Return only valid JSON. Do not add commentary."""

USER_PROMPT = """Extract entities from the following text:

Apple Inc. is planning to open a new office in Berlin by March 2026.
Tim Cook said the project will create 500 jobs."""

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": USER_PROMPT}
    ],
    response_format={"type": "json_object"},
    temperature=0.1
)

print(response.choices[0].message.content)

Running this against Oxlo.ai returns structured JSON that you can validate with Pydantic and feed directly into a database or downstream workflow.

Handling Long Documents

NER on contracts, medical records, or research papers often means inputs of ten thousand tokens or more. On token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, longer prompts raise the cost of every single request. Oxlo.ai uses flat, request-based pricing instead. One API call costs the same regardless of whether your prompt is two hundred tokens or twenty thousand.

For long-context and agentic workloads, that pricing model can be 10-100x cheaper than token-based alternatives. You can send larger windows per request, reduce the complexity of your chunking logic, and still predict your monthly bill with certainty. Oxlo.ai also offers models such as DeepSeek V4 Flash with a 1M context window and Kimi K2.6 with 131K context, so you have the headroom to process lengthy documents in a single shot.

Cost and Scaling

Predictable pricing matters when you are processing thousands of articles or running an agentic loop that calls the NER step repeatedly. Oxlo.ai keeps the model simple:

• Free: $0 per month, 60 requests per day, access to 16+ free models, plus a 7-day full-access trial.
• Pro: $80 per month, 1,000 requests per day, all models included.
• Premium: $350 per month, 5,000 requests per day, all models, priority queue.
• Enterprise: Custom contracts with unlimited requests, dedicated GPUs, and guaranteed savings against your current provider.

Because the price is per request, doubling your prompt length does not double your cost. You can see the latest details at https://oxlo.ai/pricing.

Conclusion

LLMs have made production-grade NER accessible without months of labeling and fine-tuning. The remaining barrier for many teams is inference pricing that balloons as soon as documents get long. Oxlo.ai removes that barrier with flat, request-based pricing, OpenAI SDK compatibility, and a broad catalog of models ranging from fast general-purpose workhorses to deep-reasoning flagships. If you are building an extraction pipeline, the combination of predictable costs and zero cold starts makes Oxlo.ai a genuinely strong place to run it.