Unlocking Named Entity Recognition with LLMs

Named Entity Recognition has moved past the era of hand-tuned gazetteers and rigid regex pipelines. Modern applications need to extract nested, contextual, and domain-specific entities from messy, real-world text. Large language models offer a flexible alternative. They read context, follow typed instructions, and return structured output without retraining an entire pipeline for every new entity class.

Beyond Regex and Traditional NLP

Traditional NER systems rely on conditional random fields, BiLSTMs, or transformer-based token classifiers. These approaches work well for common classes like PERSON or ORG, but they struggle with ambiguous references, evolving jargon, and long-range dependencies. Retraining or fine-tuning for a new entity type requires labeled data, compute budgets, and deployment gymnastics. LLMs sidestep most of this by treating entity extraction as an in-context reasoning task.

Why LLMs Change the Game

An LLM does not need a new output layer to recognize a novel entity schema. You describe the target labels in natural language, provide a few examples if needed, and the model generalizes. This is especially useful for extracting product codes, legal clauses, or biomedical relationships where standard corpora do not exist. Structured outputs, function calling, and JSON mode turn free-form text into machine-readable annotations without fragile post-processing.

Prompt Engineering for Structured NER

The difference between a leaky NER pipeline and a reliable one often comes down to the prompt. Be explicit about entity definitions, boundary rules, and output format. If an entity can overlap or nest, say so. If dates must follow ISO-8601, specify that.

Here is a minimal but strict prompt template that works well with JSON mode:

Extract all entities from the text below.
Allowed labels: PERSON, ORGANIZATION, LOCATION, PRODUCT, EVENT.
Rules:
- Return a JSON list of objects.
- Each object must have "text", "label", and "start" keys.
- Do not include entities that are not in the allowed list.
- If no entities are found, return an empty list.

Text:
"""{{text}}"""

Running this through an LLM with JSON mode enforces schema compliance at the API level, which removes an entire class of parsing errors.

Handling Long Documents

Real-world NER rarely happens on single sentences. You might need to process earnings call transcripts, legal contracts, or clinical notes that span tens of thousands of tokens. Chunking text introduces boundary errors where entities sit on the seam between two windows. A model with a large context window lets you feed entire documents in one shot, preserving cross-sentence and cross-paragraph context.

Oxlo.ai hosts several long-context options. DeepSeek V4 Flash supports a 1 million token context window. Kimi K2.6 handles 131K tokens with advanced reasoning and vision capabilities. Qwen 3 32B offers strong multilingual reasoning for global document sets. Because Oxlo.ai charges per request rather than per token, sending a full long-form document costs the same as a one-line query. That flat pricing removes the penalty for using large context windows in production.

Implementation with Oxlo.ai

Oxlo.ai is fully OpenAI SDK compatible, so you can point your existing client at the Oxlo.ai endpoint and start extracting entities immediately. The platform offers no cold starts on popular models, which keeps latency predictable for synchronous NER tasks.

Below is a complete Python example using the OpenAI SDK with Oxlo.ai. It uses JSON mode to enforce output structure and targets Llama 3.3 70B for general-purpose extraction:

import openai

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

def extract_entities(text: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a precise NER engine. Extract entities as JSON. "
                    "Allowed labels: PERSON, ORGANIZATION, LOCATION. "
                    "Return a list of objects with keys: text, label, start."
                )
            },
            {
                "role": "user",
                "content": text
            }
        ],
        response_format={"type": "json_object"}
    )
    return response.choices[0].message.content

document = """
Dr. Sarah Chen and Dr. James Miller of Riverdale Medical Research
collaborated with Novartis on a clinical trial in Basel.
"""

print(extract_entities(document))

For agentic NER workflows, where the model must search a document, decide on relevance, and then extract, you can combine function calling with tool definitions. Oxlo.ai supports function calling and tool use across its chat models, so you can chain extraction steps without leaving the platform.

Model Selection for NER Workloads

Not every NER task needs the largest model. Oxlo.ai offers 45+ models across categories, and several are particularly useful for entity extraction:

• Llama 3.3 70B: A strong general-purpose flagship. Use this when you need reliable English NER with broad world knowledge.
• Qwen 3 32B: Ideal for multilingual documents and agent workflows where entities cross language boundaries.
• DeepSeek R1 671B MoE: Best for deep reasoning tasks, such as extracting implicit relationships or coding entities from technical specifications.
• Kimi K2.6: Choose this when you need advanced reasoning combined with vision. It processes long documents up to 131K tokens and can extract entities from mixed text and image inputs.
• DeepSeek V4 Flash: The efficient choice for massive context. At 1 million tokens, you can NER an entire book in a single request.

If you are building a code-aware pipeline, Qwen 3 Coder 30B and DeepSeek Coder are also available under the Code category on Oxlo.ai.

Cost and Scaling Considerations

Token-based pricing punishes long-context NER. Every paragraph you add to the prompt increases cost. Over thousands of documents, that scaling factor dominates your budget. Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. Unlike token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, cost does not scale with input length, so Oxlo.ai can be 10-100x cheaper for long-context and agentic workloads. This pricing model changes how you architect pipelines. Instead of aggressively chunking text to save tokens, you can send full pages or chapters. Instead of minimizing system prompt length, you can include detailed instructions and few-shot examples. You trade token anxiety for throughput planning. See the exact plans and rates at https://oxlo.ai/pricing.

Conclusion

LLMs have turned NER from a finicky classification problem into a flexible reasoning task. With the right prompt, a capable model, and an API that does not penalize long inputs, you can deploy extraction pipelines in hours rather than weeks. Oxlo.ai provides the model variety, OpenAI SDK compatibility, and flat per-request pricing that make it a natural fit for both prototype and production NER workloads. Sign up for the free tier to test with 60 requests per day and 16+ free models, or explore the full catalog with a 7-day full-access trial.