Named entity recognition and structured information extraction have traditionally relied on fine-tuned transformers or rule-based pipelines. These approaches demand labeled training data and rigid schemas that break when your document format changes. Large language models can perform zero-shot and few-shot extraction with a single prompt, but production deployments expose a predictable problem. When you process long reports, legal contracts, or research papers, token-based billing scales linearly with input length and the cost of inference becomes unpredictable. Oxlo.ai removes that variable with flat per-request pricing and an OpenAI-compatible API, making it straightforward to extract entities from documents of any length without watching token counters.
Why Use LLMs for Extraction?
Specialized NER models such as spaCy or BERT-based taggers are fast and deterministic, but they require domain-specific fine-tuning whenever you add a new entity type. LLMs generalize across domains because they are trained on broad corpora. You can change your schema from person, organization, and location to product SKU, defect code, and manufacturing date simply by rewriting the prompt. This flexibility is useful for early-stage pipelines and for extraction tasks where training data is scarce.
The trade-off is latency and cost. Because LLMs process the entire prompt at once, long inputs generate more tokens on standard providers. Oxlo.ai charges per request, not per token, so the cost of extracting entities from a 10,000-word legal brief is the same as extracting from a 100-word email. This predictability matters when you are building agentic workflows that chunk, summarize, and extract in loops.
Enforcing Structure with JSON Mode
Raw text completions are brittle for information extraction. A model might return entities in a narrative format that breaks your downstream parser. JSON mode forces the model to emit valid JSON, which you can validate and load immediately. Oxlo.ai supports JSON mode across its LLMs, including Llama 3.3 70B and Qwen 3 32B, through the standard OpenAI SDK.
When you set response_format={"type": "json_object"}, the model is constrained to produce parseable output. You can combine this with a system prompt that defines the schema, or you can use function calling to bind the extraction to a typed signature. Both patterns reduce hallucinated keys and malformed arrays.
Designing Prompts for NER and IE
A good extraction prompt has three parts: the schema definition, the input text, and formatting instructions. Be explicit about entity types and cardinalities. If a document can contain multiple people, say so. If a date must follow ISO-8601, specify that in the prompt.
Few-shot examples improve accuracy on ambiguous cases. Include two or three examples of input text and the corresponding JSON output inside the prompt. Keep examples representative of your real data distribution. If you are processing invoices, do not use news articles as examples.
End-to-End Example with Oxlo.ai
The following Python script uses the OpenAI SDK to extract entities from a product review. Point the client at https://api.oxlo.ai/v1, set your API key, and define the desired JSON schema in the system message. Because Oxlo.ai is fully OpenAI SDK compatible, the only change from a standard implementation is the base URL.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY"),
)
system_prompt = """You are an information extraction system.
Extract the following entities from the product review and return valid JSON:
- product_name (string)
- mentioned_features (array of strings)
- sentiment (string, one of: positive, neutral, negative)
- price_mentioned (boolean)
If an entity is missing, use null."""
user_prompt = """Review: I bought the UltraPhone 14 last week for $899.
The battery life is incredible and the camera is sharp, but the charging port feels loose.
Overall I am happy despite the minor hardware issue."""
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt},
],
response_format={"type": "json_object"},
temperature=0.1,
)
print(response.choices[0].message.content)
Running this against Oxlo.ai returns a structured object such as:
{
"product_name": "UltraPhone 14",
"mentioned_features": ["battery life", "camera", "charging port"],
"sentiment": "positive",
"price_mentioned": true
}
With temperature=0.1, the model stays close to the text and avoids speculative entities. For stricter validation, you can add a Pydantic model on your side and retry if the output fails schema checks.
The Case for Flat Pricing on Long Contexts
Information extraction is often applied to long-form documents: SEC filings, medical records, or technical manuals. On token-based platforms, sending a 128K context to a large model can dominate your bill, especially when you run extraction across thousands of documents. Oxlo.ai uses flat per-request pricing, so your cost does not scale with prompt length. This makes it significantly cheaper for long-context and agentic workloads where chunks are reprocessed or passed through multiple reasoning steps.
If you are evaluating providers, compare your current invoice against Oxlo.ai's request-based plans. The pricing page outlines the free, Pro, and Premium tiers. The free tier includes 60 requests per day and access to models such as
Top comments (0)