Using LLM Models for Text Classification: A Guide

#aiinfrastructure #oxlo #ai

Text classification remains one of the most common NLP workloads, but traditional machine learning pipelines require labeled datasets, feature engineering, and retraining for every new category. Large language models change this dynamic. With zero-shot or few-shot prompting, an LLM can classify text into custom categories without gradient updates, making it practical to deploy classifiers in hours rather than weeks. The challenge shifts from model training to inference infrastructure: latency, cost, and output structure. Oxlo.ai provides an OpenAI SDK-compatible platform with request-based pricing and dozens of models suited for classification tasks.

Zero-Shot Classification with Chat Models

Zero-shot classification is the fastest way to prototype. You define categories and criteria in the system prompt, and the model returns the label. This works because models like Llama 3.3 70B and Qwen 3 32B have broad pretraining that captures semantic relationships without task-specific fine-tuning.

Below is a minimal example using the OpenAI SDK pointed at Oxlo.ai. Replace YOUR_OXLO_API_KEY and the model identifier with your own values.

import openai

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

response = client.chat.completions.create(
    model="your-model-id",  # e.g., Llama 3.3 70B or Qwen 3 32B on Oxlo.ai
    messages=[
        {"role": "system", "content": "Classify the user message into one category: Support, Billing, or Technical. Respond with a single word."},
        {"role": "user", "content": "I was charged twice for my subscription this month."}
    ],
    temperature=0.0
)

label = response.choices[0].message.content.strip()
print(label)

Setting temperature=0.0 reduces randomness, which is important when you need deterministic labels.

Improving Accuracy with Few-Shot Prompting

When zero-shot accuracy is insufficient, adding labeled examples inside the prompt often corrects edge cases. Few-shot prompting shows the model the exact format and boundary cases you care about, without any training infrastructure.

messages = [
    {"role": "system", "content": "Classify the message into Support, Billing, or Technical."},
    {"role": "user", "content": "My login fails every morning."},
    {"role": "assistant", "content": "Technical"},
    {"role": "user", "content": "Can I get a refund for last month?"},
    {"role": "assistant", "content": "Billing"},
    {"role": "user", "content": "I was charged twice for my subscription this month."}
]

response = client.chat.completions.create(
    model="your-model-id",
    messages=messages,
    temperature=0.0
)

Keep examples concise. Long example blocks increase latency on token-based providers, but on Oxlo.ai the cost per request stays flat regardless of prompt length.

Enforcing Structured Outputs with JSON Mode

For production pipelines, parsing free-text responses is fragile. Oxlo.ai supports JSON mode, which constrains the model to return valid JSON that matches your schema. This eliminates regex parsing and reduces downstream errors.

import json

response = client.chat.completions.create(
    model="your-model-id",
    messages=[
        {"role": "system", "content": "You are a classifier. Respond only with JSON containing keys: label, confidence."},
        {"role": "user", "content": "I was charged twice for my subscription this month."}
    ],
    response_format={"type": "json_object"},
    temperature=0.0
)

result = json.loads(response.choices[0].message.content)
print(result["label"], result["confidence"])

JSON mode is available across Oxlo.ai chat models and pairs well with function calling or tool use if you need to trigger downstream actions after classification.

Selecting a Model on Oxlo.ai

Oxlo.ai hosts 45+ open-source and proprietary models across 7 categories, all accessible through a single endpoint with no cold starts. For classification, the LLMs / chat and reasoning category is most relevant. Specific models to consider include:

Llama 3.3 70B: A general-purpose flagship that balances accuracy and speed for standard text classification.
Qwen 3 32B: Strong multilingual reasoning and agent workflows, ideal if your documents contain mixed languages.
DeepSeek V4 Flash: An efficient MoE with a 1M context window and near state-of-the-art open-source reasoning, useful when classifying entire reports or books in a single request.
Kimi K2.6: Advanced reasoning, agentic coding, vision, and 131K context. Use this when inputs include images or when you need deep semantic analysis.
DeepSeek V3.2: Focused on coding and reasoning, and available on the free tier. A good starting point for prototype classifiers.
GLM 5: A 744B MoE built for long-horizon agentic tasks. Consider it for complex hierarchical classification.

Because Oxlo.ai is fully OpenAI SDK compatible, switching between these models requires changing only the model string.

Cost Considerations for High-Volume Classification

Classification workloads often involve long documents: legal contracts, medical records, or customer chat histories. On token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, cost scales linearly with input length. A single long-context request can consume as many tokens as dozens of short requests.

Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. This means a 10,000-token document costs the same to classify as a 50-token tweet. For long-context and agentic classification workloads, this model can be 10-100x cheaper than token-based alternatives. You can view current plan details at https://oxlo.ai/pricing.

For development, the Free plan offers 60 requests per day across 16+ models, including a 7-day full-access trial. The Pro plan at $80 per month provides 1,000 requests per day and access to all models. The Premium plan at $350 per month includes 5,000 requests per day and priority queueing. Enterprise customers can get dedicated GPUs and unlimited volume with custom pricing.

Evaluating Classifier Performance

Treat an LLM classifier like any other model. Reserve a holdout set of labeled examples and measure precision, recall, and F1 per class. Pay attention to:

Consistency: Run the same input multiple times with temperature=0.0 and verify label stability.
Boundary cases: Test ambiguous examples that sit between two categories.
Latency: Measure end-to-end time, including network overhead. Oxlo.ai streams responses, so you can start processing the label as soon as the first tokens arrive if you use streaming for non-JSON outputs.

If you need embeddings for a hybrid retrieval-classification pipeline, Oxlo.ai also offers embedding models such as BGE-Large and E5-Large through the same API.

Conclusion

LLMs have turned text classification from a training-heavy task into a prompt-engineering problem. With zero-shot and few-shot techniques, JSON mode for structured output, and a broad catalog of models, you can build robust classifiers without managing GPUs or labeling thousands of examples. Oxlo.ai gives you OpenAI SDK-compatible access to over 45 models, flat request-based pricing that favors long documents, and no cold starts. It is a strong fit for teams that want predictable costs and model flexibility for classification at scale.

DEV Community