Building a Text Classification Tool with LLM

Text classification has been a staple of NLP pipelines for decades, but classical approaches require labeled datasets, fine-tuning, and brittle feature engineering. Large language models can perform zero-shot or few-shot classification with minimal setup, turning a multi-week ML project into an afternoon API integration. The challenge is not whether an LLM can classify text, but how to do it reliably, cheaply, and at scale.

When to Use an LLM for Classification

Traditional classifiers, such as logistic regression or fine-tuned transformers, still excel when you have thousands of labeled examples and a frozen taxonomy. However, they degrade quickly when categories change, when labeled data is scarce, or when the input contains noisy, multilingual, or domain-specific language.

An LLM is the better tool when:

• Your taxonomy evolves frequently.
• You need zero-shot or few-shot learning without retraining.
• The input includes long documents where context and reasoning matter.
• You want a single system that handles classification alongside extraction, summarization, or tagging.

The tradeoff is cost and latency. Token-based providers charge for every input and output token, which means long system prompts, few-shot examples, and lengthy documents inflate costs linearly. This is where provider choice becomes an architectural decision, not just a vendor preference.

Selecting a Model and Provider

For classification, you want a model that follows instructions precisely and outputs structured data reliably. Oxlo.ai offers several strong candidates across its catalog of 45+ models.

• Llama 3.3 70B is a solid general-purpose workhorse for English-centric classification.
• Qwen 3 32B handles multilingual reasoning and agent workflows, making it ideal if your inputs arrive in multiple languages.
• DeepSeek R1 671B MoE is useful when categories require deep reasoning or complex coding logic to disambiguate.
• Kimi K2.6 brings advanced reasoning, agentic coding, and vision support with a 131K context window, which helps when classifying entire documents rather than short snippets.

Because Oxlo.ai uses request-based pricing, the cost per classification is flat regardless of how long your prompt is. If you pack few-shot examples, detailed rubrics, or long source text into the context window, your bill does not scale with token count. For long-context workloads, this can be up to 10-100x cheaper than token-based providers such as Together AI, Fireworks AI, or OpenRouter. See the exact rates on the Oxlo.ai pricing page.

Prompt Design and Structured Output

Unstructured text output from an LLM is an operational liability. You need deterministic labels, not prose. The cleanest approach is to use JSON mode, which is supported by Oxlo.ai across its chat models.

A good classification prompt has three parts:

1. System instructions that define the task, the allowed labels, and the output schema.
2. Few-shot examples that show edge cases and disambiguation rules.
3. The target text to classify.

Here is a system prompt template that encourages consistent JSON output:

SYSTEM_PROMPT = """You are a text classification engine.
Your job is to assign exactly one label to the provided text.

Allowed labels: {labels}

Rules:
- If the text fits multiple labels, choose the single best match.
- If the text is ambiguous, set confidence to "low".
- Respond only with a JSON object containing "label" and "confidence".
"""

When you call the model, set response_format={"type": "json_object"} and keep temperature low, around 0.1 to 0.2, to reduce hallucinated labels.

Implementation with the OpenAI SDK

Oxlo.ai is fully OpenAI SDK compatible, so you can drop it into existing Python or Node.js code by changing the base URL.

Install the SDK:

pip install openai

Classify a single document:

python
import os
import json
from open