Using LLMs for Text Classification and Clustering in Business Applications

#aiinfrastructure #oxlo #ai

Text classification and clustering remain foundational workloads in enterprise AI, from routing support tickets to organizing research libraries. Classical approaches like TF-IDF with logistic regression or k-means require extensive preprocessing, labeled datasets, and brittle feature engineering. Large language models absorb much of that complexity by encoding semantic meaning directly, letting teams ship classifiers and clustering pipelines with minimal training data and no custom embeddings. The shift from classical ML to LLM-based text analytics is not just about accuracy. It is about reducing time to production and handling long, unstructured documents without rebuilding your feature stack.

Zero-Shot and Few-Shot Classification

Modern LLMs can classify text without task-specific fine-tuning. By describing categories in the system prompt and providing a few examples in the context window, you get a flexible classifier that adapts to new taxonomies in minutes rather than weeks. For production use, JSON mode keeps outputs structured and parseable.

Below is a complete example using the OpenAI SDK pointed at Oxlo.ai. It classifies a support ticket into a category, urgency level, and summary using Llama 3.3 70B.

from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

def classify_support_ticket(ticket_text):
    system_prompt = (
        "You are a classification assistant. "
        "Return only a JSON object with keys: category, urgency, summary."
    )
    user_prompt = f"Classify the following support ticket:\n\n{ticket_text}"

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt}
        ],
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)

ticket = "My account was charged twice on March 15th and I need a refund immediately."
print(classify_support_ticket(ticket))

If your workload spans multiple languages, Qwen 3 32B provides strong multilingual reasoning and agentic workflow support. For deep reasoning over complex regulatory text or code-heavy tickets, DeepSeek R1 671B MoE or Kimi K2.6 offer advanced chain-of-thought capabilities and extended context windows.

LLM Clustering Pipelines

Clustering with LLMs typically combines embedding models with classical algorithms. You generate dense vectors for each document, run k-means or HDBSCAN, then use a chat model to generate human-readable labels for each cluster. Oxlo.ai provides embedding endpoints for BGE-Large and E5-Large, which you can call through the same OpenAI-compatible client.

from openai import OpenAI
from sklearn.cluster import KMeans
import numpy as np

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

def get_embeddings(texts):
    response = client.embeddings.create(
        model="bge-large",
        input=texts
    )
    return [item.embedding for item in response.data]

documents = [
    "Invoice processing delayed by two weeks",
    "Server outage in us-east-1 region",
    "Refund request for duplicate charge",
    "VPN connection failing after update"
]

embeddings = get_embeddings(documents)
X = np.array(embeddings)

kmeans = KMeans(n_clusters=2, random_state=42, n_init=10)
labels = kmeans.fit_predict(X)

for cluster_id in np.unique(labels):
    cluster_docs = [d for d, l in zip(documents, labels) if l == cluster_id]
    prompt = f"Give a concise 3-word label for this document cluster: {cluster_docs}"
    resp = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[{"role": "user", "content": prompt}]
    )
    print(f"Cluster {cluster_id}: {resp.choices[0].message.content}")

For larger corpora, you can batch embedding requests and move the heavy lifting to Oxlo.ai's inference layer. Because there are no cold starts on popular models, clustering jobs that alternate between embedding and labeling steps stay responsive even under variable load.

Cost and Scale Considerations

Business text analytics often involves long inputs: legal contracts, customer support transcripts, or multi-page research articles. On token-based inference providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, costs scale linearly with prompt length. For classification and clustering workflows that process these documents, token bills add up quickly.

Oxlo.ai uses request-based pricing: one flat cost per API request regardless of input length. For long-context classification or embedding batches, this can be significantly cheaper than token-based alternatives. You can see current plans on the Oxlo.ai pricing page.

This pricing model changes how you architect pipelines. Instead of truncating text to save tokens, you can pass full documents. Instead of hesitating to run nightly clustering over thousands of long articles, you can batch them without cost scaling by the word. Oxlo.ai also offers no cold starts on popular models, so batch jobs and real-time classification endpoints stay responsive.

Model Selection for Business Workloads

Oxlo.ai hosts more than 45 models across seven categories, all accessible through the same OpenAI-compatible endpoint. For text classification and clustering, the following are particularly relevant:

Llama 3.3 70B: General-purpose flagship, ideal for reliable zero-shot classification.
Qwen 3 32B: Multilingual reasoning and agent workflows for global support queues.
DeepSeek R1 671B MoE: Deep reasoning and complex coding for regulatory or technical document classification.
Kimi K2.6: Advanced reasoning, agentic coding, and vision with a 131K context window for multimodal document pipelines.
BGE-Large / E5-Large: Embedding models for clustering and semantic search.

Because Oxlo.ai is fully OpenAI SDK compatible, switching between these models is a single parameter change. You can benchmark Llama 3.3 70B against Qwen 3 32B on your own dataset without rewriting client code.

Production Tips

When moving from prototype to production, keep the following in mind:

Use JSON mode. It eliminates parsing errors and makes downstream automation trivial.
Batch embeddings. Pass arrays of texts to the embeddings endpoint to reduce network overhead.
Monitor context windows. Even with flat per-request pricing, model context limits still apply. For very long documents, chunk before embedding or summarizing.
Prioritize queue position. If you are running large nightly clustering jobs, the Premium plan includes priority queue access to keep latency predictable.

Conclusion

LLMs have turned text classification and clustering from weeks-long ML projects into afternoon API integrations. The remaining bottleneck is usually cost and infrastructure: long documents, variable traffic, and the need to switch models without rewriting clients. Oxlo.ai addresses these with request-based pricing, no cold starts, and a broad model catalog exposed through a standard OpenAI-compatible API. For teams building business analytics pipelines, it is a relevant option that keeps costs predictable as document length grows.

DEV Community