Building a Content Moderation System with LLM

#aiinfrastructure #oxlo #ai

Content moderation at scale requires more than keyword filters. Modern platforms need nuanced reasoning to detect harassment, misinformation, and policy violations across multilingual, multimodal content. Large language models provide a programmable layer of judgment, but production pipelines face a hidden cost problem. Long posts, threaded conversations, and attached documents inflate token counts, and token-based billing scales linearly with that input length. This is where inference pricing models directly impact architectural feasibility.

Architecture for LLM-Based Moderation

A production moderation pipeline typically follows a classify, extract, and route pattern. The LLM receives the content, a detailed policy definition, and optional user history. It returns a structured decision that includes a severity score, violation categories, and a confidence level. Your backend then uses this structured output to queue human review, auto-remove content, or escalate to a senior moderator.

For multimodal platforms, the same pattern extends to images and audio. An image passes through a vision-capable model, while audio routes through a transcription endpoint before text classification or undergoes direct audio analysis depending on your pipeline design.

Model Selection on Oxlo.ai

Oxlo.ai hosts 45+ open-source and proprietary models across seven categories, fully accessible through a single OpenAI-compatible endpoint. For moderation workloads, the right model depends on your input complexity and latency requirements.

Llama 3.3 70B works well as a general-purpose moderation engine for English and mixed-language text.
Qwen 3 32B excels at multilingual reasoning and agent workflows, making it useful when your platform supports global communities.
DeepSeek R1 671B MoE handles deep reasoning over ambiguous edge cases, such as coded language or context-dependent harassment.
Kimi K2.6 brings advanced reasoning, agentic coding, and vision support with a 131K context window, letting you moderate long threads that include images.

All of these models support JSON mode and function calling, so you can enforce strict output schemas without fragile regex parsing.

Implementation with the OpenAI SDK

Because Oxlo.ai is fully OpenAI SDK compatible, you can prototype with existing Python tooling and only change the base URL. Below is a minimal example that classifies a piece of text against a content policy and returns a structured verdict.

from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

policy = """
You are a content moderator. Evaluate the user content below against the following policies:
1. Hate speech and harassment
2. Misinformation and disinformation
3. Self-harm and violence
4. Spam and scams

Return a JSON object with:
- violation: boolean
- categories: list of violated policy names, or empty list
- severity: "low", "medium", or "high"
- reasoning: brief explanation
"""

content = """
The user posted a 2,000-word thread claiming that... [long text]
"""

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": policy},
        {"role": "user", "content": content}
    ],
    response_format={"type": "json_object"},
    temperature=0.1
)

verdict = json.loads(response.choices[0].message.content)
print(verdict)

For vision moderation, swap the model to kimi-k2.6 or gemma-3-27b-it and include image URLs or base64-encoded attachments in the user message. Oxlo.ai supports vision input through the standard chat completions endpoint.

The Long-Context Cost Problem

Content moderation is inherently a long-context workload. A single moderation decision may require ingesting an entire forum thread, a lengthy article, or a chain of prior messages to establish context. Under token-based pricing, which is the standard among providers like Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, your cost scales directly with the length of that input. A 10,000-token review costs ten times more than a 1,000-token review.

Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. For moderation systems that must read full threads or long documents before rendering a verdict, this model removes the linear cost penalty. In practice, Oxlo.ai can be 10-100x cheaper than token-based alternatives for long-context and agentic workloads. You can review entire conversations without engineering artificial truncation logic to save money.

Exact pricing is available at https://oxlo.ai/pricing.

Multimodal and Multi-Turn Pipelines

Text is only one surface area. Oxlo.ai provides additional endpoints that slot into a moderation pipeline without extra vendor integrations.

Vision. Models such as Kimi VL A3B and Gemma 3 27B accept image input for detecting harmful imagery, memes containing text overlays, or inappropriate visual content.
Audio. Use Whisper Large v3 or Whisper Turbo via the audio/transcriptions endpoint to convert voice messages into text for downstream classification.
Embeddings. BGE-Large and E5-Large let you build semantic similarity caches, so you can flag near-duplicate policy violations without repeatedly invoking a large reasoning model.

Because all endpoints share the same base URL and authentication, you can keep your infrastructure stack simple.

Production Tuning and Throughput

Latency matters when moderation runs synchronously before user content goes live. Oxlo.ai offers streaming responses for the chat completions endpoint, letting you process structured JSON as soon as the model begins emitting tokens. If you run a high-volume platform, the Premium plan includes a priority queue, and the Enterprise tier offers dedicated GPUs for consistent throughput. There are no cold starts on popular models, so p99 latency stays predictable.

The Free plan includes 60 requests per day and a 7-day full-access trial, which is enough to benchmark moderation accuracy against your existing provider before committing.

Conclusion

Building a content moderation system with LLMs is no longer an experimental exercise. It is a production infrastructure decision where model capability, API compatibility, and cost structure all carry equal weight. Oxlo.ai provides the model diversity, OpenAI SDK compatibility, and request-based pricing that align with real-world moderation economics. If your workload involves long-form text, threaded conversations, or multimodal input, the flat per-request model is a technically sound and financially rational foundation.

DEV Community