DEV Community

Sonam
Sonam

Posted on

Build an AI Moderation Classifier in Python

#python #ai #flask #moderation

Moderation is one of those product features where "just ask an LLM" sounds tempting, but it is not always the best workflow.

Some content is obvious. Known spam, repeated abuse patterns, and common scam text should be caught quickly and consistently.

Other content needs judgment. It may be contextual, vague, sarcastic, or borderline.

I put together a Python Flask example that handles both paths with Telnyx AI Inference:

https://github.com/team-telnyx/telnyx-code-examples/tree/main/moderation-classifier-python

The app uses a two-stage pipeline:

  1. Embeddings pre-filter against a known-bad blocklist.
  2. LLM moderation judgment when there is no strong blocklist match.

The flow

Before moderating content, build the blocklist index:

curl -X POST http://localhost:5000/blocklist/index
Enter fullscreen mode Exit fullscreen mode

That embeds the bundled sample_blocklist.json entries through:

POST /v2/ai/openai/embeddings
Enter fullscreen mode Exit fullscreen mode

Then you can classify a single piece of content:

curl -X POST http://localhost:5000/moderate \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Great product, really enjoyed using it.",
    "source": "review",
    "author_id": "user-123"
  }'
Enter fullscreen mode Exit fullscreen mode

The app returns structured fields:

{
  "category": "safe",
  "confidence": 1.0,
  "flags": [],
  "blocklist_match": false,
  "recommended_action": "allow",
  "reason": "Benign positive product review with no policy violations."
}
Enter fullscreen mode Exit fullscreen mode

If content matches the blocklist strongly enough, the app skips the LLM and returns the matched category directly.

For anything else, it calls:

POST /v2/ai/chat/completions
Enter fullscreen mode Exit fullscreen mode

The default models are:

AI_MODEL=moonshotai/Kimi-K2.6
EMBEDDING_MODEL=thenlper/gte-large
Enter fullscreen mode Exit fullscreen mode

Endpoints included

The example includes:

  • POST /blocklist/index
  • POST /moderate
  • POST /moderate/batch
  • POST /blocklist
  • GET /blocklist
  • GET /moderations
  • GET /moderations/<id>
  • GET /stats
  • GET /health

Batch moderation supports up to 20 items:

curl -X POST http://localhost:5000/moderate/batch \
  -H "Content-Type: application/json" \
  -d '{
    "items": [
      {"content": "Great product!", "source": "review"},
      {"content": "Buy cheap followers now!", "source": "message"}
    ]
  }'
Enter fullscreen mode Exit fullscreen mode

Why this shape is useful

The important idea is separating obvious moderation cases from nuanced ones.

Known-bad patterns can be handled with embeddings similarity.

Ambiguous content can go to an LLM for classification and reasoning.

The response is structured enough to plug into product logic:

  • allow
  • flag
  • remove
  • escalate

For production, I would add persistence, audit logs, human review queues, rate limiting, and a feedback loop so reviewers can tune the blocklist and thresholds over time.

Resources:

Top comments (0)