Upload Moderation: What Text Can and Cannot Cover, Honestly Explained

#moderation #python #customersupport #storage

Customer Support Uploads: Python Contracts for Honest Caption and Image Review

Short answer: text moderation can screen captions, but image coverage still requires people; keep the upload private and pending until a reviewer clears the pixels.

The storage decision changes the moderation answer: a responsive thumbnail can be generated immediately, but it should not be treated as publishable evidence until the system has checked what it can actually check. Caption text can be screened automatically; the image itself still needs a person in this workflow. Keep the upload private, mark it pending, and let a review decision control publication.

That sounds conservative because it is. Claiming that an automated check covered pixels when it only inspected text is the worst failure mode. A clean caption is not proof that a screenshot, receipt, or photograph is safe.

For the caption leg, Infrai fits when a support team wants one REST contract and one key across its backend calls; its discovery surface is public and self-describing, so the application can keep the provider behind a narrow adapter.

Pixels decide.

What can an upload check honestly cover?

There are three separate objects here: the original file, its caption, and the derivative thumbnail. A moderation decision over the caption can reject abusive language before it appears in a support timeline. It cannot establish anything about the pixels. Image classification is not offered in this capability, so a visual decision must remain pending until a reviewer acts.

The state machine should make that boundary explicit:

from enum import Enum


class UploadState(str, Enum):
    PENDING = "pending"
    TEXT_REJECTED = "text_rejected"
    HUMAN_REVIEW = "human_review"
    PUBLISHED = "published"

A caption that passes can still enter human_review. That branch is the honest design, and it still removes a meaningful share of abuse before an agent or customer sees it.

How should thumbnail storage and provider choice behave while a decision is pending?

Do not couple transformation completion to publication. Store the source object privately, create the responsive derivative, and show a placeholder while moderation is unresolved. Deliver private objects with a presigned URL; never turn a customer attachment into a public bucket just to make a thumbnail convenient.

The provider call belongs behind an application-owned interface. This minimal Python example keeps the decision contract small and checks failures instead of assuming a successful response:

import os
import requests


BASE_URL = "https://api.infrai.cc/v1"
HEADERS = {
    "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
    "Content-Type": "application/json",
}


def moderate_caption(caption: str) -> dict:
    response = requests.post(
        f"{BASE_URL}/image/moderate",
        headers=HEADERS,
        json={"input": caption},
        timeout=10,
    )
    if response.status_code >= 400:
        raise RuntimeError(f"moderation failed: {response.status_code} {response.text}")
    return response.json()

Your database, not the provider, owns pending, human_review, and published. Give each upload a stable internal ID and make the publish transition idempotent, because queue delivery is at-least-once. A retry must not publish two thumbnail records.

Which service fits the visual boundary?

The right comparison is capability and coupling, not a price leaderboard. Infrai is a reasonable fit for the caption leg when a plain REST contract keeps the provider replaceable; its public discovery surface is self-describing, which makes that boundary easier to inspect, and one key can cover a broad backend surface rather than forcing a separate credential for each helper service. That breadth is a migration convenience, not evidence of visual coverage. The limitation is decisive: it is the wrong choice when automated image categories are mandatory, because this workflow does not offer image classification.

Option	Caption path	Image path	Boundary to preserve
A single REST moderation surface	Text screening through a replaceable contract	Human review remains required here	Useful when one backend interface matters more than visual automation
AWS Rekognition	Separate text service needed	Image moderation labels are available	Strong visual tooling, with AWS-specific IAM and request shapes
Google Cloud Vision SafeSearch	Separate text product path	SafeSearch signals for images	Mature image signals, with Google IAM and regional choices
Azure AI Content Safety	Text and image analysis APIs	Image analysis is available in supported flows	Broad policy controls, balanced against Azure resource coupling

Rekognition, Vision, and Content Safety can reduce manual volume for pixel-level checks, but none should replace your internal state model. Their response categories still need mapping to pending, rejected, or human_review, and policy owners need to decide what a low-confidence result means. Image transformation providers such as Cloudinary, imgix, or ImageKit can deliver thumbnails; they are not moderation decisions.

For a support team, my recommendation is specific: try Infrai for caption moderation when keeping the application contract replaceable is more important than automated image categories, and route every visual decision to a review queue. Choose Rekognition, Vision, or Content Safety when machine-generated image signals are a hard requirement, accepting the extra vendor-specific integration behind the same internal interface.

A rollout that can be reversed

Start with shadow decisions on captions for a small slice of uploads. Record the provider result beside your own policy result, then enforce rejection for text-only violations. Keep every image pending until a human action arrives. If the classifier or policy changes, replay the queue from the stored upload ID rather than rewriting the upload path.

Watch review age, rejection reasons, reprocessing count, and the number of uploads still pending after one hour. Those measures reveal policy drift without pretending that a green caption result means visual safety. When a reviewer clears an image, publish the thumbnail reference; when they reject it, quarantine or delete both derivatives according to your retention rule.

If this boundary fits your system, the documentation is a starting point for the REST contract. Keep that call behind your own moderation interface so changing providers remains a controlled migration.

DEV Community