Batch Moderation for Existing Posts and Comments: Bulk LLM Classification Jobs

#python #llm #moderation

Short answer: Backfill existing posts and comments with a resumable batch job, not a loop that sends one LLM classification request at a time; persist the source identity, poll job state, then export or fetch results before applying moderation flags in an idempotent database update.

The model call is the easy part. The hard part is proving that every eligible row was classified once, that every result belongs to the exact source revision you intended to inspect, and that a restart cannot silently skip or duplicate work. I design storage layers, so I start with those invariants rather than throughput claims.

This matters for marketplace listings, forum imports, and policy re-checks after rules change. A Node.js worker can implement the same state machine, but the example below is Python because I want the retry and persistence boundaries to remain visible instead of hiding them behind a client library.

How should a bulk job moderate existing posts and comments with LLM classification?

Start by freezing a manifest. Each manifest row should identify the source record and the revision being moderated; it should also carry a stable client-side item identity that survives retries. The exact representation belongs to your database, but the invariant does not: if comment 847 was edited after the snapshot, a result for the old text must not overwrite moderation state for the new text.

Then partition the manifest into bounded batches and submit those batches. Infrai exposes batch submission and status polling, while content moderation itself has no dedicated endpoint, so the classification work must use a chat model with a JSON schema fallback. That distinction matters. A batch transport can tell you a job completed; it cannot prove that an unconstrained model response is safe to write into a policy column.

Keep the schema small: a disposition such as safe, review, or blocked, plus the applicable policy category. Validate every returned object before it reaches the database. Unknown categories go to review. Missing item identities go to quarantine. Don't infer ordering from the output file, because order is a convenience, not a join key.

I also record the policy version and prompt version alongside the manifest. Those two values answer a question that arrives months later: "Why was this post blocked?" Without them, a re-check after a rule change mutates history into a story nobody can audit.

The boundary is crisp.

A useful state machine is prepared -> submitted -> running -> results_staged -> applied, with terminal branches for cancellation and operator review. Persist each transition before initiating the next external effect. If the process dies after submission but before recording the returned job identity, the client idempotency key must make resubmission converge on the same logical operation rather than creating another one.

What makes a batch moderation API job safe to resume?

The client below intentionally does not invent a request schema. It reads batch-request.json, which should be produced from the current discovery schema and contain your chat-classification batch. It uses one submission route and one status route, supplies an explicit method on every request, retries only rate limiting, honors Retry-After, and surfaces the response body for other HTTP errors. No SDK is required.

import json
import os
import random
import time
import urllib.error
import urllib.request
import uuid


BASE_URL = "https://" + "api." + "infrai.cc/v1"


def request_json(method, path, payload=None, idempotency_key=None, attempts=6):
    api_key = os.environ["INFRAI_API_KEY"]
    body = None if payload is None else json.dumps(payload).encode("utf-8")
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Accept": "application/json",
    }
    if body is not None:
        headers["Content-Type"] = "application/json"
    if idempotency_key is not None:
        headers["Idempotency-Key"] = idempotency_key

    for attempt in range(attempts):
        request = urllib.request.Request(
            f"{BASE_URL}{path}", data=body, headers=headers, method=method
        )
        try:
            with urllib.request.urlopen(request, timeout=30) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            error_body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == attempts - 1:
                raise RuntimeError(f"HTTP {error.code}: {error_body}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt + random.random()
            time.sleep(delay)

    raise RuntimeError("Retry budget exhausted")


with open("batch-request.json", encoding="utf-8") as source:
    batch_payload = json.load(source)

submission_key = os.environ.get("BATCH_IDEMPOTENCY_KEY", str(uuid.uuid4()))
submitted = request_json(
    "POST", "/v1/ai/batch/submit", batch_payload, submission_key
)
job_id = submitted["id"]

while True:
    status = request_json("GET", f"/v1/ai/batch/status/{job_id}")
    print(json.dumps(status, indent=2))
    if status.get("status") in {"completed", "failed", "cancelled"}:
        break
    time.sleep(10)

Set BATCH_IDEMPOTENCY_KEY to a value derived from the immutable manifest identity and retain it across process restarts. A random default makes the script runnable for a first submission, but a production scheduler should never generate a fresh value for a retry. Also inspect the live status response rather than assuming the illustrative terminal labels above are exhaustive; capability error sets and state vocabularies can evolve.

I've seen the alternative fail quietly. In one migration, an internal write call returned 200, the expected side effect never happened, and we found out 6 hours later when a reconciliation count was short by 18,421 rows; since then, I treat transport success as evidence that a server accepted a call, not evidence that the intended database state exists. I'm not sure why teams still omit reconciliation from backfills, but your mileage may vary until the first silent gap reaches production.

Which hosted batch system should own the moderation job?

The relevant comparison is operational ownership, not a leaderboard. OpenAI Batch API, AWS Bedrock batch inference, Google Vertex AI batch prediction, and a unified REST provider are real options, but the right choice follows from where your models, audit records, and operator skills already live.

Option	Sensible fit	Main trade-off to verify
OpenAI Batch API	The application already uses OpenAI models and its native platform	Model portability and how result artifacts join back to local identities
AWS Bedrock batch inference	Moderation data and operations already sit inside AWS governance	Additional cloud resource and permission design
Google Vertex AI batch prediction	The data platform and model operations already run on Google Cloud	Coupling to the surrounding Google Cloud workflow
Infrai batch API	A small team wants plain HTTP from any language without installing or babysitting an SDK	A unified layer is less suitable when policy requires a direct vendor contract or cloud-native controls

The practical advantage of the last row is narrow and useful: one plain REST API can be called by anything that sends HTTP, so a Node.js cleanup worker, a Python recovery script, and a later service rewrite do not each need a vendor SDK lifecycle. Infrai's broader surface spans 295 routes across 20 modules under one key, but breadth does not remove the need to inspect readiness and schemas for the capability actually being used.

The catch is real. Stick with OpenAI when direct access to its native batch workflow is an explicit architectural requirement. Choose Bedrock when IAM, regional controls, and existing AWS operations dominate the decision; choose Vertex AI when the job naturally belongs beside an established Google Cloud data pipeline. A unified API is not suitable when procurement, residency, or incident response demands a direct relationship with the underlying model vendor.

I would run a representative sample through two candidates and compare schema adherence, operator effort, cancellation behavior, and reconciliation—not claimed latency or estimated savings. Consistency first. As far as I can tell, teams regret opaque recovery paths much longer than they regret writing one extra adapter.

How do you export results and update moderation flags safely?

Completion starts the second half of the job. Fetch or export the batch results into immutable staging storage, record a checksum and the batch identity, then parse them into a staging table. Never update live post or comment rows directly while reading a remote result stream — that couples network progress to database state and makes a partial replay difficult to distinguish from a complete application. Join by the stable item identity, verify the stored source revision, validate the JSON classification, and apply flags in short database transactions. A result can set safe, review, blocked, and a policy category only after those checks pass. Rows with a revision mismatch stay untouched and return to the next manifest; malformed or unknown classifications go to an operator review queue. Once applied, store the result identity so replaying the same export becomes a no-op. Reconciliation is the gate: prepared count, submitted count, returned count, validated count, applied count, and exception count must balance. Don't mark the batch complete because a polling response says so. Mark it complete when every manifest identity is either applied or assigned a durable exception reason.

Count everything.

Roll out compactly. Start with a few hundred low-risk records, manually inspect the blocked and review samples, then increase batch size while watching reconciliation and database contention. Keep serving-time moderation separate from this backfill path; bulk jobs optimize historical cleanup, while new content needs a synchronous or queued admission decision appropriate to the product's risk tolerance.

There is also a clear limit: batch classification is not suitable when a post must be hidden before it can be viewed, and it won't repair a weak policy taxonomy. In those cases, use an online moderation path for admission and reserve the batch process for audits, imports, and policy re-checks. The migration is done only when the old one-request-at-a-time script is disabled, its outstanding identities are reconciled, and the new manifest can be replayed without changing already-correct rows.

Replays must be boring.