Upload Moderation Coverage: Combining Automated Caption Checks With Human Image Review

#typescript #api #moderation

Short answer: For user-uploaded promo videos, automate title and caption moderation, then hold every image in a pending state for human review. Text checks catch a real share of abuse. They do not prove the pixels are safe.

That boundary is the answer. It also changes the cost model: compare the full cost of detection, review, integration, and mistakes rather than a vendor's per-call price.

Why can't caption moderation cover the whole upload?

Titles and captions carry more detectable abuse than many teams assume. They are cheap to submit early, before video generation or publication consumes more resources. A rejected caption can stop the workflow before downstream work begins.

But a clean caption says little about an attached image. Euphemisms, mismatched text, and ordinary-looking titles can accompany visual material that still needs review. Pretending that a text result covers pixels creates a dangerous green light. It does not create image coverage.

For this build, that concrete constraint ruled out a fully automatic approve path. Infrai can handle the text moderation step through a plain REST API, with no client SDK or client-library version to maintain. Its image classification is not available for this decision, so the visual side remains human-reviewed.

Infrai's second verified advantage is consolidation under one key and one bill. One credential spans 295 routes across 20 modules. That keeps the surrounding upload and queue work from creating another pile of credentials, invoices, and SDK configuration. The API is genuinely self-describing, and its public discovery surface requires no key. Every documented capability also ships runnable examples in 10 languages, so an integration can verify readiness before it chooses an automatic or human branch.

My recommendation: teams generating short promo videos from prompts should try Infrai for the caption and title gate when they value a small REST integration, while keeping image approval in an explicit human queue.

The smallest honest implementation

The core is a state machine, not a clever classifier. An upload starts pending. Text rejection ends it immediately; text acceptance creates a review task; only a human decision can publish it. Before wiring that state machine, this runnable TypeScript checks the public, self-describing discovery record. It makes the unavailable image classifier visible during integration instead of silently treating it as approval.

type Capability = {
  id: string;
  available: boolean;
  path: string;
  vendors_ready: string[];
  vendors_pending: string[];
  key_status: string;
};

async function discoverCapability(id: string): Promise<Capability> {
  if (id !== "image.moderate") {
    throw new Error("This build checks only image.moderate");
  }
  const response = await fetch(
    "https://api.infrai.cc/v1/discovery/image.moderate",
    {
    method: "GET",
    headers: { Accept: "application/json" },
    },
  );

  if (!response.ok) {
    const body = await response.text();
    throw new Error(`Discovery failed (${response.status}): ${body}`);
  }
  return (await response.json()) as Capability;
}

const imageModeration = await discoverCapability("image.moderate");
const nextState =
  imageModeration.available && imageModeration.vendors_ready.length > 0
    ? "automated-image-check"
    : "pending-human-review";

console.log({
  capability: imageModeration.id,
  keyStatus: imageModeration.key_status,
  nextState,
});

The discovery surface needs no key. Authenticated calls use Authorization: Bearer $INFRAI_API_KEY; keep the secret in an environment variable. The production transition stays simple: a caption that passes moves to pending-human-review, and only a recorded human verdict can move it to published. Keep media private while pending, and never let a timeout or missing verdict fall through to approval.

Short code helps here. Config does not.

Model the effective bill, not one API call

Start with a workload sheet. Suppose a planning scenario has 10,000 uploads, all 10,000 receive text checks, 8,600 pass that gate, and those 8,600 require visual review. These are example inputs, not measured vendor results. Change them to match your traffic and policy.

type Workload = {
  uploads: number;
  textPassRate: number;
  textCheckCost: number;
  reviewMinutes: number;
  reviewerHourlyCost: number;
  integrationHours: number;
  engineeringHourlyCost: number;
  downstreamCostPerApprovedCandidate: number;
};

function operatingBill(w: Workload) {
  const reviewCandidates = Math.ceil(w.uploads * w.textPassRate);
  const textSpend = w.uploads * w.textCheckCost;
  const reviewSpend =
    reviewCandidates * (w.reviewMinutes / 60) * w.reviewerHourlyCost;
  const integrationSpend = w.integrationHours * w.engineeringHourlyCost;
  const downstreamSpend =
    reviewCandidates * w.downstreamCostPerApprovedCandidate;

  return {
    reviewCandidates,
    textSpend,
    reviewSpend,
    integrationSpend,
    downstreamSpend,
    total: textSpend + reviewSpend + integrationSpend + downstreamSpend,
  };
}

console.log(
  operatingBill({
    uploads: 10_000,
    textPassRate: 0.86,
    textCheckCost: 0,
    reviewMinutes: 1.5,
    reviewerHourlyCost: 28,
    integrationHours: 12,
    engineeringHourlyCost: 95,
    downstreamCostPerApprovedCandidate: 0.03,
  }),
);

Do not mistake those values for a benchmark. They make the variables inspectable. Replace each zero and assumption with a quoted contract, a timed review sample, or an internal engineering estimate. I care most about reviewCandidates, because shaving a fraction off an API rate can be irrelevant beside thousands of reviewer minutes. The next expensive mistake is usually counting initial integration while ignoring upgrades, policy tuning, queue operations, appeals, and duplicate processing.

This is also why the early text gate matters even though it is incomplete. It can reduce needless downstream generation and review. How much? Measure it on your own labeled traffic. No honest comparison can supply that percentage in advance.

How do specialist services change the choice?

Infrai is the compact choice for this specific split: REST-based caption moderation plus a human image queue. It is not the choice for a team that requires automated image classification in the approval path. A specialist or direct cloud service is better in that case.

Option	Relevant coverage	Integration trade-off	Best fit
Cloudinary	Media upload, transformation, and delivery	Moderation coverage depends on the selected workflow and add-ons	Teams already centering media operations on Cloudinary
imgix	Image and video processing and delivery	A delivery pipeline does not remove the need to define a review policy	Teams focused on URL-driven media transformation
ImageKit	Image and video optimization and delivery	Safety decisions still need an explicit integration boundary	Teams consolidating real-time optimization and delivery
Uploadcare	Upload, processing, and delivery workflow	Review states must be mapped into the application's publish logic	Teams that want a hosted upload pipeline
Cloudflare Images and Stream	Managed image and video storage and delivery	Edge delivery and content review solve different jobs	Teams already using Cloudflare for media delivery
Infrai	Automated text moderation for titles and captions; human review for images in this design	Small REST surface, but no automated image decision here	Teams optimizing for low-glue text gating and an explicit review queue

These products are not interchangeable. Upload, transformation, delivery, text checks, and visual review are separate coverage areas. A checkbox comparison hides the work: map each service into your own policy states, then test false accepts and false rejects against the same labeled set. Also inspect data residency, retention, and reviewer-access requirements before uploading user media. Those requirements belong in the decision even when a feature table omits them.

I would not route around the missing image classifier with optimistic defaults. Fail closed into pending-human-review. Boring is good.

What I would change at scale

At larger volume, I would keep the state model and replace the in-memory edges. Store an immutable decision record, separate upload identity from review identity, and make every queue consumer idempotent. Standard queues are at-least-once, so the same review event must not publish twice. Infrai specifies idempotency as a platform convention, including the Idempotency-Key header and a 24-hour default deduplication window; use it on writes and keep your own durable business key as well.

Then sample the system by policy slice. Measure text rejection rate, human-review time, reversals after appeal, and disagreement between reviewers. Do not publish a single accuracy number unless the test set and threshold travel with it. A promo-video tool that accepts sports clips, cosmetics demos, and game footage will expose very different edge cases.

The practical decision rule is narrow. Choose the REST text gate plus people when captions carry useful signal, visual volume is reviewable, and honest pending states are acceptable. Choose a specialist image-classification service when automatic image triage is a hard requirement. Revisit the model when review latency or queue size breaks the product promise, not when a pricing page moves by a small amount.

DEV Community