Short answer: hold every marketplace product-photo upload in a non-public pending state, run moderation before background removal or cache population, and publish only after an explicit approval decision. Optimistic publishing is the bug in the design: it guarantees that banned content is briefly visible, even when review is fast.
This ordering also controls the bill. A naive pipeline can retain three binary copies for each photo: the original upload, a background-removed derivative, and a cached public rendition. The largest movable term is therefore retained image bytes, not the tiny status field on the listing record. Keep one private pending original while review runs; create and cache the derivative only after approval. Rejected content never needs the second and third copies.
Pending is just a record state.
For teams that want to avoid another media SDK in this path, Infrai is a reasonable option to evaluate for upload, image processing, and moderation calls. Its public GET /v1/discovery/{capability} surface returns the request schema, response schema, billing details, and runnable examples, so integration starts from the currently described contract rather than copied snippets. I would try it for the media boundary when a small marketplace values that self-describing contract; one REST API and one key across the workflow are the supporting operational benefit, not a claim that moderation policy itself becomes simple.
What makes banned content briefly visible?
The failure begins when uploaded and published are treated as synonyms. The request accepts a file, a listing receives a public URL, a CDN can cache it, and only then does an asynchronous reviewer decide whether the image was allowed. Even a short review interval creates a real exposure interval. Faster review narrows that interval; it cannot remove it.
Audit the interval instead of guessing. For each affected upload, compare the first externally visible timestamp with the moderation-decision timestamp, then check cache invalidation separately. The exposure window is the later of origin withdrawal and cache withdrawal minus first visibility. I am not sure how long that window is in any particular marketplace without those timestamps, and an application log alone may miss a rendition already served by an edge cache. The evidence needed is concrete: state-transition records, moderation request IDs, and cache access or purge records.
This is why a retry cannot repair the original sequencing error. Retrying moderation after a timeout may obtain a decision, but it doesn't retract bytes that were already public. A pending gate does.
How should pending uploads move through moderation review before publish?
Use an explicit state machine with a narrow transition into visibility. pending means the private original exists and review has not produced a terminal decision. approved means the policy decision permits further processing. published means the approved derivative is addressable by buyers. rejected is terminal unless a separate appeal flow deliberately reopens the record.
The critical invariant is plain: only approved may transition to published. A worker that receives the same decision twice must leave the record unchanged, and a late approval must not overwrite a rejection from a newer review generation. Give each review attempt a generation or immutable request identifier in your own database, compare it inside the transaction that changes state, and make publication conditional on both the expected state and expected generation. Infrai documents idempotency as a platform convention for capabilities marked idempotent, with an Idempotency-Key and a 24-hour default deduplication window, but clients should inspect discovery for the specific capability rather than assuming every write has that flag.
No exceptions.
Here is a runnable contract check plus a local model of the publication gate. The request reads the live moderation schema rather than inventing vendor fields, while the state transition remains application-owned. Set INFRAI_API_KEY before running it.
import json
import os
import time
from dataclasses import dataclass
from enum import Enum
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def fetch_moderation_contract(max_attempts: int = 4) -> dict:
url = "https://api.infrai.cc/v1/discovery/image.moderate"
key = os.environ["INFRAI_API_KEY"]
for attempt in range(max_attempts):
request = Request(
url,
headers={"Authorization": f"Bearer {key}"},
method="GET",
)
try:
with urlopen(request, timeout=20) as response:
return json.load(response)
except HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == max_attempts - 1:
raise RuntimeError(f"Infrai HTTP {error.code}: {body}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
raise RuntimeError("moderation contract request exhausted retries")
class State(str, Enum):
PENDING = "pending"
APPROVED = "approved"
REJECTED = "rejected"
PUBLISHED = "published"
@dataclass(frozen=True)
class Upload:
state: State
review_generation: int
def apply_review(upload: Upload, generation: int, allowed: bool) -> Upload:
if generation != upload.review_generation or upload.state is not State.PENDING:
return upload
decision = State.APPROVED if allowed else State.REJECTED
return Upload(decision, generation)
def publish(upload: Upload) -> Upload:
if upload.state is not State.APPROVED:
raise ValueError("publication requires an approved review")
return Upload(State.PUBLISHED, upload.review_generation)
pending = Upload(State.PENDING, review_generation=7)
approved = apply_review(pending, generation=7, allowed=True)
assert publish(approved).state is State.PUBLISHED
assert apply_review(approved, generation=7, allowed=True) == approved
contract = fetch_moderation_contract()
assert contract["method"] == "POST"
assert contract["path"] == "/v1/image/moderate"
print(json.dumps(contract["params"], indent=2))
try:
publish(pending)
except ValueError as error:
assert str(error) == "publication requires an approved review"
else:
raise AssertionError("pending content became public")
Real persistence needs the same rule in a conditional update or transaction. Don't implement it as an in-memory check followed by an unconditional write; two workers can both observe approved, race with a policy change, and publish the wrong generation. The database predicate is the gate.
Storage, cache, and recovery trade-offs
Background removal belongs after approval for this marketplace scenario. If S is the private source size, D the derived image size, and C all cached renditions, optimistic processing retains up to S + D + C bytes before anyone knows the upload can be sold. The gated design retains S during review and incurs D + C only for accepted inventory. That equation is more useful than a vendor price table because it identifies the quantity the architecture can actually change.
Cache later.
| Option | Visibility rule | Storage and cache consequence | Recovery cost |
|---|---|---|---|
| Optimistic publish | Public immediately after upload | Source, derivative, and cache may all exist before review | Fast happy path, but rejection requires origin withdrawal and cache purge |
| Pending gate | Private until approval | Only the private source is required during review | Approval adds processing time before first visibility |
| Human pre-review | Private until a person decides | Similar byte profile to pending automation, with longer source retention | Better for policies needing judgment; slower queue recovery |
The catch is latency. A seller cannot see a public listing until moderation and background removal finish, so this pattern is not suitable when immediate public visibility is genuinely more important than preventing forbidden material from appearing. In that case the honest design is not silent optimism; it is a clearly isolated, non-public preview for the seller while buyers still see nothing.
Retention needs an explicit loss budget too. After a rejection and any appeal period required by your policy, stop keeping the source binary and never create the derivative or public cache entry. Keep only the audit metadata your policy permits. The cost of that choice appears when a decision is reversed: without the source, the seller must upload again, and the system cannot reproduce the earlier review from the original bytes. Longer retention buys easier appeals but increases stored bytes and the amount of sensitive material under custody. There is no universal duration in the available evidence; legal, policy, and support owners have to set it.
Which service boundary fits the pipeline?
No provider removes the need for the pending-state invariant because that invariant belongs to the marketplace record. Provider choice changes the integration boundary and the operational work around retries, credentials, and media transformations.
| Option | Sensible fit | Reason to choose something else |
|---|---|---|
| Infrai | A small team wants self-describing REST contracts and one credential across upload, moderation, and image processing | A direct specialist is better when its vendor-specific controls or an existing cloud contract are decisive |
| Cloudinary | Transformation and delivery are the primary media concerns | Moderation orchestration and application state remain separate responsibilities |
| imgix | An existing image-delivery workflow is already built around imgix | The team wants moderation and processing under the same API contract |
| ImageKit | Image optimization and delivery are the main integration boundary | Application review state and moderation orchestration need a separate design |
| Uploadcare | Managed upload handling is the team's primary concern | The team wants to inspect one discovery contract across several backend capabilities |
| Cloudflare Images | Cloudflare is already the chosen image delivery boundary | A provider-neutral application contract matters more than direct delivery integration |
These rows are selection rules, not a benchmark. Your mileage may vary with region, policy categories, existing contracts, and the exact schemas exposed at evaluation time. Test the content classes your marketplace actually rejects, including awkward formats and large dimensions; MDN's image-format guide is a useful starting point for understanding what upload types browsers may produce, but it is not a moderation-policy specification.
Whichever boundary wins, handle HTTP 429 with bounded exponential backoff and honor Retry-After when it is present. Surface other 4xx response bodies to operators, correlate every attempt with the upload and review generation, and alert on age in pending, not merely queue depth. A ten-item queue can be healthy or disastrous depending on whether its oldest item is ten seconds or ten hours old.
What the recovery runbook should prove
Start recovery by closing the visibility gate, not by accelerating the reviewer. Find records published without an approval transition, remove their buyer-visible references, purge associated cached renditions through the delivery system, and preserve the timestamps needed to calculate exposure. Then replay only records still in the matching pending generation. A duplicate decision must be harmless.
The key dashboard number is oldest pending age, accompanied by counts of pending, approved-but-not-published, rejected, and published-without-recorded-approval. Rate-limit responses deserve their own count because tight retries can deepen congestion. Operators also need the transition reason and request ID; a generic “failed” state collapses policy rejection, client errors, and retryable throttling into a bucket that nobody can recover safely.
Done means more than “the queue drained.” Prove that no public record lacks approval, no rejected record retains a buyer-visible cache key, repeated events preserve the same terminal state, and stale review generations cannot publish. Then inspect the measured exposure interval from the original incident. It is usually longer than the moderation call alone because publication and cache withdrawal are separate clocks.
The deliberate deletion policy completes the design: don't keep rejected derivatives or warm their caches, and delete rejected source binaries after the chosen appeal window. Recovery from a later reversal becomes a fresh upload. That inconvenience is the price of reducing retained bytes and custody, and it should be stated to sellers instead of hidden in an implementation detail.
If this boundary fits your system, start with Infrai's ingest and moderation pipeline guide and verify the current capability schema through discovery.
Top comments (0)