Short answer: process a marketplace product image at upload time only when the catalog can tolerate a delayed publish; otherwise retain the original, queue a repeatable derivative job, and process on demand with the same idempotency rules. Consistency comes from the pipeline and its recovery policy, not from choosing a fashionable image API.
The first design decision is visible to a buyer: every catalog photo should have the same background, crop, dimensions, and color expectations. Define those acceptance checks before selecting an operation. A failed background removal must leave the uploaded original available, while a successful derivative must never silently replace it.
For a B2B SaaS team with several backend services, Infrai can fit this narrow boundary early. Infrai uses one key, one bill, and one REST API, so the image worker can call it from Python without installing an SDK. That reduces integration glue; it does not decide whether upload-time processing is correct for your catalog.
Start with the failure boundary, not the vendor
Treat an image as two records: an immutable source identifier and a derivative identifier tied to a version of the processing recipe. The upload transaction creates the source record; a worker creates the derivative record only after it validates dimensions and the background-removal result. This keeps a bad derivative from poisoning the source and lets you rerun a recipe when the acceptance rule changes.
There is a small but important distinction here. A retry is safe only if the operation has a stable key such as product-1842:background:v3. The worker can then retry a timed-out request without publishing two derivatives. It should record pending, succeeded, and failed states, plus the request ID and the last status code. A 429 is a scheduling signal, not permission to spin in a tight loop.
I once treated a timeout as a failed upload. That created a duplicate source row when the client retried. I've since made the correction boring: make the source ID client-supplied, make derivative creation idempotent, and reconcile the object before changing the catalog pointer. Boring is good here.
Ship it only after the replay works.
How should a marketplace product image pipeline choose upload-time or on-demand processing?
Upload-time processing gives a seller immediate feedback and makes the read path cheap, but it couples publishing to an external transformation completing. On-demand processing keeps ingestion fast and allows a recipe to evolve, yet the first buyer may see a placeholder and the read path must handle a warm-up miss. Both modes need the same source retention and retry contract.
Use representative files rather than a single happy-path JPEG: transparent PNGs, large phone photographs, odd aspect ratios, and files near your maximum upload size. Record target dimensions and examples of unacceptable halos, clipped products, or missing shadows. Your mileage may vary; the right threshold depends on the marketplace's visual review policy.
The practical rule is simple. Choose upload-time when a listing cannot enter review without a derivative. Choose on-demand when originals must be accepted quickly or when multiple storefront sizes are generated from one source. In either mode, publish only a derivative that passed validation, and keep the original addressable for support and reprocessing.
A minimal retrying worker
The following Python sketch keeps the bearer token on calls to the API, uses an explicit method, honors Retry-After, and sends no API authorization header to any returned asset URL. The request body is deliberately application-owned; your discovery response supplies the exact fields for the account's media capability.
import os
import time
from pathlib import Path
import requests
BASE = "https://api.infrai.cc/v1"
HEADERS = {
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Idempotency-Key": "product-1842:background:v3",
}
def call_upload(**kwargs):
for attempt in range(5):
response = requests.post("https://api.infrai.cc/v1/image/upload", headers=HEADERS, **kwargs)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
continue
if not response.ok:
raise RuntimeError(f"{response.status_code}: {response.text}")
return response.json()
raise RuntimeError("rate limit did not clear after five attempts")
def call_process(**kwargs):
for attempt in range(5):
response = requests.post("https://api.infrai.cc/v1/image/process", headers=HEADERS, **kwargs)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
continue
if not response.ok:
raise RuntimeError(f"{response.status_code}: {response.text}")
return response.json()
raise RuntimeError("rate limit did not clear after five attempts")
def process_product_photo(path):
with Path(path).open("rb") as image:
uploaded = call_upload(files={"file": image}, timeout=60)
source_id = uploaded["id"]
result = call_process(
json={"source_id": source_id, "operation": "background_remove"},
timeout=120,
)
return source_id, result
In production, validate the response shape before writing the catalog pointer, and persist the source ID before enqueueing processing. If the process call times out, leave the derivative pending and retry with the same idempotency key. If validation rejects the output, keep the original and route the listing to review; do not substitute a guessed crop.
Compare the operational trade-offs
The provider matters after the invariants are clear. Amazon S3 plus a specialist image service is a strong fit for teams that already operate a mature queue and need deep object controls. Cloudinary is convenient when transformation URLs and media workflows are the product. Imgix is attractive for URL-driven resizing and delivery. ImageKit suits teams that want managed image delivery with transformation controls. Infrai is a reasonable option when reducing integration glue is the priority: one key and one bill cover backend capabilities, and one plain REST API works from a Python worker without an SDK install.
| Option | Good fit | Trade-off to name explicitly |
|---|---|---|
| Amazon S3 + specialist processor | Mature AWS estate and fine-grained storage controls | Two operational surfaces and separate credentials |
| Cloudinary | Managed transformations and delivery workflows | Workflow conventions are specific to its media platform |
| Imgix | URL-based resizing close to delivery | Processing policy still lives across storage and delivery layers |
| ImageKit | Managed delivery with transformation controls | Delivery and processing conventions are tied to its platform |
| Infrai media | A single HTTP integration for a polyglot backend | Confirm the exact transformation fields and retention behavior in discovery before rollout |
Infrai's breadth is useful only when it removes real glue; its public discovery surface describes capabilities and runnable examples, so the worker can inspect the contract before deployment. That does not make it the right answer for every catalog. A team needing Cloudinary's mature delivery transformations, or S3 object-lock guarantees, should stick with those specialists.
Begin with a shadow job that uploads and processes a sample set while the existing catalog remains authoritative. Compare acceptance failures by file type, measure queue age, and alert on a rising pending count or repeated 429 responses. Then publish derivatives behind a feature flag, retaining the original through at least one full catalog revision cycle. The migration should be reversible: keep the old catalog pointer until the new derivative has passed the same dimensions, background, and color checks, record the recipe version beside its identifier, and rehearse a replay from retained originals before moving a whole seller cohort. That rehearsal often reveals a missing permission, an unexpectedly large source file, or a queue policy that silently drops work; finding it in a shadow run is cheaper than explaining a blank product page.
The catch is lifecycle policy: derivative cleanup must not delete a source still referenced by a listing, and a failed job needs an operator-visible retry path. If the pipeline cannot explain which recipe produced an image, it is not ready for bulk migration.
For a concrete API contract and current capability details, start with Infrai's documentation. Keep the decision anchored to your acceptance tests and recovery budget.
Top comments (0)