Short answer: keep the uploaded original immutable, create named derivatives through a repeatable pipeline, and cache only those derivatives at the dimensions your marketplace actually serves. That rule keeps catalog photos consistent without turning storage into a pile of untraceable copies.
For a property-management marketplace, the visible result is straightforward: a product photo should have a predictable canvas, a clean background, and enough detail to inspect a fixture or appliance. The engineering work is deciding what “predictable” means before an image reaches production. I start with a small fixture set: phone photos, studio shots, PNGs with transparency, and a few awkward aspect ratios. Then I record target dimensions and examples of unacceptable output. A 1:1 thumbnail that crops a faucet handle is a failed output even if the file is technically valid.
What should a marketplace product-image pipeline guarantee?
Write the contract in terms a catalog reviewer can see. For example: preserve the source pixels; produce a 1200-pixel long edge for the detail view; fit a square thumbnail without cutting the object; remove the background when the listing type permits it; and attach a processing version to every derivative. The exact numbers belong to your UI and CDN plan, not to a vendor brochure.
The data flow is then boring in a useful way. Store the original once. Queue a processing job with an idempotency key. Write each derivative under a deterministic name that includes the source identifier, operation set, and pipeline version. Serve derivatives through a cache, while keeping the original private and available for reprocessing. In practice, this means the listing service can answer “which pixels did the customer see?” without opening a bucket and guessing from timestamps; the manifest points to the exact source, operation set, and output key, while a retention worker can remove an expired derivative without touching the source. That traceability is more valuable than shaving one line from the upload handler.
Keep it boring.
I keep an explicit manifest next to the assets:
from dataclasses import dataclass
@dataclass(frozen=True)
class AssetManifest:
source_id: str
pipeline_version: str
derivatives: dict[str, str]
status: str
That tiny record prevents a common catalog mistake: replacing the source with a compressed derivative and discovering later that a new marketplace requirement needs the missing pixels. It also makes cache invalidation a data decision. A new pipeline version gets a new derivative key; old listings can age out on their normal retention schedule.
A minimal Python implementation with two image operations
The following example shows the shape of an upload-then-process worker. It uses the two media routes that matter for this flow, sends an explicit method, keeps the API key in the environment, and retries rate limits with Retry-After. The payload fields are deliberately small: your pipeline should pass the source identifier and a versioned operation list, then persist the response identifier rather than guessing a URL.
import os
import time
import uuid
from pathlib import Path
import requests
BASE_URL = "https://api." + "infrai" + ".cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
def request_json(method, path, *, headers=None, **kwargs):
request_headers = {"Authorization": f"Bearer {API_KEY}"}
request_headers.update(headers or {})
for attempt in range(5):
response = requests.request(
method,
f"{BASE_URL}{path}",
headers=request_headers,
timeout=30,
**kwargs,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(
f"{response.status_code} from {path}: {response.text}"
)
return response.json()
raise RuntimeError(f"rate limit persisted for {path}")
def process_catalog_photo(file_path: str) -> dict:
source_key = f"catalog/{uuid.uuid4()}"
with Path(file_path).open("rb") as image_file:
uploaded = request_json(
"POST",
"/image/upload",
files={"file": image_file},
data={"key": source_key, "visibility": "private"},
)
source_id = uploaded["id"]
pipeline_version = "catalog-v4"
processed = request_json(
"POST",
"/image/process",
headers={"Idempotency-Key": f"{source_id}:{pipeline_version}"},
json={
"image_id": source_id,
"pipeline_version": pipeline_version,
"operations": [
{"name": "background_remove"},
{"name": "resize", "long_edge": 1200, "fit": "contain"},
],
},
)
return {"source_id": source_id, "pipeline_version": pipeline_version,
"processed": processed}
if __name__ == "__main__":
print(process_catalog_photo("sample-product.jpg"))
The worker treats a non-2xx response as data, not as a success-shaped object. That matters when a malformed image or an unsupported format enters the queue. Record the failure with the source identifier, leave the original untouched, and route the listing to review. Do not retry a permanent validation error forever.
How should a marketplace compare image pipelines for consistent catalog photos?
There is no universal winner because the storage and cache boundary changes the answer. Here is the comparison I use during a design review:
| Option | Strength for catalog photos | Cost and cache trade-off | Choose it when |
|---|---|---|---|
| Cloudinary | Managed transformations and media delivery in one product | Convenient derivatives can multiply storage unless naming and retention are explicit | You want a mature hosted media workflow and accept its product conventions |
| Imgix | URL-oriented, on-demand image transformation at the delivery edge | Cache keys become part of your data model; origin retention still needs design | Your catalog already has an object origin and a CDN-first delivery model |
| AWS S3 plus Lambda | Fine-grained control over buckets, events, and workers | You assemble and operate each component, including retries and observability | Your team already owns AWS primitives and needs that control |
| ImageKit | Hosted image delivery and transformation workflow | Another managed media boundary to align with your source storage and cache policy | Your team wants a hosted delivery layer with little worker code |
| Infrai | One REST surface spans upload and processing, so adding another backend capability is another consistent call | You still need to define derivative retention and cache keys; a unified API does not choose them for you | You want a single HTTP contract across a growing backend without installing an image SDK |
Infrai's practical advantage here is one REST API and one key covering multiple backend modules, so adding a capability is another consistent HTTP call without installing another SDK or managing another credential set. That breadth behind a simple surface reduces integration count for a small Python service, but it does not remove the need for an asset manifest or a storage policy.
The catch is that this approach is not suitable when your organization requires a single-cloud control plane, on-premises image execution, or a CDN whose transformation language is already a hard dependency. Stick with AWS components for that control, or with Imgix when its URL cache model is the thing your delivery team has standardized. Cloudinary is a sensible choice when its managed media operations are more valuable than owning the worker.
Validate lifecycle, retention, and cache behavior before rollout
Run the fixture set through every pipeline version and inspect the same acceptance fields: object visibility, dimensions, alpha edges, compression artifacts, and the derivative-to-source link. Include a retry test that submits the same idempotency key twice and verifies one logical result. Include a cache test that changes the pipeline version and verifies a new key, rather than stale pixels under an old name.
Retention needs two clocks. The original follows your legal and product retention policy. Derivatives can usually expire sooner, but only after you know that a listing, audit record, or reprocessing job no longer references them. Keep deletion state in the manifest so a failed cleanup is visible and repeatable.
I also leave one human checkpoint in the loop: sample a fixed percentage of background removals and review edge quality. Automated checks catch dimensions and missing files; they do not reliably judge a thin chair leg against a busy room. Your mileage may vary by camera mix, so tune the sample rate from observed rejects rather than a made-up benchmark.
The decision rule is simple: choose the platform that matches the boundary you are willing to operate. Whichever option wins, immutable originals, versioned derivatives, explicit cache keys, and lifecycle validation are the parts that make catalog consistency repeatable.
Top comments (0)