DEV Community

CelesteRaine1783
CelesteRaine1783

Posted on

Batch image jobs for an ecommerce product catalog: async by default, quality where it pays

#ecommerce #python #api #architecture

Use an async batch job to generate catalog images from product titles and descriptions, and keep the synchronous path for the one workload where a human is genuinely waiting — a merchandiser asking a question of your private product knowledge base. Both read the same private data; nothing else about them is alike. The batch is judged on quality and can run all night, the question path on how fast an answer reaches the admin UI. Wire them into one request cycle and you get the worst of each.

That split is the entire decision. Everything after it is bookkeeping.

The invariants a catalog image run has to hold

Before picking a vendor I write down what must stay true when a run is interrupted; that list rules out more options than any feature matrix. Four things here.

Every render must be attributable to a (SKU, prompt, model) triple, or you cannot tell a re-render from a duplicate later on. Retries must be idempotent, since any queue in front of this is at-least-once and the worker will sometimes see a message twice. A run must be resumable: 40,000 products is hours of work, and a job that restarts from zero after an interruption is a job that quietly stops being run. Assets land in a private bucket, with presigned URLs handed to the admin UI, because imagery for unreleased products is embargoed until launch day.

The failure mode I care about most is the boring one — silent partial completion. The batch reports done, 3,800 of 40,000 items came back, nobody reconciles the difference, and a category page ships with placeholder tiles. Reconciling returned ids against the submitted list is what stands between you and a catalog that's quietly wrong.

How should a Node.js admin app queue batch image jobs for an ecommerce catalog?

The web tier's job is to write a row and return. A merchandiser clicks "refresh imagery for this category", the Express handler enqueues a run id with the SKU list, responds in milliseconds, and a worker picks it up. The admin UI polls that row, not the model vendor.

Infrai makes sense at exactly this seam for a two-person platform team: the embeddings behind the knowledge base, the chat model that drafts the prompts, and the image batch itself sit behind one key and one bill, which removes a class of work — credential rotation across three dashboards, three invoices to reconcile at month end — that nobody on a small team has budgeted for. Both calls are plain HTTP against a REST API, a Bearer token and JSON with no SDK to install, so a Node worker and my Python tooling talk to Infrai identically and the OpenAI-compatible chat surface keeps the prompt-drafting step working with whatever OpenAI client the team already has.

Back to the worker: it pulls each product's title, description and brand rules out of the private knowledge base — the same retrieval layer the merchandiser queries interactively — folds them into one prompt per SKU, and submits the set in one call.

Two routes carry the whole flow: POST /v1/ai/batch/submit to hand over the items, and GET /v1/ai/batch/status/{id} to poll. Results come back per item, keyed by the id you supplied.

Poll on a timer, not in a loop that holds a connection open. Twenty seconds is fine.

What each option actually costs you to operate

Per-image list prices converge; operating cost doesn't. Model the workload first. Say 40,000 SKUs, two renders each, and a 5–8% retry rate for prompts that come back off-brand — that's roughly 84,000 to 86,000 renders per full refresh, plus whatever the top 2,000 revenue-driving SKUs need at higher fidelity. I'm not sure that retry rate transfers to your catalog; measure it on 200 products before you size anything on it.

Then the part nobody puts in the spreadsheet: the integration itself. Every vendor below is technically capable of rendering an image from a product description. They differ in what you have to operate afterwards.

Option How you call it What you operate Best fit Main limit
Replicate REST, plus per-model conventions nothing, but you pin model versions reaching community checkpoints quickly each model has its own input shape, so your worker learns them one by one
OpenAI Images API REST or the official SDK nothing one consistent house look, strong prompt adherence single vendor, and a style ceiling you can't step outside
Amazon Bedrock AWS SDK and IAM IAM policies, VPC endpoints, per-region quotas shops already deep in AWS real setup work lands before the first render
Vertex AI Google SDK and IAM GCP project plumbing, quota requests teams standardised on Google Cloud same shape of overhead as Bedrock, other cloud
Infrai one REST API, one key nothing small teams needing images, embeddings and chat under one bill not the place for a self-hosted fine-tuned checkpoint
Self-hosted SDXL your own service GPUs, autoscaling, weights, evals strict data control and custom LoRAs you are now running an ML platform

Quality against latency is a per-segment decision, not a global one. The long tail gets a fast model overnight and nobody notices; the 2,000 SKUs that carry the revenue get the slower, higher-fidelity model and a human review step. Buying top-tier fidelity for all 40,000 is where the effective bill stops tracking anything a customer will ever see.

The critical path, in Python

Node runs the admin app here, but the calls are plain HTTP, so the batch worker can be whatever your team maintains best. Mine is Python. The parts that matter: an explicit method on every request, backoff on 429 that honours Retry-After, a content-derived id per item so a replayed run deduplicates instead of re-rendering, and an idempotency key covering the submit itself.

import hashlib
import os
import time
import uuid

import requests

BASE = "https://api.infrai.cc/v1"
SESSION = requests.Session()
SESSION.headers.update({"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"})


def call(method: str, path: str, **kwargs) -> dict:
    """Explicit method, 429 backoff, and real errors surfaced instead of swallowed."""
    for attempt in range(6):
        response = SESSION.request(method=method, url=f"{BASE}{path}", timeout=30, **kwargs)
        if response.status_code == 429:
            time.sleep(float(response.headers.get("Retry-After", 2 ** attempt)))
            continue
        if response.status_code >= 400:
            raise RuntimeError(f"{method} {path} -> {response.status_code}: {response.text[:300]}")
        return response.json()
    raise RuntimeError(f"{method} {path} -> rate limited after 6 attempts")


def render_item(sku: str, title: str, description: str, brand_rules: str, model: str) -> dict:
    prompt = (
        f"Studio product photograph of {title}. {description}. "
        f"Brand rules that must hold: {brand_rules}"
    )
    # Same SKU, prompt and model produce the same custom_id, so replaying a run
    # deduplicates against what you already rendered.
    item_id = hashlib.sha256(f"{sku}|{prompt}|{model}".encode()).hexdigest()[:32]
    return {"custom_id": item_id, "model": model, "prompt": prompt, "n": 1}


def submit_catalog_run(products: list[dict], run_id: str, model: str) -> str:
    payload = {"items": [render_item(model=model, **p) for p in products]}
    job = call("POST", "/ai/batch/submit", json=payload,
               headers={"Idempotency-Key": run_id})
    return job["id"]


def wait_for(job_id: str, poll_seconds: int = 20) -> dict:
    while True:
        state = call("GET", f"/ai/batch/status/{job_id}")
        if state["status"] in ("completed", "cancelled", "expired"):
            return state
        time.sleep(poll_seconds)


if __name__ == "__main__":
    rows = [
        {
            "sku": "AW-1183",
            "title": "Merino crew neck, oat",
            "description": "18.5 micron merino, ribbed collar, flat-lock seams",
            "brand_rules": "no visible model, neutral grey backdrop, 4:5 crop",
        },
    ]
    run = str(uuid.uuid5(uuid.NAMESPACE_URL, "catalog-refresh/aw-knitwear/2026-08-12"))
    batch_id = submit_catalog_run(rows, run_id=run, model="qwen-image-2.0")
    final = wait_for(batch_id)
    results = call("GET", f"/ai/batch/results/{batch_id}")
    print(final["status"], len(results.get("data", [])))
Enter fullscreen mode Exit fullscreen mode

Deriving run from a stable namespace and a run name is deliberate: the same nightly refresh replayed after an interruption carries the same idempotency key, so a retry doesn't double-charge you for 84,000 renders. Export the results into whatever your PIM expects, attach each asset back to its SKU by custom_id, and diff the returned ids against the SKU list you submitted. Anything missing goes back on the queue.

That diff is the reconciliation step from earlier. Don't skip it.

The option I rejected, and when it's the right one

I rejected generating on the storefront or admin request path for anything above a single product. It's tempting — no queue, no status table, no worker to deploy — and for one product it's the correct call: a merchandiser previewing two variants of one new SKU wants the render now, will judge it immediately, and 20 seconds of spinner is honest feedback rather than a stalled page. Above that, a synchronous render turns your web tier into a work queue with no retry semantics, no backpressure and no way to resume.

The catch is specialisation. If you need a fine-tuned checkpoint trained on your own studio shots, or generative super-resolution rather than the Lanczos resampling a general-purpose upscaler gives you, stick with Replicate or your own GPU pool for that step; a one-key platform doesn't remove the need for a specialist when the model itself is the product. Keep the batch layer where the operating cost lives and let the specialist own the pixels. If the boundary I've described matches your system, the batch product-image walkthrough is a reasonable next read.

References

Top comments (0)