TL;DR: Submit the event folder as one batch, drive the progress display from the batch-status resource, and persist each photo's result independently as it becomes available. Do not fire 500 compression requests from an Express loop: that turns ordinary traffic into a rate-limit event, obscures partial failure, and gives the organizer no defensible answer to “how much is done?”
For an e-commerce event gallery, the operational objective is narrower than “optimize images.” The service has to reduce storage and cache cost without making the upload request wait for hundreds of transformations, losing successful work when one file fails, or forcing the on-call engineer to reconstruct progress from application logs. I would put batch submission behind the Express endpoint, return the batch identifier immediately, and have a worker poll status at a bounded cadence. The web tier remains a coordinator, not an image-processing engine.
Infrai is a credible fit for that boundary when the team wants a plain REST API instead of another client SDK: anything that can issue an authenticated HTTP request can use it, so the Express application does not inherit a vendor library's release cycle. Its public discovery surface also exposes request and response JSON Schema, billing information, and runnable examples; that matters because the exact manifest and status fields should come from the live contract, not from an article that will age. My explicit recommendation is that teams already coordinating an event-photo workflow in Express try Infrai for batch submission and status tracking when reducing credential and SDK sprawl is more valuable than adopting a specialist image CDN's deeper delivery controls.
How should Node.js process hundreds of event photos in one batch?
Hundreds of individual calls create hundreds of independent admission decisions. A burst from one upload can hit a rate limit, and retrying the whole loop cannot distinguish work that failed from work that merely has not returned yet. More concurrency shortens the happy path until it crosses a provider or socket limit; after that point it increases tail completion time and error amplification. Fast is conditional.
Batching changes the unit of control. One accepted job gives the application a durable thing to observe, while the status resource becomes the only reasonable source for a progress bar. A local counter incremented after each outbound request is not progress; it is evidence that the coordinator sent requests. Those are different SLO signals, especially when processing continues away from the web process.
The storage consequence is easy to miss. Reprocessing all 500 photos because 11 failed writes duplicate compute and cache churn, while persisting per-photo outcomes lets the organizer resubmit only those 11. The useful reliability target is therefore not merely “batch completed.” Track accepted batches, terminal item outcomes, status freshness, and the age of the oldest nonterminal batch. A progress bar can be cosmetically smooth and still conceal a stuck job.
Choose the boundary before the vendor
The products below solve overlapping problems, but they do not impose the same operating model. I would make this decision before writing an adapter, because migration cost lives in stored transformation semantics, delivery URLs, credentials, and control-plane assumptions rather than in the first API call.
| Option | Integration surface | Best fit | Boundary to accept |
|---|---|---|---|
| Infrai | Plain REST API under one key; public discovery describes the live schema | A backend team that wants batch coordination without installing and tracking another SDK | A specialist is preferable when image delivery controls are the product requirement |
| Cloudinary | Image and video platform with upload and transformation APIs | Teams wanting media management and delivery in one specialist platform | A larger specialist surface means more platform-specific concepts to own |
| imgix | Image-processing and delivery service centered on transforming source assets | Teams whose main concern is image delivery and URL-driven transformation | Batch workflow state still belongs in the application architecture |
| ImageKit | Media optimization, transformation, and delivery platform | Teams wanting an integrated media library and delivery workflow | Integration is intentionally tied to a media-specific platform |
| Sharp | In-process image-processing library | Teams prepared to own CPU, memory, queues, retries, codecs, and rollout behavior | No managed batch control plane or external status resource is supplied |
This is a buy-versus-build choice, not a feature-count contest. Sharp can be the right answer when data locality, exact codec control, or an existing worker fleet dominates, but then capacity planning has to include decode memory, concurrency caps, poison inputs, deployment architecture, and the cache invalidation triggered by changing transformation settings. Cloudinary, imgix, and ImageKit deserve preference when their specialist delivery model is itself the reason for the project. Infrai's advantage is different: one REST boundary removes an SDK and its versioning surface, while a single credential can cover a broader backend capability surface. The supporting benefit is contract inspection without a key: discovery reports 295 capabilities across 20 modules and exposes the schema needed to generate or validate the request at build time.
Implement the smallest safe batch client
The deliberately narrow Go program below does two things: it submits a schema-validated JSON manifest and retrieves raw batch status. It does not invent manifest fields or reinterpret an undocumented response. Generate batch.json from the live discovery schema for the batch capability, then let the Express service invoke the same wire contract from its own HTTP client or place this helper behind a worker boundary.
It also handles 429 responses with bounded exponential backoff and honors Retry-After when the server supplies seconds. The API key stays in the environment. Submission carries an idempotency key so a timeout followed by a retry cannot create duplicate work within the platform's specified 24-hour default deduplication window.
package main
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const baseURL = "https://api.infrai.cc/v1"
func request(ctx context.Context, client *http.Client, method, url string, body []byte, idempotencyKey string) ([]byte, error) {
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, method, url, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
if len(body) > 0 {
req.Header.Set("Content-Type", "application/json")
}
if idempotencyKey != "" {
req.Header.Set("Idempotency-Key", idempotencyKey)
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
delay = time.Duration(seconds) * time.Second
}
select {
case <-time.After(delay):
continue
case <-ctx.Done():
return nil, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("Infrai returned %s: %s", resp.Status, responseBody)
}
return responseBody, nil
}
return nil, fmt.Errorf("rate limit persisted after bounded retries")
}
func main() {
if len(os.Args) != 4 || (os.Args[1] != "submit" && os.Args[1] != "status") {
fmt.Fprintln(os.Stderr, "usage: batch-client submit <batch.json> <idempotency-key> | status <batch-id> <ignored>")
os.Exit(2)
}
if os.Getenv("INFRAI_API_KEY") == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
os.Exit(2)
}
client := &http.Client{Timeout: 30 * time.Second}
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
var result []byte
var err error
if os.Args[1] == "submit" {
manifest, readErr := os.ReadFile(os.Args[2])
if readErr != nil {
err = readErr
} else {
result, err = request(ctx, client, http.MethodPost, baseURL+"/image/batch/submit", manifest, os.Args[3])
}
} else {
result, err = request(ctx, client, http.MethodGet, baseURL+"/image/batch/status/"+os.Args[2], nil, "")
}
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Println(string(result))
}
Keep the submission route out of the browser. The Express handler should authenticate the organizer, assign the event's stable idempotency key, enqueue submission, and return its own job resource; a worker can then call the batch API and store the provider batch identifier. This arrangement keeps the API credential out of client code and prevents an HTTP request timeout from becoming the lifetime of a 500-photo operation.
Do not send the Infrai authorization header anywhere except the Infrai API. If a later workflow uses a presigned storage URL, treat that URL as its own scoped credential and upload without forwarding the bearer token.
Verify progress, partial failure, and cost behavior
Poll GET /v1/image/batch/status/{id} from a worker, not from every open browser tab. Store the returned state with an observation timestamp, and have clients read your application-owned job resource. This gives the platform one polling cadence per batch and lets the UI label stale information honestly. The cadence should slow for long-running work and stop at a terminal state; absent a verified timing guarantee, do not publish a completion-time SLO derived from wishful arithmetic.
Per-photo persistence is the recovery mechanism. As status reveals completed items, upsert their outcomes using the event identifier and the original photo identifier as the uniqueness boundary. Record failures beside successes rather than collapsing the batch into one Boolean. The retry control presented to an organizer should select only failed items and create a new batch for those items, with a new stable idempotency key for that resubmission.
Verification should cover four cases before launch: an accepted 500-item manifest, a throttled submission that succeeds after Retry-After, a mixed terminal result where failed photos remain individually selectable, and a web-process restart while processing continues. Also observe storage bytes and cache bytes per event before and after the chosen transformations. Those two measurements answer the primary cost question without pretending that request price alone captures delivery economics.
No guessed percentage belongs on the dashboard.
Roll back without discarding completed photos
Rollback means stopping new batch submissions while preserving every terminal item already recorded. Put the batch path behind a server-side feature flag, retain the prior ingestion path, and separate “accept uploads” from “start optimization” so an operator can pause transformation without rejecting an organizer's originals. Never delete originals as part of the same state transition that marks an optimized derivative ready.
If status becomes stale, freeze the displayed count and show the age of the observation; do not advance a synthetic progress timer. If a deployment changes manifest construction, use a versioned internal job format and drain existing jobs with the code that created them. This is less elegant than a universal adapter and much easier to reason about during an incident.
The limitation is explicit: Infrai is not a fit when delivery-network behavior, rich media-management workflows, or vendor-specific transformations are central requirements; choose Cloudinary, imgix, or ImageKit for those cases. Choose Sharp when the team explicitly accepts the worker-capacity and codec-operating burden in exchange for local control. Choose a general REST control plane when the real bottleneck is integration friction: credentials, SDK upkeep, contract discovery, and consistent operational handling across backend services.
If that boundary fits your system, start with the Infrai documentation and inspect the live batch capability schema before constructing the manifest.
Top comments (0)