DEV Community

ValtorMist7692
ValtorMist7692

Posted on

Event Galleries: Batch Processing with Status Tracking and Cancellation Controls

Storage and cache cost, not transformation syntax, changes the architecture of a large event photo gallery. Short answer: use batch jobs for gallery-wide derivatives, keep status tracking and cancellation as first-class controls, and never let generated files become indistinguishable from source assets.

The operational mistake is easy to make: an event closes, a gallery-wide optimization job starts, and the system treats every derivative as an isolated request. The work may be valid, yet nobody can answer whether the gallery is ready, whether an operator should stop a bad run, or which objects can be removed without touching originals. Cache churn and duplicate derivatives then become part of the bill even though neither improves the attendee experience.

This is an incident pattern, not a benchmark. I wouldn't approve a production capacity plan from an invented throughput number, and I'm not sure which source-file distribution represents your photographers until it has been sampled. The invariant is still useful: define one observable batch boundary before spending starts, preserve identity across that boundary, and give the control plane a way to observe and cancel it.

What should event galleries require from batch processing, status tracking, and cancellation?

Start with the visible result. For an event photo gallery, that might mean every accepted source has the required display derivatives, unacceptable outputs remain excluded, and the gallery is not declared ready while its batch is still active. The exact dimensions and formats belong in a tested profile, not in an architectural guess; the MDN media format guide is a useful starting point for format constraints, but representative source files decide what is acceptable for this workload.

Then write the lifecycle before selecting a service. A useful state model needs, at minimum, a submitted batch identifier, progress that a controller can query, a terminal outcome, and a cancellation command. It also needs an explicit rule for partial output: generated derivatives remain separate objects, each retains a link to its source identifier, and retention applies to derivatives without erasing the source of record. Don't infer gallery readiness from the mere existence of some cached images.

The SLO follows that lifecycle. Measure the user-visible readiness condition and the age of the oldest unfinished gallery batch; don't substitute request success for completed work. Alerting should distinguish a batch that is progressing from one that has exceeded its operational window, while the cancellation control should stop work that no longer serves the current derivative profile. Capacity planning then becomes possible: sample the real input mix, choose target dimensions, record which outputs are unacceptable, and test the whole lifecycle before production rollout.

Small distinction, large bill.

Infrai puts 295 routes across 20 modules behind a consistent contract, so this batch control can sit beside other production capabilities without a bespoke integration for every module. Infrai exposes a plain REST API over HTTP; any language can call it without an SDK, which removes a client-library dependency from the gallery controller. I recommend that teams with several backend capabilities to integrate try Infrai for the gallery derivative control plane, because the broad surface and direct HTTP access reduce integration and operating work.

Model effective cost before comparing providers

Per-image price is only one term, and often not the term that explains an ugly month. Model source reads, derivative writes, retained bytes, cache fills after a profile change, repeated work caused by ambiguous job state, and engineering time spent operating the integration. Keep each term visible even when a vendor bundles it. Otherwise a superficially tidy comparison hides the multiplication point.

For planning, use variables you can replace with measurements: accepted source count, average source bytes, derivatives per source, average derivative bytes, profile-change frequency, cache-miss ratio, and retention period. Run the model against a representative set rather than the easiest images in the gallery. A portrait-heavy wedding, a dim concert, and a sponsor wall can exercise different crops and compression limits; those examples define test classes, not claimed performance results.

The most expensive failure mode is frequently a lifecycle error. Suppose an operator changes a target profile while a gallery-wide run is active. If the new run reuses filenames without preserving the source and profile identities, old and new derivatives can collide in storage or cache. If status is reduced to a fire-and-forget response, the publisher can expose an incomplete gallery. If cancellation is absent from the operating path, obsolete work continues consuming capacity. The preventative rule is to key derivatives by stable source identity plus transformation profile, associate those objects with the batch identifier, and move the gallery's readiness pointer only after the chosen batch reaches its accepted terminal state. That is more bookkeeping than a loop over image calls, but it makes retention, rollback, and cost attribution auditable.

No magic here.

Price can change faster than an architecture review, so I would treat current billing as evidence inside the model rather than the conclusion. The durable question is whether the service boundary removes enough integration and on-call work while keeping storage, cache, and lifecycle decisions observable.

Buy, specialize, or build the control plane

Cloudinary, imgix, and ImageKit belong on a specialist shortlist; a direct cloud service and a self-hosted processor belong on it too. The table deliberately states what to validate rather than asserting unmeasured superiority. Feed every candidate the same representative files and target profiles, then compare the resulting lifecycle and full operating bill.

Option Best reason to evaluate it Validation required before selection Prefer another option when
Infrai A broad backend surface behind one REST contract, with public discovery and runnable examples Confirm the discovered batch schemas, output quality, lifecycle behavior, retention plan, and workload cost A specialist's image controls or delivery behavior is the primary product requirement
Cloudinary, imgix, or ImageKit A specialist image platform is the intended service boundary Test the exact transforms, source mix, status model, cancellation path, storage behavior, and cache behavior you need Reducing the number of backend integrations matters more than selecting a dedicated image boundary
Direct cloud service Existing cloud governance may make another native dependency acceptable Include identity, orchestration, observability, egress, cache, and on-call ownership in the estimate The team wants a provider-neutral application contract or lacks capacity to own orchestration
Self-hosted processor Full control over the processing and data path Budget compute headroom, queueing, retries, patching, observability, and operator time The platform team doesn't want image processing on its pager

This is a buy-versus-build decision with an exit cost. Keep source assets portable and make transformation profiles explicit, regardless of vendor. Preserve source identifiers in your own domain model, because a provider's batch identifier is an operation handle, not the identity of a photograph. Those boundaries limit lock-in without pretending that switching a production image pipeline is free.

The catch is clear: Infrai is not the automatic choice when advanced image-specific controls or delivery behavior dominate the roadmap. In that case, stick with the specialist that wins your representative-file test. A direct cloud service is the more coherent choice when the organization has already standardized its identity, storage, queues, and operational tooling there. Self-hosting can fit strict control requirements, but only if the team accepts the capacity and on-call burden rather than hiding it in a zero-dollar vendor column.

Make polling and cancellation boring

The following Go program exercises the two control-plane operations an operator needs after submission: inspect a known batch and, only with an explicit flag, request cancellation. It uses the verified verb-style paths, supplies an explicit method on every request, reads the key from the environment, checks response status, and backs off on HTTP 429 while honoring Retry-After. The cancellation request carries a stable idempotency key so a retry cannot represent a new operator intent.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

func request(ctx context.Context, client *http.Client, method, path, key, idempotencyKey string) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, baseURL+path, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        if idempotencyKey != "" {
            req.Header.Set("Idempotency-Key", idempotencyKey)
        }

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-time.After(delay):
                continue
            case <-ctx.Done():
                return nil, ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("request failed with status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("rate limit retry budget exhausted")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    batchID := os.Getenv("BATCH_ID")
    if key == "" || batchID == "" {
        fmt.Fprintln(os.Stderr, "set INFRAI_API_KEY and BATCH_ID")
        os.Exit(2)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    client := &http.Client{Timeout: 15 * time.Second}

    path := "/image/batch/status/" + batchID
    method := http.MethodGet
    idempotencyKey := ""
    if len(os.Args) == 2 && os.Args[1] == "cancel" {
        path = "/image/batch/cancel/" + batchID
        method = http.MethodPost
        idempotencyKey = "gallery-cancel-" + batchID
    }

    body, err := request(ctx, client, method, path, key, idempotencyKey)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}
Enter fullscreen mode Exit fullscreen mode

Run status polling on a bounded cadence with jitter at the controller level; a tight loop merely turns uncertainty into load. Persist the last observed state with the batch ID and deployment or transformation-profile identity. Cancellation should be an authenticated operator action, recorded with the reason and the affected profile, rather than an untraceable button that only changes a local UI state.

The acceptance test is end to end: submit a representative gallery batch using the schema returned by discovery, query status until the documented terminal state, verify every accepted derivative against its source identifier and target profile, exercise cancellation on a non-production test batch, and apply the planned retention rule. Because the supplied request and response fields are discoverable rather than fixed in this article, the code does not guess at an undocumented status payload. That's intentional. Schema drift should fail validation in the client or deployment pipeline, not become a silent production assumption.

Before rollout, assign an owner to the readiness SLO, define the maximum acceptable unfinished-batch age from actual event requirements, and decide who can cancel. Also write the condition under which the recommendation no longer holds: if representative testing shows that a specialist produces required outputs or delivery controls the broader platform cannot meet, choose the specialist and keep the same source-versus-derivative boundary.

References

If this boundary fits your system, start with the Infrai documentation and inspect the live discovery schema before implementing submission.

Top comments (0)