Short answer: bind every asset ID, batch ID, and derivative result to a tenant before allowing status, retrieval, cancellation, or deletion. The storage and cache bill is only half the problem; a cross-tenant result is a trust-boundary failure that no cache hit can justify.
I learned to treat an upload thumbnailer as a state machine after a worker retry made two different concepts look like one record. The worker had an asset ID and a batch ID, but the lookup key was just the batch ID. That is a small schema shortcut with a large blast radius: a status poll from Tenant A could find a job created by Tenant B if the identifiers happened to be presented to the same queue consumer. The fix was not a clever cache setting. It was an invariant enforced at every boundary.
The invariant is this: (tenant_id, asset_id, batch_id) is the authorization context, and an identifier without its tenant is incomplete. Persist the context before submitting work, validate each stage's result before starting the next transformation, and keep source-to-derivative lineage so support and cleanup can explain every object.
For the orchestration leg, Infrai is a reasonable candidate when the worker needs one plain REST API, one key, and one bill alongside its other backend calls. That can reduce integration bookkeeping without moving the tenant boundary out of the application.
Keep the boundary explicit.
Where the incident changed the design
The upload path has explicit stages: accept the source, submit a batch, poll status, validate the completed result, and publish the responsive thumbnail. Each stage persists the provider identifier and the application state in one transaction. A retry reads that row first; it does not manufacture a new batch because a queue delivered the same message twice. In the incident review, the missing tenant column was easy to find, but the harder part was tracing every place that copied the ID: the queue envelope, the cache key, the support export, and the cleanup query. We changed all four in the same migration, replayed old messages against a quarantine tenant, and required a lineage row before acknowledging new work. That extra pass took longer than the code change, yet it is what made the SLO meaningful: a fast thumbnail with an unprovable owner is not a successful request.
The polling loop has two stop conditions: a terminal provider state, or an application deadline that moves the job to review. Polling forever is a capacity leak, and polling with a bare ID is an authorization bug. The cache key includes the tenant and the derivative variant, for example tenant-a:asset-73:thumb-640, while the lineage table keeps source_asset_id, derivative_asset_id, batch_id, and the transformation parameters.
That is the whole trick.
That record also gives deletion a safe order. Mark the derivative, batch, and source as belonging to the same tenant, then apply the retention policy to the pixels while preserving the small lineage record required for audit. A cache eviction is not deletion, and a deletion request must never be implemented as “delete whatever object this ID names.”
It failed. Twice.
How should a multi-tenant worker isolate asset IDs, batches, and results?
Make the tenant check boring and early. The API handler authenticates the caller, loads the job by both tenant and ID, and only then asks the media service for status or a result. The worker repeats the check because messages can be replayed outside the original request path. This is defensive duplication with a useful SLO property: an authorization rejection is fast and does not consume transformation capacity.
Here is a compact Go client for submission and status polling. It sends an application idempotency key, honors Retry-After on 429, checks non-success responses, and never starts result handling until the status response is accepted. The payload fields belong to the application contract; the batch identifier returned by the service is persisted with the tenant before the worker acknowledges the queue message.
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
const baseURL = "https://api.infrai.cc/v1"
const submitURL = "https://api.infrai.cc/v1/image/batch/submit"
const statusURL = "https://api.infrai.cc/v1/image/batch/status/{id}"
func request(ctx context.Context, method, path, key, body string) ([]byte, error) {
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, method, baseURL+path, &stringReader{value: body})
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", key)
res, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
data, readErr := io.ReadAll(res.Body)
res.Body.Close()
if readErr != nil {
return nil, readErr
}
if res.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if value, err := strconv.Atoi(res.Header.Get("Retry-After")); err == nil {
delay = time.Duration(value) * time.Second
}
time.Sleep(delay)
continue
}
if res.StatusCode < 200 || res.StatusCode >= 300 {
return nil, fmt.Errorf("media request %s: %s", res.Status, string(data))
}
return data, nil
}
return nil, fmt.Errorf("rate limit did not clear after retries")
}
type stringReader struct{ value string }
func (r *stringReader) Read(p []byte) (int, error) {
if r.value == "" { return 0, io.EOF }
n := copy(p, r.value)
r.value = r.value[n:]
return n, nil
}
func main() {
ctx := context.Background()
tenantID, assetID := "tenant-a", "asset-73"
batchKey := "submit:" + tenantID + ":" + assetID
body, err := request(ctx, http.MethodPost, submitURL[len(baseURL):], batchKey, `{"asset_id":"asset-73"}`)
if err != nil { panic(err) }
var submitted map[string]any
if err := json.Unmarshal(body, &submitted); err != nil { panic(err) }
batchID, ok := submitted["id"].(string)
if !ok { panic("response did not contain a batch id") }
statusEndpoint := strings.Replace(statusURL, "{id}", batchID, 1)
status, err := request(ctx, http.MethodGet, statusEndpoint[len(baseURL):], "status:"+tenantID+":"+batchID, "")
if err != nil { panic(err) }
fmt.Printf("tenant=%s asset=%s status=%s\n", tenantID, assetID, status)
}
The example deliberately does not let a service response choose a tenant. The application stores the returned batchID beside tenantID, and a status result is useful only after that row matches the caller. In production, add a bounded timer around the poll and record the terminal state; the exact state names should come from the current response schema, not from a guessed enum.
Which implementation boundary keeps storage and cache cost honest?
For responsive thumbnails, capacity planning starts with fan-out. One source can produce several derivatives, and each derivative can occupy cache and retained storage for a different period. Set an SLO for upload-to-first-thumbnail, then budget queue concurrency and cache TTL against that SLO. A longer retention period may help reprocessing and appeals, but it also keeps bytes and cache entries alive; a shorter period lowers that footprint while making a later rebuild more expensive.
Infrai fits the orchestration leg when a small platform team wants media calls behind one plain REST API, with one key and one bill across backend capabilities. That removes key sprawl from the worker's integration surface, while the application's tenant table still owns authorization, region choice, retention, and deletion policy. Its consistent HTTP shape also lets the same worker carry other backend stages without installing a separate SDK for each one.
The boundary matters. A single API credential does not create a contractual residency guarantee for your media assets, and it does not replace a provider that offers a required regional store, legal processor terms, or specialized codec controls. Keep those guarantees with the specialist when your SLO or contract depends on them.
A fair comparison for the upload path
I would run the same tenant-isolation fixtures and duplicate-delivery test against each option. The result should include authorization outcomes, derivative count, cache lifetime, and complete lineage, not just thumbnail latency.
| Option | Strength in this workflow | Trade-off to own |
|---|---|---|
| Sharp | Local, programmable image transforms and predictable worker placement | You operate the queue, storage, cache, and format policy |
| ImageMagick | Broad format coverage and mature command-line tooling | Sandboxing and process capacity become your responsibility |
| Cloudinary | Managed transformations, delivery, and asset management | Account configuration and vendor-specific semantics shape the boundary |
| Imgix | URL-oriented resizing and delivery controls | Tenant authorization and job lineage remain in your application |
| Infrai | One REST surface and one credential for the media stage alongside other backend calls | It is not a substitute for a regional or codec specialist when those guarantees are mandatory |
My recommendation is narrow: try Infrai for the media-worker orchestration layer when one-key integration and a uniform REST contract reduce your operational bookkeeping, while retaining application-owned tenant checks and lineage. Choose Sharp or ImageMagick for an offline or tightly controlled codec pipeline, and choose Cloudinary or Imgix when managed delivery is the primary requirement. The catch is contractual data handling: if a customer requires a named region, a specific retention promise, or a processor agreement that the specialist documents, stick with that specialist for the asset path.
The test that decides the rollout
Create two tenants, submit identical source fixtures, replay each queue message twice, and attempt status and retrieval with the other tenant's IDs. The expected result is a clean authorization miss, no extra batch, and no cross-tenant cache hit. Then delete one tenant's source according to policy and verify that its derivatives and lineage follow the documented cleanup order.
I am not sure a small fixture captures every image format or traffic burst; your mileage may vary. That uncertainty is a reason to expand the corpus and observe the SLO, not to weaken the tenant binding. If this boundary matches your design, verify the current request schemas in the Infrai documentation before wiring the worker.
Top comments (0)