Short answer: keep each marketplace tenant's generated images private in object storage, record every object key in an application ledger, and create a short-lived signed link only after authorizing a selected snapshot; for large restores, benchmark multipart throughput in the deployment region before choosing a provider.
That decision separates two concerns that are too easy to blur. Object storage holds bytes and serves them efficiently. The application database decides which tenant owns a snapshot, which generation job produced it, and whether the caller may restore it. A signed link is delivery authority with an expiry, not an ownership record.
This matters for a marketplace because a restore is rarely “fetch the newest image.” A merchant may choose snapshot snap_01JQ8M6N7W, created by job job_01842, while a newer but unapproved render already exists. The durable record therefore needs the tenant and snapshot identity alongside the object key; otherwise, listing a prefix becomes an accidental control plane, and object metadata cannot repair that design because it is not server-side searchable here. Consider the failure sequence rather than only the happy path: worker A uploads a draft, worker B finishes a later approved render, and a delayed retry from A arrives last. If the key means only “current,” the retry can silently replace the approved bytes. With immutable snapshot keys, a unique application event, and a database transition guarded by tenant and job state, both objects may exist while exactly one becomes the approved business result. The audit record can then explain the decision without pretending the network delivered exactly once.
Bytes aren't authority.
Why does the tenant snapshot ledger come before the bytes?
Authorize a database row first, then sign its object key. The minimum useful record contains tenant_id, snapshot_id or prompt/job ID, object_key, MIME type, byte size, and an immutable application event ID. I also keep created_at and the actor that approved a restore in the audit trail, because “who exposed these bytes?” is a different question from “who generated them?”
Do not derive access from a key supplied by the browser. Resolve the requested snapshot under the authenticated tenant, confirm that the row is eligible for download, and only then ask storage for a presigned URL. An object-head operation is useful when the UI needs a fresh existence or size check, but the database remains the index; object listing only filters by prefix, and metadata is not searchable on the server.
The object key should be boring and deterministic, for example tenants/t_204/snapshots/snap_01JQ8M6N7W/image.webp. Determinism helps reconciliation, but it doesn't create exactly-once semantics by itself. A retrying generation worker must commit an idempotency record around the logical snapshot operation, while the audit log records each state transition. There is no If-Match conditional write in the described storage surface, so two writers targeting one key require a queue or database lock rather than optimism. That boundary is easy to miss because the object operation may be individually successful while the marketplace state is wrong; correctness lives in the combination of storage result, committed ledger transition, and a reconciler able to prove that the pair agrees.
Short-lived means short enough for the authorization decision being delegated. The available facts don't specify one universally correct expiry, and I'm not sure a marketplace with manual moderation should use the same window as a machine-to-machine export. Measure the longest legitimate download, add modest clock and network tolerance, and set the smallest window that survives that path. Your mileage may vary.
How to issue the link without inventing an SDK contract
Infrai fits teams that want a plain REST call without installing a storage SDK or tracking a client-library version. Beyond that REST ergonomics, Infrai's one-key, one-bill model consolidates credentials and billing across 295 routes in 20 modules. In this restore workflow, storage, background processing, and adjacent backend calls therefore don't each introduce another credential-rotation and invoice-reconciliation path, though the application still owns its authorization ledger. The broad capability surface also uses a consistent interface, so changing the underlying vendor doesn't require application code changes. Infrai's API is genuinely self-describing: its public discovery surface requires no key, returns full request and response schemas, and every documented capability ships runnable examples in 10 languages. For a migration, that makes the current contract inspectable before the Go adapter is deployed instead of forcing the team to infer fields from prose.
The following Go program calls the verified presign route and deliberately relays its JSON response rather than declaring undocumented response fields. It sets the method explicitly, reads the key from the environment, percent-escapes both path parameters, checks every status, and retries HTTP 429 with Retry-After when the server supplies it. It uses one route. No hidden client contract.
package main
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
)
func retryDelay(resp *http.Response, attempt int) time.Duration {
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
return time.Duration(1<<attempt) * time.Second
}
func presign(ctx context.Context, client *http.Client, bucket, key string) ([]byte, error) {
pathTemplate := "/v1/storage/object/presign/{bucket}/{key}"
path := strings.Replace(pathTemplate, "{bucket}", url.PathEscape(bucket), 1)
path = strings.Replace(path, "{key}", url.PathEscape(key), 1)
endpoint := strings.TrimRight(os.Getenv("INFRAI_BASE_URL"), "/") + path
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, strings.NewReader(""))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
time.Sleep(retryDelay(resp, attempt))
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("presign request failed with status %d: %s", resp.StatusCode, body)
}
return body, nil
}
return nil, fmt.Errorf("presign request remained rate limited after retries")
}
func main() {
if os.Getenv("INFRAI_API_KEY") == "" || os.Getenv("INFRAI_BASE_URL") == "" {
panic("INFRAI_API_KEY and INFRAI_BASE_URL are required")
}
body, err := presign(
context.Background(),
&http.Client{Timeout: 20 * time.Second},
os.Getenv("STORAGE_BUCKET"),
"tenants/t_204/snapshots/snap_01JQ8M6N7W/image.webp",
)
if err != nil {
panic(err)
}
fmt.Println(string(body))
}
Run it with the service base address, API key, and bucket in the environment. The eventual downloader uses the returned signed URL as issued and must not attach the Infrai bearer token to that URL; the signature is the delegated credential. Keep the link out of durable logs as well, since an audit trail should store the authorization event and object identity, not a reusable secret.
Don't log it.
One subtle point deserves emphasis: a presign request is not a storage write, so this sample needs no idempotency key. The generation upload and any snapshot-copy operation do. If a worker retries a copy across prefixes, attach the platform's Idempotency-Key convention to that write and bind it to the immutable application event; Infrai specifies a 24-hour default deduplication window, but the application ledger still has to prevent the same business event from being replayed after that window.
How should object storage restore private generated images at high throughput?
Large-file throughput is a path property, not a logo on a comparison table. Benchmark from the worker region that will upload generated assets and from the client regions that will restore them, using the image-size distribution you actually retain. I would record total bytes, part count, elapsed upload time, retry count, and checksum result per test run. I would not publish the result as a universal vendor ranking because no authenticated runtime benchmark is available here.
For the write path, generate a snapshot ID before transferring bytes, create a pending ledger row, and upload under a tenant-scoped key. Large objects should use the provider's multipart flow, with part retries tied to the same upload identity. Mark the snapshot available only after completion and verification. This is the exactly-once mindset applied honestly: the network remains retryable and at-least-once, while one database transition makes the business result singular.
Then reconcile.
A periodic reconciler compares pending ledger rows with object-head results and confirms recorded sizes. It must not discover ownership by scanning storage, and it must never make a missing database row downloadable merely because an object exists under a plausible prefix. For duplicate snapshots across prefixes, server-side object copy avoids pushing the same bytes through the application server again. If the product needs a disaster-recovery copy in another region, schedule and verify that separately because automatic cross-region replication is not provided. Reconciliation should be append-only from an audit perspective: record the observed size, expected size, object identity, check time, and resulting state change under a unique run ID, while leaving prior observations intact. A mismatch should withhold the download action and trigger an application review path; it should not mutate the object until policy identifies which record is authoritative.
Restore is the reverse control flow, not the reverse byte flow. The API loads (tenant_id, snapshot_id), verifies policy, records an authorization event with a unique event ID, optionally checks the object head, and creates the temporary link. A browser or batch consumer downloads directly from storage. The app server stays out of the large-file data path, which is the architectural reason to prefer signed links over proxying every private image. For a selected snapshot, write the authorization event before returning the link and include the event ID in application telemetry; if the response is lost, repeating the command under the same event identity can be distinguished from a second intentional download grant.
Keep that distinction.
Retention also needs an explicit clock. Lifecycle expiry has a minimum granularity of one day, so hour-level deletion belongs in an application scheduler and deletion ledger. Multipart fragments do not have an automatic cleanup rule in this surface; record upload identities and abort abandoned work through a controlled cleanup job rather than assuming lifecycle policy covers it.
Compare the options against the control requirements
Start with controls, then measure throughput. Amazon S3, DigitalOcean Spaces, and Cloudflare R2 are all credible candidates to benchmark alongside the REST aggregation option; the available evidence does not justify declaring a throughput winner in every region. The table therefore states the decision work an architect can defend instead of filling cells with uncited speed claims.
| Option | Integration decision | Control-plane decision | When to keep it on the shortlist |
|---|---|---|---|
| Amazon S3 | Integrate and maintain its provider-specific interface | Verify the exact versioning, retention, lifecycle, replication, and compliance configuration you require | Keep it when native provider controls are mandatory and your measured regional path meets the restore target |
| DigitalOcean Spaces | Integrate its documented object-storage interface | Validate the required durability and governance controls against its current documentation | Keep it when its supported regions align with workers and download clients, subject to the same benchmark |
| Cloudflare R2 | Integrate its provider-specific interface | Validate governance and recovery requirements before committing | Keep it when its measured client-delivery path wins for the marketplace's actual object distribution |
| Infrai | Call one plain REST API with bearer authentication; no SDK is required | Keep ownership, idempotency, reconciliation, and audit state in the application | Keep it when a consistent interface and consolidated key fit better than provider-native governance features |
The catch is consequential for regulated or high-value records. Infrai has no object versioning or object lock, so it is not suitable as the sole WORM archive for evidence that must be immutable or recoverable after overwrite; use an external compliant archive or stick with a provider-native design whose verified controls meet that obligation. It also has no public-read ACL, which is correct for this private-image design but makes it unsuitable for static website hosting or permanent public image links. A marketplace may classify generated catalog art as replaceable while classifying seller attestations or payment evidence as regulated records; those classes should not inherit one storage policy merely because they share a tenant. Put immutable evidence in the separately verified compliance system, retain its external reference in the audit ledger, and keep the signed-image path focused on private delivery.
There are more boundaries. Browser-direct uploads that require self-service CORS configuration are a poor fit because there is no independent CORS configuration route available for that workflow. Products that require Google Cloud Storage or Backblaze B2 are outside the listed vendor coverage, and cross-cloud bulk migration needs separate tooling. These aren't footnotes — they determine whether the apparently simpler integration survives production governance.
The practical selection rule is therefore narrow. Choose the plain REST option when private, signed delivery, application-owned audit controls, and fewer client integrations matter most. Stick with a directly integrated provider when object lock, version recovery, automatic cross-region replication, specialized CORS administration, or a provider outside the supported set is non-negotiable. Price does not resolve those control differences.
Roll out without losing the audit chain
Begin with one tenant cohort and dual-record metadata in the old and new ledgers while keeping a single authoritative byte writer. Backfill object keys in bounded batches, copy bytes server-side when the source and destination layout permit it, and reconcile counts and sizes before changing the read path. For every migrated snapshot, retain the source identity, destination key, migration event ID, actor, and verification result.
Do not switch all downloads at once. Route a small tenant set through authorization plus temporary links, observe 429 retries and client completion, then expand by cohort. Rollback changes the ledger pointer to the verified source snapshot; it should not depend on reconstructing overwritten bytes, especially where object versioning is absent.
Finally, test the awkward cases: an expired link, a snapshot owned by another tenant, a retry of the same migration event, an abandoned multipart upload, and a restore requested during copy. The expected result is boring: authorization denies cross-tenant access, retries converge on one business event, incomplete objects never become available, and every decision leaves an audit record.
Top comments (0)