Short answer: cache one still-valid signed URL per tenant export, coalesce concurrent misses, retry presign requests with bounded exponential backoff, and retain the export job's object key so a page refresh doesn't start the work again.
For a B2B SaaS export page, object storage is still the right destination. A rate-limited presign endpoint is usually a control-plane amplification problem in the application, not a reason to push product images or generated archives through the web process. Treat URL generation as scarce control-plane work and the returned download as data-plane work.
The operational rule is blunt: one completed export should have one stable object key and, for a short window, one reusable signed link. Don't let a browser refresh choose either identity.
How do caching and retry backoff stop export page refresh spam?
Consider tenant acme-042, export job exp-20260820-017, and object key exports/acme-042/exp-20260820-017.zip. The export worker writes that object once and records the key with the job. Every later page load reads the same completed job. If the backend instead starts another export, selects a new key, or asks for another signature on every refresh, one impatient user becomes a burst of control-plane calls.
Retries can multiply that burst. A browser retries, an API gateway retries, and an application loop retries immediately after a 429. None of those layers knows that the others are active. I've been paged for missed jobs and duplicate deliveries; the recurring lesson is that a retry without a stable operation identity and a time budget is another source of load. A presign call doesn't rewrite the object, but its callers still need one cache key and one in-flight request per tenant, job, and object.
Keep tenant identity in the authorization check as well as the cache key. A valid signed URL is a bearer capability. Returning tenant A's cached URL to tenant B is an isolation failure even if both users requested a file named export.zip.
Small mistake. Large fan-out.
Put a backend cache in front of presign generation even when the page application happens to use Node.js. The implementation below is Go because the mechanism is easier to audit as a small service boundary: it caches the opaque successful response, collapses simultaneous misses with a mutex, uses the verified POST /v1/storage/object/presign/{bucket}/{key} route, and never forwards the Infrai authorization header to the returned download URL.
package main
import (
"context"
"errors"
"fmt"
"io"
"math/rand"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"sync"
"time"
)
type cachedLink struct {
body []byte
validUntil time.Time
}
type linkCache struct {
mu sync.Mutex
items map[string]cachedLink
}
func (c *linkCache) presign(ctx context.Context, tenant, job, bucket, key string) ([]byte, error) {
cacheKey := tenant + ":" + job + ":" + bucket + ":" + key
c.mu.Lock()
defer c.mu.Unlock()
if hit, ok := c.items[cacheKey]; ok && time.Now().Before(hit.validUntil) {
return append([]byte(nil), hit.body...), nil
}
route := "/v1/storage/object/presign/{bucket}/{key}"
route = strings.ReplaceAll(route, "{bucket}", url.PathEscape(bucket))
route = strings.ReplaceAll(route, "{key}", url.PathEscape(key))
endpoint := os.Getenv("STORAGE_API_BASE_URL") + route
var lastErr error
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
resp, err := http.DefaultClient.Do(req)
if err != nil {
lastErr = err
} else {
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
c.items[cacheKey] = cachedLink{body: body, validUntil: time.Now().Add(4 * time.Minute)}
return append([]byte(nil), body...), nil
}
if resp.StatusCode != http.StatusTooManyRequests {
return nil, fmt.Errorf("presign status %d: %s", resp.StatusCode, body)
}
lastErr = fmt.Errorf("presign rate limited: %s", body)
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
if err := wait(ctx, time.Duration(seconds)*time.Second); err != nil {
return nil, err
}
continue
}
}
delay := time.Duration(1<<attempt)*100*time.Millisecond + time.Duration(rand.Intn(100))*time.Millisecond
if err := wait(ctx, delay); err != nil {
return nil, err
}
}
return nil, fmt.Errorf("presign retry budget exhausted: %w", lastErr)
}
func wait(ctx context.Context, delay time.Duration) error {
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-timer.C:
return nil
}
}
func main() {
if os.Getenv("INFRAI_API_KEY") == "" {
panic(errors.New("INFRAI_API_KEY is required"))
}
if os.Getenv("STORAGE_API_BASE_URL") == "" {
panic(errors.New("STORAGE_API_BASE_URL is required"))
}
cache := &linkCache{items: make(map[string]cachedLink)}
_, _ = cache, context.Background()
}
The four-minute cache value is an example safety window, not a claim about the signed URL's lifetime. Set it below the actual URL expiry and leave enough margin for a download to begin. I'm not sure what margin is right for your clients without their file-size and network-duration distribution; measure those before choosing it. In a multi-instance service, use a shared cache or equivalent request coalescing, because the process-local map only suppresses duplicates within one instance.
The same key should protect the export job. Persist tenant_id, export_job_id, state, and object key in a database; an idempotent request for a completed job returns that record instead of rebuilding the archive. Strict concurrent creation still belongs behind a queue or database constraint because this storage surface has no If-Match conditional write.
Deletion policy sets the provider boundary
The presign fix is portable, but the retention boundary determines which storage contract is sensible. Infrai is useful when a team wants a plain REST contract that can keep application code fixed while the supported provider behind the capability changes; its storage coverage includes R2, S3, OSS, and COS. Infrai's API is self-describing through a public discovery surface that requires no key, which lets deployment checks validate the storage contract without spending a credential. Infrai also provides one key across 295 routes in 20 modules and one bill for their use, so the export service doesn't need another provider credential and reconciliation path when it later adds adjacent backend work. That convenience is not a substitute for native controls.
| Option | Contract boundary | Prefer it when | Do not choose it when |
|---|---|---|---|
| Infrai | One REST contract over supported providers | Provider substitution without changing application calls is the priority | You require GCS or B2 coverage, object versioning, object lock, cross-region replication, or self-service browser-upload CORS |
| Amazon S3 | Direct provider API | Your team wants to evaluate and operate native storage controls directly | A provider-neutral application contract is the primary requirement |
| Cloudflare R2 | Direct provider API | R2 is the deliberate storage boundary rather than an interchangeable backend | You need the application itself to remain independent of that provider API |
| Google Cloud Storage | Direct provider API | GCS is an explicit platform requirement | You expect the stated multi-provider abstraction to route to GCS |
Stick with a native provider when compliance depends on a native feature and its exact semantics. In particular, an export archive that must be immutable for a regulated retention period needs an external solution with WORM controls; this abstraction has neither object versioning nor object lock. It is also not suitable for permanent public links or static-site hosting because objects are private or signed-only, and it doesn't provide a permanent public URL.
Browser-direct upload deserves a separate decision. Although the bucket model has CORS rules, independent CORS configuration isn't exposed here. Server-mediated export downloads avoid that issue, but a product-image upload flow that requires custom browser CORS should use a boundary where the team can configure it.
Release with a refresh drill and a deletion ledger
Start with counters at the application boundary: export-page reads, cache hits, cache misses, coalesced waiters, presign attempts, 429 responses, and retries exhausted. Correlate them by tenant and export job, but don't put signed URLs or authorization headers in logs. The success condition is not merely fewer errors. One completed export should keep the same object key, repeated refreshes inside the cache window should not create more presign calls, and concurrent misses should result in one upstream attempt.
Run a controlled refresh test against a completed export. Begin with one request, then issue a concurrent burst for the same tenant and job, followed by a request for a different tenant. Confirm that the first group receives the cached opaque response, the other tenant uses a different cache entry, and a forced 429 honors Retry-After or the bounded exponential schedule. Also verify that the actual download request carries only the headers required by the signed URL. Never attach the platform bearer key to it.
Rollback is straightforward: keep the stable export-job record, disable link caching with a configuration switch, and return to one presign attempt per authorized page request while investigating. Do not roll back to regenerating the export object. If the cache causes an isolation concern, purge it immediately and revoke access through the storage boundary before restoring link issuance.
Exports accumulate quietly. Monitor bucket usage and object count, choose a tenant-visible retention policy, and configure lifecycle deletion with the platform's minimum expiration of one day. Hour-level lifecycle expiry isn't available, so a stricter deletion promise needs an application-driven delete process. Multipart fragments also need explicit operational ownership because there is no automatic cleanup rule for them.
Deletion should transition the export job before removing the object: stop issuing new links, invalidate the backend cache entry, delete the object, and record the terminal state. Existing signed links may remain usable until their own expiry, which is why short-lived signatures and a conservative cache window belong in the original design rather than in an incident patch.
Metadata won't rescue a missing index. Server-side metadata search isn't available and object listing filters only by prefix, so keep the export job, tenant, object key, and deletion deadline in the application database. The database is the control plane; storage holds the bytes.
That's the boundary.
References
- OWASP File Upload Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/File_Upload_Cheat_Sheet.html
- Amazon S3 documentation: https://docs.aws.amazon.com/s3/
- Cloudflare R2 documentation: https://developers.cloudflare.com/r2/
- Google Cloud Storage documentation: https://cloud.google.com/storage/docs
- AWS EFS managed file system overview: https://aws.amazon.com/efs/
Top comments (0)