Short answer: persist one export object per tenant, cache its still-valid signed URL, and coalesce concurrent cache misses in the backend. Treat HTTP 429 as a bounded retry problem: honor Retry-After, use exponential backoff with jitter, and never let a page refresh create a second export.
Here is the decision matrix I would use for a marketplace export page:
| Approach | Good fit | Main trade-off |
|---|---|---|
| Private object plus short-lived signed URL | Tenant-scoped files delivered after an export job completes | Requires authorization, cache-key discipline, and URL renewal |
| Backend download proxy | Strict control over every byte and a stable application URL | Adds bandwidth, connection handling, and application scaling work |
| Public object URL | Truly public, immutable assets | It is unsuitable for tenant data or per-user authorization |
| Managed file system | Workloads needing file-system semantics | It does not solve presign request storms by itself |
For a marketplace, choose the first approach when the export is an asynchronous file artifact and the browser only needs temporary read access. The hard part is tenant isolation, not generating a URL. Keep the export identity, object identity, and access grant as separate records with separate lifetimes.
Why do SaaS export download links hit rate limits after a page refresh?
An export page often combines several retry sources: the browser refreshes, a component remounts, a polling timer fires, and a user clicks Download again. If each path asks the backend to generate an export and presign its object, one completed file can cause a surprising number of control-plane requests. A 429 response followed by an immediate retry makes the burst larger.
It gets loud fast.
That multiplication is easy to miss in a code review because each individual call looks reasonable. The page asks for the export status, sees complete, asks for a link, receives a 429, and retries. A component remount repeats the sequence. A second tab has no knowledge of the first tab's in-flight promise. If the API turns a missing link into a new export job, the storage object count rises too; if it only repeats signing, the presign control plane still sees the same tenant and object over and over. The fix is to name the state transitions explicitly: pending means poll the job, complete means reuse the recorded object, and link-cache-miss means perform one bounded signing attempt. A refresh may revisit all three states, but it must not create a new state transition merely because the browser was reloaded. That distinction gives the backend something it can enforce and gives the test suite a finite set of cases.
A refresh did not change the tenant, export, object key, or requested access policy. It should not require a new export. Store an export record such as tenantId, exportId, status, objectKey, and expiresAt. Once the job is complete, the link endpoint should read that record, authorize the caller, and reuse the object. The signing operation is a derivative step.
The cache key must include every value that changes the authorization result. In practice that usually means tenant, export or object identity, requested method, and the relevant expiry or response options. A key based only on a file name can cross tenants when names are reused. That's a security failure before it is a performance failure.
Cache the signing response for less time than the signed URL remains valid. The gap gives the application room to stop handing out a URL just before its actual expiry. I am not sure there is a universal TTL for this: an export page open for thirty seconds and one open for an hour have different renewal pressure. Replay real refresh intervals and measure cache misses before choosing the window.
The backend should also collapse concurrent misses. Ten requests for the same cache key should wait on one signing promise, not start ten signing calls. This is often called single-flight or request coalescing. It belongs behind the application endpoint because browser tabs do not reliably share state, and a client retry can continue after the original page has disappeared.
What should Node.js do for rate limit, presign, export, and signed URL retries?
Keep the policy in one small service function. It should check the cache, join an in-flight request when one exists, call the storage adapter on a true miss, and cache only a successful result. Do not cache a 429, authorization failure, or malformed response as if it were a link.
This TypeScript example uses an interface instead of a vendor-specific path. The adapter is where a particular object-storage SDK or HTTP contract belongs. That separation makes the rate-limit behavior testable without turning the article into a setup guide for one provider.
type SignRequest = {
tenantId: string;
objectKey: string;
expiresInSeconds: number;
};
type SignedLink = {
url: string;
expiresAt: number;
};
type RateLimitError = Error & {
status?: number;
retryAfterMs?: number;
};
interface Signer {
createSignedReadUrl(request: SignRequest): Promise<SignedLink>;
}
const cache = new Map<string, { value: SignedLink; cacheUntil: number }>();
const inFlight = new Map<string, Promise<SignedLink>>();
function cacheKey(request: SignRequest): string {
return JSON.stringify([
request.tenantId,
request.objectKey,
request.expiresInSeconds,
]);
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function retryDelay(error: RateLimitError, attempt: number): number {
const serverDelay = error.retryAfterMs;
if (typeof serverDelay === "number" && serverDelay >= 0) {
return serverDelay;
}
const exponential = Math.min(8_000, 250 * 2 ** attempt);
const jitter = Math.floor(Math.random() * 100);
return exponential + jitter;
}
async function signWithBackoff(
signer: Signer,
request: SignRequest,
maxAttempts = 4,
): Promise<SignedLink> {
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
try {
return await signer.createSignedReadUrl(request);
} catch (unknownError) {
const error = unknownError as RateLimitError;
if (error.status !== 429 || attempt === maxAttempts - 1) {
throw error;
}
await sleep(retryDelay(error, attempt));
}
}
throw new Error("unreachable retry state");
}
export async function getSignedLink(
signer: Signer,
request: SignRequest,
cacheTtlMs: number,
): Promise<SignedLink> {
const key = cacheKey(request);
const now = Date.now();
const hit = cache.get(key);
if (hit && hit.cacheUntil > now && hit.value.expiresAt > now) {
return hit.value;
}
const existing = inFlight.get(key);
if (existing) return existing;
const pending = signWithBackoff(signer, request)
.then((value) => {
const safeTtl = Math.min(cacheTtlMs, value.expiresAt - Date.now());
if (safeTtl > 0) {
cache.set(key, { value, cacheUntil: Date.now() + safeTtl });
}
return value;
})
.finally(() => {
inFlight.delete(key);
});
inFlight.set(key, pending);
return pending;
}
There are two details worth keeping. First, the cache lifetime is capped by the URL's real expiry, so a stale grant cannot be served because an operator picked an overly generous cache setting. Second, the finally handler removes the in-flight entry on both success and failure. A failed request must not poison every later request forever.
In production, add a distributed cache or a fleet-level coalescing mechanism when multiple Node.js processes serve the same tenant. A process-local map only collapses requests that land on one process. Bound the map as well; expired entries and abandoned export records should not become a second memory leak.
How does tenant isolation change the export download architecture?
Tenant isolation starts before signing. Resolve the caller's tenant from authenticated server-side context, load the export by both tenantId and exportId, and verify that the export belongs to that tenant before reading its object key. Never accept an arbitrary bucket, object key, or tenant identifier from an untrusted browser as the authorization decision.
Use opaque object keys with a tenant-scoped prefix, for example tenant/<tenant-id>/exports/<export-id>/result.json, while still enforcing authorization in the application. A prefix is useful for listing and cleanup; it is not an access-control substitute. Keep the bucket private, and return only the temporary URL after the application check succeeds. The browser does not need the storage service's bearer credential.
Make the export job idempotent too. A client request can carry a stable export request identifier, and the database can map that identifier to one job for one tenant. On a refresh, return the existing pending, complete, or failed state. Do not start another worker merely because the page mounted again.
Observability should expose the boundary between useful work and noise. Record tenant-safe hashes rather than raw signed URLs, then measure export jobs created, signing attempts, cache hits, coalesced waiters, 429 responses, retry delay, and link-expiry failures. Alert on a rising miss ratio and retry budget exhaustion. A dashboard that counts page views but hides signing misses will point the on-call engineer at the wrong bottleneck.
Validate the design with tests that run concurrently. The most important case starts ten calls with the same tenant and object and asserts one adapter invocation. Add a cross-tenant case with the same object name and assert separate keys. Add a Retry-After case, a final-attempt 429 case, an already-expired URL case, and a refresh after the export record is complete. These are cheap tests. The production incident is not.
When is a signed URL design the wrong choice?
The catch is that signed URLs are temporary credentials, not a complete file-delivery architecture. A backend proxy is a better fit when every byte must pass through application policy, when a stable URL must hide storage details, or when downstream audit requirements need a single controlled stream. Pay for that control with application bandwidth and connection capacity.
Public assets are another boundary. If an image is intentionally public and immutable, a signed-link round trip adds policy machinery without adding useful protection. Use a public delivery design with an explicit publication workflow instead. Do not make tenant exports public to avoid a rate limit.
Large files and long-lived downloads also need a separate decision. A short-lived grant can expire during a slow transfer; a very long grant increases the impact of a leaked URL. Test the chosen validity against real file sizes, client behavior, and network conditions. Your mileage may vary, especially for mobile clients that suspend background work.
Finally, object storage is not a managed file system. If the workload needs shared file-system semantics, locking, or tools that require mounted paths, evaluate a managed file system or another file-oriented service. That choice changes the operational model; it does not remove the need to prevent duplicate export jobs and refresh-triggered retries.
For this marketplace scenario, I would ship the idempotent export record, tenant-bound lookup, short cache, single-flight map, and bounded 429 backoff first. Then I would benchmark cache-miss traffic and download completion. Change the storage architecture only when the measured workload or a required capability says to.
Top comments (0)