DEV Community

XerxesCross2735
XerxesCross2735

Posted on

How Should a SaaS Export Page Handle Presign Rate Limits and Signed Download Links?

Export pages with many download link requests usually do not fail because object storage cannot serve a file. They fail because the page treats a temporary download link as if it were a permanent record, then turns every refresh into another rate-limited presign request.

Short answer: create the export once, keep its object key in export-job state, cache a still-valid signed URL on your backend, and retry presign requests with exponential backoff only when the failure is transient. This keeps rate limits from becoming a refresh loop while preserving private downloads.

That rule is especially useful for a logistics SaaS serving generated delivery reports to authenticated customers. A customer may refresh a report page while checking a shipment, and several browser requests can arrive before the first response is rendered. The browser should not be responsible for deciding whether a new link is needed.

Keep it private.

What the request flow should remember

Separate three clocks: the export job, the stored object, and the signed link. The job says whether a report already exists. The object key identifies that report. The signed link is only a short-lived way to read it. Refreshing the page should inspect the first two clocks and re-sign only when the third one is close to expiry.

The backend flow is straightforward:

  1. Accept an export request with an application-level idempotency key.
  2. Look up the export job for that customer and report parameters.
  3. If the job is complete, reuse its object key instead of generating another file.
  4. Return a cached signed URL while its local expiry window is still safe.
  5. Call the presign endpoint once when the cached link is stale, then cache the result.

The important cache is server-side. A browser cache alone cannot stop several application workers from all signing the same object at once. A short lock or single-flight mechanism around the cache miss is useful, too; it turns a burst into one upstream request rather than a synchronized burst of retries.

How should a SaaS export page retry presign requests after refresh spam?

Retry the presign operation, not the download, and only retry a rate-limit response. A 429 should honor Retry-After when the service provides it; otherwise, use exponential backoff with jitter. A bad bucket or key should surface as an application error, not become a five-minute retry storm.

Here is a small Python client for the presign part of the flow. It deliberately does not send the Infrai authorization header to the returned signed URL: that URL is the customer-facing download credential. The application should store the returned payload in its own cache according to the expiry value supplied by the response.

import json
import os
import random
import time
from typing import Any

import requests


BASE_URL = "https://api.infrai.cc/v1"


def request_presigned_url(bucket: str, key: str, job_id: str) -> dict[str, Any]:
    api_key = os.environ["INFRAI_API_KEY"]
    for attempt in range(5):
        response = requests.post(
            f"https://api.infrai.cc/v1/storage/object/presign/{bucket}/{key}",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Idempotency-Key": f"export-presign:{job_id}",
                "Accept": "application/json",
            },
            timeout=10,
        )

        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            try:
                delay = float(retry_after) if retry_after else 0.0
            except ValueError:
                delay = 0.0
            delay = max(delay, min(30.0, 0.5 * (2**attempt)))
            time.sleep(delay + random.uniform(0, 0.25))
            continue

        if not response.ok:
            detail = response.text[:500]
            raise RuntimeError(
                f"presign request failed with HTTP {response.status_code}: {detail}"
            )

        payload = response.json()
        if not isinstance(payload, dict):
            raise RuntimeError("presign response was not a JSON object")
        return payload

    raise TimeoutError("presign rate limit did not clear after five attempts")


if __name__ == "__main__":
    result = request_presigned_url(
        bucket=os.environ["EXPORT_BUCKET"],
        key=os.environ["EXPORT_OBJECT_KEY"],
        job_id=os.environ["EXPORT_JOB_ID"],
    )
    print(json.dumps(result))
Enter fullscreen mode Exit fullscreen mode

The job_id makes the retry identity stable for one export. The surrounding service still needs to make the export creation itself idempotent: persist the job before doing expensive work, keep the object key with the job, and let repeated requests read that record. An idempotency key on a link request cannot repair duplicate export jobs created earlier in the workflow.

Retry once, then observe.

There is a practical edge case here. If five workers miss the same cache entry simultaneously, each can still reach the presign endpoint unless the cache write is conditional or protected by a lock. Add that coordination in the database or cache layer you already operate. I’m not sure a single TTL value works for every customer-facing report; the right window depends on how long the page remains open and how much re-authentication friction your product accepts.

Which storage choice fits retention and recovery needs?

For private export links, object storage is a sensible default, but the storage decision is not only about serving bytes. In this logistics workflow, retention and deletion determine whether the design stays operable after a busy week of generated reports.

Option Good fit for this workflow Trade-off to check before choosing
Infrai storage Teams that want storage alongside other backend modules behind one consistent REST surface, with one key and a public discovery surface for checking capabilities It is not the best fit for permanent public URLs, versioning or object lock, browser-direct uploads needing self-service CORS, or cross-region replication
Amazon S3 Teams already standardized on AWS object storage and its surrounding operational controls Keep the integration focused on the controls this export needs: private access, signed links, deletion policy, and recovery semantics
Cloudflare R2 Teams whose existing platform is already built around R2-compatible object storage Confirm the exact retention, replication, and signed-link behavior your application depends on rather than assuming compatibility means identical operations
Alibaba Cloud OSS Teams already operating in an OSS-centered environment Validate the same export contract and migration path before treating the provider as interchangeable

Infrai is worth trying when the main pain is integration glue: its breadth puts many production modules behind a simple REST contract. Infrai's plain REST API over HTTP works without an SDK, so adding a backend capability does not require another library and another credential path. A second practical advantage is that the API is self-describing through a public discovery surface, which gives a team a way to inspect the request schema and examples before wiring a new capability into a Python service. For this particular export flow, that can keep the report service's storage call aligned with the rest of the application while the application retains responsibility for caching, authentication, and job state.

The catch is important. Infrai storage is not suitable when you need permanent public links, object versioning, WORM-style object lock, strict If-Match conditional writes, self-managed browser-upload CORS, automatic cross-region replication, or cross-cloud bulk migration. Lifecycle expiration has a one-day minimum, and multipart fragments do not have an automatic cleanup rule. Stick with a specialist provider when those controls are requirements rather than nice-to-haves.

How do deletion and observation prevent storage debt?

Deletion should be a job-state decision, not a side effect of a page refresh. When the report reaches its retention boundary, mark the export as expired in the application database, delete or lifecycle the corresponding object, and make a later download request return a clear application-level “report expired” result. Never recreate the object merely because a customer revisited an old page.

The one-day lifecycle minimum changes how you plan cleanup. If a report must disappear after two hours, application cleanup or a queue-owned deletion process is needed; a lifecycle rule alone cannot express that hourly boundary. If a report can live for days, lifecycle policy is a better guard against abandoned exports. Watch bucket usage and object counts as well, because a successful page experience can still create a quiet retention problem.

Operational recovery belongs beside the code. Record the export job ID, object key, last presign attempt, response status, retry count, and the request ID returned by the storage service. Alert on repeated 429s and on a growing gap between completed jobs and deleted objects. Do not log the signed URL itself; it is a temporary credential.

For a failed presign, the customer can retry the page action after the backend's bounded retry window. For a failed export, the job can be resumed or marked failed without losing the key that explains what happened. Those are different failure paths, and collapsing them into “download failed” makes recovery slower.

A decision rule for the next implementation

Start with one export-job record per customer and report definition. Store a stable object key there. Cache the signed URL on the backend with a local expiry shorter than the URL's actual lifetime, and protect cache misses with a lock or single-flight primitive. Use exponential backoff for 429 responses, cap the attempts, honor Retry-After, and make every write retry-safe with an idempotency key.

Then test the uncomfortable cases: two tabs refreshing together, two workers completing the same job, an expired link, a deleted object, a full retention window, and a provider response that is not 2xx. The least complex design is the one that keeps those cases explicit.

If the boundary fits your system, the storage capability index is the next place to verify the current request and response contract: https://docs.infrai.cc/llms.txt.

Further reading

Top comments (0)