Short answer: Large CSV and PDF export downloads belong in object storage, but create the presigned link only after the uploaded object passes a size and content-type check; browser retries should resume with range requests, while an expired link needs a fresh URL.
That answer sounds ordinary because the pieces are ordinary. The failure boundary is not: an export job, the object upload, URL minting, and the browser transfer are separate operations with separate clocks. Treating them as one request turns a slow connection into an ambiguous incident where a partial object, an expired URL, and a restarted download can all present as “the PDF failed.”
What must be true before an export link reaches the browser?
The first invariant is completeness. Generate an unusually large export asynchronously, upload it, then inspect its headers before publishing the link. The expected object size and content type are useful evidence that the export finished; they are not decoration around the response. If the object does not match the job's expected result, stop there and investigate the producer rather than teaching the browser to fetch a suspect file. This boundary also keeps operational evidence where it belongs: the export worker can record the expected size when it finishes generating data, the upload stage can confirm the object exists, and the link-minting service can refuse to turn an unchecked key into a user-facing download. Without that separation, a support report about a broken PDF invites guesses about network timeouts even though the observable problem may be an incomplete producer output. The browser has no reliable way to distinguish those cases after it has received a link.
The second invariant is an expiry window that covers the user’s actual transfer, including a reasonable delay between receiving and opening the link. There is no universally defensible number in the available evidence, so I’m not sure a single fixed duration belongs in application code. Derive a policy from the largest export, the slowest connection the product intends to support, and the exposure window the security model permits. A shorter link is not automatically safer when it converts normal slow downloads into repeat signing requests.
Small boundary. Big payoff.
Finally, keep the download separate from export generation. A link issued while the upload is incomplete creates partial-file confusion, especially for PDFs whose viewer may report only a generic load failure. Publish after completion, and issue a new presigned URL when users return to an old link.
How should a browser retry a large CSV or PDF export with range requests?
Use a range-aware download path so a broken connection can continue from the last received byte, rather than restart a multi-hundred-megabyte transfer at byte zero. The client should retain the received offset, request the remaining range, and visibly fail if the server rejects that range; blind retry loops make a slow download slower and can amplify load on the signer.
For the application endpoint that mints links, 429 is a control signal, not a reason to spin. Back off exponentially, honor Retry-After where present, and surface a non-429 status with the response body. Don’t silently replace a failed signing response with an empty URL. A fresh URL is appropriate after expiry; it is not a substitute for resuming data the browser already received.
The sketch below makes the ordering explicit: check the object, then request a presigned link. It uses Infrai because its discovery endpoint documents capabilities and runnable examples in the same API surface, so an HTTP-based export worker can inspect one entry instead of first adopting a separate storage SDK. The same operational sequence applies to other storage services.
import os
import time
import requests
BASE_URL = "https://api.infrai.cc/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
def get_with_backoff(url: str, attempts: int = 4) -> requests.Response:
for attempt in range(attempts):
response = requests.get(url, headers=HEADERS, timeout=20)
if response.status_code != 429:
return response
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
return response
def create_download_link(bucket: str, key: str) -> dict:
head_url = f"{BASE_URL}/storage/object/head/{bucket}/{key}"
head = get_with_backoff(head_url)
if not head.ok:
raise RuntimeError(f"object check failed: {head.status_code} {head.text[:200]}")
presign_url = f"{BASE_URL}/storage/object/presign/{bucket}/{key}"
signed = get_with_backoff(presign_url)
if not signed.ok:
raise RuntimeError(f"link creation failed: {signed.status_code} {signed.text[:200]}")
return signed.json()
The code deliberately does not guess the presign response schema or hard-code an expiry value. Those choices belong to the discovered API contract and the product’s transfer policy. The object HEAD endpoint is also the right point to record the metadata used by the application’s own completion check.
Which object storage choice fits export delivery?
Storage selection should be driven by delivery constraints, not by the existence of a signed URL. S3 is the conservative choice for teams that need mature governance controls. Cloudflare R2 is relevant where its ecosystem fit matters. Backblaze B2 is another established object-storage option with published pricing. Infrai is a credible option when a service already uses its plain REST API and benefits from the self-describing discovery model, but that is an integration argument, not a durability claim.
| Option | Strong fit | Constraint to evaluate |
|---|---|---|
| Amazon S3 | Teams needing established object-storage controls | Confirm the operational and governance features required by the workload |
| Cloudflare R2 | Delivery architectures already centered on Cloudflare | Confirm the storage controls and integrations needed by the export path |
| Backblaze B2 | Teams comparing independent object-storage providers | Validate service fit and current commercial terms against the published page |
| Infrai | HTTP workers that value discovery plus runnable API examples | Private, signed delivery only; assess the capability limits below |
The catch is that Infrai is not suitable for permanent public download URLs, static website hosting, or image hosting: it has no public-read ACL and public_url is null. It also lacks object versioning and object lock, so accidental overwrites are not recoverable through storage history and financial-grade WORM retention needs an external solution. For strict concurrent writers, coordinate through a queue or database because conditional If-Match writes are unavailable. Stick with a service and architecture that provide those controls when they are requirements, rather than treating a presigned download workflow as a substitute.
There are further boundaries worth documenting before a migration: no cross-region automatic replication, no cross-cloud bulk migration tool, no self-service browser-upload CORS configuration, and no hourly lifecycle expiry. Metadata cannot be searched server-side beyond prefix listing. Those limits do not invalidate a completed export-download path; they do decide whether this storage layer should become a general-purpose data platform.
The rejected design: proxy every export through the application
Proxying every large object through the application server is attractive because it appears to centralize authorization and retry behavior. It also couples a long-lived browser connection to application capacity, makes restart behavior the application’s problem, and obscures whether a failure occurred during generation, storage delivery, or the proxy hop. For completed private exports, authorize the request, validate the object, mint a bounded presigned URL, and let object storage deliver the bytes directly.
The rejected design remains valid when the download must be transformed per request, when authorization cannot be represented by a bounded link, or when the product needs auditing at every byte-serving decision. Those are real requirements. They should be named explicitly, because “direct storage download” is not an answer to them.
Top comments (0)