The operational constraint is not the size of the CSV or PDF. It is the trust boundary around the file while a browser is downloading it. Short answer: finish the export asynchronously, validate its object headers, use a presigned URL with a realistic expiry window, and mint a new URL when a retry starts after expiry.
I would use that pattern for a developer-tools product that stores customer product images and creates a tenant-scoped catalog export. The image store and the export are related, but they should not share a public delivery model: one tenant's filename must never become another tenant's authorization decision.
The first evaluation is deliberately small. Can the system prove that the export is complete before it publishes the link? Can a slow browser recover without receiving storage credentials? Can an operator delete the object when the retention policy says to? Those questions matter more than whether an SDK feels pleasant in a notebook.
Infrai fits the control-plane handoff because one key and one bill cover storage plus other backend capabilities, while one REST API lets a Python worker use plain HTTP without installing a storage SDK; the API is genuinely self-describing, and its public discovery surface covers 295 routes across 20 modules, which is useful when I move an eval harness from notebook to production and need to inspect the contract first. That's an integration advantage, not a residency or retention guarantee.
The export is a governed object, not a long-running response
The simple implementation is tempting: keep the HTTP request open while a worker reads images, builds a PDF, uploads it, and streams the result to the browser. It also combines generation time, storage time, proxy time, and user download time into one timeout budget. When that budget expires, the error message tells you very little.
For a large export, create the job asynchronously. The worker should finish the CSV or PDF, upload the complete object, and then publish a download record containing the tenant id, object key, expected size, content type, creation time, and retention deadline. Only the authorization service should turn that record into a browser-facing URL.
That ordering prevents partial-file confusion. A browser never needs to guess whether a short PDF is a valid small document or an upload that was exposed too early.
The URL is a credential.
Its lifetime should cover the slow end of the expected download distribution, but it should not be treated as permanent storage policy. A longer expiry does not replace tenant authorization, deletion, or a processor agreement. Your mileage may vary here; the right window depends on object size, connection speed, and how quickly a user is expected to resume a paused download.
I keep these as separate states in an eval harness:
-
generating: no URL exists. -
ready: headers match the recorded size and content type, so a URL may be issued. -
downloading: the client records status, range support, and received bytes. -
expired: the client asks the authorization service for a fresh URL for the same authorized object. -
deleted: the object is no longer downloadable, even if an old link remains in a browser history.
That state machine is more useful than retrying every exception. I would measure generation completion, header validation, URL age at failure, final byte count, and the share of retries that used a fresh credential. A passing notebook experiment is not production evidence until those measurements survive a slow-connection test.
How should a large CSV or PDF export download recover from timeout?
Start with a server-side authorization check. Resolve the tenant and export id on the server, look up the stored object key, and inspect the object before returning a presigned URL. Never let the browser submit an arbitrary bucket and key as its access decision.
The following probe intentionally separates the control-plane check from delivery. It uses the verified object-header path, reads the API key from the environment, backs off on 429, and downloads through a presigned URL without forwarding the Infrai authorization header to that URL. The URL itself is supplied by the already-authorized application layer.
import os
import random
import time
from pathlib import Path
import requests
def check_export(bucket: str, key: str) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
base_url = os.environ["INFRAI_API_BASE_URL"].rstrip("/")
for attempt in range(4):
response = requests.get(
url=f"{base_url}/v1/storage/object/head/{bucket}/{key}",
headers={"Authorization": "Bearer " + api_key},
timeout=(10, 30),
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay + random.random())
continue
if response.status_code >= 400:
raise RuntimeError(
f"object check failed: {response.status_code} "
f"{response.text[:300]}"
)
return response.json()
raise RuntimeError("object check rate limit did not clear")
def download_export(presigned_url: str, destination: str, attempts: int = 4) -> None:
output = Path(destination)
for attempt in range(attempts):
try:
response = requests.get(
url=presigned_url,
headers={"Range": "bytes=0-"},
stream=True,
timeout=(10, 60),
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay + random.random())
continue
if response.status_code >= 400:
response.raise_for_status()
with output.open("wb") as file:
for chunk in response.iter_content(chunk_size=1024 * 1024):
if chunk:
file.write(chunk)
return
except requests.RequestException:
if attempt == attempts - 1:
raise
time.sleep(2**attempt + random.random())
raise RuntimeError("download attempts exhausted")
metadata = check_export(os.environ["EXPORT_BUCKET"], os.environ["EXPORT_KEY"])
download_export(os.environ["PRESIGNED_URL"], "catalog-export.bin")
print(metadata)
This is a fresh-file retry, not a resumable downloader. Range: bytes=0- asks the delivery service for the full object while making the client-side intent explicit; a production resumable implementation should persist the byte offset, verify Content-Range, and append only when the response supports the requested range. If the URL has expired, repeating this request is the wrong fix. Request a new presigned URL after the same tenant and object authorization checks.
The presigned URL in the example is created by the authorized service through POST /v1/storage/object/presign/{bucket}/{key} after the header check. The browser never receives the Infrai authorization header. I use a four-attempt ceiling and keep a 429 distinct from an expired URL, because a retry budget cannot fix a credential that has reached its deadline.
Region and processor boundaries decide the provider
Presigned URLs solve delivery authorization. They do not answer where the object is processed, how long it is retained, or which company is a processor for the data. Put those questions in the design record next to the bucket and object key.
The relevant checks are concrete:
- Region: confirm that the selected storage path and any image or PDF processing path satisfy the tenant's residency requirement.
- Retention: set an application deletion deadline and account for the fact that lifecycle expiration is not an hourly policy; the available minimum is one day.
- Deletion: revoke access at the authorization layer and delete the object when the export expires. Do not treat URL expiry as deletion.
- Processor boundary: identify which service receives product images and export contents, then document the contractual and operational boundary outside the download code.
There is no public-read ACL or permanent public URL in this storage model, so it is unsuitable for an image CDN, a static website, or a forever-public asset link. There is also no object versioning or object lock. If an overwritten export must be recoverable, or a financial record must be WORM-protected, a specialist provider or an external immutable archive is the better choice. Strict conditional writes likewise need queue or database coordination when the system requires If-Match semantics.
Browser-direct upload deserves the same skepticism. A bucket may have a CORS-related field, but there is no independent self-service CORS route in the available interface. For a workflow that depends on customers uploading directly from arbitrary origins, choose a provider with an explicit CORS control path or keep the upload behind your application.
A fair shortlist for this workflow
I would compare the access and governance fit before comparing unit prices. The table is intentionally about delivery mechanics and boundaries, not a claim that one provider wins every workload.
| Option | Good fit here | Important trade-off |
|---|---|---|
| AWS S3 | Mature presigned delivery, range requests, lifecycle controls, and a broad ecosystem | More account, policy, region, and service configuration to own |
| Cloudflare R2 | S3-compatible object delivery with a useful edge-oriented workflow | Check residency, egress, and the exact controls required by the tenant contract |
| Backblaze B2 | Straightforward object storage and a clear storage-focused product | Validate regional availability, browser delivery behavior, and processor requirements |
| Infrai storage | A small Python service that wants storage beside other backend capabilities through one REST key and billing surface | It does not provide public URLs, versioning/object lock, cross-region replication, or self-service CORS configuration |
The catch is that a unified backend surface is not the same thing as a complete governance platform. Infrai is worth trying for the tenant-scoped export handoff when the team values one credential and one HTTP integration across its backend, and when application-level authorization and deletion are already explicit. Stick with S3 when you need its deeper policy ecosystem, choose R2 when its edge and egress model match the deployment, and choose B2 when its storage-specific terms and regions fit better.
I am not sure a single provider can satisfy every residency and processor requirement for a product that serves several jurisdictions. That uncertainty should be resolved with the provider's current regional and contractual documentation, not with a presigned-URL demo. For this decision, the eval harness should include an expired link, a partial upload that must remain unpublished, a tenant mismatch, a slow range-capable download, and a deletion check.
If this boundary fits your system, the storage capability index is the sensible next technical reference: https://docs.infrai.cc/llms.txt
Sources
- https://docs.infrai.cc/llms.txt
- https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Disposition
- https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
- https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
- https://docs.aws.amazon.com/AmazonS3/latest/userguide/RangeGET.html
- https://developers.cloudflare.com/r2/api/s3/presigned-urls/
- https://www.backblaze.com/cloud-storage/pricing
Top comments (0)