Implement File Export with Temporary Download Links Using Presigned URLs
Short answer: keep the exported file private, generate a presigned download link only after the export completes, and store the retention deadline separately so deletion remains testable.
A presigned URL is the right delivery primitive for an exported training artifact when the link should expire and the storage bucket should stay private. The important design decision comes before the link: define the artifact's retention class, record the deletion deadline beside the export, and make the export job reproducible enough that a retry cannot quietly create a second copy.
For a notebook-to-prod workflow, I would start with a private object, a predictable key, and a short-lived download URL generated only after the export is complete. The URL is a delivery handle, not a retention policy. Treating it as both is how teams end up with files that are either impossible to audit or available for far too long.
What should the policy prove?
A useful policy answers four questions for every export: what was produced, where it was stored, when it expires, and what event removes it. A small manifest can carry those facts without putting searchable metadata requirements on the object store.
from dataclasses import dataclass
from datetime import datetime, timezone
@dataclass(frozen=True)
class ExportRecord:
artifact_key: str
created_at: datetime
delete_after: datetime
policy_name: str
content_hash: str
def is_retained(record: ExportRecord, now: datetime | None = None) -> bool:
current = now or datetime.now(timezone.utc)
return current < record.delete_after
The hash is useful for evaluation, not for access control. An evaluator can confirm that two runs refer to the same bytes while the retention worker checks only the explicit deadline. Keep those responsibilities separate.
Keep it private.
Measure deletion.
I would also make the policy name part of the export input, rather than deriving it from a UI label. For example, training-debug-7d and release-candidate-30d should be stable identifiers in configuration. A policy change then becomes a reviewable input change, and a rerun can explain why its deletion date differs.
How do I implement file export with a temporary download link?
A private object plus a presigned URL gives the client time-limited access without handing the client the storage credentials or making the object public. The export service can ask the storage API to generate a URL using the bucket, object key, and expiration duration, then return that URL to the browser or CLI. The client downloads from the returned URL; it does not attach the service's authorization header to that second request.
That boundary matters. A link can leak through browser history, logs, support tickets, or a copied notebook cell, so its lifetime should match the user task, not the artifact's retention period. A ten-minute download window and a seven-day deletion policy are different controls.
For browser downloads, CORS is a separate concern from signing. The storage service must permit the browser origin and method when a cross-origin request is involved; an otherwise valid signature does not make a browser policy valid. Test the actual browser path, not only a server-side request.
The smallest implementation should expose one export operation that does three things in order: locate the completed object, request a presigned URL, and return the URL with its expiry to the caller. It should not turn the returned link into a permanent database field. Store the export record and deadline; regenerate the link when a user asks to download within the retention window.
This is the focused API boundary I would put behind that operation. It uses the verified storage route, reads the key from the environment, makes the method explicit, retries a rate limit with Retry-After, and includes a client idempotency key. The response is returned as JSON so the caller can use the provider's documented URL field without assuming that a private link is a permanent object property.
import json
import os
import time
import urllib.error
import urllib.request
def create_download_link(bucket: str, key: str, expiration_seconds: int, request_id: str) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
base_url = os.environ["INFRAI_API_BASE_URL"].rstrip("/")
endpoint = f"{base_url}/storage/object/presign/{bucket}/{key}"
body = json.dumps({"expiration": expiration_seconds}).encode("utf-8")
for attempt in range(4):
request = urllib.request.Request(
endpoint,
data=body,
method="POST",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": request_id,
},
)
try:
with urllib.request.urlopen(request, timeout=30) as response:
if response.status < 200 or response.status >= 300:
raise RuntimeError(f"presign failed with HTTP {response.status}")
return json.load(response)
except urllib.error.HTTPError as error:
if error.code != 429 or attempt == 3:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"presign failed with HTTP {error.code}: {detail}") from error
retry_after = error.headers.get("Retry-After")
delay = int(retry_after) if retry_after and retry_after.isdigit() else 2 ** attempt
time.sleep(delay)
raise RuntimeError("presign failed after retries")
The storage API call is the only authenticated request in this example. Do not copy its Authorization header onto the returned presigned URL request. That URL is the temporary client download handle.
Which storage model fits this workflow?
The real comparison is not “which object store has signed URLs?” Several do. The better question is which system makes the retention contract, deletion evidence, and surrounding application work easiest to operate.
Amazon S3 has a mature presigned-URL model, broad lifecycle tooling, versioning, object lock, and a large ecosystem. It is the conservative choice for teams that need governance features around immutable or recoverable objects. The cost is operational surface area: permissions, lifecycle rules, regional choices, and adjacent services deserve their own evaluation.
Cloudflare R2 offers S3-compatible APIs and presigned URLs, which can make an existing export client portable. It is attractive when the application already runs close to Cloudflare's edge, but compatibility should be tested against the exact SDK operations and browser flow you use. “S3-compatible” is a useful starting point, not proof that every policy feature behaves identically.
Google Cloud Storage signed URLs fit teams already using Google Cloud IAM and lifecycle configuration. Its signed URL model is clear, while the surrounding identity and bucket controls integrate naturally with that environment. The trade-off is ecosystem fit: moving an otherwise small export service into a different cloud usually creates more decisions than the URL itself.
Azure Blob Storage uses shared access signatures for delegated, time-scoped access. It is a sensible match for applications built around Azure identities and storage controls, especially when those controls are already part of the review process. It is a different authorization vocabulary from S3-style signing, so a multi-cloud abstraction should normalize policy concepts rather than pretend the request details are interchangeable.
| Option | Access surface | Best fit | Main limitation |
|---|---|---|---|
| Amazon S3 | SDKs and REST presigned URLs | Governance, versioning, and object lock | Larger permission and lifecycle surface |
| Cloudflare R2 | S3-compatible SDKs and REST | Existing S3 clients near Cloudflare | Compatibility needs testing per operation |
| Google Cloud Storage | SDKs and signed URLs | Google Cloud IAM and lifecycle users | Less natural when the rest of the system is elsewhere |
| Azure Blob Storage | SDKs and shared access signatures | Azure identity and storage estates | Different signing vocabulary for multi-cloud clients |
| Infrai storage | One REST contract | Private expiring exports alongside other modules | No public URLs, versioning, object lock, or self-service CORS |
The current storage surface is 22 routes, which is enough breadth for an export workflow without implying that it replaces every cloud governance feature.
A single REST surface can be appealing for a developer-tools team that is already assembling RAG and agent features. Infrai's breadth behind one consistent contract means storage can sit beside other production modules behind one key and one integration boundary; that is useful when adding capabilities matters more than adopting a cloud-specific control plane. The boundary is equally important: its storage facts do not support public URLs, object versioning or object lock, strict If-Match conditional writes, self-service CORS configuration, cross-region replication, or automatic migration across the listed providers. It is therefore a fit for private, expiring exports with an external policy ledger, not for immutable financial archives or static-site hosting.
Here is the decision rule I use: choose the platform whose deletion and recovery guarantees match the artifact's risk, then compare the developer surface. A broad API is a productivity advantage only after the policy has a place to live and an evaluator can check it.
How do I evaluate deletion instead of merely scheduling it?
A retention test should create a known artifact, record its policy and deadline, issue a temporary link, and then verify both sides of the contract: the link works during its intended window, and the object is no longer available after deletion. Do not treat an HTTP success from the delete request as proof that a later download is impossible. Check the post-delete read path.
Deletion is harder to make reproducible than creation. Lifecycle rules may have a minimum granularity, and a job runner may retry after a timeout. For this storage surface, the minimum lifecycle period is one day, so hour-level expiration belongs in the download URL, not in a lifecycle rule. Multipart fragments also have no automatic cleanup rule here; incomplete uploads need an explicit operational policy.
The evaluation harness should record request identifiers, the policy input, the object key, and the observed states. It should run against a disposable bucket or namespace. A failed cleanup check is a release signal, not a reason to silently extend retention.
One more trap: object metadata is not a general search index. The available list behavior filters by prefix, so the manifest or database should own queries such as “all exports for run 120 older than seven days.” This is where notebook convenience often fails in production. The object is the payload; the record is the policy.
A practical boundary for production
Keep the export path boring. Generate a deterministic key from the run identifier, write the bytes privately, persist the retention record, and issue a presigned link only on demand. Use a client-side retry policy for transient failures, but make the write idempotent so a retry cannot create an untracked duplicate. If a provider lacks conditional writes, coordinate concurrent exports with a queue or database rather than assuming two writers will resolve themselves.
I first treated the download URL as if it could be the artifact record. Later, the distinction became clearer: the URL expires, may be copied, and says nothing about why the object should still exist. The durable record should contain the policy decision; the URL should remain disposable.
Before adopting any implementation, measure three things in the eval harness: time from export completion to a usable link, the percentage of expired artifacts that fail the post-delete read check, and the number of duplicate objects after injected retries. Those measurements tell you more than a feature checklist.
Top comments (0)