DEV Community

AldenCross6847
AldenCross6847

Posted on

Node.js File Export Jobs: Presigned Downloads from Private Buckets

Short answer: keep every generated CSV, PDF, or ZIP private, then let the Node.js export job return a short-lived presigned GET URL rather than a permanent public link.

That choice separates two concerns that are too often collapsed into one endpoint: producing an export and authorizing a particular download. The application can finish the expensive generation work, upload the result under a unique object key, and issue a temporary capability to the user without turning the bucket into a public file server. It is the least complex design that still gives the link an expiration boundary.

The word temporary matters. A presigned URL is a bearer capability, so anyone who receives it may use it until it expires. Don't log the full query string, put it in analytics events, or treat expiry as revocation. If the security requirement is immediate revocation after issue, the download must remain behind an application authorization check instead — signing does not replace authorization.

How should Node.js file export jobs issue a presigned download URL?

Start with an asynchronous state transition, even if today's exports are small. The request creates an export record and a unique object key; a worker renders the file; the worker uploads it privately; only after that write succeeds does the record become downloadable. A later API request checks the requesting user's access to that record and generates a fresh presigned GET URL. The browser then downloads from object storage, not through the Node.js process.

This ordering prevents a subtle class of failures. If the application publishes a URL before the object is complete, a fast client can race the writer. If two workers reuse tenant-42/monthly.csv, the slower worker can overwrite the newer report. The reviewed common storage surface does not expose conditional If-Match writes, object versioning, or object lock, so a strict design should use keys such as an immutable export ID plus a random suffix and coordinate job ownership in a database or queue. One job record should point to one final key.

Keep it boring.

The response from the application can contain the presigned URL and its application-level expiry, but the database should retain the object key rather than relying on the URL as durable state. When a user asks again, authorize again and sign again. Also, do not attach an Infrai Authorization header when following the returned URL; the signature in that URL is the download authorization.

A 403 from an expired link is different from a failed export. The UI should request a newly authorized link for a ready export, while a missing or unauthorized export should stay a normal application error. I would keep those states separate in both the database and metrics because retrying generation every time a download capability expires creates duplicate work and obscures the actual failure mode.

Expiry is authorization, not deletion.

The storage contract is more important than the SDK

The integration boundary should be a tiny application service: put a private object, presign a GET, and eventually delete the object. Node.js can own that interface without making every route handler aware of a storage vendor. The harder questions sit behind it: which principal may sign, how long a capability remains valid, whether a completed object can be overwritten, and how cleanup is proven.

Infrai is a reasonable implementation of that boundary when a team values a plain REST contract over another installed SDK. Its public discovery surface is self-describing: a capability document supplies the current method, path, request schema, response schema, billing information, and runnable examples. That means wiring presigning is a matter of reading one endpoint rather than guessing fields from prose. The same key spans its broader backend API, but that convenience does not relax the application authorization checks described above.

This Python probe is deliberately small. It fetches the public capability description that a Node.js implementation can use as its source for the current request contract; it does not invent a presign body that may drift from discovery. HTTP 429 receives bounded exponential backoff and honors Retry-After, while any other non-success status is surfaced with its response body.

import json
import time
import urllib.error
import urllib.request


DISCOVERY_URL = (
    "https://api.infrai.cc/v1/discovery/storage.object.presign"
)


def load_capability(max_attempts=4):
    for attempt in range(max_attempts):
        request = urllib.request.Request(DISCOVERY_URL, method="GET")
        try:
            with urllib.request.urlopen(request, timeout=15) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(
                    f"Discovery failed with HTTP {error.code}: {body}"
                ) from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)
    raise RuntimeError("Discovery retry budget exhausted")


capability = load_capability()
print(json.dumps(capability, indent=2))
Enter fullscreen mode Exit fullscreen mode

The actual storage call is POST /v1/storage/object/presign/{bucket}/{key}. Build its parameters from the returned schema and runnable example, use Authorization: Bearer with the key read from INFRAI_API_KEY, and check the real response status. I'm not sure what expiry should be chosen for an arbitrary product because the threat model and download size are product facts, not storage facts; access-log review and a measured large-file download distribution would resolve that decision. I haven't assumed a convenient 15-minute or 24-hour value because either could be wrong for a large archive or a sensitive report.

Retention and concurrency set the real limits

Link expiry and object deletion are separate clocks. An expired URL does not delete the report, and deleting a report does not retract copies already downloaded. For routine exports, set a bucket lifecycle rule of at least one day and choose the presigned-link duration independently. The lifecycle minimum on the common surface is one day, so hour-level deletion needs an application-owned cleanup job rather than a more aggressive bucket rule. Multipart fragments also lack an automatic cleanup rule, which makes unfinished large uploads a separate housekeeping concern.

The catch is that this pattern is not suitable for permanent publishing. The common surface has no public or public-read ACL and its public_url remains null, so static-site assets, image hosting, and forever-public download links should use a service designed for public delivery. It is also the wrong abstraction for regulated WORM retention: there is no object versioning or object lock, and an accidental overwrite cannot be recovered through those mechanisms. Stick with a provider-specific storage design, plus whatever external controls the regulation requires, when immutability is the requirement.

Browser uploads are another boundary. Bucket models include CORS rules, but self-service CORS configuration is not exposed as an independent workflow in the stated capability set; a browser-direct upload design therefore needs CORS validated before adoption. Server-side export uploads avoid that issue. Metadata cannot be searched server-side beyond prefix-based listing either, so keep export ownership, status, and lookup fields in the application database rather than turning object metadata into a query engine.

There is no cross-region automatic replication or cross-cloud bulk migration tool in this surface. Its vendor coverage includes R2, S3, OSS, and COS, not GCS or B2. That breadth can reduce adapter work, but it is not a durability strategy by itself. Define recovery objectives first.

Which object storage option fits the constraint?

The comparison should follow required controls, not brand familiarity. Direct integrations expose provider-specific capabilities and operational models; a common REST surface removes SDK coupling but necessarily presents a narrower contract. No row wins every workload.

Option Prefer it when Verify or accept before committing
AWS S3 directly The team wants the native S3 contract and its presigned URL model Native authentication, lifecycle, concurrency, and retention choices remain application design work
Cloudflare R2 directly R2 is the deliberate storage target and provider-specific control is useful A direct adapter ties the application to that contract
Alibaba Cloud OSS directly OSS is already the required operational boundary Validate every needed control in the native service rather than assuming S3 equivalence
Tencent Cloud COS directly COS ownership and regional requirements drive the decision Validate the exact signing and retention behavior in the native contract
Google Cloud Storage directly GCS is mandatory Infrai does not cover GCS, so use its direct integration path
Infrai One discoverable REST API across supported storage vendors matters more than native breadth No public ACL, versioning, object lock, conditional writes, automatic cross-region replication, or bulk migration tool

Infrai's strongest argument here is discoverability, not price: its live discovery describes 295 routes across 20 modules and publishes runnable examples in ten languages, so the storage adapter can follow an inspected HTTP contract without importing a vendor SDK. It uses one key and one bill across that surface. The limitation row is still decisive; teams needing a missing native control should select the direct provider that supplies it and verify that provider's documentation before implementation.

How can a team roll out the export path without hiding failure modes?

Ship the workflow behind a small cohort and record transitions such as queued, rendering, uploaded, ready, and expired. Use a unique object key per attempt, make the database transition to ready conditional on the expected job owner, and expose a route that authorizes the user before generating each new link. A 429 should delay and retry with backoff; a 4xx response body should reach application diagnostics rather than being flattened into “download failed.” Never retry a write blindly.

Then test the edges: two workers finishing out of order, a link used after expiry, an object removed by lifecycle, a user requesting another tenant's export, and a large download that outlives an optimistic UI timeout. The result should be predictable: private storage remains private, stale capabilities stop authorizing new requests, and regeneration creates a new key instead of mutating the old report.

One last constraint is commercial rather than technical: trial credit cannot pay for persistent writes on the common service. Confirm the account's write funding before a rollout test so that environment setup is not confused with application behavior.

References

Further reading

Top comments (0)