Logistics Document Exports: Node.js Object Storage Download Header Design
Short answer: give each export a policy record, keep the signed documents private, and treat the download response headers as a tested contract rather than an incidental storage setting. For large logistics files, the application should authorize and describe the export while object storage carries the bytes.
That sounds tidy until a deletion deadline meets a 900 MB route archive. The file can download successfully and still be governed incorrectly: the user sees an opaque object key, a link remains usable after the deadline, or a retry starts a second expensive export. My decision axis here is reliability of the whole lifecycle, not the convenience of making one URL.
Start with the deletion policy
Create the export record before work starts and assign its deletion deadline then. Write the archive under a temporary key. Validate size, checksum, and metadata, and transition the record to ready only after the final object is complete. The finalization operation needs one owner so two workers cannot publish different content under one export ID.
Cleanup can then select records whose delete_at has passed, revoke issuance in the application, remove the object, and record the result. Alert on expired objects that remain, expired exports that are still downloadable, cleanup lag, abandoned multipart uploads, and scratch space approaching its limit.
Deletion is an observed state.
Retention controls deserve separate treatment. A deletion deadline is not proof of immutable legal retention. If the workflow needs object lock, legal hold, version history, or an audit trail that the application cannot rewrite, those are requirements for the storage design and operating procedure. Signed URLs solve time-bounded access; they do not settle every records-management question.
Three states are enough to make the first review concrete: pending, ready, expired. Deletion is the observed outcome of the fourth transition.
The incident lesson: delivery success is not lifecycle success
Consider a carrier portal that creates a ZIP of signed delivery receipts. A dispatcher asks for the records from one route, a worker assembles the archive, and the API returns a link. The visible filename should be something like route-1842-receipts.zip; the storage key should be an opaque, collision-resistant identifier; and the policy record should know when the artifact must be deleted. Those are three separate concerns.
The failure mode is easy to miss. If the cleanup process derives its target from the browser filename, a rename can make the artifact unfindable. If the signer checks only that the object exists, an expired export can still receive a fresh URL. If the worker writes directly to the final key, a client may observe an incomplete object unless publication is coordinated.
The review I would run on that route archive is deliberately chronological. At request time, the service records the tenant, route, requested filename, and deletion deadline. During generation, it writes to a temporary key and reports progress without exposing that key to the dispatcher. At finalization, it verifies that the archive is complete, records the stable key and response metadata, and changes the export to ready in one controlled transition. At download time, authorization checks the record rather than trusting the existence of the object, and signing is refused once delete_at has passed. During cleanup, the job records both the attempted deletion and the remaining object state, so an alert can distinguish a late worker from an access-control mistake. This sequence also makes capacity review concrete: the worker needs scratch space for its largest intermediate artifact, the API needs enough capacity for short authorization calls, and the object path needs to absorb concurrent readers and retries without making those readers compete with ordinary control-plane traffic.
That is the contract.
I would make export_id, storage_key, download_name, content_type, ready_at, and delete_at explicit fields. The record is the source of truth for authorization and lifecycle; the object key is an implementation detail; the filename is a user-facing response value.
One boundary matters most: never make the web handler proxy a large artifact by accident. Estimate peak concurrent downloads, maximum object size, worker scratch space, retries, and the download SLO before choosing the path. A test that passes for a 2 MB PDF says little about a multi-gigabyte route archive competing with ordinary API connections.
What should a Node.js object storage export return for a download?
The application should first verify the caller, tenant, export state, and deletion deadline. Only a ready, unexpired export should produce a short-lived signed URL for the exact storage key. The browser then requests the object directly, while the application keeps the session credential away from the object host.
Content-Disposition supplies the attachment behavior and suggested filename. Keep the storage key and the display name independent. For names containing spaces or non-ASCII characters, send a conservative ASCII filename fallback alongside correctly encoded filename*; then test with the browser and embedded clients that matter to the business. I’m not sure every client applies the same precedence rules, so that uncertainty belongs in a compatibility test, not in an assumption hidden inside a helper.
The backend must provide a way to set the response header, either through object metadata or through a signing-time response override. If the selected storage interface cannot express that contract, a proxy can set it, but the large-file throughput and connection budget move back to Node.js. That is a deliberate architecture choice, not a harmless formatting fix.
Here is the small, vendor-independent assertion I would keep in the integration suite. The signing SDK stays outside this helper because its method names and response-override options vary.
package downloadcheck
import (
"fmt"
"mime"
"net/http"
)
func RequireAttachment(h http.Header, wantType, wantName string) error {
if got := h.Get("Content-Type"); got != wantType {
return fmt.Errorf("content type = %q, want %q", got, wantType)
}
mediaType, params, err := mime.ParseMediaType(h.Get("Content-Disposition"))
if err != nil {
return fmt.Errorf("invalid content disposition: %w", err)
}
if mediaType != "attachment" || params["filename"] != wantName {
return fmt.Errorf("disposition = %q, want attachment filename %q", h.Get("Content-Disposition"), wantName)
}
return nil
}
The integration test should also verify the policy boundary: a pending export cannot be signed, an expired export is denied even if its object still exists, and the URL names the expected object. Download a representative large fixture and verify its byte count or checksum. Exercise a filename with spaces and a non-ASCII character. Short tests are cheap; false confidence is not.
How should teams compare storage paths for large-file reliability?
Use a failure matrix before selecting an implementation. A managed object-storage path may reduce the amount of data-plane infrastructure the platform team operates, while a self-hosted path may offer more control over network placement and recovery design. The relevant question is which team owns each failure and whether its SLO can be measured.
| Decision area | Test or question | Reliability consequence |
|---|---|---|
| Large-file path | Can clients download directly, and how do retries and multipart uploads behave? | Protects the API connection budget and exposes partial-transfer risks. |
| Header contract | Can the response carry the intended Content-Disposition and content type? |
Keeps recipient names stable without coupling them to object keys. |
| Deadline enforcement | Can the application deny expired exports before cleanup finishes? | Prevents a lifecycle race from becoming an access-control gap. |
| Recovery | Who restores objects, records, and cleanup state after an incident? | Makes the export SLO operationally meaningful. |
| Governance | Are immutability, audit, versioning, and deletion semantics sufficient? | Separates ordinary expiry from regulated retention. |
The catch is that direct signed delivery is unsuitable when every byte must be transformed in flight, when permanent public URLs are required, or when the backend lacks controls needed for a strict legal-retention regime. For tiny dynamic responses, adding an object lifecycle can also create more moving parts than the SLO needs. Keep the simple application response there; reserve this design for artifacts with a real lifecycle and meaningful size.
My launch checklist is narrow: prove the largest expected archive, concurrent download behavior, a missed cleanup cycle, duplicate finalization, an expired link, and internationalized filenames. Instrument generation duration, bytes written, signing latency, denied expired downloads, transfer initiation failures, and cleanup lag separately. When an alert fires, the on-call engineer should be able to identify whether the worker, authorization boundary, storage path, or lifecycle controller owns it.
The decision rule after the review
For signed logistics documents with deletion deadlines, make the policy record authoritative, publish only complete objects, and test Content-Disposition at the actual client boundary. Use direct signed delivery when large-file throughput is the limiting concern and the storage interface can express the required headers and governance controls; choose an application proxy or a different backend when those conditions do not hold.
Top comments (0)