DEV Community

knoxblackwood2375
knoxblackwood2375

Posted on

How to Export Signed Files with Download Links from Storage Buckets in Node.js (Retention)

#node #storage #security #saas

The decision rule is straightforward: generate an export into private object storage, issue a short-lived signed GET URL only after authorization, and make the deletion deadline part of the export record rather than a promise hidden in a worker. For a B2B SaaS product retaining signed documents, this keeps large-file throughput away from the application process while preserving a clear audit trail.

The link is not the export.

The examples below use a small Go reference implementation for the storage boundary. The same boundary is what a Node.js service should call: CreateExport, SignDownload, and DeleteExpired. Keeping those operations narrow matters more than choosing a particular bucket API. The tradeoff is real: direct-to-storage transfer reduces pressure on the application tier, but it adds lifecycle state, reconciliation work, and another policy boundary to operate. For a small file with no retention requirement, a normal application response can be easier to reason about.

How can Node.js export files with signed download links?

Start with an export record. It needs a tenant identifier, object key, byte count when known, creation time, expiration time, deletion deadline, and state. The object key should be generated by the service, not accepted verbatim from a browser. A useful key has no customer-controlled path segments that can become confusing during incident response.

Authorization happens before signing. A signed URL is a bearer credential, so anyone who obtains it can use it until its signature expires or the object is removed. The endpoint should check the requesting principal against the export record, then return a URL with a lifetime shorter than the business retention window. Those are different clocks.

A deletion deadline is also not the same as URL expiry. A link can stop working in ten minutes while the document remains retained until the contractual deadline. I keep both timestamps because conflating them produces either accidental early deletion or an archive that outlives its policy.

Step 1: Define a storage-neutral export contract

The application owns policy; the storage adapter owns bytes and signatures. That split makes the throughput decision measurable. If a 2 GB export is streamed through the Node.js process, memory may remain bounded, but application connections, egress, retries, and deployment interruptions still become part of the transfer path. A direct upload or multipart writer to the bucket moves the large-file path out of the request handler.

Here is a runnable contract sketch. It deliberately leaves signing primitives behind an interface because signature formats and canonical-request rules belong to the selected storage protocol.

package main

import (
    "context"
    "fmt"
    "time"
)

type Export struct {
    TenantID     string
    ObjectKey    string
    CreatedAt    time.Time
    URLExpiresAt time.Time
    DeleteAfter  time.Time
}

type Storage interface {
    SignGet(ctx context.Context, key string, expiresAt time.Time) (string, error)
    Delete(ctx context.Context, key string) error
}

func issueDownload(ctx context.Context, store Storage, export Export, tenantID string, now time.Time) (string, error) {
    if export.TenantID != tenantID {
        return "", fmt.Errorf("export is outside tenant scope")
    }
    if now.After(export.URLExpiresAt) {
        return "", fmt.Errorf("download link expired")
    }
    return store.SignGet(ctx, export.ObjectKey, export.URLExpiresAt)
}

func main() {
    fmt.Println("storage boundary ready")
}
Enter fullscreen mode Exit fullscreen mode

The decision check is intentionally boring. Tenant scope, export state, and URL expiry are policy checks; the adapter should not silently invent them. In production, persist the export before returning a link and record the signing event without recording the bearer token itself.

Step 2: Stream the large file without making Node.js the bottleneck

The producer should write to a multipart-capable upload stream, finalize it, and only then mark the export available. For a Node.js implementation, use backpressure and do not collect chunks in an array. The Go example below shows the state transition and a bounded copy; it is also a useful executable model for tests around the adapter.

package main

import (
    "context"
    "fmt"
    "io"
)

type Uploader interface {
    Open(ctx context.Context, key string) (io.WriteCloser, error)
}

func writeExport(ctx context.Context, up Uploader, key string, src io.Reader) error {
    dst, err := up.Open(ctx, key)
    if err != nil {
        return fmt.Errorf("open upload: %w", err)
    }
    defer dst.Close()

    if _, err := io.Copy(dst, src); err != nil {
        return fmt.Errorf("stream export: %w", err)
    }
    if err := dst.Close(); err != nil {
        return fmt.Errorf("finalize upload: %w", err)
    }
    return nil
}
Enter fullscreen mode Exit fullscreen mode

There is a small trap here: a successful copy is not necessarily a durable, readable export. The adapter must treat finalization as a separate operation, and the record should move from writing to ready only after that operation succeeds. If a worker dies between those events, a reconciler can inspect old writing records and either resume or remove them according to policy.

For capacity planning, measure bytes per minute, concurrent uploads, average and p99 completion time, retry volume, and the temporary storage consumed by unfinished multipart uploads. A 2 GB export that takes 90 seconds in a quiet test says little about a queue carrying 40 concurrent exports, so capacity review should exercise the queue, the worker pool, the network path, and cleanup together. Set an SLO for link issuance separately from the export-completion SLO. A user waiting for a URL and a user waiting for a 2 GB report are observing different services.

Keep it measurable.

Which failure signals change the design?

A rising p99 upload duration with flat application CPU points toward the storage path or network, not a need to add web workers. A growing count of writing exports points toward worker interruption, finalization, or reconciliation. Repeated authorization failures for otherwise valid records deserve a tenant-scope investigation; they should not be “fixed” by making links longer-lived.

The operational table I use during review is deliberately plain:

Concern Application responsibility Storage boundary responsibility
Tenant isolation Authorize export ownership Enforce object-key access policy
Large-file throughput Schedule and retry work Stream, multipart, finalize
Link exposure Set short URL lifetime Validate signature and method
Retention Persist deletion deadline Remove the object when instructed
Recovery Reconcile state transitions Report object and multipart status

Test the ugly paths. Expire the URL before fetching it. Kill the worker after the last byte but before finalization. Retry deletion after a timeout. Send a request from an untrusted tenant. Verify that a browser download from a separate origin has the intended CORS response headers, because CORS controls browser access and does not replace authorization.

Step 3: Verify deletion and keep rollback possible

Deletion is a workflow, not a cron expression. A periodic job can select records whose delete_after is in the past, claim them, delete the object, and mark the record deleted. Make the operation idempotent: an already absent object should not make the record permanently stuck. Keep a small reconciliation report with counts for ready, writing, expired-link, deletion-pending, and deletion-failed records.

Before rollout, run a canary tenant through a complete export, download, expiry, and deletion cycle. Compare the stored byte count with the source count, verify that the returned content disposition is suitable for the filename, and inspect audit events for tenant, export ID, actor, and timestamps. Never put the signed URL in normal request logs.

Rollback should disable new export creation or route it to the previous adapter while leaving already-ready objects governed by their existing deadlines. Do not bulk-delete during rollback. That separation gives the platform team room to correct signing or scheduling logic without changing retention outcomes.

The important boundary is therefore not the URL helper. It is the record that ties authorization, throughput, expiry, and deletion together. Once that record is durable, Node.js can remain the control plane while the storage service carries the large file.

References

Top comments (0)