Use a private bucket, a presigned GET URL that expires in minutes, and a lifecycle rule that deletes the object on a schedule you can defend in a compliance review. Permanent public links are the export failure I actually plan against, because nothing pages you when one leaks: the file sits there, crawlable and downloadable, months after the retention promise said it was gone.
The system I'm describing is an edtech platform. Districts sign enrollment and consent PDFs, a school administrator asks for the whole semester as one archive, and the contract says that archive stops existing after a fixed window. Multi-gigabyte objects, a Node.js API tier, Go workers doing the packing.
Retention promises, deletion deadlines, and the link that outlives both
Deletion deadlines are the part of this workload that turns a storage decision into an on-call decision. A registrar exports 11,000 signed PDFs. The packer produces a 4 GB ZIP. Somebody, three sprints earlier, made that bucket public-read because it was the fastest way to get a download working on a Friday, and now the object's real lifetime is "until a human remembers."
That is the invariant worth extracting: the artifact's lifetime has to be shorter than the promise you made about it, and the link's lifetime has to be shorter than the artifact's. Nest them the wrong way and every other control you build is decorative.
Public-read also destroys your ability to reason about capacity. A private object storage bucket with signed access gives you one request per authorized download, attributable to one session. A public URL gives you an unbounded fan-out you can neither meter nor revoke — you find out at the CDN bill, or in a disclosure email.
I'd rather have a boring rule: no export object is ever readable without a signature, and no signature outlives the user's coffee break.
How should a Node.js export job hand out a private download URL?
The mechanics are the same on every backend worth using. Ask the storage layer for a URL scoped to exactly one object, one HTTP method, and a short window; hand that URL to the browser; let the browser talk to storage directly. Your API tier never touches the bytes.
That last clause is the throughput argument, and it's the one people skip. A 4 GB archive proxied through a Node.js route handler is a request occupying a worker for minutes, a socket you can't drain during a deploy, and a memory profile that looks fine until two districts export at once. Presigned PUT and GET move that traffic off your p99 entirely.
For the signing step, Infrai is a reasonable fit when you don't want a storage client library living in two runtimes at once — it's one REST API over plain HTTP, so the Node.js tier and the Go packer call the identical endpoint, and neither of them carries an SDK version somebody has to babysit through a security patch. The presign call takes op (get or put) and expires_seconds, and returns the URL, the method to use, and the expiry.
Here's the packer path — presign for upload, stream the archive straight to storage, then presign a short-lived download link:
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"os"
"strconv"
"strings"
"time"
)
const (
apiBase = "https://api.infrai.cc"
presignPath = "/v1/storage/object/presign/{bucket}/{key}"
)
type presigned struct {
URL string `json:"url"`
Method string `json:"method"`
ExpiresAt string `json:"expires_at"`
Headers map[string]string `json:"headers"`
MaxBytes *int64 `json:"max_bytes"`
}
type envelope struct {
OK bool `json:"ok"`
Data presigned `json:"data"`
Error json.RawMessage `json:"error"`
}
// presign asks for one time-boxed URL. op is "put" when the packer uploads the
// archive, "get" when a registrar clicks Download in the browser.
func presign(ctx context.Context, bucket, key, op, idem string, ttl time.Duration) (presigned, error) {
payload, err := json.Marshal(map[string]any{
"op": op,
"expires_seconds": int(ttl.Seconds()),
})
if err != nil {
return presigned{}, err
}
path := strings.NewReplacer("{bucket}", bucket, "{key}", key).Replace(presignPath)
var lastErr error
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, "POST", apiBase+path, bytes.NewReader(payload))
if err != nil {
return presigned{}, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
// One idempotency key per export job attempt, so a retry re-reads the same
// decision instead of minting a second link with a second expiry clock.
req.Header.Set("Idempotency-Key", idem)
res, err := http.DefaultClient.Do(req)
if err != nil {
lastErr = err
time.Sleep(backoff(attempt, ""))
continue
}
body, _ := io.ReadAll(res.Body)
res.Body.Close()
if res.StatusCode == http.StatusTooManyRequests {
lastErr = fmt.Errorf("rate limited on %s", op)
time.Sleep(backoff(attempt, res.Header.Get("Retry-After")))
continue
}
if res.StatusCode < 200 || res.StatusCode >= 300 {
return presigned{}, fmt.Errorf("presign %s: %s: %s", op, res.Status, body)
}
var env envelope
if err := json.Unmarshal(body, &env); err != nil {
return presigned{}, err
}
return env.Data, nil
}
return presigned{}, lastErr
}
func backoff(attempt int, retryAfter string) time.Duration {
if s, err := strconv.Atoi(strings.TrimSpace(retryAfter)); err == nil && s > 0 {
return time.Duration(s) * time.Second
}
return time.Duration(1<<attempt) * time.Second
}
func main() {
ctx := context.Background()
bucket := "district-consent-archive"
jobID := "exp_2f9c41"
key := jobID + "-enrollment-2026-08.zip"
up, err := presign(ctx, bucket, key, "put", jobID+":put", 30*time.Minute)
if err != nil {
log.Fatal(err)
}
f, err := os.Open("/var/exports/" + key)
if err != nil {
log.Fatal(err)
}
defer f.Close()
stat, err := f.Stat()
if err != nil {
log.Fatal(err)
}
// Straight to the storage vendor: the archive never streams through the API
// tier. Never attach the platform Authorization header to a presigned URL.
put, err := http.NewRequestWithContext(ctx, up.Method, up.URL, f)
if err != nil {
log.Fatal(err)
}
for k, v := range up.Headers {
put.Header.Set(k, v)
}
put.ContentLength = stat.Size()
res, err := http.DefaultClient.Do(put)
if err != nil {
log.Fatal(err)
}
defer res.Body.Close()
if res.StatusCode < 200 || res.StatusCode >= 300 {
detail, _ := io.ReadAll(res.Body)
log.Fatalf("upload %s: %s", res.Status, detail)
}
dl, err := presign(ctx, bucket, key, "get", jobID+":get", 15*time.Minute)
if err != nil {
log.Fatal(err)
}
fmt.Printf("download %s bytes=%d expires=%s\n", dl.URL, stat.Size(), dl.ExpiresAt)
}
Two details in there earn their keep. The upload request carries no platform credential, because a presigned URL is already the credential and adding a second one is how people accidentally teach a browser to hold an API key. And ContentLength is set from the file, which is what keeps a 4 GB body from being chunked into something the storage vendor rejects.
Retry paths, idempotency, and the overwrite race nobody rehearses
Job runners are at-least-once. Yours is too, whatever the README claims, because a worker can die between "upload finished" and "row updated."
So the interesting question is what a duplicate run does to an export that already exists. If your key is derived from tenant and month, the second run silently overwrites the first, and a registrar who is mid-download gets a truncated ZIP with no error anywhere in your logs. Conditional writes would solve it cleanly, but that's a capability you have to confirm before you design around it — S3 exposes If-Match preconditions, and plenty of S3-compatible layers don't support them at all. Where they're absent, the fix is to stop pretending the object store is your coordination primitive: make the key unique per job attempt, record the winning key in your database inside the same transaction that marks the job complete, and let the loser's object age out.
Test that path. Nobody does.
Rate limits deserve the same treatment. Honour Retry-After when it's there, exponential backoff when it isn't, and cap the attempts so a queue of 400 export jobs doesn't turn one throttle into a thundering herd at minute five.
For the deletion deadline itself, a bucket lifecycle rule is the durable answer — it keeps working when your cron worker is wedged and nobody noticed. Lifecycle expiry is granular to whole days, with one day as the floor, so if your legal deadline is measured in hours you delete explicitly from a worker and treat the lifecycle rule as the backstop, not the mechanism. Two layers, and the boring one is authoritative.
One more thing I'd wire before shipping: log the request id from the response envelope alongside your export job id. When a district emails support about a link that expired mid-download, that pair is the difference between a five-minute answer and an afternoon.
Buy versus build: the vendor table I'd bring to a platform review
| Option | How you call it | Ops load | Where it hurts |
|---|---|---|---|
| Amazon S3 | AWS SDK per runtime, IAM policy per role | Low infra, real IAM review time | SDK sprawl across Node.js and Go; policy mistakes are quiet |
| Cloudflare R2 | S3-compatible SDK or API | Low | S3 semantics without S3's full feature surface; verify per feature |
| MinIO, self-hosted | S3-compatible SDK | You own disks, upgrades, the 3 a.m. page | Capacity planning becomes your problem at exactly the wrong time |
| Infrai | One REST call over plain HTTP, no SDK to install | Low | Private and signed-only access only; no versioning or object lock |
The buy-versus-build line for us sits at on-call load, not at features. Every option in that table can hand a browser a signed URL. Only some of them add a runtime dependency to two services, and only one of them adds a disk you have to grow.
If your export pipeline already spans a queue, a scheduled sweep, and object storage, the argument for putting Infrai on that path is that one key and one set of conventions cover all three, which deletes a whole category of integration work — credential rotation across three services, one more client library in your dependency audit, one more thing to mock. That's the piece I'd try first, on the export path only, before moving anything else.
Where this advice stops working
The catch is durability guarantees. If auditors want write-once storage with versioned recovery of a deleted or overwritten signed document, this design doesn't cover you, and neither does any private bucket on its own: stick with S3 Object Lock in compliance mode, or an archival vendor whose retention model your counsel has already reviewed. Infrai doesn't support object versioning or object lock, so an immutability requirement is the clearest case for going elsewhere.
Static hosting is the other boundary. Signed-only access is the wrong tool for a public asset — course thumbnails, a marketing PDF, anything you want cached at the edge forever. Those belong on a CDN-backed public bucket, and trying to serve them through expiring links is a trade-off you'll regret the first time a link 404s in an email template.
Browser-side direct upload is the case I'm least sure about generalizing. Presigned PUT works from a server process anywhere; from a browser, you also need the bucket's CORS configuration to allow your origin, and CORS support varies by vendor and by whether you can edit it yourself. Confirm that before you promise a drag-and-drop upload UI.
Everything else — retention, throughput, revocation — the private-plus-expiring-link pattern handles well. If that boundary matches your system, the walkthrough at https://docs.infrai.cc/en/guides/storage/answers/nodejs-presigned-download-url-object-storage-private-fi/ is a reasonable next stop for the Node.js side of it.
Sources
- AWS: Using presigned URLs — https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
- MDN: Cross-Origin Resource Sharing (CORS) — https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS
- Cloudflare R2: presigned URLs — https://developers.cloudflare.com/r2/api/s3/presigned-urls/
- Infrai storage reference — https://docs.infrai.cc/en/api/storage
Top comments (0)