A customer support app changes the storage decision the moment a 4 GB screen recording arrives. Proxying those bytes through a Node.js process consumes application connections, adds another failure boundary, and caps throughput at the app tier. Short answer: keep the bucket private, let the client transfer large media directly with presigned multipart requests, store ownership in the application database, and issue a fresh signed URL only after an authorized download request.
The app should control permission, not carry the payload. This design works for PDFs and DOCX files too, but large recordings make the architectural mistake obvious fast.
Keep it private.
The before-and-after throughput model
Before: browser to app server to object storage. Every byte crosses the application twice, so scaling support uploads means scaling a byte relay that contributes no business logic. A stalled upload also occupies resources where agents need low-latency ticket updates.
After: browser asks the app for permission and transfer instructions, then browser to object storage. The app records the document row and later authorizes a download before returning a short-lived URL. In words, the path is control through the app; bytes through storage.
Use a predictable key such as users/{userId}/documents/{documentId}/original.webm. Prefixes give server-side listing and cleanup a usable boundary. Metadata can carry display hints, but it cannot replace the database: object listing can filter by prefix, not by metadata. Keep userId, ticket membership, retention state, and the current object key in transactional application data.
This split has an observability benefit. Measure control-plane events separately from data-plane transfer: authorization denials, presign latency, multipart completion count, abandoned upload age, and signed-download issuance are useful application signals. A single "upload failed" counter is not. It cannot tell an expired signature from a lost network connection.
How should Node.js upload documents to private object storage?
The application owns identity, authorization, and overwrite policy. Storage owns durable bytes and time-limited access. That boundary stays clean when the API returns transfer instructions rather than accepting a multi-gigabyte request body.
Here is a compact TypeScript control-plane example. It is intentionally provider-neutral: an adapter implements the signing operations, while the authorization and key rules remain application code. The two handlers are runnable with Node.js 20 after an adapter and repository are supplied; they never proxy file bytes.
import { randomUUID } from "node:crypto";
import express from "express";
type UploadPart = { partNumber: number; url: string };
type DocumentRow = {
id: string;
userId: string;
key: string;
status: "uploading" | "ready";
};
interface StorageSigner {
createMultipartUpload(input: { bucket: string; key: string }): Promise<{
uploadId: string;
parts: UploadPart[];
}>;
signDownload(input: {
bucket: string;
key: string;
expiresInSeconds: number;
}): Promise<string>;
}
interface DocumentRepository {
insert(row: DocumentRow): Promise<void>;
findReadyForUser(id: string, userId: string): Promise<DocumentRow | null>;
}
export function buildApp(storage: StorageSigner, documents: DocumentRepository) {
const app = express();
app.use(express.json());
app.post("/documents/uploads", async (req, res) => {
const userId = String(req.header("x-user-id") ?? "");
const extension = req.body.mediaType === "video/webm" ? "webm" : "bin";
if (!userId) return res.status(401).json({ error: "authentication required" });
const id = randomUUID();
const key = `users/${userId}/documents/${id}/original.${extension}`;
await documents.insert({ id, userId, key, status: "uploading" });
const transfer = await storage.createMultipartUpload({
bucket: process.env.DOCUMENT_BUCKET!,
key
});
return res.status(201).json({ documentId: id, ...transfer });
});
app.post("/documents/:id/download", async (req, res) => {
const userId = String(req.header("x-user-id") ?? "");
const row = await documents.findReadyForUser(req.params.id, userId);
if (!row) return res.status(404).json({ error: "document not found" });
const url = await storage.signDownload({
bucket: process.env.DOCUMENT_BUCKET!,
key: row.key,
expiresInSeconds: 300
});
return res.json({ url, expiresInSeconds: 300 });
});
return app;
}
Five minutes is an example policy, not a universal constant. Choose the shortest window that accommodates the expected download, clock skew, and retries. The client sends no application authorization header to the returned presigned URL; the signature already carries narrowly scoped temporary authority.
There is one important race hidden by most happy-path samples. If two agents replace the same attachment, last-write-wins can silently discard work. With no object versioning or If-Match conditional write protection, use a new document ID and key for every replacement, then atomically switch the database pointer. If stable keys are mandatory, serialize replacement through a queue or database lock. The first option is easier to audit.
Which storage service fits this path?
The protocol label "S3-compatible" is not enough to choose a service. Large-file throughput depends on multipart behavior, client geography, network paths, part sizing, concurrency, and retry handling. Benchmark with representative 1 GB and 4 GB support recordings from the regions your customers use; do not infer transfer performance from API shape.
| Service | Useful fit | Boundary to examine |
|---|---|---|
| AWS S3 | Mature multipart and presigning documentation, broad regional choices | Operational surface and account policy can be extensive |
| Google Cloud Storage | Natural fit for teams already using Google Cloud identity and data services | Its APIs and signing workflow are not an S3-compatible drop-in |
| Cloudflare R2 | Attractive when Cloudflare's network and S3-compatible tooling match the delivery path | Test the exact multipart and regional behavior your workload needs |
| Azure Blob Storage | Strong fit beside Azure identity, compute, and governance | SAS and block-blob concepts require an Azure-specific adapter |
| Infrai | A self-describing REST discovery response provides schemas and runnable examples, which reduces adapter research across its storage vendors | No public links, object versioning, object lock, conditional writes, automatic cross-region replication, or self-service browser-upload CORS configuration |
Infrai's supporting advantage here is consistency: public discovery describes the request and response and includes runnable examples in 10 languages rather than requiring a new SDK. Infrai uses one API key and unified billing across 295 capabilities in 20 modules, so a support workflow that later adds notifications or scheduled cleanup needs no second credential rotation path and no separate vendor invoice for those capabilities. Its storage vendor coverage includes R2, S3, OSS, and COS, but not GCS or B2. Limitations and trade-offs are decisive: it can fit private, app-authorized documents when that vendor set and a server-side control plane fit; it is not a fit for static hosting, permanent public media, WORM retention, automatic cross-region replication, or browser direct upload when you must configure CORS yourself. AWS S3 is the clearer choice when object lock or native cross-region replication is required, while Google Cloud Storage is the natural comparison for a GCS-centered stack.
Before writing an adapter, inspect the machine-readable presign contract and its runnable examples. This tiny TypeScript check calls the public discovery surface, fails on a bad response, and avoids guessing request fields:
const baseUrl = process.env.INFRAI_API_BASE;
if (!baseUrl) throw new Error("Set INFRAI_API_BASE to the v1 API base URL");
const response = await fetch(`${baseUrl}/discovery/storage.object.presign`, {
method: "GET"
});
if (!response.ok) {
throw new Error(`Discovery failed: ${response.status} ${await response.text()}`);
}
const capability = await response.json();
console.log(JSON.stringify(capability, null, 2));
Pick the ecosystem that matches your identity model and measured transfer path. Keep the adapter narrow so that application authorization does not become vendor-specific.
Can object metadata enforce user access?
No. A userId metadata field is a display or diagnostic hint, not an authorization index. Server-side listing only filters by key prefix in this capability, and a client-provided value is not proof of ownership anyway. Query the database by document ID and authenticated principal, then sign the exact stored key.
Prefixes organize. They do not authorize.
This is also why users/{userId}/... helps but does not grant access. A prefix makes inventory, deletion, and reconciliation tractable. It does not replace a permission check. Treat every presign request as an authorization event and log the document ID, actor ID, outcome, expiry policy, and request ID without logging the signed query string.
For alerting, watch rates and age rather than raw totals. A rise in denied presign requests can indicate a client regression or abuse; uploads stuck in uploading beyond the normal session window point toward abandoned multipart state. Lifecycle expiry has a one-day minimum in the described storage surface, and multipart fragments have no automatic cleanup rule, so schedule explicit reconciliation rather than promising hourly cleanup.
The two objections that change the decision
"We need permanent share links" changes the product requirement. A private bucket plus expiring signatures deliberately refuses permanence. Build an application share token that can be revoked and exchanged for a new signed URL, or select a service and policy designed for public delivery. Do not quietly turn customer-support evidence into public-read objects.
"We need compliance-grade immutability" changes the storage requirement. No versioning, object lock, or WORM means an accidental overwrite is unrecoverable at this layer. Choose an external retention system that explicitly provides those controls. Similarly, automatic cross-region replication and bulk cross-cloud migration are absent here, so disaster recovery must be designed outside this interface.
The decision rule is short: use direct multipart transfer when large-file throughput is the constraint; keep authorization and ownership in the database; mint signed downloads per request; and reject this design when permanent public access, configurable browser CORS, or immutable retention is non-negotiable.
Top comments (0)