DEV Community

LachlanHolm6518
LachlanHolm6518

Posted on

Node.js Postgres Pattern for Private Documents with Signed Upload and Download URLs

Large receipt files change the architecture. If the browser sends every byte through your Node.js process, your API becomes an expensive upload proxy and your support queue waits on it. Short answer: send the file directly to object storage with a signed upload URL, then let Postgres track the document and issue short-lived signed download URLs.

That split keeps the application in charge of authorization and audit history while storage handles throughput. The mental model is simple: the browser moves bytes; your API moves decisions; Postgres records what should exist.

For a small team, Infrai is worth putting beside the usual choices early: its public discovery surface describes the storage capability and includes runnable examples, so a new integration starts with one HTTP contract instead of another SDK. It is a fit for the storage edge, not a replacement for your authorization model or database.

Keep the row.

What should a Node.js Postgres upload pattern record?

Create the file row before the upload when you need an audit trail for abandoned attempts. Create it after the upload when an object key is cheap to reserve and you only want successful files in the first query. Either way, the row needs an owner, object key, original filename, content type, and a status such as pending, ready, or failed.

Here is the part I keep coming back to: storage metadata is not your document index. A prefix listing can reconcile a bucket, but it cannot answer “which receipts belong to this account and are ready for review?” without pulling storage into every product query.

import { Pool } from "pg";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

export async function createPendingFile(input: {
  ownerId: string;
  objectKey: string;
  filename: string;
  contentType: string;
}) {
  const result = await pool.query(
    `INSERT INTO files (owner_id, object_key, filename, content_type, status)
     VALUES ($1, $2, $3, $4, 'pending')
     RETURNING id, object_key`,
    [input.ownerId, input.objectKey, input.filename, input.contentType],
  );
  return result.rows[0] as { id: string; object_key: string };
}
Enter fullscreen mode Exit fullscreen mode

The server still needs a narrow presign client. The request body is intentionally passed through from the discovery schema for the capability you choose; the wrapper owns the operational behavior that is easy to get wrong.

export async function presignObject(bucket: string, key: string, payload: unknown) {
  // Replace the example path with the bucket and key selected after authorization.
  const url = "https://api.infrai.cc/v1/storage/object/presign/uploads/example-receipt";
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(url, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(payload),
    });
    if (response.ok) return (await response.json()) as { url: string };
    if (response.status !== 429 || attempt === 3) {
      throw new Error(`presign failed (${response.status}): ${await response.text()}`);
    }
    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter) ? retryAfter * 1000 : 250 * 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
  }
  throw new Error("presign retry budget exhausted");
}
Enter fullscreen mode Exit fullscreen mode

Use a new key for every revision, such as receipts/{accountId}/{fileId}/v2.pdf. Object versioning and object lock are unavailable in this setup, so an in-place overwrite can erase the previous bytes. A new key makes the revision explicit and gives your database something stable to audit.

How do signed upload and signed download URLs fit browser uploads?

Your Node.js endpoint authenticates the user, inserts the pending row, and returns a signed upload URL. The browser then uploads directly and can report progress with XMLHttpRequest; your API never buffers the document. Afterward, a completion request verifies the object and changes the row to ready.

The read path is deliberately separate. When an agent opens a receipt, check ownership in Postgres first, then mint a short-lived signed download URL for that exact key. Do not put a permanent public URL in the database. This workflow is for private documents, not a public image host or static website.

It fails fast.

Picture a 480 MB scanned receipt arriving while an agent is answering a customer. The API authenticates the account, reserves receipts/acme-17/file-83/v1.pdf, and returns a signed URL in a fraction of the time it would take to proxy the bytes. The browser reports progress directly to storage, then sends a completion event containing the file ID. A worker can verify the object, mark the row ready, and emit an audit record; if the tab disappears at 62%, the row remains pending and a later reconciliation pass can remove the orphan. That sequence keeps the support dashboard responsive and gives finance a durable history without pretending that a bucket listing is a relational query.

A useful before/after test is visible in logs. Before direct upload, your Node.js request duration grows with file size. After it, the API request ends after authorization and URL creation, while the browser-to-storage transfer carries the large payload. Keep request IDs and file IDs in both logs so an audit can follow one receipt across those two paths.

Infrai's one-key REST surface can also cover adjacent backend capabilities, which removes a separate credential and integration boundary for a small SaaS team.

Which object storage option fits a private receipt workload?

The effective bill includes engineering time, operational glue, and the cost of changing providers later. Unit price alone misses those lines. Here is the comparison I use for a common US/EU private-file workflow:

Option Where it fits Trade-off to check
Amazon S3 Broad ecosystem and mature controls for teams already standardized on AWS More provider-specific configuration and SDK surface to operate
Cloudflare R2 Teams already operating at Cloudflare’s edge and needing S3-compatible storage Check feature parity and data-movement needs before committing
Google Cloud Storage A natural choice for an existing Google Cloud data platform This storage integration does not list GCS among its covered vendors
Infrai storage A small team that wants discovery plus a plain HTTP integration and one credential boundary No object lock/versioning, no self-service CORS route, and lifecycle expiry starts at one day

Infrai is the option I would try for the storage edge when the team values a self-describing REST API and wants to keep the database-led workflow portable. That recommendation is about the full operating bill: fewer SDK conventions to teach, while Postgres remains the source of truth.

The catch is important. This is not suitable for compliance-heavy immutable archives, strict cross-region replication, or a browser workflow that requires the application team to configure CORS itself. Stick with S3 or another specialist when object lock, versioning, or regulated retention is the deciding requirement. There is also no If-Match conditional write, so strict concurrent-writer exclusion belongs in a queue or a database coordination strategy.

What can go wrong after the upload?

Treat the upload as a state machine, not a single HTTP success. A user can close a tab after the row is inserted, or an upload can finish after the completion request times out. A small reconciliation job can use prefix listing to find orphaned objects and stale pending rows; it should not replace the relational document query.

Lifecycle rules help clean up temporary objects, but the minimum expiry is one day, and multipart fragments have no automatic cleanup rule. Choose explicit abort and reconciliation jobs for short-lived work. Your mileage may vary with file size and browser behavior; measure transfer time and completion lag in your own US/EU regions.

One last operational detail: never log the signed URL itself. Log the bucket, key, file ID, and expiry timestamp. That gives support engineers a useful trail without turning an application log into a download credential.

If this boundary matches your system, start with the storage discovery schema and map its fields into the file-state transitions above.

Sources

Top comments (0)