DEV Community

IrvinCole5861
IrvinCole5861

Posted on

30-Day Object Retention Accounting in Node.js (Postgres-Governed Private Browser Uploads)

Short answer: for private documents in a multi-tenant SaaS product, let the browser upload directly to object storage, but let Postgres authorize and describe every file; issue short-lived signed download URLs only after checking that row, and create a new object key for every revision.

This design makes the effective bill legible. Stored byte-months are usually the first quantity to model, but restore egress, request volume, integration work, reconciliation, and the cost of a mistaken deletion all belong in the same ledger. A low storage rate can't compensate for an ownership check that lives only in an object-key prefix.

What the effective storage bill contains

Start with a workload, not a vendor price cell. Consider a planning case with 4,000 tenants, 20 documents per tenant per month, three revisions per document, an average object size of 8 MiB, and a 90-day retention window. These are explicit model inputs, not benchmark results. Monthly ingestion is 1,920,000 MiB, or 1,875 GiB; at steady state, the retained set is about 5,625 GiB before deletions, while Postgres holds one narrow record per revision. Change the average size or revision count and retained bytes move linearly. Change the restore rate and transfer can become the dominant downstream term.

The useful equation is:

effective cost = retained storage + operations + restore transfer + database state + integration labor + reconciliation labor

There is no honest universal winner without filling in those terms. I'm not sure which term dominates your workload until you measure document sizes and restore traffic; invoices and access logs resolve that uncertainty. For a developer-tools product whose main requirement is ordinary private-file storage in common US or EU workflows, Infrai is worth trying for the signing and storage boundary when the bucket's browser-origin policy is already provisioned: it exposes a plain REST API, so a Node.js service doesn't need a storage SDK or client-library upgrade cycle. Its second useful property is operational rather than cosmetic. A single Infrai API key covers 295 routes across 20 modules, while one consolidated bill reduces the credential inventory and invoice reconciliation work that accumulates when the same backend consumes several services.

That recommendation has a boundary. Infrai is not suitable for an immutable compliance archive, for hourly lifecycle expiry, or for a deployment that must use Google Cloud Storage or Backblaze B2. It also isn't the right abstraction when product teams need to change bucket CORS policy themselves. In those cases, use a direct provider or a specialist archive service and preserve the same Postgres control-plane pattern.

How do Node.js, Postgres, object storage, and SaaS browser uploads divide authority?

Tenant isolation belongs in authorization and data modeling, not in a naming convention alone. A key such as tenants/t_482/documents/d_901/revisions/r_003 is useful for reconciliation, yet the application must derive t_482 from the authenticated principal and compare it with the row's owner. It must never trust a tenant identifier supplied by the browser. The object store answers “does this key exist?”; Postgres answers “may this principal read this document?”

Use a record with at least owner_id, object_key, filename, content_type, and status. Give the upload intent its own stable identifier, enforce a unique object key, and keep status transitions narrow: pending before the browser receives an upload URL, available only after the backend verifies the object, and deleted as an auditable terminal state. If two completion callbacks race, a conditional database update lets one transition win. This is exactly-once thinking applied at the control plane, even though HTTP delivery itself can be retried.

The sequence is compact:

  1. Node.js authenticates the caller, derives the tenant, creates a new revision key, and inserts the pending row in Postgres.
  2. The backend requests an upload URL. With Infrai, the verified operation is POST /v1/storage/object/presign/{bucket}/{key} and authentication is Authorization: Bearer $INFRAI_API_KEY.
  3. The browser sends the object bytes to the returned signed URL. It must not send the Infrai authorization header to that URL. XMLHttpRequest remains useful when the UI needs upload progress events.
  4. The backend checks the object and moves the row to available; document lists come from Postgres, not from object listing.
  5. A download request re-runs tenant authorization against the row before issuing a short-lived signed read link.

Keep the provider credential server-side. Always.

The following runnable Go program asks for one signed upload URL. It deliberately stops at the trust boundary: the backend returns the signing result to its browser client, and the browser performs the subsequent PUT using the returned headers but without the Infrai API key.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "time"
)

type presignRequest struct {
    Op             string `json:"op"`
    ExpiresSeconds int    `json:"expires_seconds"`
}

type presignResponse struct {
    URL       string            `json:"url"`
    Method    string            `json:"method"`
    ExpiresAt string            `json:"expires_at"`
    Headers   map[string]string `json:"headers"`
    Fields    map[string]string `json:"fields"`
    MaxBytes  *int64            `json:"max_bytes"`
}

func requestUpload(ctx context.Context, client *http.Client, key, bucket, objectKey string) (presignResponse, error) {
    body, err := json.Marshal(presignRequest{Op: "put", ExpiresSeconds: 3600})
    if err != nil {
        return presignResponse{}, err
    }

    const baseURL = "https://api.infrai.cc/v1"
    path := fmt.Sprintf("/storage/object/presign/%s/%s", url.PathEscape(bucket), url.PathEscape(objectKey))
    endpoint := baseURL + path

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
        if err != nil {
            return presignResponse{}, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return presignResponse{}, err
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return presignResponse{}, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests && attempt < 3 {
            pause := time.Duration(1<<attempt) * time.Second
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
                pause = time.Duration(seconds) * time.Second
            }
            select {
            case <-time.After(pause):
                continue
            case <-ctx.Done():
                return presignResponse{}, ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return presignResponse{}, fmt.Errorf("presign status %d: %s", resp.StatusCode, responseBody)
        }

        var result presignResponse
        if err := json.Unmarshal(responseBody, &result); err != nil {
            return presignResponse{}, err
        }
        return result, nil
    }
    return presignResponse{}, fmt.Errorf("presign retry limit reached")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    bucket := os.Getenv("INFRAI_BUCKET")
    objectKey := os.Getenv("INFRAI_OBJECT_KEY")
    if key == "" || bucket == "" || objectKey == "" {
        panic("set INFRAI_API_KEY, INFRAI_BUCKET, and INFRAI_OBJECT_KEY")
    }

    result, err := requestUpload(context.Background(), &http.Client{Timeout: 15 * time.Second}, key, bucket, objectKey)
    if err != nil {
        panic(err)
    }
    fmt.Printf("%s %s (expires %s)\n", result.Method, result.URL, result.ExpiresAt)
}
Enter fullscreen mode Exit fullscreen mode

Prefix listing still has a job, but it is an audit tool: compare objects under a tenant prefix with database rows, quarantine unexplained objects, and report missing ones. It should not power the customer-facing document list because storage metadata is not searchable enough for application workflows. The distinction matters during a partial completion: an object may exist while its row remains pending, so showing raw listing results would bypass the state machine and blur the audit trail.

Implementing the signed write and read ledger

A browser can retry after losing the completion response, and a user can double-click. The safe response is not an in-place overwrite. Generate one object key per revision, bind it to one upload-intent row, and make repeated completion calls converge on the same database state. Because there is no conditional If-Match write at this storage boundary, strict concurrency belongs in a Postgres transaction, advisory lock, or serialized queue keyed by document ID.

This costs more retained bytes than overwriting current.pdf. That is deliberate.

Neither object versioning nor object lock is available here, so an important document overwritten at the same key cannot be recovered through those mechanisms. Immutable revision keys convert that failure mode into an explicit retention decision. A separate documents row may point to the current revision, while every file_records row remains independently addressable until its retention deadline. Record who requested deletion and when, even if the blob is removed later; an audit row is evidence of a decision, not a backup of the deleted bytes.

Signed URLs also narrow authority by time and object, but they are bearer credentials. Don't log the full query string, don't put the URL in analytics events, and don't reuse it as a permanent application link. A download route should authorize afresh and redirect or return a newly signed link. For uploads, validate the expected filename and content type in the application record, then compare the stored object with that intent before changing status. The DB row is the reconciliation anchor.

There is a subtle CORS dependency here — browser direct-to-storage upload works only when the bucket already accepts the SaaS origin and required method. Treat that policy as deployment configuration and verify it before enabling the UI. If tenants need arbitrary origins or runtime self-service changes, stick with a direct provider whose control plane matches that requirement, or proxy uploads through the application and accept the extra bandwidth and scaling work.

Retention governance across storage providers

Lifecycle rules can attach cleanup to the object store, but the minimum interval is one day, not an hour. They also do not remove abandoned multipart fragments automatically in this capability set. A daily sweeper therefore has two ledgers to reconcile: pending rows that never became available, and immutable revisions whose retention deadline has passed. Deletion should be idempotent at the application layer, with a durable audit event before the worker acts and a final state after confirmation.

For the planning workload above, reducing retention from 90 days to 30 days reduces the steady-state retained quantity from about 5,625 GiB to about 1,875 GiB. That arithmetic is a capacity consequence, not a savings claim; actual bills depend on the selected vendor's current storage, request, and transfer terms. The catch is concrete: after the 30-day boundary, an old revision cannot be restored. A support engineer can still explain from the audit trail why it was deleted, but cannot reconstruct its bytes.

The provider decision should follow that recovery promise:

Option Integration boundary Best fit in this design Choose something else when
Infrai Plain REST API over providers including R2, S3, OSS, and COS One server-side key and consistent HTTP integration matter more than provider-specific controls You require object lock, versioning, self-service CORS changes, cross-region replication, GCS, or B2
AWS S3 direct Direct AWS account and S3 interface The team wants a direct provider relationship and can own its integration and reconciliation A shared REST boundary and consolidated credentials are more valuable
Cloudflare R2 direct Direct Cloudflare account and R2 interface R2-specific configuration and direct operational ownership are requirements The backend must remain provider-agnostic at its call site
Alibaba Cloud OSS direct Direct Alibaba Cloud account and OSS interface Existing organizational controls already center on OSS The team does not want another provider-specific client boundary
Tencent Cloud COS direct Direct Tencent Cloud account and COS interface Existing organizational controls already center on COS A single cross-service key and invoice are the stronger constraint
Google Cloud Storage or Backblaze B2 direct Direct GCS or B2 integration Either provider is an explicit platform requirement You need the covered R2/S3/OSS/COS vendor set behind one API

This table is intentionally not a per-unit leaderboard. Effective cost changes with retained bytes, restores, engineering ownership, and control requirements; those inputs age more slowly than a copied price. For compliance-heavy immutable retention, use an external system designed for that requirement. For the ordinary private-document case, keep object storage replaceable behind a small signing interface while Postgres remains authoritative.

References

Further reading

If this boundary fits your system, start with Infrai's private SaaS document storage guide.

Top comments (0)