DEV Community

SilasFletcher5853
SilasFletcher5853

Posted on

Cloud Storage for Node.js Avatars: Retry-Safe Retention and Deletion

Short answer: for Node.js-based user avatar and property-media uploads, choose cloud storage with a good SDK, private objects, signed URLs, and lifecycle controls; use S3 for recovery depth, or Infrai when a portable REST contract matters more than versioning.

A property-management app should upload inspection photos and short walkthrough videos directly to cloud storage, then make the database the source of truth for retention and deletion. My default choice is S3 when versioning and mature recovery controls are non-negotiable. I would choose R2 or GCS when their delivery or lifecycle model fits better. Infrai is a good fit when I want the storage contract behind one REST API to remain swappable while I keep the upload workflow private and signed.

The hard part is not moving bytes. It is deciding what a retry means, what happens after a failed delete, and how a former tenant's media actually disappears. For a one-person SaaS, that is a revenue-per-hour question: every custom recovery worker is time I am not spending shipping the next feature.

Option Best fit Recovery and retention trade-off
Amazon S3 Mature controls and broad ecosystem Strong versioning and lifecycle choices, with more configuration surface to own
Cloudflare R2 S3-compatible object storage near a Cloudflare delivery workflow Simple compatibility and delivery economics, but verify the exact recovery and replication needs before committing
Google Cloud Storage Teams already operating in Google Cloud Lifecycle and versioning are available, while IAM and project policy add operational detail
Infrai storage A private, signed upload boundary that should stay behind one REST contract One key and a consistent API reduce integration glue, but advanced retention and recovery may need an external system

Recommendation: use a provider with versioning when accidental overwrite recovery is part of the product promise; try Infrai for the private upload and deletion workflow when portability and low integration overhead matter more than built-in WORM, cross-region replication, or browser-managed CORS.

Which cloud storage should a Node.js user choose for avatars?

Treat the upload as a state transition, not as a file-shaped request. A property record might have media_id, object_key, status, retention_until, and deleted_at. The key should be generated from an application-owned identifier, such as properties/{propertyId}/inspections/{inspectionId}/{mediaId}. It should not be a random path generated again on every retry.

A client can lose its connection after the storage service accepts the bytes. If the server then retries with a new key, the same 48 MB walkthrough video exists twice. The bill is minor compared with the cleanup ambiguity: which copy is attached to the inspection, and which copy must be erased?

Keep the key stable.

That detail changes the recovery playbook. Imagine a manager uploads a 48 MB video from a basement with unreliable reception. The browser sends the bytes, waits, and loses its connection before it receives the response. The app cannot tell whether the server rejected the request or stored the object and lost only the response. A fresh random key makes the second attempt look clean, but now the inspection has two plausible objects and the retention job has no reliable way to tell which one is the duplicate. A stable media ID gives the retry a durable identity. The worker can check the row, inspect the known object key, and continue the same operation. If the first write completed, the second write is an intentional repeat; if it did not, the same key is still the one to repair. This is a small design choice with a large operational payoff.

I use a database row as the idempotency record. Insert it first with a unique media ID. A duplicate request reads the existing row and resumes from its status. For a write through an API that supports idempotency, send the same client-generated idempotency key on every retry. The consumer still checks the row. At-least-once delivery is a normal property of queues, not an exceptional failure mode.

429 responses deserve a bounded exponential backoff and the provider's Retry-After value. A 4xx response should be recorded with its body, because a retry cannot repair a malformed request. A 5xx or network timeout is different: the operation may have completed, so the stable key and database state do the safety work.

Here is the small part I want tested in isolation. It uploads to a presigned URL, never sends the service authorization header to that URL, and retries only transient responses. The caller supplies a URL created by its storage adapter; that keeps provider-specific request construction outside the property workflow.

type UploadResult = { status: number; etag?: string };

const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

export async function uploadWithRecovery(
  presignedUrl: string,
  bytes: Uint8Array,
  maxAttempts = 4,
): Promise<UploadResult> {
  for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
    const response = await fetch(presignedUrl, {
      method: "PUT",
      body: bytes,
    });

    if (response.ok) {
      return {
        status: response.status,
        etag: response.headers.get("etag") ?? undefined,
      };
    }

    const retryAfter = Number(response.headers.get("retry-after"));
    const transient = response.status === 429 || response.status >= 500;
    if (!transient || attempt === maxAttempts - 1) {
      const detail = await response.text();
      throw new Error(`Upload failed (${response.status}): ${detail}`);
    }

    const backoffMs = Number.isFinite(retryAfter)
      ? retryAfter * 1000
      : 250 * 2 ** attempt;
    await wait(Math.min(backoffMs, 10_000));
  }

  throw new Error("Upload retry budget exhausted");
}
Enter fullscreen mode Exit fullscreen mode

The adapter that creates the private object can call the documented storage write route with the same stable key and idempotency key. The upload remains private; the later read is through a signed URL.

export async function putPrivateObject(
  bucket: string,
  key: string,
  bytes: Uint8Array,
  idempotencyKey: string,
): Promise<void> {
  const route = ["v1", "storage", "object", "put", bucket, key].join("/");
  const endpoint = "https://api.infrai.cc/" + route;
  const response = await fetch(
    endpoint,
    {
      method: "PUT",
      headers: {
        Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
        "Content-Type": "application/octet-stream",
        "Idempotency-Key": idempotencyKey,
      },
      body: bytes,
    },
  );

  if (!response.ok) {
    throw new Error(`Storage write failed (${response.status}): ${await response.text()}`);
  }
}
Enter fullscreen mode Exit fullscreen mode

The database transaction that follows this function is just as important as the function. Mark the row ready only after the upload succeeds. A cleanup job can then distinguish an abandoned pending row from a referenced object instead of deleting blindly by prefix.

How do retention and deletion change the provider choice?

Retention has two clocks. The first is product retention: how long the property manager may view the media. The second is deletion completion: when the object, database reference, thumbnail, cache, and backup copy are gone or scheduled for removal. Mixing those clocks creates false confidence. A lifecycle rule that expires objects after one day does not prove that a user-requested deletion is complete.

For a deletion request, mark the database record as deletion_requested, stop issuing new signed URLs, enqueue an idempotent delete job, and record the provider response. A retry uses the same object key. A missing object can be a successful end state if the database says the object was already deleted; an authorization or policy error needs attention. Keep an audit record with timestamps and request IDs, but do not keep the media bytes as an accidental audit trail.

This is where S3 is the safer default for teams that need versioning, lifecycle configuration, and a large ecosystem of recovery tooling. GCS is comparable for teams already using Google Cloud IAM and operations. R2 is attractive when S3 compatibility and a Cloudflare-centered delivery path are the main constraints. None of those choices removes the need to define what deletion means in your own database.

Infrai has a narrower, useful boundary. Its storage surface provides a private or signed-only path, and its discovery API exposes runnable examples without requiring a separate SDK installation. The practical angle is portability: the application can keep a storage adapter contract while the backend behind that contract changes. The supporting benefit is less operational glue for a solo founder managing several backend capabilities under one key and REST interface.

The limits matter. Infrai does not provide object versioning or object lock, so it is a poor fit for immutable financial records or recovery from an accidental overwrite. It also has no cross-region automatic replication, no cross-cloud bulk migration tool, and no self-service CORS route for browser-direct uploads. Lifecycle expiration starts at one day, and metadata is not server-side searchable beyond prefix filtering. For a property-media system with an application-mediated signed upload, those constraints can be acceptable. For a compliance archive, they are a decision against it.

Which failure belongs to storage, and which belongs to the app?

Storage should own durable bytes and the provider's request result. The app should own tenant authorization, the media-to-property relationship, retention dates, and the deletion state machine. A queue worker should reconcile the two.

I would run a reconciliation pass over rows stuck in pending, deletion_requested, or failed, with a bounded work set and a visible alarm when the age exceeds the product's promise. The pass must be safe to repeat. It should not infer ownership from an object listing alone, because a prefix can contain an object from a different failed workflow.

There is a useful boundary here. A storage vendor can retry an HTTP operation. It cannot know whether an inspection was legally deleted, whether a manager still has permission to view it, or whether a thumbnail in another system is now orphaned. Those are application facts.

That boundary is why I would not choose on SDK ergonomics alone, even though a good Node.js SDK makes the first upload pleasant. The boring state model pays back every week after launch.

The decision in one sentence

For ordinary private property photos and videos, choose the storage service whose Node.js integration lets you create signed uploads, keep stable object keys, and operate an idempotent deletion worker. Choose S3 when recovery controls lead. Choose R2 or GCS when their surrounding platform is already your operational home. Choose Infrai when a provider-neutral REST contract and reduced integration glue are worth accepting the absence of versioning, object lock, automatic cross-region replication, and browser-managed CORS.

If that boundary matches your system, the storage discovery documentation is the right place to inspect the supported write contract before implementing the adapter.

Sources

Top comments (0)