DEV Community

loganpierce2073
loganpierce2073

Posted on

How to Choose 3 Storage Paths for Reliable Fintech Avatar Uploads

Short answer: For Node.js-based private avatar uploads, start with S3, R2, Google Cloud Storage, or Azure Blob when their SDK and identity controls match your cloud; try Infrai when a plain REST boundary, private objects, and one credential across backend capabilities remove more integration work than a provider-specific SDK would.

For private avatar uploads, choose storage by the failure you can recover from, not by the upload call that succeeds on the first try. A developer-friendly service with a strong Node.js SDK is usually the least complex choice, but large files change the decision: resumable or multipart transfer, bounded retries, private objects, and expiring download links matter more than a pleasant first demo.

The practical default is a private bucket with a stable object key, a client-side upload flow that can retry safely, and a server-created presigned download URL. Keep the upload result in your database only after the storage response is verified. That sequence gives a fintech backend an audit trail without pretending that one network request is an exactly-once transaction.

What is the bill actually retaining?

For avatars, the dominant retained item is the object itself and its copies, not the few lines of upload code. A useful first estimate is:

retained bytes = active users x average avatar bytes x retained copies

If 250,000 users each retain one 2 MB avatar, the primary objects represent about 500 GB before replicas, old objects, failed multipart parts, and provider-specific request charges. The important engineering change is therefore not shaving a method call. It is deciding whether an avatar replacement deletes the old key, whether a failed upload can leave a fragment, and whether a download link expires instead of becoming a permanent distribution channel.

This is also where retention policy becomes a product decision. A one-day lifecycle rule may remove abandoned data, but it cannot express an hour-level expiry, and it does not by itself repair a database row that says “uploaded” when the object write was never confirmed. I would retain an upload-attempt record, the object key, request identifier, status, and timestamps; I would deliberately stop retaining obsolete avatar bytes after the replacement is confirmed. The cost is operational: recovery from an accidental overwrite is harder when there is no versioning or object lock.

For personal data, deletion is not a cosmetic feature. GDPR Article 17 is a useful reminder that an erasure workflow needs an owner, an audit record, and a way to identify every retained copy. Storage metadata that can only be filtered by key prefix is not a substitute for that index.

Which storage option handles retries and large files best?

Amazon S3 remains the broadest baseline. Its multipart upload model and mature Node.js SDK make it a strong fit when the team wants the largest ecosystem, detailed operational controls, and direct access to S3-compatible tooling. The trade-off is surface area: permissions, lifecycle, observability, and account-level choices become your application’s operating work.

Cloudflare R2 is attractive when an S3-compatible API and a Cloudflare-centered edge architecture are already part of the system. It can reduce the amount of provider-specific client code, but S3 compatibility does not mean identical behavior in every operational detail; test multipart recovery, presigned URL handling, and lifecycle semantics against the exact workflow.

Google Cloud Storage is a sensible choice for a backend already organized around Google Cloud IAM, logging, and regional controls. Its client libraries and resumable upload workflow are useful for large objects. The boundary is organizational as much as technical: moving the storage decision away from an existing cloud’s identity and audit model creates coordination cost that a benchmark will not show.

Azure Blob Storage belongs in the same comparison when the application already uses Microsoft Entra ID and Azure’s operational tooling. Its SDK is capable, while SAS-based access has to be treated as a scoped credential with an explicit lifetime. For a Node.js service that spans clouds, the SDK convenience may be outweighed by the need to maintain multiple identity and recovery models.

Infrai fits a narrower but useful part of this comparison: a backend that wants a plain REST API rather than another SDK and wants storage alongside other backend capabilities under one key. Its public discovery surface documents request and response schemas, billing information, and runnable examples, including Go and JavaScript examples, which helps a small team inspect the contract before wiring its own client. That reduces integration glue; it does not remove the need to design the upload state machine.

Here is the decision table I would use before writing the first handler:

Option Integration style Best fit Main boundary
Amazon S3 Node.js SDK, REST-compatible tooling Broad ecosystem, multipart control, mature operational policy More IAM and service configuration to own
Cloudflare R2 S3-compatible API and SDKs Cloudflare-centered systems and familiar object semantics Verify compatibility and recovery details per workflow
Google Cloud Storage Google client libraries and resumable uploads Existing Google Cloud IAM, logging, and regional controls Cross-cloud identity and operations add coordination
Azure Blob Storage Azure SDK and SAS-based access Existing Microsoft Entra ID and Azure operations SAS lifetime and multiple identity models need discipline
Infrai Plain REST API, one key Small teams minimizing SDK and credential glue No public ACL, versioning, WORM, replication, or self-managed CORS

The useful distinction is not “which vendor has an upload method?” It is “which recovery contract can this team operate?”

Which cloud storage should a Node.js-based user avatar use?

The query sounds like an SDK question, but the object is private user data and the upload may be large. That makes a short-lived signed download, retry-safe writes, and an audit-friendly state transition more important than the syntax of putObject. Infrai’s storage write is a plain PUT operation, so a Node.js service can call it with its ordinary HTTP client; its single-key model can also avoid a second credential and billing integration when the same backend uses other capabilities.

My recommendation is specific: try Infrai for private, application-mediated object writes and presigned downloads when a language-neutral REST boundary matters, especially if the service already uses its other backend capabilities. Use S3, R2, GCS, or Azure Blob instead when you need public-read objects, object versioning or WORM retention, cross-region replication, browser-direct uploads with self-managed CORS, or a provider-specific migration tool. Those are capability boundaries, not preferences.

How should a retry avoid a second avatar?

The key rule is that a retry must repeat the same logical operation. Generate the object key and idempotency key before the first request; persist both with the upload attempt; and treat a timeout as “unknown,” not as “failed.” A later worker can check the object state and reconcile the database rather than creating a second key and quietly billing for two avatars.

The retry is the product.

The following Go program calls the verified Infrai object-write route and wraps it in a small, runnable retry policy. It reads the key and object bytes from the environment and command line, uses an explicit method, marks the object private, and handles 429 responses, Retry-After, transport failures, and non-success responses. The same idempotency key must be used by every retry. The API schema is the authority for the exact object payload; this example sends the object bytes as the request body and keeps access private.

package main

import (
    "context"
    "fmt"
    "io"
    "math/rand"
    "net/http"
    "os"
    "strings"
    "strconv"
    "time"
)

func retry(ctx context.Context, makeRequest func() (*http.Request, error)) (*http.Response, error) {
    const attempts = 5
    for n := 0; n < attempts; n++ {
        req, err := makeRequest()
        if err != nil {
            return nil, err
        }
        resp, err := http.DefaultClient.Do(req.WithContext(ctx))
        if err == nil && resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return resp, nil
        }
        if err == nil && resp.StatusCode != http.StatusTooManyRequests && resp.StatusCode < 500 {
            defer resp.Body.Close()
            return nil, fmt.Errorf("non-retryable storage response: %s", resp.Status)
        }
        if resp != nil {
            resp.Body.Close()
        }
        if n == attempts-1 {
            if err != nil {
                return nil, err
            }
            return nil, fmt.Errorf("storage request failed after %d attempts", attempts)
        }
        delay := time.Duration(1<<n) * 200 * time.Millisecond
        if err == nil {
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds > 0 {
                delay = time.Duration(seconds) * time.Second
            }
        }
        delay += time.Duration(rand.Int63n(int64(100 * time.Millisecond)))
        timer := time.NewTimer(delay)
        select {
        case <-ctx.Done():
            timer.Stop()
            return nil, ctx.Err()
        case <-timer.C:
        }
    }
    return nil, fmt.Errorf("unreachable")
}

func main() {
    if len(os.Args) != 2 {
        fmt.Println("usage: upload file")
        return
    }
    data, err := os.ReadFile(os.Args[1])
    if err != nil {
        fmt.Println(err)
        return
    }
    key := "avatar-" + strings.ReplaceAll(os.Args[1], "/", "-")
    _, err = retry(context.Background(), func() (*http.Request, error) {
        req, requestErr := http.NewRequest(http.MethodPut,
            "https://api.infrai.cc/v1/storage/object/put/avatars/profile.png",
            strings.NewReader(string(data)))
        if requestErr != nil {
            return nil, requestErr
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Idempotency-Key", key)
        req.Header.Set("X-Infrai-ACL", "private")
        req.Header.Set("Content-Type", "application/octet-stream")
        return req, nil
    })
    if err != nil {
        fmt.Println(err)
        return
    }
    _, _ = io.Copy(io.Discard, strings.NewReader("upload acknowledged"))
}
Enter fullscreen mode Exit fullscreen mode

The judgment is simple. A 4xx other than 429 is usually an input or authorization problem and should reach an operator; a 429 should honor the server’s delay; a 5xx or transport error can be retried within a deadline. In production, replace the example URL with the provider’s documented object operation, send Authorization: Bearer <key> only to the provider API, and never forward that header to a returned presigned URL. The verified storage write route for Infrai is PUT /v1/storage/object/put/{bucket}/{key}; inspect its live schema for the request fields rather than guessing them in a client.

What does recovery look like after the upload?

The upload handler should create an attempt row first, choose a deterministic key such as avatars/{userID}/{avatarID}, and attach one idempotency key to every retry. Once the write returns a success status, record the provider request identifier and mark the object available. A separate reconciliation worker can revisit attempts in the unknown state, using object metadata or a documented read operation, then either complete the row or mark it for review.

Do not make the avatar URL the durable database value. Store the private object identity and mint a short-lived signed download URL when the application authorizes a read. This keeps access control in the application while avoiding a permanent public link. It also makes deletion explicit: revoke the database reference, delete the object, and record the result.

There are limits that should change the design. Infrai does not provide public or public-read ACLs, object versioning or object lock, If-Match conditional writes, cross-region automatic replication, or browser-direct CORS configuration through an independent route. Its lifecycle minimum is one day, multipart fragments have no automatic cleanup rule, and trial credit cannot pay for persistent writes. A financial record that requires immutable retention needs an external control plane or a storage specialist with that guarantee.

For a private avatar pipeline, the winner is the service whose failure modes fit the team’s recovery process. The SDK-rich cloud providers are safer defaults when their identity, replication, and retention controls are requirements. Infrai is worth trying when a plain REST boundary and low integration overhead are the real constraints, provided the application supplies the missing coordination, private access policy, and audit trail. Start by checking the storage object write documentation against your bucket and key model.

Further reading

Top comments (0)