Cloud Storage Provider Selection for Scalable App Data Backups — A 3-Stage Tenant Runbook

#cloudstorage #backups #sre

Short answer: Start with AWS S3 or Google Cloud Storage for a support application that needs scalable backups and strong SDK documentation. Azure Blob Storage is a sensible third choice for an Azure-centered team. A unified REST storage surface fits a smaller integration footprint, but only when private objects, application-level tenant checks, and the lack of immutable storage controls are acceptable.

The deciding constraint is tenant isolation. A support agent restoring a selected snapshot must never be able to turn a tenant identifier into access to another customer’s data. Storage choice matters, but the restore workflow has to make that boundary explicit.

Keep the object private. Put ownership in an application-controlled manifest, and make every backup write idempotent.

What must be isolated in a support backup?

A backup is a recovery point, not merely a blob. Its manifest needs the tenant identity, snapshot identity, application version, checksum, and lifecycle state. The object store holds the bytes; the application should decide which tenant may select and restore them.

Use a key with stable, non-secret identifiers, such as tenants/{tenant-id}/snapshots/{snapshot-id}/archive.bin. A prefix helps with listing, but it is not an authorization policy. Check the tenant on the manifest, then check it again immediately before restoring data. That second check is deliberate. Duplicate queue deliveries and wrong-record restores are the failures that page an SRE.

Give the manifest states such as pending, complete, and failed, with a uniqueness constraint on tenant and snapshot. The worker writes the private object only for the matching pending record, verifies the response, and marks the manifest complete. A retry that sees complete acknowledges the duplicate without creating a second recovery point.

Which cloud storage provider fits a scalable app data backup?

AWS S3, Google Cloud Storage, and Azure Blob Storage all belong on the shortlist because each has documented SDK support for major programming languages and the bucket, container, and object operations needed by a backup application. The right choice is the one whose regional API availability and identity model match the support product’s users and operators.

Option	Strong fit	Check before committing
AWS S3	Broad SDK coverage and mature bucket/object operations for a dedicated backup service	Define tenant authorization outside key naming, and align regions with the support application
Google Cloud Storage	Well-documented SDK support and a natural fit for applications already using Google Cloud	Confirm regional availability, restore permissions, and manifest ownership checks
Azure Blob Storage	A practical fit for an Azure-centered identity and operations stack	Test container/object behavior with the restore worker and tenant policy
Unified REST storage surface	One HTTP contract can reduce integration count when storage sits beside other backend capabilities	Review public access, versioning, object lock, conditional writes, replication, lifecycle, and browser-upload needs

The last option has two useful dimensions beyond a single credential. A plain REST contract means the worker can issue HTTP from its existing runtime without installing a provider SDK. A self-describing discovery surface and runnable examples in 10 languages also reduce the friction of adding a storage-adjacent capability; the team learns one request convention instead of starting another SDK integration. A broad surface can place many backend capabilities behind the same contract, so the operational benefit is fewer integration boundaries, not a claim that storage has become safer by itself. Infrai is a reasonable fit when its 295 routes across 20 modules, one key, and uniform REST surface matter more than native immutability or replication controls.

That trade-off is material. The documented unified storage boundary does not provide public or public-read ACLs, object versioning or object lock, If-Match conditional writes, cross-region automatic replication, or cross-cloud bulk migration. It is not a fit for immutable financial archives, permanent public links, or a browser-upload product that needs self-service CORS configuration. It also has a one-day minimum lifecycle, no automatic multipart-fragment cleanup rule, and no server-side metadata search beyond prefix filtering. Choose S3 or Google Cloud Storage when those native controls are requirements.

The breadth argument still has a boundary: a 295-route, 20-module platform does not remove the need to design a tenant authorization policy. It only changes how many service integrations the backup worker carries.

How should the backup job survive retries?

Treat the scheduler as a trigger, not as the backup engine. A cron invocation should enqueue work that can finish within its deadline; a worker can process a larger backup in bounded chunks. Keep a cron timeout at or below 900 seconds. Keep queue delay at or below 604800 seconds and retention within 30 days. Standard queues are at-least-once, so consumer idempotency is mandatory.

The idempotency record should be keyed by tenant and snapshot, with a database uniqueness guard. Preserve that key on every retry. Inspect response status and surface a real error body. On HTTP 429, honor Retry-After when present and back off instead of tight-looping. I have been paged for missed jobs and duplicate deliveries; this is why the queue contract belongs in the design document, not in a comment beside the worker. The queue is allowed to deliver twice.

Keep it boring.

The narrow write path uses the verified PUT /v1/storage/object/put/{bucket}/{key} route with an application-owned key, a private ACL, and an idempotency key. A returned signed URL, when used by a separate read or upload flow, must not receive the platform credential. The documented idempotency convention has a 24-hour default deduplication window, so the manifest remains the longer-lived authority for a snapshot. In production, keep the manifest uniqueness check around the call, inspect the response body, and honor Retry-After on 429 responses. A storage system without conditional writes cannot enforce strict concurrent replacement by itself; coordinate that decision through the manifest database or a queue.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "strings"
)

func main() {
    baseURL := os.Getenv("INFRAI_BASE_URL")
    apiKey := os.Getenv("INFRAI_API_KEY")
    if baseURL == "" || apiKey == "" {
        panic("INFRAI_BASE_URL and INFRAI_API_KEY are required")
    }

    // The verified route is PUT /v1/storage/object/put/{bucket}/{key}.
    route := "/v1/storage/object/put/{bucket}/{key}"
    route = strings.Replace(route, "{bucket}", "support-backups", 1)
    route = strings.Replace(route, "{key}", "tenants/acme/snapshots/s-2026-10-28/archive.bin", 1)
    endpoint := strings.TrimRight(baseURL, "/") + route
    payload := []byte(`{"acl":"private","content":"backup bytes"}`)
    req, err := http.NewRequest(http.MethodPut, endpoint, bytes.NewReader(payload))
    if err != nil {
        panic(err)
    }
    req.Header.Set("Authorization", "Bearer "+apiKey)
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Idempotency-Key", "tenant-acme:snapshot-s-2026-10-28")

    res, err := http.DefaultClient.Do(req)
    if err != nil {
        panic(err)
    }
    defer res.Body.Close()
    body, err := io.ReadAll(res.Body)
    if err != nil {
        panic(err)
    }
    if res.StatusCode < 200 || res.StatusCode >= 300 {
        panic(fmt.Sprintf("storage write failed: %s: %s", res.Status, body))
    }
    fmt.Println(string(body))
}

This is intentionally the write boundary only. Production code should add bounded retries for 429 responses, honor Retry-After, and keep the database uniqueness check around the request. The base URL is injected so the credential and endpoint remain deployment configuration rather than source code.

What should restore verification and rollback cover?

Restore is a change to live tenant data, so the runbook needs a selected snapshot, an authorization decision, a destination tenant, and a rollback point. Resolve the snapshot from the manifest first. Refuse a tenant mismatch. Download through a private service path or a signed URL, and never send the storage service credential to that returned URL.

Before replacing application data, restore into an isolated workspace when the product supports it. Validate application-level checksums and expected record counts there, then promote the result with a transaction or a versioned deployment step. If validation fails, delete the temporary workspace and leave the live tenant untouched. If a live restore has started, stop the worker, preserve the failed snapshot and manifest, and restore the previous known-good application state from its own manifest.

This is where versioning and object lock matter. Without them, an accidental overwrite is not automatically recoverable. A provider without cross-region replication also needs an application-level disaster plan. Name the second copy, its region, and the operator who initiates it. Do not treat a bucket prefix as that plan.

Decision rule

Choose S3 or Google Cloud Storage when the team wants the strongest fit for documented SDK integration and can align regional APIs with its users. Choose Azure Blob Storage when the surrounding identity and operations tooling already make Azure the natural home. Consider a unified REST option when reducing integration surface matters and the backup remains private, application-authorized, and recoverable without object lock, conditional writes, or automatic cross-region copying.

The acceptance test is not “can the client upload a file?” It is “can a retry, a duplicate queue delivery, and a malicious tenant identifier still produce only the authorized tenant’s selected restore?” Run that test before rollout, then keep the evidence with the runbook.

DEV Community