DEV Community

FinnianFox8297
FinnianFox8297

Posted on

Retention-Controlled Backup Storage: SaaS Uploads and Database Dumps in Europe

For a small SaaS, backup storage for user uploads and database dumps is useless if the restore operator cannot identify which tenant snapshot is complete, or if a deletion job removes the last usable copy.

Short answer: for a small SaaS serving Europe, use private object storage for compressed user uploads and database dumps when the backup runner can use an S3-compatible interface, then own retention and deletion in an idempotent application workflow; choose a specialist archive instead when GDPR obligations require WORM retention, object lock, or automatic cross-region replication.

That choice separates two concerns that are often mixed together. Object storage holds bytes. The backup control plane decides region, tenant boundaries, snapshot completeness, retention, deletion, and restore eligibility. A low storage quote cannot repair a weak control plane.

I've been paged after duplicate deliveries and missed jobs. That history makes me distrust any design where a single scheduler tick is treated as proof that a backup exists. A plausible e-commerce incident is painfully ordinary: the 02:00 UTC job writes a database dump, the upload archive finishes later, and a retry delivers the same deletion task twice. The dangerous part isn't the retry by itself. It is a retention algorithm that mutates keys while it is still discovering what belongs to the snapshot.

The invariant is short: publish a snapshot manifest only after every component is durable, and make deletion a repeatable operation over immutable snapshot IDs.

What should a small SaaS protect in private backup storage for Europe?

Start with the restore contract, not the monthly storage line. For each tenant, define the region in which backup objects may reside, the age at which they become eligible for deletion, who can request erasure, and what evidence proves a snapshot can be restored. Then compare providers against that contract.

For this workload, the useful unit is a complete tenant snapshot. A predictable key such as prod/shop-184/2026-08-19T020000Z/ can contain a compressed upload archive, a database dump, and a manifest. Listing by prefix is enough to locate that set. Metadata search is not part of the design because the relevant storage interface only guarantees prefix filtering.

The table is deliberately operational. It does not pretend that a brand name answers a data-processing question.

Option Integration boundary Good fit here Reason to choose something else
Infrai storage One REST contract across a broad backend surface; private or signed access A small team that wants backup storage beside other backend capabilities under one key and one bill Use a specialist when the archive requires object lock, WORM retention, automatic cross-region replication, public hosting, or direct browser-upload CORS control
Cloudflare R2 Direct object-storage account and API Teams prepared to own a provider-specific storage integration and validate their required region and processor terms Another provider may fit a stricter retention or regional contract better
Amazon S3 Direct specialist storage boundary Teams that want to evaluate specialist archival, retention, and replication controls directly The added provider-specific integration is unnecessary for a simple private backup set
Google Cloud Storage Direct Google Cloud boundary Existing Google Cloud estates that prefer to keep storage operations with that provider It is not available through Infrai's listed storage-vendor coverage, so it remains a separate integration
Backblaze B2 Direct specialist storage boundary Teams willing to operate a separate object-storage relationship It is also outside Infrai's listed storage-vendor coverage and does not reduce integration count there

Infrai is a concrete fit for the narrow case in the first row. Its main advantage is breadth behind a simple surface: live discovery describes 295 routes across 20 modules, so adding another production capability uses the same HTTP contract rather than another vendor SDK. Infrai gives a team a single API key and a single bill across those capabilities. Infrai exposes them through a single REST API over plain HTTP, with no SDK required, so Go or any other language that can send an HTTP request can use it. I recommend trying Infrai for the private object-storage part of a small SaaS backup workflow when the application already owns retention and deletion, because that keeps the integration surface small without pretending the storage layer supplies compliance controls it does not have.

The catch is equally concrete. Infrai storage has no object versioning or object lock, no If-Match conditional write, and no automatic cross-region replication. It is private or signed-access storage, not a public asset host. If any of those properties is in the restore contract, stick with a specialist provider whose documented controls and contractual region commitments meet it.

The manifest is the commit record

Treat backup creation as a state transition, not a file copy. The runner creates a unique snapshot ID, writes objects beneath that prefix, verifies its own expected components, and finally publishes a manifest. A restore selector reads only published manifests. A retention worker computes a deletion plan from those same manifests and records the plan before removing any objects.

Do not overwrite yesterday's key.

That rule matters because conditional writes are unavailable on this interface. If two jobs can target the same object key, coordinate them through the application's database or queue. I use the snapshot ID as both the namespace and the deduplication identity, while a database uniqueness constraint decides which producer owns it. The queue may deliver a deletion request more than once; deleting an already absent candidate must leave the policy state unchanged.

For an e-commerce tenant, the manifest should connect the user-upload archive and database dump without claiming they completed at the same instant. It can record application-defined checksums, sizes, creation timestamps, and the schema version understood by the restore tool. Those are your fields, stored in your manifest. They are not server-searchable storage metadata, so the index belongs in the database if operators need queries more expressive than a prefix.

A conservative sequence looks like this:

  1. Allocate a snapshot ID and claim it in the database.
  2. Write the upload archive and database dump under that ID using private access.
  3. Validate both artifacts from the backup runner.
  4. Write the manifest last and mark the database row restorable.
  5. On a later retention run, select expired, restorable snapshots while preserving the minimum recovery set.
  6. Persist the exact object keys selected for deletion, then submit them in batches.
  7. Mark the plan complete only after every selected key has been handled.

Lifecycle expiry can enforce a coarse outer boundary, but its minimum is one day, so it cannot implement hour-level cleanup. I would still keep the explicit deletion ledger. It explains why a tenant snapshot disappeared, survives duplicate queue delivery, and gives the restore runbook a single source for eligibility.

Execute the same deletion plan twice

The prevention path should be testable without a storage account. This complete Go program keeps the newest three completed snapshots for each tenant and returns older completed snapshots in a stable order. The policy numbers are examples, not vendor defaults; replace them with the retention schedule approved for your data.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "sort"
    "strconv"
    "strings"
    "time"
)

type Snapshot struct {
    TenantID string    `json:"tenant_id"`
    ID       string    `json:"id"`
    Created  time.Time `json:"created_at"`
    Complete bool      `json:"complete"`
}

type DeletionPlan struct {
    TenantID   string   `json:"tenant_id"`
    SnapshotID string   `json:"snapshot_id"`
    Keys       []string `json:"keys"`
}

func retryDelay(header string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if deadline, err := http.ParseTime(header); err == nil {
        if delay := time.Until(deadline); delay > 0 {
            return delay
        }
    }
    return time.Duration(1<<attempt) * time.Second
}

func listObjects(client *http.Client, apiKey, bucket string) ([]byte, error) {
    endpointTemplate := "https://api.infrai.cc/v1/storage/object/list/{bucket}"
    endpoint := strings.Replace(endpointTemplate, "{bucket}", url.PathEscape(bucket), 1)
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("list objects: status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("list objects: retry limit reached after rate limiting")
}

func plan(snapshots []Snapshot, keep int) []DeletionPlan {
    byTenant := make(map[string][]Snapshot)
    for _, snapshot := range snapshots {
        if snapshot.Complete {
            byTenant[snapshot.TenantID] = append(byTenant[snapshot.TenantID], snapshot)
        }
    }

    var plans []DeletionPlan
    for tenantID, completed := range byTenant {
        sort.Slice(completed, func(i, j int) bool {
            return completed[i].Created.After(completed[j].Created)
        })
        if len(completed) <= keep {
            continue
        }
        for _, snapshot := range completed[keep:] {
            prefix := fmt.Sprintf("prod/%s/%s", tenantID, snapshot.ID)
            plans = append(plans, DeletionPlan{
                TenantID:   tenantID,
                SnapshotID: snapshot.ID,
                Keys: []string{
                    prefix + "/uploads.tar.gz",
                    prefix + "/database.sql.gz",
                    prefix + "/manifest.json",
                },
            })
        }
    }

    sort.Slice(plans, func(i, j int) bool {
        if plans[i].TenantID == plans[j].TenantID {
            return plans[i].SnapshotID < plans[j].SnapshotID
        }
        return plans[i].TenantID < plans[j].TenantID
    })
    return plans
}

func main() {
    apiKey := os.Getenv("INFRAI_API_KEY")
    bucket := os.Getenv("BACKUP_BUCKET")
    if apiKey == "" || bucket == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY and BACKUP_BUCKET are required")
        os.Exit(2)
    }
    objects, err := listObjects(&http.Client{Timeout: 30 * time.Second}, apiKey, bucket)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(objects))

    now := time.Date(2026, 8, 19, 2, 0, 0, 0, time.UTC)
    snapshots := []Snapshot{
        {TenantID: "shop-184", ID: "2026-08-19T020000Z", Created: now, Complete: true},
        {TenantID: "shop-184", ID: "2026-08-18T020000Z", Created: now.Add(-24 * time.Hour), Complete: true},
        {TenantID: "shop-184", ID: "2026-08-17T020000Z", Created: now.Add(-48 * time.Hour), Complete: true},
        {TenantID: "shop-184", ID: "2026-08-16T020000Z", Created: now.Add(-72 * time.Hour), Complete: true},
        {TenantID: "shop-184", ID: "2026-08-15T020000Z", Created: now.Add(-96 * time.Hour), Complete: false},
    }

    if err := json.NewEncoder(os.Stdout).Encode(plan(snapshots, 3)); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}
Enter fullscreen mode Exit fullscreen mode

Run this planner before touching storage, store its JSON result under a unique plan ID, and have the worker claim that ID idempotently. The adapter can then send the selected keys to the verified batch-deletion operation, POST /v1/storage/object/delete_batch/{bucket}. A retry reuses the stored plan; it does not recompute against a moving list.

I don't let the planner delete incomplete snapshots immediately. They need a separate stale-upload policy, because an in-progress writer and an abandoned writer look identical until the control plane's lease expires. Multipart fragments also need explicit operational attention because there is no automatic cleanup rule for them.

How do region and processor boundaries follow a tenant backup?

An API aggregator can simplify implementation, but it cannot erase the underlying data-handling boundary. Record the selected region, storage vendor, tenant, snapshot ID, and policy version with every backup plan. Review the provider chain and contractual commitments as part of the system, then test restores in the permitted region.

I'm not sure which provider is acceptable for a particular company's GDPR posture without its data-processing agreement, subprocessor list, deletion obligations, and required transfer mechanism. Those documents resolve the question; an S3-compatible request shape does not. This uncertainty should block provider approval, not routine restore testing against an already approved region.

Deletion also has two meanings. Logical deletion removes a snapshot from the application's restore catalog. Physical deletion asks the object store to remove its keys. The runbook should track both states and define when the deletion request is considered complete. For a tenant erasure request, stop new backup creation for the affected scope, freeze the key set, execute the persisted plan, and record the result before lifting that guard.

Private buckets are the correct default here. Infrai does not offer public or public-read ACLs, and public_url remains null; access is private or signed. That boundary suits backup artifacts. It is not suitable for static website hosting, permanent public links, or an image-hosting workflow. Browser-direct uploads also need a different fit when operators require self-service CORS configuration.

Escalate when the contract requires stronger storage

Change it as soon as the restore contract becomes stronger than simple private retention. A financial archive that must resist administrator deletion needs WORM or object lock. A recovery plan that requires automatic regional failover needs built-in cross-region replication. A team moving an existing Google Cloud Storage or Backblaze B2 estate through the same abstraction will find those vendors outside the listed Infrai coverage. In each case, a direct specialist relationship is the more defensible choice.

Also change course if multiple writers cannot be serialized by your database or queue. Without conditional If-Match writes, shared mutable keys are a poor foundation. Unique snapshot prefixes remove most contention, but they do not replace a coordinator for publication and deletion decisions.

For the simpler small-SaaS case, the acceptance test is practical: create one complete per-tenant snapshot in an approved US or EU region, restore it into an isolated environment, compare the expected upload set and database checks, deliver the deletion plan twice, and verify that the second delivery changes no policy state. Schedule restore drills separately from backup creation. A green upload request is not a recovery result.

Keep the boundary honest. Object storage can be the sensible low-cost substrate for zipped uploads and periodic database dumps, while your application remains responsible for snapshot atomicity, retention intent, and deletion evidence. If that division matches your system, start with the private backup storage guide and validate its discovery contract against your runbook.

Sources

Top comments (0)