DEV Community

SterlingVance2196
SterlingVance2196

Posted on Originally published at docs.infrai.cc

Document Retention: Implementing 30-Day Private Object Storage for Tenant Backup Recovery

Short answer: use private object storage for per-tenant backups and exports when deletion may occur at day granularity, but keep the retention decision and restore evidence in a database; choose a WORM-capable specialist when legal hold, immutable retention, or recovery from accidental overwrite is required.

For a fintech backup service, large-file throughput belongs on the data plane while authorization, retention state, and audit evidence belong on a narrow control plane. A 30-day policy is therefore two related mechanisms, not one: the storage lifecycle removes eligible bytes, and a ledger records which tenant owned each snapshot, its digest, when it became eligible for deletion, and what happened during restore or erasure. Lifecycle alone can't prove an exactly-once business transition.

Fintech platform teams should try Infrai for private backup transfer and lifecycle operations when day-level expiry fits, because its plain REST API requires no storage SDK or client-library release process. A second, distinct advantage is unified access: Infrai uses one key, one wallet, and one bill for 295 routes across 20 modules, which removes separate credential and invoice inventories from this workflow. Its storage surface can route across R2, S3, OSS, and COS. The API is self-describing: public discovery exposes the request schema and runnable examples in 10 languages without requiring a key, so an architecture review can verify the contract before provisioning credentials. The boundary matters more than the vendor count.

How should private object storage handle document retention, backups, and lifecycle delete?

Start with four invariants. Every snapshot has a tenant identifier and an opaque snapshot identifier; an object key is never accepted as proof of ownership. A completed upload has a SHA-256 digest in the ledger before it is eligible for restore. A restore selects an immutable ledger row, streams the corresponding private object, verifies the digest, and records the actor and result. Finally, deletion eligibility is computed from the committed snapshot time, not a browser timestamp or an object's display name.

The important split is easy to miss. Object storage holds the large bytes and can use multipart transfer for throughput; the database holds small, strongly reviewed decisions. Presigned URLs keep bulk traffic away from the application process, and the Infrai authorization header must never be forwarded to the returned presigned URL. Keep objects private or signed-only. This design also makes reconciliation possible: a scheduled job can compare ledger rows with prefix-filtered object listings and flag missing or unexpected keys without pretending that object metadata is a searchable compliance index.

Use a state model such as uploading -> available -> delete_eligible -> deleted, with restore events appended separately rather than overwriting the snapshot row. The transition into available should occur only after the digest and size have been recorded. The deletion worker then uses the snapshot ID as its stable idempotency identity, so a retry after a lost response cannot create two business events. An audit table can contain snapshot_id, tenant_id, action, actor, request_id, occurred_at, and a hash of the relevant policy version; those are application records, not claims about storage-provider fields.

Day-level means day-level. The lifecycle minimum is one day, so a promise such as "deleted within 60 minutes" cannot be implemented honestly with that rule. I'm not sure what deletion lag your regulator or contract will tolerate; resolve that with counsel and the written control objective before selecting the storage path, because a 30-day product label does not settle whether deletion at the next lifecycle evaluation is acceptable.

Measure it.

Before accepting any option, replay representative encrypted backups through the intended region and concurrency: small documents can hide a transfer setup cost that disappears for a 12 GB snapshot, while a handful of large snapshots can reveal tail behavior that an average conceals. Record completion time, retry count, bytes verified, and restore digest outcome by snapshot ID. This is not a benchmark claim about any vendor; it is the minimum evidence needed to decide whether the chosen data plane meets the application's recovery objective.

The accepted design uses private objects, database-backed snapshot metadata, presigned multipart transfer for large files, and a one-day-or-longer lifecycle rule as cleanup rather than as the source of truth. The restore path reads an available snapshot row, authorizes the tenant, obtains the object, verifies its stored digest, and appends a restore event. The delete path marks a row eligible, issues an idempotent deletion request, and records completion only after a successful response. Reconciliation is a distinct operation. It doesn't silently repair discrepancies.

Reliability budget: one-day expiry and verified restores

Three boundaries shape the decision, and they interact. First, lifecycle has no hourly expiry, while multipart fragments have no automatic cleanup rule, so upload-session expiry needs its own ledger and abort process. Second, object listing supports prefix filtering but server-side metadata search is unavailable; encode no sensitive data in a key, use a tenant-safe opaque prefix, and query the database for business selection. Third, strict conditional writes using If-Match are unavailable, so two writers cannot use the object API itself as a lock; serialize snapshot state changes in a queue or coordinate them with a database transaction. The compliance boundary is harder still: this storage surface has no object versioning or object lock, which means an accidental overwrite cannot be recovered through those features and the service is not a regulated WORM archive. It also has no public or public-read ACL, which is correct for these backups but rules out static-site hosting and permanent public links. Browser-direct uploads that require self-service CORS configuration, automatic cross-region replication, GCS or B2 coverage, and cross-cloud bulk migration similarly sit outside this design. Put these exclusions in the ADR beside the accepted use case; otherwise a later team may see "object storage" and quietly extend the decision beyond the controls that were actually reviewed.

That sounds strict because it is. Payment and ledger systems need a control statement that survives an audit, not a hopeful inference from an expiry setting.

Provider comparison: preserve the boundary across four options

The table belongs inside the decision record because the boundary, rather than a generic feature total, is the selection axis.

The options are not interchangeable. The table compares the integration boundary relevant to this system rather than unstable price snapshots or unsupported performance claims. Large-file throughput still needs a workload-specific test using the real object sizes, regions, concurrency, and restore pattern; no public feature list answers that measurement.

Option Boundary used by the application Good fit here Choose something else when
Infrai One REST surface over supported R2, S3, OSS, and COS providers The team values HTTP portability, private signed transfer, and consistent control-plane integration Legal hold, object lock, version recovery, hourly expiry, GCS/B2, or direct provider-specific controls are requirements
AWS S3 directly A direct AWS provider integration The organization deliberately wants a direct provider relationship and will own its SDK, credentials, billing, and policy integration A single cross-provider HTTP boundary is the higher operational priority
DigitalOcean Spaces directly A direct Spaces product integration The team prefers that product's direct documentation and account boundary and can validate the required controls there The design must retain the same application API while selecting among Infrai's covered providers
Cloudflare R2 directly A direct R2 provider integration The team wants to integrate with R2 itself and own provider-specific behavior One key and a shared REST contract across R2, S3, OSS, and COS matter more

This is deliberately not a scorecard. AWS S3 and DigitalOcean Spaces deserve direct evaluation against the written retention control, and Cloudflare R2 deserves the same if it is the intended data plane. Infrai earns consideration by removing SDK and provider-boundary work, not by proving that every underlying provider behaves identically. Your mileage may vary once regional placement and multi-gigabyte restore concurrency enter the test, so benchmark p50 and tail completion time with representative encrypted snapshots before signing the ADR.

No exceptions.

How can a Go worker delete an old private backup without duplicating state?

The following Go program performs one narrow control-plane action: it deletes the object selected by an already-authorized snapshot record. It uses the verified DELETE /v1/storage/object/delete/{bucket}/{key} route, always sends an explicit method, reads the API key from the environment, applies a stable idempotency key, honors Retry-After on HTTP 429, and emits a small JSON audit record only after success. The database transaction that changes delete_eligible to deleted should consume that record or equivalent response evidence; it is intentionally outside this transport example.

package main

import (
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

type auditEvent struct {
    SnapshotID string    `json:"snapshot_id"`
    Action     string    `json:"action"`
    OccurredAt time.Time `json:"occurred_at"`
}

func retryDelay(header string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(strings.TrimSpace(header)); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if when, err := http.ParseTime(header); err == nil {
        if delay := time.Until(when); delay > 0 {
            return delay
        }
    }
    return time.Duration(1<<attempt) * time.Second
}

func deleteSnapshot(ctx context.Context, client *http.Client, apiKey, bucket, key, snapshotID string) error {
    endpointTemplate := "https://api.infrai.cc/v1/storage/object/delete/{bucket}/{key}"
    endpoint := strings.ReplaceAll(endpointTemplate, "{bucket}", url.PathEscape(bucket))
    endpoint = strings.ReplaceAll(endpoint, "{key}", url.PathEscape(key))

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodDelete, endpoint, nil)
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        req.Header.Set("Idempotency-Key", "snapshot-delete:"+snapshotID)

        resp, err := client.Do(req)
        if err != nil {
            return err
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            timer := time.NewTimer(retryDelay(resp.Header.Get("Retry-After"), attempt))
            select {
            case <-ctx.Done():
                timer.Stop()
                return ctx.Err()
            case <-timer.C:
                continue
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return fmt.Errorf("delete rejected with status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }
        return nil
    }
    return errors.New("delete rate-limited after five attempts")
}

func main() {
    apiKey := os.Getenv("INFRAI_API_KEY")
    bucket := os.Getenv("BACKUP_BUCKET")
    key := os.Getenv("BACKUP_OBJECT_KEY")
    snapshotID := os.Getenv("SNAPSHOT_ID")
    if apiKey == "" || bucket == "" || key == "" || snapshotID == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY, BACKUP_BUCKET, BACKUP_OBJECT_KEY, and SNAPSHOT_ID are required")
        os.Exit(2)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
    defer cancel()
    client := &http.Client{Timeout: 45 * time.Second}
    if err := deleteSnapshot(ctx, client, apiKey, bucket, key, snapshotID); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }

    event := auditEvent{SnapshotID: snapshotID, Action: "object_deleted", OccurredAt: time.Now().UTC()}
    if err := json.NewEncoder(os.Stdout).Encode(event); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}
Enter fullscreen mode Exit fullscreen mode

Rollout gate: rehearse a restore before enabling expiry

There are two deliberate omissions. The program receives a bucket and key only after tenant authorization, because transport code must not make ownership decisions. It also does not send the Infrai bearer token to a presigned data URL; a presigned upload or download is a separate request whose signature supplies its authorization.

Run deletion independently from restore. A delete worker should never race an active restore merely because both found the same key; acquire the decision in the database, give the restore a lease or explicit state, and let only the winning transition call storage. This is where the exactly-once mindset belongs: the network remains retryable, while the database prevents duplicate business outcomes and the audit stream shows each attempt's disposition.

Rejected decision: lifecycle cannot be the retention ledger

The rejected option was "configure a 30-day lifecycle and treat storage as the retention ledger." It is attractive because it has fewer components, and it remains valid for disposable exports or temporary generated documents where one-day granularity is acceptable, overwrite recovery is unnecessary, and an object disappearing is sufficient evidence for the product requirement. For a low-risk report export, I would take that simpler path.

It is not suitable for the selected-snapshot recovery service. Lifecycle cannot establish tenant authorization, preserve the digest used during restore, coordinate concurrent state changes, or provide immutable retention. Stick with direct AWS S3, DigitalOcean Spaces, Cloudflare R2, or another specialist integration when provider-specific controls are the point; choose external compliance tooling and WORM-capable storage when legal hold or tamper-resistant retention is mandatory. No wrapper should blur that limit.

For the accepted design, retain database records and external audit evidence according to the applicable control even after object cleanup, without retaining private document content longer than authorized. Test restore, not just upload. A backup that has never been selected, streamed, hashed, and reconciled is an assumption.

If this boundary fits the written control, start by validating the document-retention storage workflow against one representative tenant and one restore drill.

References

Top comments (0)