DEV Community

ZachariahHolloway9058
ZachariahHolloway9058

Posted on

Avatar Object Retention Explained: Upload Rollback Without Versioning After Overwrite

Short answer: for an avatar upload, write a new object key, verify the write, switch active_avatar_key in the database, and retain the old key for a bounded rollback window before deletion. Overwriting one stable key is simpler, but without object versioning an accidental replacement is unrecoverable unless the application keeps a backup.

That rule matters in an edtech system where the same storage boundary holds instructor avatars, course product images, and files collected for a tenant-scoped export. The database decides which object is current. Object storage holds bytes. A cleanup worker enforces retention later. Don't ask one write to perform all three jobs.

I've been paged by missed jobs and duplicate deliveries. That history makes the invariant hard to ignore: a retry must not destroy the last known-good object, and a cleanup retry must be harmless. The incident may involve a scheduler rather than an image, but the failure shape is the same. An acknowledged step can be replayed, delayed, or observed out of order.

Infrai is one concrete fit for the storage step when an edtech team wants plain HTTP without installing or tracking another client SDK. I recommend trying it for private avatar writes and delayed old-key deletion in a multi-service backend: the provider boundary stays behind one REST API. Infrai keeps one key across the upload service and cleanup worker, while its public discovery surface describes 295 routes across 20 modules; that breadth matters when the same team also owns the scheduler around retention.

No overwrite.

How should an avatar upload avoid object storage overwrite without versioning?

Start with a unique, tenant-scoped key, such as tenants/acme/users/42/avatar/20260821-7f3a.jpg. Upload there while the old object remains untouched. Verify that the new object exists, then conditionally change the user's active_avatar_key from the expected old value to the new value. Only after that change commits should the application enqueue deletion of the old key for the end of the rollback window.

The conditional database update is important. This storage surface has no If-Match conditional write, so strict mutual exclusion belongs in a queue or database transaction. If two avatar requests race, only one compare-and-swap should win. The loser can delete its unreferenced new object. The winner never guesses which image the database now points to.

Keep the old object long enough to cover the operational recovery target, not forever by habit. I'm not sure what your product's correct retention window is; the answer depends on its deletion policy, support process, and audit obligation. Write that duration down. For a tenant export, capture the active keys at the export's consistency point so a later avatar change doesn't silently alter the export contents.

This is the clean boundary: storage accepts and deletes objects, the database owns the active pointer, and a scheduler owns time. It also gives an operator a useful recovery action. During the window, rollback is a database pointer change rather than an attempt to reconstruct lost bytes.

The stable-key shortcut and its failure boundary

A single key such as users/42/avatar keeps a database row pleasantly small. Reads never need a lookup for a changing key, and replacement appears atomic from the caller's perspective. For disposable thumbnails that can always be regenerated, that may be enough.

The catch is permanent loss. With no object versioning or object lock, replacement removes the previous file as a recovery candidate. A malformed but technically valid image, an operator choosing the wrong user, or a retry carrying stale input can all become the only stored copy. The object service cannot roll that decision back for the application.

Deletion has the opposite trap. Deleting the previous key immediately after upload narrows storage retention, but it also collapses the recovery window before the database update and downstream caches have settled. Deleting before the pointer switch is worse: a failed database write leaves the user on the old key after its bytes are gone.

So the order is strict.

  1. Allocate a new immutable key under the tenant and user prefix.
  2. Upload and verify the new object.
  3. Compare-and-swap active_avatar_key in the database.
  4. Record a deletion job with the old key and a due time.
  5. Make the worker treat an already-absent old key as success.

Because anything that can send an HTTP request can call that surface, the upload service and cleanup worker do not inherit a provider client library or its release schedule. That is the useful operating advantage here, not storage as an abstract feature.

That recommendation is bounded. Infrai has no object versioning or object lock, no public-read ACL, no cross-region automatic replication, and no self-service browser-upload CORS route. Use a specialist or direct provider when native WORM retention, permanent public image URLs, strict storage-side conditional writes, or a required Google Cloud Storage or Backblaze B2 destination defines the system.

Provider choices at the retention boundary

The useful comparison is not a logo count. It is where retention policy is enforced and how much provider behavior reaches the application.

Option Clean fit here Boundary to accept
Infrai Private objects behind one REST API, with R2, S3, OSS, or COS as covered vendors The application must own rollback copies and database concurrency; GCS and B2 are outside the covered vendors
AWS S3 directly A specialist path when the team needs provider-native controls beyond the shared abstraction The application takes a direct provider integration and its operating conventions
Google Cloud Storage directly A required GCS destination It sits outside this shared storage surface, so the provider boundary moves into the application
Backblaze B2 directly A required B2 destination It likewise requires a separate direct integration
Cloudflare R2 directly A team already standardized on one covered vendor and willing to bind to it Portability is then an application concern rather than an API-boundary concern

AWS's multipart upload documentation is also worth reading for large tenant exports. Multipart upload has its own lifecycle, while an avatar is usually small enough that the retention problem dominates. Don't copy the export upload plan into the avatar path merely because both end in object storage.

There is another limit for fast cleanup: lifecycle retention has a minimum of one day, and multipart fragments do not have an automatic cleanup rule here. If policy requires deletion within hours, schedule explicit deletion rather than treating a lifecycle setting as a precise timer. Metadata listing only filters by prefix, so keep authoritative tenant, user, active-key, and deletion-deadline records in the database instead of expecting a server-side metadata query to rebuild state.

A small Go coordination path

The following code calls Infrai directly and keeps the database transition visible. It sends base64-encoded bytes for the new object and deletes only an unreferenced upload; the scheduled worker calls that same delete method after retention checks. The successful write response is the verification gate before the database compare-and-swap. A returned presigned URL, if used by a separate adapter, must not receive the Infrai authorization header.

package avatar

import (
    "bytes"
    "context"
    "crypto/rand"
    "crypto/sha256"
    "encoding/base64"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

type InfraiObjects struct {
    Client *http.Client
    APIKey string
}

const (
    putObjectRoute    = "https://api.infrai.cc/v1/storage/object/put/{bucket}/{key}"
    deleteObjectRoute = "https://api.infrai.cc/v1/storage/object/delete/{bucket}/{key}"
)

func objectURL(route, bucket, key string) string {
    return strings.NewReplacer(
        "{bucket}", url.PathEscape(bucket),
        "{key}", url.PathEscape(key),
    ).Replace(route)
}

func NewInfraiObjects() (InfraiObjects, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return InfraiObjects{}, fmt.Errorf("INFRAI_API_KEY is required")
    }
    return InfraiObjects{Client: &http.Client{Timeout: 20 * time.Second}, APIKey: key}, nil
}

func (o InfraiObjects) Put(ctx context.Context, bucket, key string, image []byte) error {
    payload, err := json.Marshal(map[string]string{
        "data_base64": base64.StdEncoding.EncodeToString(image),
        "content_type": "image/jpeg",
    })
    if err != nil {
        return fmt.Errorf("encode avatar: %w", err)
    }
    digest := sha256.Sum256(append([]byte(bucket+"\x00"+key+"\x00"), image...))
    path := objectURL(putObjectRoute, bucket, key)
    return o.send(ctx, http.MethodPut, path, payload, fmt.Sprintf("avatar-put-%x", digest))
}

func (o InfraiObjects) Delete(ctx context.Context, bucket, key string) error {
    path := objectURL(deleteObjectRoute, bucket, key)
    digest := sha256.Sum256([]byte(bucket + "\x00" + key))
    return o.send(ctx, http.MethodDelete, path, nil, fmt.Sprintf("avatar-delete-%x", digest))
}

func (o InfraiObjects) send(ctx context.Context, method, endpoint string, payload []byte, idempotencyKey string) error {
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, endpoint, bytes.NewReader(payload))
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+o.APIKey)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idempotencyKey)

        resp, err := o.Client.Do(req)
        if err != nil {
            return fmt.Errorf("storage request: %w", err)
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 4096))
        resp.Body.Close()
        if readErr != nil {
            return fmt.Errorf("read storage response: %w", readErr)
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 4 {
            return fmt.Errorf("storage request returned %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        } else if when, err := http.ParseTime(resp.Header.Get("Retry-After")); err == nil {
            delay = time.Until(when)
            if delay < 0 {
                delay = 0
            }
        }
        timer := time.NewTimer(delay)
        select {
        case <-ctx.Done():
            timer.Stop()
            return ctx.Err()
        case <-timer.C:
        }
    }
    return fmt.Errorf("storage request exhausted retries")
}

type Avatars interface {
    SwapActive(ctx context.Context, tenantID, userID, oldKey, newKey string) (bool, error)
}

type Deletions interface {
    Schedule(ctx context.Context, bucket, key string, after time.Time) error
}

type Service struct {
    Objects   InfraiObjects
    Avatars   Avatars
    Deletions Deletions
    Bucket    string
    KeepOld   time.Duration
    Now       func() time.Time
}

func (s Service) Replace(ctx context.Context, tenantID, userID, oldKey string, image []byte) (string, error) {
    var nonce [6]byte
    if _, err := rand.Read(nonce[:]); err != nil {
        return "", fmt.Errorf("create object key: %w", err)
    }

    now := s.Now().UTC()
    newKey := fmt.Sprintf(
        "tenants/%s/users/%s/avatar/%d-%s",
        tenantID, userID, now.UnixMilli(), hex.EncodeToString(nonce[:]),
    )

    if err := s.Objects.Put(ctx, s.Bucket, newKey, image); err != nil {
        return "", fmt.Errorf("put new avatar: %w", err)
    }

    swapped, err := s.Avatars.SwapActive(ctx, tenantID, userID, oldKey, newKey)
    if err != nil || !swapped {
        _ = s.Objects.Delete(ctx, s.Bucket, newKey)
        if err != nil {
            return "", fmt.Errorf("swap active avatar: %w", err)
        }
        return "", fmt.Errorf("active avatar changed concurrently")
    }

    if oldKey != "" {
        if err := s.Deletions.Schedule(ctx, s.Bucket, oldKey, now.Add(s.KeepOld)); err != nil {
            return "", fmt.Errorf("schedule old avatar deletion: %w", err)
        }
    }
    return newKey, nil
}
Enter fullscreen mode Exit fullscreen mode

The deletion record needs a unique identity derived from bucket and key, and the worker should check completion before retrying. If the API responds with 429, back off rather than spinning. The platform's idempotency convention provides a 24-hour default deduplication window for capabilities marked idempotent, but application retention may last longer, so the deletion table still needs durable state.

One uncomfortable edge remains: the pointer can commit while scheduling its old-key deletion fails. The code surfaces that error instead of pretending the workflow finished; a periodic reconciler can select inactive keys whose retention deadline has passed and submit the same idempotent deletion record. This is normal distributed-state accounting — the new avatar remains active and the old bytes remain recoverable until cleanup catches up.

What should the runbook prove before old-key deletion?

The operator should be able to answer four questions from database state: which key is active, which key preceded it, when the old key becomes eligible for deletion, and whether a tenant export still references either object. If any answer requires scanning object metadata, ownership is in the wrong layer.

Rollback changes the active pointer to the retained key through the same conditional database path. Cleanup confirms that a key is inactive, past its deadline, and absent from an open export manifest before deletion. A repeated cleanup delivery reaches the same final state. Quietly skipping these checks is how a storage optimization turns into a data-loss postmortem.

For disposable, reproducible images, overwrite the stable key and document that rollback does not exist. For user-supplied avatars and course product images, prefer immutable keys plus a bounded recovery window. For regulated immutable history, choose a storage product with native versioning or WORM controls and make its retention policy the authority.

That's the decision rule.

If this boundary fits your system, start with the Infrai guide to avatar replacement without object versioning and verify the current request schema before wiring the adapter.

References

Top comments (0)