DEV Community

JorisRhodes8286
JorisRhodes8286

Posted on

Marketplace Avatar Upload Overwrite — Old File Storage Rollback for Signed-Document Deadline Deletion

Short answer: don't overwrite the old file key; upload a new object, verify it, atomically switch the pointer, and delete the predecessor only after rollback and retention checks pass.

The hard part of replacing an avatar is not writing the new bytes. It is deciding when the old object is no longer needed, especially when the same upload path also holds signed marketplace documents with a contractual deletion deadline. My rule is simple: write a new immutable key, switch the database pointer only after a verified upload, and delay deletion of the previous key until the rollback window and retention policy both allow it.

That ordering keeps access control explicit without making delivery fragile. A failed upload leaves the current avatar alone; a successful replacement has a recorded predecessor; a scheduled deletion can prove why an object is being removed.

Keep it explicit.

Start with the retention constraint, not the bucket

An object store is a byte-addressable system, not a transaction log for your marketplace. An overwrite at avatars/user-42 can hide the previous bytes before your application has recorded a recovery point. Object versioning can help, but it does not answer who may read a version, how long it must remain, or which signed document is allowed to disappear on a deadline.

Model those decisions in your database. Store an object key, content digest, media type, size, uploader, and delete_after timestamp. For a signed document, also store the signature or envelope identifier and the legal retention reason. The key should be opaque and immutable, such as tenant-7/doc-981/rev-03, rather than a user-controlled filename.

The deletion worker should be boring. It selects records whose deadline has passed, checks that no hold or active dispute exists, then deletes the exact key and records the result. A retry must be idempotent: deleting an already-absent object is a completed state, while deleting a newly referenced key is a safety violation that the database check must prevent.

How should avatar upload, overwrite, rollback, and old-key deletion work?

Use a two-phase application flow. First create a pending revision and upload to a fresh key. Then verify the object metadata and digest, and atomically move the user’s current_key pointer. Only after that transaction succeeds should cleanup become eligible.

from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from hashlib import sha256


@dataclass
class Upload:
    key: str
    digest: str
    size: int


def replace_avatar(store, db, user_id: str, payload: bytes) -> str:
    digest = sha256(payload).hexdigest()
    revision = db.create_pending_revision(user_id, digest=digest)
    new_key = f"avatars/{user_id}/{revision.id}"
    store.put(new_key, payload, content_type="image/*")

    head = store.head(new_key)
    if head.size != len(payload) or head.digest != digest:
        store.delete(new_key)
        db.fail_revision(revision.id, reason="metadata mismatch")
        raise ValueError("uploaded object did not verify")

    old_key = db.promote_revision(revision.id, new_key=new_key)
    if old_key:
        db.schedule_cleanup(old_key, not_before=datetime.now(timezone.utc) + timedelta(hours=24))
    return new_key
Enter fullscreen mode Exit fullscreen mode

The 24-hour delay is an example policy, not a universal number. Pick it from your support response time, moderation workflow, and document hold rules. The important property is that rollback has a real predecessor to point at. If the marketplace needs immediate legal erasure, that policy outranks a convenience rollback and the old key must be removed after the required audit record is written.

Multipart upload adds another failure surface. A client can abandon an upload after several parts; configure lifecycle cleanup for incomplete multipart uploads and expose an upload identifier to your worker. AWS documents this lifecycle concern in its multipart overview, and the same principle applies to other object stores.

Access control is a separate decision from delivery

Public avatar delivery and signed-document access should not share a policy merely because both use object storage. Avatars can be served through a cache with a short-lived URL. Signed documents should use private objects, narrowly scoped signed requests, and an audit trail containing actor, purpose, and expiry.

The application should authorize before it generates a download URL. Checking authorization only at upload time is a common leak: an old document key may remain valid in logs, caches, or a copied link. Keep the object key out of user-facing identifiers where practical, and set response headers that prevent browsers from treating a document as executable content.

I've seen teams chase a delivery timeout by widening a bucket policy. That fixes the symptom and quietly changes the threat model. In messaging systems I have also chased 429 rate-limit responses; the lesson carries over here: a retry queue, bounded timeouts, and a dead-letter record are safer levers than making every object readable. Your mileage may vary with provider defaults, so verify them in a staging account before setting a retention promise.

What do storage services trade off for versioning and deletion?

The major services expose similar primitives with different operational details. Amazon S3 supports multipart uploads and optional versioning; Google Cloud Storage offers object generations; Azure Blob Storage offers blob versioning and snapshots. None of these features replaces application metadata for retention deadlines or authorization. Their lifecycle syntax, replication behavior, minimum billing units, and restore semantics differ, so test the exact provider configuration you operate.

For a marketplace, compare these capabilities:

Decision Ask before adopting it
Immutable keys Can a database transaction identify the exact predecessor and successor?
Native versioning Can operators list, authorize, and permanently delete a specific version?
Lifecycle rules Do rules cover incomplete multipart uploads and expired noncurrent versions?
Restore path Can a support engineer roll back without guessing a key?
Evidence Will access and deletion events satisfy your audit requirement?

Native versioning is useful when recovery is frequent or operators need provider-level protection. It is less suitable when legal erasure must guarantee that old versions, replicas, and derived thumbnails are removed on a defined schedule. Immutable application keys are easier to reason about, but they create more metadata and cleanup work.

Roll out the policy with observable, reversible steps

Begin by writing the state machine: pending, verified, current, superseded, held, and deleted. Add metrics for verification failures, orphaned objects, cleanup age, and denied downloads. Alert on a cleanup backlog and on any object referenced by two tenants.

Then test the awkward cases: duplicate client retries, a database commit timeout after a successful put, a worker crash between promotion and scheduling, and a deletion deadline that arrives during a legal hold. One particularly expensive class of mistake is a commit timeout after the storage write: the client retries, creates a second pending revision, and a cleanup job later sees one key as orphaned even though a delayed transaction will reference it. Give each request an idempotency key, reconcile pending rows before purging them, and retain the object listing needed to explain the decision. Property-based tests are handy for asserting that current_key never points to a deleted object. In production, sample object listings against database rows and quarantine anything unreferenced before deletion.

The catch is operational cost. This design is not suitable when you need a single flat namespace with no metadata store or when every upload must be overwritten in place for an external protocol. In those cases, keep provider versioning enabled and document who can restore or purge versions; do not pretend an overwrite gives you rollback. Stick with immutable revisions when access control and deadline deletion matter more than minimizing object count.

References

Top comments (0)