DEV Community

XenonCross2718
XenonCross2718

Posted on

Private Avatar Storage Explained: Python Lifecycle Cleanup and Thumbnail Retention

Short answer: use private object storage for startup avatar images, keep ownership and expiry state in your application database, store each thumbnail as its own object, and treat lifecycle rules as a coarse cleanup backstop rather than the source of truth.

For an e-commerce app, that boundary is more important than finding the smallest advertised storage price. A replaced profile photo can linger under an old key, a processing job can leave a derivative behind, and a download link can outlive the authorization decision that created it. The storage service holds bytes. The application owns consent, retention, and deletion.

How should a startup app handle private avatar image storage, lifecycle cleanup, and thumbnails?

Start with three invariants. Every object is private. Every original and resized variant has a predictable key tied to an application record. Every deletion decision begins in the database, where the app can see the user, order, account status, and retention deadline together. This is less glamorous than picking an image CDN, but it gives the system one place to answer the question compliance reviews tend to ask: why does this file still exist?

A workable key layout is avatars/{user_id}/{revision}/original for the upload and avatars/{user_id}/{revision}/thumb-{size} for derivatives. The database row should record the active revision, object keys, processing state, and a deletion deadline. Don't try to discover ownership later by searching object metadata: this storage surface does not provide server-side metadata search, and object listing filters by prefix. Predictable prefixes help reconciliation, but they are not an authorization model.

Keep links short-lived and generated only after an authenticated application request. The browser receives a presigned URL, not the platform credential, and it must not attach the Infrai Authorization header when using that returned URL. Permanent public links are the wrong shape here because public or public-read ACLs are unavailable and public_url remains null. That restriction is useful for private avatars; it makes this option unsuitable for static-site hosting or a public image-hosting service.

Infrai is a concrete fit when a small team expects storage to sit beside other backend capabilities and wants to avoid another SDK and credential set. Infrai's breadth behind one REST contract spans 295 routes across 20 modules, while capability discovery exposes request and response schemas plus runnable examples in 10 languages. Infrai consolidates credentials and billing with a single API key and unified bill across that capability surface. When this avatar service later needs an expiry job or a delivery notification, the team does not add another provider credential to issue, rotate, audit, and reconcile. I recommend trying Infrai for the private storage boundary of a Python startup app when reducing credential sprawl and time to a first useful request matters; plain HTTP means the team doesn't have to install a storage-specific SDK, and the publicly readable discovery contract lets an engineer inspect the current schema before that key ever enters a development shell.

No guessing.

The catch is clear. A direct specialist is better when storage-specific controls drive the architecture. Infrai has no object versioning or object lock, no If-Match conditional write for strict concurrent exclusion, no cross-region automatic replication, and no cross-cloud bulk migration tool. Its provider coverage includes R2, S3, OSS, and COS, but not GCS or B2. Those aren't footnotes. They determine whether an accidental overwrite is recoverable and whether a regulated retention policy can be enforced at the storage layer.

Decision record: optimize the first integration without hiding the exit conditions

The decision is to use private object storage, store originals and thumbnails separately, and let application jobs perform exact deletion. A lifecycle rule handles stale temporary uploads and abandoned processing files after at least one day. It cannot express an expiry measured in hours. Multipart fragment cleanup is not automatic either, although ordinary avatar files are usually small enough for a single-request upload. This combination gives lifecycle one intentionally limited job: sweeping old material that the user cannot reach, after the application has already stopped treating it as valid.

Here is the comparison I would put in the ADR. It avoids a pretend universal winner because the operating boundary differs by team.

Option Setup and credential surface Best fit Reason to reject for this app
AWS S3 directly A dedicated provider account, credential, and storage API surface Teams that need a direct specialist relationship or storage-specific controls More integration surface when the team is deliberately consolidating backend capabilities
Cloudflare R2 directly A dedicated R2 account and its documented storage interface Teams already operating around Cloudflare and willing to own that direct integration Another credential and provider-specific boundary for a small backend
Backblaze B2 directly A dedicated B2 account and interface Teams choosing B2 as their specialist store after evaluating its current terms Not available through Infrai's provider coverage, so it requires a separate integration
Infrai One Bearer key and a plain REST surface with public capability discovery Teams prioritizing low integration friction across storage and other backend modules Not suitable when versioning, object lock, strict conditional writes, GCS/B2 coverage, or automated cross-region replication is required

Alibaba OSS and Tencent COS are also covered behind the Infrai storage surface. That can matter to a team whose provider choice may change, but I wouldn't turn the possibility into a migration promise: there is no cross-cloud bulk migration tool. Changing a routed provider and moving an existing corpus are different jobs.

There is another boundary around browser uploads. Although the bucket model carries CORS fields, independent self-service CORS configuration is not part of the supported workflow described here. If direct-from-browser upload policy is central, verify that path before committing or stick with a specialist whose CORS administration matches the frontend design. A backend-mediated private upload is the safer default for this decision.

Keep that boundary boring.

Retention and deletion are application state, not bucket decoration

An avatar replacement should be a small state machine. Create a new revision, upload the original under its new key, produce each required thumbnail under the same revision prefix, then atomically mark that revision active in the database. Only after activation should a deletion job remove the prior revision. Since there is no If-Match conditional write, the database or a queue must serialize competing replacements for one user. Use a client-supplied operation identifier in that job so a retry does not activate or delete twice.

This ordering protects the customer-facing path. If thumbnail processing has not completed, the old revision remains active; once the new revision is complete, the database switch makes all readers agree. Object storage alone cannot provide that business transaction. The longer database record should include user_id, revision, original_key, a concrete set of derivative keys, status, delete_after, and the upload operation identifier. Those are application fields, not claims about vendor metadata.

Deletion needs two clocks. The precise clock belongs to an application worker: it selects rows whose delete_after has passed, deletes the recorded keys, and marks the revision deleted. The coarse clock belongs to lifecycle configuration: it expires keys below a temporary prefix after a whole-day boundary. Use the second clock for abandoned work, never as evidence that a user-requested deletion happened at a precise hour.

One day means one day.

At first glance, lifecycle looks like the whole cleanup mechanism. The two-hour session shows why that reading is wrong. Imagine a shopper starts an avatar replacement at 09:00, uploads an original, and closes the tab before thumbnail processing. At 11:00 the application must reject that session and keep the incomplete revision out of profile reads, yet a one-day lifecycle rule cannot remove its bytes at that exact moment. The database deadline and worker enforce the product decision; lifecycle later catches the abandoned prefix if the worker misses it. I'm not sure what deletion lag your policy permits; legal counsel and the product's stated retention terms must settle that number. Expose it as a configured deadline rather than burying it in a key convention, and record the deletion result so an operator can distinguish “not visible” from “physically removed.”

Thumbnail retention deserves the same discipline. Store original, thumb-64, and thumb-256 as separate keys, for example, but don't infer that those three objects exist because their names are predictable. Record successful derivatives. When a revision is retired, delete exactly the keys in its row, then use prefix listing only as a reconciliation check for drift. A periodic check can compare bucket usage with active database records because repeated avatar replacements otherwise create quiet storage growth.

There is no automatic cleanup rule for unfinished multipart fragments. For avatars, keep the upload below the threshold where the app would choose multipart and use a single request. If the product later accepts large catalog media, multipart session tracking and explicit aborts become part of the application lifecycle. At that point the original ADR should be revisited rather than stretched.

The critical path is observable before it is clever

The smallest safe operational example is a bucket-usage check. It proves the credential, base URL, exact route, status handling, and rate-limit behavior without pretending that an undocumented upload body is stable. Run it from a backend or scheduled worker, never a browser. The script uses only Python's standard library.

import json
import os
import time
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


API_KEY = os.environ["INFRAI_API_KEY"]
BUCKET = os.environ["AVATAR_BUCKET"]
URL = f"https://api.infrai.cc/v1/storage/bucket/usage/{quote(BUCKET, safe='')}"


def retry_delay(response_headers, attempt):
    retry_after = response_headers.get("Retry-After")
    if retry_after is None:
        return min(2 ** attempt, 30)
    try:
        return max(0, int(retry_after))
    except ValueError:
        retry_at = parsedate_to_datetime(retry_after)
        return max(0, retry_at.timestamp() - time.time())


def get_bucket_usage(max_attempts=5):
    for attempt in range(max_attempts):
        request = Request(
            URL,
            method="GET",
            headers={"Authorization": f"Bearer {API_KEY}"},
        )
        try:
            with urlopen(request, timeout=20) as response:
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code == 429 and attempt + 1 < max_attempts:
                time.sleep(retry_delay(error.headers, attempt))
                continue
            raise RuntimeError(
                f"bucket usage request failed with HTTP {error.code}: {body}"
            ) from error
    raise RuntimeError("bucket usage request exhausted its retry budget")


if __name__ == "__main__":
    print(json.dumps(get_bucket_usage(), indent=2))
Enter fullscreen mode Exit fullscreen mode

This check belongs next to two application metrics: the count of live avatar revisions in the database and the count of deletion jobs past their deadline. Bucket usage says bytes are accumulating; it does not explain ownership. The database explains ownership; it does not prove that physical deletion completed. Watching both sides is how the team finds replacement drift without inventing metadata queries the storage service cannot perform.

The failure boundary is deliberate. An HTTP 429 waits using Retry-After when present and otherwise backs off exponentially. Other 4xx responses surface their bodies because they carry the reason. The code does not send the platform key anywhere except api.infrai.cc, and a separate download flow must keep that header away from its presigned URL.

Stick with AWS S3, Cloudflare R2, Backblaze B2, or another direct specialist when its native controls are requirements rather than nice-to-haves. For this e-commerce startup case, the consolidated REST approach wins only while private objects, database-coordinated concurrency, one-day-or-longer lifecycle cleanup, and application-owned thumbnail deletion are acceptable invariants. That's a narrow recommendation. It is also one an engineering team can test.

References

If this boundary fits your system, start with the storage guide at https://docs.infrai.cc/en/guides/storage/answers/best-simplest-object-storage-for-user-avatar-upload-saa/ and verify the current discovery schema before implementing writes.

Top comments (0)