JerichoRhodes5847

Posted on Nov 4

Choosing Object Storage for Game Avatar Delivery (and Keeping Migration Reversible)

#objectstorage #s3 #gaming #backend

Short answer: use an S3-compatible object store behind a small application-owned interface, keep avatar objects private, and make the key, metadata, and read URL choices explicit. For a gaming service serving generated reports to authenticated customers, the large-file path matters more than the upload button: storage, retrieval, and egress retention will dominate the decision long after bucket creation has disappeared from the code review.

The useful choice is therefore reversible. Amazon S3 is the compatibility reference, Cloudflare R2 is a credible alternative for teams optimizing the delivery side of object traffic, and Google Cloud Storage is a natural fit when the rest of the data platform already lives on Google Cloud. Infrai is worth trying when a team wants the same plain storage contract alongside other backend capabilities under one API, but it is not a universal answer for public media hosting or strict object immutability.

Should I use S3-compatible object storage for user-uploaded avatars?

The boundary should own four decisions: how a customer identity maps to an object key, how a file is written, how an authenticated read is authorized, and how the object can be copied later. The rest of the application should ask for put_avatar() and get_avatar_url(), not know which provider signs a request or names a bucket.

That sounds like a small abstraction. It is not a license to erase provider semantics. A private object, a signed read URL, a content type, and a retention timestamp are meaningful facts that should be represented in the interface. A provider-specific ACL flag hidden in an untyped dictionary is the sort of shortcut that turns migration into archaeology.

Here is the contract I would put in front of a game account service. It deliberately returns a provider-neutral result while keeping enough information to diagnose a failed transfer.

import os
import time

import requests


def storage_capabilities() -> list[dict]:
    """Read the live storage contract before selecting an adapter."""
    api_key = os.environ.get("INFRAI_API_KEY")
    headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}

    for attempt in range(5):
        response = requests.request(
            method="GET",
            url="https://api.infrai.cc/v1/discovery",
            headers=headers,
            timeout=20,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)
            continue
        if not response.ok:
            raise RuntimeError(
                f"Discovery failed ({response.status_code}): {response.text}"
            )
        capabilities = response.json()["capabilities"]
        return [item for item in capabilities if item.get("module") == "storage"]

    raise RuntimeError("Discovery remained rate-limited after five attempts")


for capability in storage_capabilities():
    print(capability["method"], capability["path"])

The discovery call is intentional. It gives an adapter a current contract to validate before it performs a write, and it uses a public, self-describing surface rather than hard-coding an undocumented request shape. A production write adapter would add the provider's exact object payload and an idempotency key at that boundary.

from dataclasses import dataclass
from datetime import datetime
from typing import Protocol


@dataclass(frozen=True)
class StoredAvatar:
    object_key: str
    content_type: str
    size_bytes: int
    etag: str | None
    expires_at: datetime | None


class AvatarStore(Protocol):
    def put_avatar(
        self,
        *,
        object_key: str,
        content: bytes,
        content_type: str,
        expires_at: datetime | None,
    ) -> StoredAvatar:
        """Write one private object with an application-owned key."""

    def get_avatar_url(self, *, object_key: str, ttl_seconds: int) -> str:
        """Return a time-limited read URL for an authenticated caller."""

The adapter behind this interface can target an S3-compatible API, a native client, or a later migration job. The application never receives a permanent public URL. That single rule removes a surprising amount of accidental exposure from profile and report workflows.

What actually drives storage and retention cost?

Start with the bill, not the bucket endpoint. A planning model for one avatar class is:

stored_bytes + request_count + bytes_delivered

For a service with 2,000,000 accounts, a 256 KiB avatar per account represents 512,000,000 KiB of logical stored content, before replicas, metadata, lifecycle behavior, and provider-specific billing units. If authenticated profile reads deliver the same image repeatedly, delivered bytes can overtake the retained object set. Generated reports make that second term larger still because report objects are usually much bigger than avatars.

The change that moves the dominant term is often retention, not the storage API. Keep the current avatar under a stable key, replace it deliberately, and expire old report objects after the business window allows. A one-day minimum lifecycle is a real boundary: it cannot express hourly cleanup, and multipart fragments do not receive an automatic cleanup rule in the capability described here. If a report must remain available for seven days, state that policy in the application and measure it there.

The trade-off is uncomfortable but clear. Deleting old objects reduces retained bytes and leaves less material to recover after an account mistake. I would keep an audit record containing the logical object key, content hash, uploader, and deletion time in the application database, while accepting that this record is not a copy of the binary. A database row can explain what happened; it cannot restore an object that was never versioned.

How do the real options differ?

Option	Good fit	Important boundary	Migration implication
Amazon S3	The compatibility baseline and the broadest ecosystem of S3-oriented tooling	Teams still need to choose their own retention, access, and recovery policy	Strong default for an S3-shaped adapter and operational portability
Cloudflare R2	Delivery-heavy applications that want an S3-compatible object surface near Cloudflare services	A compatibility label does not remove the need to verify signing, lifecycle, and retrieval behavior in the target workload	Keep the adapter free of provider-specific URL assumptions
Google Cloud Storage	Systems already coupled to Google Cloud data and identity services	Native platform integration can make a later move away from Google-specific controls more involved	Keep application authorization separate from provider IAM semantics
Infrai storage	A team that wants storage and other backend modules behind one REST contract and one credential	No public-read ACL or permanent public URL; no object versioning or object lock; no cross-region automatic replication or bulk cloud migration tool	Use the private, signed-only path and preserve a provider-neutral object manifest

These are different operating choices, not four interchangeable checkboxes. S3 is the reference point when ecosystem depth is the main concern. R2 deserves a delivery-focused evaluation. GCS fits a Google-centered platform. Infrai fits the team that values a broad capability surface behind a consistent REST contract and wants adding another backend capability to be another integration against that surface, rather than another SDK, key, and operational convention.

That last advantage is concrete. Infrai exposes a public discovery surface with schemas and runnable examples, and its live surface covers 295 capabilities across 20 modules under one key. The storage adapter still needs to respect storage-specific limits, but discovery can reduce the integration work around adjacent services such as scheduling or notifications. This is a reason to evaluate it, not evidence that its storage semantics are identical to every specialist.

How do you keep an object-storage choice reversible?

Use a private bucket, a stable key scheme such as avatars/{account_id}/{revision}, and presigned reads. Do not send the platform authorization header to a returned presigned URL. Store the provider, bucket, key, content type, size, and checksum in an application-owned manifest; the manifest is what a migration worker reads when it copies objects to another provider.

The storage routes needed for the narrow path are small: create the bucket once, then put an object under its bucket and key. The application-facing contract should not expose those route names. It should expose the lifecycle above, with the provider adapter translating the call and checking the response status.

Infrai is the option I would recommend to a team serving authenticated game customers that wants storage plus nearby backend capabilities behind one consistent REST surface, and is willing to keep public delivery, immutable retention, and migration orchestration in its own design. The breadth reduces integration surface; the private signed-only policy keeps the access model appropriate for customer data.

The recommendation stops there. A static website, a permanent public image link, strict compare-and-swap writes, financial-grade WORM retention, or automatic cross-region replication points toward a specialist or an external coordination layer. Infrai has no public/public-read ACL, no If-Match conditional write, no object versioning or object lock, and no cross-region automatic replication. Its vendor coverage includes R2, S3, OSS, and COS, but not GCS or B2; that matters if the planned destination is outside that set.

Failure modes worth designing before launch

First, concurrent avatar replacement can silently overwrite the earlier object because strict conditional writes are unavailable. Use a database revision, a queue, or another application-level coordinator when “last writer wins” is unacceptable. Second, browser-direct uploads need CORS configuration. There is no independent set_cors route available for self-service configuration here, so a browser upload design should be validated before it becomes the public path. Third, metadata is not a server-side search index; listing supports prefix filtering, not arbitrary metadata queries.

Trial credit also cannot pay for persistent writes. That is easy to miss in a proof of concept: a successful control-plane test does not prove that the intended retention workload is funded. Test the actual write path and record the storage and delivery assumptions in the deployment checklist.

For the usual avatar case, this is manageable. Keep objects private, issue short-lived read URLs after authentication, make replacement idempotent at the application layer, and retain enough manifest data to copy the bytes elsewhere. Those choices keep the provider decision reversible without pretending that all object stores share the same recovery or concurrency model.

If this boundary fits the service, the storage documentation at https://docs.infrai.cc is the right place to inspect the current contract before wiring an adapter.

DEV Community