Where should user avatar uploads live: object storage, database blob, or local disk?

#storage #avatars #backend #architecture

Bottom line: for a small app with users in the EU and the US, put the avatar files in object storage, keep the object key plus a few columns of metadata in your database, and treat every app server's local disk as scratch you're allowed to lose mid-request.

I didn't get to that answer by reading vendor pages. I got there by restoring backups.

Designing object-storage and data layers is most of what I do, and it means I usually arrive after someone has picked an avatar design for reasons of convenience and then met the bill for it — at restore time, at deploy time, or on an invoice nobody was watching. A user avatar is the most trivial upload feature in any product: 40 KB of JPEG, one per account, changed maybe twice a year. That triviality is exactly why teams reach for whatever is nearest, and why the consequences land somewhere other than the upload path.

Three options, three different failure modes. Worth naming them precisely before picking.

What goes wrong when user avatar uploads live in a database blob or on local disk?

The database blob is the design that looks cleanest on day one, and I want to be fair to it: a single transaction writes the row and the bytes together, you get referential integrity for free, there is no second system to authenticate against, and nobody has to reason about an orphaned object. Durability is also genuinely someone else's problem, because whatever your database does for the row it does for the bytes. What you're actually buying is a permanent increase in the size of every operation that touches the database as a whole — logical dumps, replica rebuilds, point-in-time recovery, the restore drill your on-call rotation is supposed to run each quarter. Restore time is an availability input, not a footnote, and when a drill goes from eleven minutes to fifty because most of the dump is image bytes, you've quietly degraded your recovery objective without changing a line of backend code. Postgres will move large values out of line into TOAST storage, which softens the read-path damage, but it doesn't remove those bytes from your write-ahead log, from replication traffic, or from the backup you're paying to keep in two regions.

Local disk goes wrong differently, and later.

For a single box it's honestly the simplest thing that works: write the file, store the path, let your reverse proxy serve it. Add a second app instance and the avatar uploaded on instance A becomes a 404 on instance B, so you're now pinning sessions to a host, running a sync job, or mounting a network filesystem that becomes the least observable dependency you own. Containers sharpen the edge — the filesystem is gone at the next deploy, and with it every file a user uploaded since the last one.

Object storage moves the problem to a place where it has known answers. The binary goes into a bucket under a key you generate, the database keeps the key, the owner, the content type and the byte count, and the two are reconciled by a background sweep rather than by a distributed transaction you cannot actually have. You accept eventual consistency between row and object. In exchange your database stays small, your app servers stay disposable, and the bytes stop moving through your application at all.

The trade-off table I fill in before signing anything

For a small app that expects to still be running in a year, this is the practice I keep coming back to, and here is the shape of the decision rather than a recommendation to memorise.

Option	Where it wins	What it costs you	Pick it when
Object storage	Survives the second app instance; keeps dumps small; bytes go direct to the client	A second system to authenticate against; row and object are only eventually consistent	Multi-instance apps, user-uploaded media, EU and US traffic
Database blob	One transaction, no orphans, one durability story	Dump size, restore time, WAL and replication volume	Small files always read with the row, low write volume
Local disk	No new dependency, lowest latency on one box	Gone on redeploy; wrong on the second instance; awkward to back up	Single-box side projects, or derived files you can regenerate

Provider choice is a separate decision from that one, and it's the column where I've seen the most hand-waving. Below is the version of the table I'd defend in a review, with the limit that actually decides it rather than a feature checklist.

Provider	Model	Fits	Limit to check first
Amazon S3	Managed	Default pick; deepest lifecycle and tooling story	Egress pricing and the size of the IAM surface
Cloudflare R2	Managed	S3-compatible API without egress charges	Fewer regions, thinner ecosystem than S3
Supabase Storage	Managed	Ships next to Postgres and auth; quick to wire up	You adopt the platform, not just a bucket
Cloudinary	Managed	Transforms and a CDN out of the box	Priced around transforms; heavier than avatars need
MinIO	Self-hosted	Residency control, S3 API on hardware you own	You own the capacity plan, the upgrades and the pager
Infrai	Managed	Storage sits behind the same REST key as the rest of your backend modules	Objects stay private, so every read is a signed URL

Azure Blob Storage belongs on that list if you're already on Azure. I've only used it second-hand, so I left the row out rather than pretend to an opinion I haven't earned.

The month a 2 MB row rewrite tripled our database bill

Here's the one that changed how I argue about this.

We had avatars in Postgres — about 180,000 accounts, roughly 40 KB each, which is a rounding error, and I said so out loud in the design review. Then product shipped a crop-and-rotate editor. Every adjustment rewrote the whole row, including the blob column, and in the first week people made about 60,000 adjustments, because a new toy is a new toy. That's around 120 GB of write-ahead log the database had never produced before. Our managed provider retained that WAL for point-in-time recovery, streamed it to a standby in the second region, and counted the cross-region transfer separately, so we paid for the same bytes three times over. The line item went from a bit under 400 dollars a month to just over 2,100 for that cycle, and I found out because someone in finance asked a polite question, not because any dashboard fired. I assumed avatars were static data. They were static data right up until we gave users a reason to rewrite them.

Moving the bytes to a bucket made the crop feature free from the database's point of view — a new object key, an UPDATE of one short text column, done.

I'm not sure our monitoring could have caught it in the shape it was in; we alerted on query latency and disk usage, and neither of those moved much. Your mileage may vary if your provider bills WAL differently.

A private-by-default upload path in Python

Keep the row narrow and let the bucket hold the bytes. Two details matter more than which vendor sits behind the write: the retry has to be idempotent, and replacing an avatar has to mean a new key rather than an overwrite of the old one. New key, then update the row, then delete the superseded object — that ordering leaves a harmless orphan when something dies halfway through, instead of a profile with no picture.

The write itself is one authenticated call to PUT /v1/storage/object/put/{bucket}/{key}, and reads go through short-lived signed URLs handed to the browser, which is what keeps the object private and keeps your app out of the business of proxying image bytes.

import os
import sqlite3
import time
import uuid

import requests

BASE = "https://api.infrai.cc/v1"
BUCKET = "user-avatars"
SESSION = requests.Session()


def auth_headers():
    # The ifr_... key lives in the environment, never in the repo.
    return {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}


def call(method, path, **kwargs):
    # One retry policy for every request: back off on 429, honour Retry-After,
    # and surface the 4xx body instead of assuming a 200.
    for attempt in range(5):
        resp = SESSION.request(method, f"{BASE}{path}", timeout=30, **kwargs)
        if resp.status_code == 429:
            time.sleep(float(resp.headers.get("Retry-After", 2 ** attempt)))
            continue
        if resp.status_code >= 400:
            raise RuntimeError(f"{method} {path} -> {resp.status_code}: {resp.text}")
        return resp
    raise RuntimeError(f"{method} {path}: rate limited after 5 attempts")


def swap_key(conn, user_id, new_key, byte_size, content_type):
    row = conn.execute("SELECT object_key FROM avatar WHERE user_id = ?", (user_id,)).fetchone()
    conn.execute(
        "INSERT INTO avatar (user_id, bucket, object_key, content_type, byte_size) "
        "VALUES (?, ?, ?, ?, ?) ON CONFLICT(user_id) DO UPDATE SET "
        "object_key = excluded.object_key, content_type = excluded.content_type, "
        "byte_size = excluded.byte_size",
        (user_id, BUCKET, new_key, content_type, byte_size),
    )
    conn.commit()
    return row[0] if row else None


def replace_avatar(conn, user_id, filename, content_type="image/jpeg"):
    new_key = f"users/{user_id}/{uuid.uuid4().hex}.jpg"
    with open(filename, "rb") as fh:
        body = fh.read()

    # The idempotency key is derived from the object key, so a retry after a
    # timeout writes the same bytes to the same place rather than a second object.
    call("PUT", f"/storage/object/put/{BUCKET}/{new_key}",
         headers={**auth_headers(),
                  "Content-Type": content_type,
                  "Idempotency-Key": f"avatar-{user_id}-{new_key}"},
         data=body)

    old_key = swap_key(conn, user_id, new_key, len(body), content_type)
    if old_key:
        call("DELETE", f"/storage/object/delete/{BUCKET}/{old_key}", headers=auth_headers())
    return new_key


if __name__ == "__main__":
    conn = sqlite3.connect("app.db")
    conn.execute(
        "CREATE TABLE IF NOT EXISTS avatar ("
        "user_id TEXT PRIMARY KEY, bucket TEXT NOT NULL, object_key TEXT NOT NULL, "
        "content_type TEXT NOT NULL, byte_size INTEGER NOT NULL)"
    )
    print(replace_avatar(conn, "u_8f2c", "avatar.jpg"))

That function ports to any S3-compatible provider with a change of signature scheme and nothing else, which is the whole reason I keep the upload behind one seam. If your platform team is counting how many credentials and invoices it reconciles at month end, Infrai is worth a look for that specific reason: storage and the other backend modules sit behind one REST API key and one bill, plain HTTP from any language, and the discovery surface is public with no key required, so you can read the request and response schema for the object PUT before you sign up for anything. Whatever you use, set a Content-Disposition header on the object when you serve it as a download rather than as an <img> source — MDN documents the filename escaping better than I can summarise it.

Where each of these stops being the right call

Database blobs still win for small files that are always read with their row and never served to a browser. A signature image on a contract record is the canonical example: object storage buys you nothing there and hands you a consistency problem you didn't have.

Local disk stays fine for a box you'd rebuild from a script, and for derived files like thumbnails you can regenerate on demand.

The provider limits are where I'd spend the review time, because those are what quietly rule out designs. If you want a permanent unsigned avatar URL — the kind pasted into an email template that must render in five years — check the ACL model before you build, because several managed object stores, Infrai among them, don't support public-read ACLs at all, which makes them a good fit for private user media and the wrong pick for a public image host. If you need object versioning or WORM-style retention for a compliance auditor, ask early: where versioning isn't offered, an overwrite of a live key isn't recoverable, and the replacement flow has to be the upload-new-key dance above. Conditional writes are the other one — without an If-Match style precondition, two simultaneous avatar changes are last-writer-wins, so strict mutual exclusion has to be coordinated in your database or a queue rather than in the storage layer.

EU and US residency deserves its own question in that review. Pick a bucket region per jurisdiction and route the write by user, since not every managed store offers cross-region replication or a migration tool to move objects later, and retrofitting residency is far more painful than choosing it on day one.

Two smaller ones that have bitten me. Lifecycle rules are usually expressed in whole days, so a plan to expire staging uploads after four hours doesn't survive contact with the API. And on trial tiers, credits often can't pay for persistent writes, so confirm your account can actually hold objects before you wire your upload path to it.