Originally published on my blog. Cross-posted here with a canonical link.
Prefer audio? Telegram
Instagram stores billions of photos and videos. Somewhere, a decision was made about where the actual bytes of every one of those files live — inside the database, or in object storage like S3. It's one of the easiest calls to get wrong on day one, and one of the most brutal to undo once you're at scale.
The one-line mental model:
- Blob in the database is keeping the whole parcel inside the filing cabinet — everything is in one drawer, one lock, one place. Convenient until the cabinet is full of bricks.
- Object storage is keeping the parcel in a warehouse and filing only the shelf number — the database stays a thin index, the heavy stuff lives where heavy stuff belongs.
Blobs in the database: everything in one place
The tempting path is to add a bytea (Postgres) or BLOB (MySQL) column and shove the file's bytes straight into the row:
CREATE TABLE photos (
id BIGSERIAL PRIMARY KEY,
user_id BIGINT NOT NULL,
created_at TIMESTAMPTZ DEFAULT now(),
data BYTEA NOT NULL -- the entire JPEG lives here
);
For a weekend project this genuinely works, and it has real appeal:
- One place. One database, one backup, one restore.
- Transactions. The row and its bytes commit or roll back together — no orphaned files, no "the DB says it exists but the file is missing."
- Access control for free. Whoever can read the row can read the file; you already have that logic.
So why doesn't Instagram do this?
Why it falls apart
A relational database is engineered for small, structured rows that it can index, cache, and join. Multi-megabyte media is the opposite of that, and every part of the system pays for it:
- Backups and restores balloon. Every dump and every restore now drags terabytes of image bytes through your most expensive, hardest-to-scale tier. A logical backup that used to take minutes takes hours.
- Reads pull huge binaries through the connection pool. A single request for a 4 MB photo occupies a database connection while megabytes stream out of it. Connections are a scarce resource; you're spending them on plumbing bytes.
- The cache thrashes on pixels. Your database's buffer cache is precious RAM meant for hot indexes and rows. Fill it with JPEG bytes that are read once and it evicts the very data that makes queries fast.
-
No CDN, no easy streaming. The bytes sit behind your application and your database. You can't put a global CDN in front of a
SELECT, and range requests for video seeking become your problem to reinvent.
You end up scaling your most costly component to do a job it was never built for.
Object storage (S3): the file is an object, the DB keeps a reference
Object storage flips the layout. The file becomes an object in a bucket, addressed by a key. The database keeps only a tiny reference — a key or URL, a few bytes per row:
CREATE TABLE photos (
id BIGSERIAL PRIMARY KEY,
user_id BIGINT NOT NULL,
created_at TIMESTAMPTZ DEFAULT now(),
object_key TEXT NOT NULL -- e.g. "photos/2026/07/ab12cd.jpg"
);
What you get in return:
- Cheap storage priced for bulk bytes, not for a transactional engine.
- Eleven nines of durability (99.999999999%) — the platform replicates and repairs objects for you.
- It scales itself. No sharding media across DB nodes; the bucket just grows.
- CDN-ready. Object storage sits naturally behind a CDN, so bytes are served from an edge near the user instead of from your origin.
Your database goes back to doing what it's good at: indexing, joining, and answering queries fast — over rows that are now a few bytes each.
The piece that ties it together: pre-signed URLs
The obvious worry: if the file lives in a bucket, isn't it just... public? No. The clean pattern keeps the bucket private and uses pre-signed URLs for access.
When a user needs to upload or view a file, your server generates a temporary, cryptographically-signed link that grants access to exactly one object, for just a few minutes:
import boto3
s3 = boto3.client("s3")
# Hand the browser a link to GET one object, valid for 5 minutes.
url = s3.generate_presigned_url(
"get_object",
Params={"Bucket": "user-media", "Key": "photos/2026/07/ab12cd.jpg"},
ExpiresIn=300, # seconds
)
The browser then talks straight to S3 with that link. This is the whole point:
- The bytes never touch your server. You offload the bandwidth entirely — your app returns a short string, not a 4 MB payload.
- The bucket stays locked down. Nothing is public; access is granted one object at a time.
- The link expires on its own. No revocation bookkeeping — a leaked URL is dead in minutes.
Uploads use the same trick in reverse: hand the browser a pre-signed PUT (or a pre-signed POST policy), and the client uploads directly to the bucket without the bytes ever passing through your API.
The comparison at a glance
| Dimension | Blob in the database | Object storage + reference |
|---|---|---|
| Row size | Megabytes per row | A few bytes (a key/URL) |
| Backups & restores | Drag terabytes through the DB tier | Stay small and fast; media backed up by the object store |
| Reads | Huge binaries through the connection pool | Browser fetches bytes straight from S3 |
| Cache behavior | Buffer cache thrashes on pixels | Buffer cache stays hot on indexes/rows |
| Cost per byte | Most expensive tier you run | Cheap bulk storage |
| Durability | Your backup discipline | ~11 nines, managed for you |
| CDN & streaming | You reinvent it | Native — CDN in front, range requests included |
| Access control | Free via row permissions | Private bucket + short-lived pre-signed URLs |
| Transactional consistency | Atomic with the row | Eventual — needs cleanup for orphaned objects |
Object storage isn't free of trade-offs. You lose the neat atomicity of "the row and the file commit together," so you need a small amount of discipline: write the object first, then the reference; and run a background sweep to garbage-collect objects whose rows never materialized (or rows whose objects were deleted). That bookkeeping is trivial next to the cost of scaling a database full of media.
The same pattern shows up in AI/LLM serving
If you're building anything with models, this isn't a niche web concern — object storage is the backbone of how model artifacts and generated media move around:
-
Model weights and checkpoints. A single set of LLM weights is tens to hundreds of gigabytes. They live as objects in a bucket; your serving nodes pull them by key at startup. You would never put a checkpoint in a
byteacolumn — same reasoning, one more order of magnitude. - Generated media. An image or video generation endpoint produces large binaries on the fly. The clean flow is: write the output to object storage, store the key in your database next to the request record, and return a pre-signed URL so the user's browser downloads straight from the bucket — the bytes never round-trip through your inference server, which you want doing GPU work, not byte-shuffling.
- RAG source documents. The raw PDFs, images, and transcripts behind a retrieval system belong in a bucket; your vector store and metadata DB keep only the chunk text, embeddings, and the object key pointing back to the source. Retrieval stays fast because the heavy originals are out of the hot path.
- Training and eval datasets. Multi-terabyte datasets stream from object storage into training jobs. The database (or a catalog) tracks manifests and references, not the samples themselves.
The rule scales all the way up: heavy bytes in object storage, a thin reference in the database, moved with short-lived signed URLs. Whether it's an Instagram photo or a 70B checkpoint, the shape is identical.
The verdict
For a prototype where nothing will ever grow, a bytea column is fine — don't over-engineer a weekend project. For anything that will hold real user media or serve real traffic, the answer is settled:
Bytes in object storage. Reference in the database. Moved with short-lived pre-signed URLs behind a CDN.
Your database stays lean and fast, your storage stays cheap and durable, and your files stay private without your server ever sitting in the byte path. That's not a clever trick — it's simply how the big apps actually do it.
▶ Want the 90-second version? Watch the Short · and follow @software-engineer-blog for daily software & systems breakdowns.
Top comments (1)
The object lifecycle is where most implementations need more than “write object, then row.” I’d use server-generated opaque keys and an upload state machine: pending → uploaded/checksum verified → scanned/quarantined → active → deleting. Pre-signed POST policies can bound content type and size; verify actual bytes, checksum, and metadata after upload before exposing the object. Treat the URL as a bearer credential: short expiry reduces risk, but an issued URL is generally usable until expiry unless you add a revocation mechanism through credentials/policy/CDN. For downloads, authorize the current principal immediately before signing and keep filenames/PII out of keys and logs. Reconciliation should work both ways with a grace period and version-aware deletes, so a slow upload or retry is not mistaken for garbage and an old worker cannot delete a replacement object.