DEV Community

Magevanta
Magevanta

Posted on • Originally published at magevanta.com

Magento 2 Media Storage Performance: Database, Filesystem or S3?

Every Magento store starts with a simple media/ folder and nobody thinks about it. Then the catalog grows, image resizes multiply, the deploy takes twenty minutes, and the server starts throwing "No space left on device" — while df -h shows 80% free. That's the moment you discover you've run out of inodes, not bytes. Media storage is one of the most underestimated performance bottlenecks in Magento 2, because it doesn't show up in MySQL slow logs or Blackfire traces. It shows up as slow deploys, stalled image requests, NFS timeouts and sync processes that never finish.

This post breaks down how Magento actually stores and generates media, where the performance traps are hiding, and how to choose between filesystem, database and object storage — without breaking your store.

How Magento 2 Stores Media (Under the Hood)

Magento's media catalog lives in <store_root>/pub/media, and the interesting parts are:

  • catalog/product/ — original product images, organized by first letter of the filename (a/b/, c/d/, ...). This scatter is intentional: it keeps directory sizes manageable on filesystems that struggle with huge flat directories.
  • catalog/product/cache/generated, resized images. Every resized variant (category pages, product pages, cart thumbnails, widgets) gets written here, keyed by the resize parameters.
  • catalog/category/, theme/, wysiwyg/, tmp/, import/ — the rest of the ecosystem.
  • media_storage_file and media_gallery rows in the database — the metadata layer (only fully used in DB storage mode, see below).

The critical thing to understand: a typical Magento store ends up with millions of tiny files. Each product image spawns anywhere from 5 to 30 resized variants depending on your theme. With 50,000 products and 3 images each, you're looking at 750K–4.5M files. That's not a storage problem — it's a filesystem problem.

Performance Trap #1: Inodes and Directory Scalability

On a default ext4 filesystem, every file and directory consumes one inode. df -h reports bytes, df -i reports inodes. A busy store with 4 million media files can exhaust the inode pool on a 100GB disk that's only half full. When that happens, any write fails — logs, sessions, deploys, image generation — and the failure mode is confusing because the disk looks fine.

Check it before you ever get close to the edge:

df -i /var/www/magento
# Filesystem      Inodes  IUsed   IFree IUse% Mounted on
# /dev/sda1      6553600 6500210  53390   99% /
Enter fullscreen mode Exit fullscreen mode

Fixes, in order of preference:

  1. Use XFS instead of ext4 for new volumes (XFS allocates inodes dynamically — no fixed pool to exhaust).
  2. Move pub/media to a dedicated volume (own inode pool, own I/O, own backup policy).
  3. Never store media on NFS with default settings — NFS is the classic cause of "image requests hanging for 30 seconds". If you must, use NFSv4 with actimeo=600 and a dedicated export, or better: move to object storage (below).

Also check your filesystem's directory index. ext4 uses hashed directory entries, which is fine, but on shared/NAS filesystems even reading a directory with 50,000 entries can take seconds. Run ls -U (unsorted, no stat) to see raw read speed — if that's slow, the filesystem is the bottleneck, not Magento.

Performance Trap #2: On-the-Fly Image Resizing Races

Here's the dirty secret of Magento's image pipeline: when a resized variant doesn't exist in catalog/product/cache/, the frontend generates it on demand during the request that needs it. The first visitor to hit a category page pays the resizing cost — and in a multi-server setup, every server races to generate the same images, because there's no shared lock.

That's why you sometimes see flock() warnings in var/log/ about image cache locks, and why a cache flush can spike your CPU to 100% for an hour while the "warm" cache rebuilds itself on live traffic.

Two things to do about it:

  1. Pre-generate during deploy, not on first request:
bin/magento catalog:images:resize
Enter fullscreen mode Exit fullscreen mode

Run this on your build/deploy server for all themes and locales, and ship the generated cache/ directory with the release (or sync it to a shared location). Combined with a CDN in front, subsequent deploys become much cheaper because the CDN already holds most variants.

  1. Consider dynamic image resizing only where it pays off — some CDNs (Cloudflare, Fastly, imgix-style services) can resize on the edge. If you go that route, you can point catalog:images:base_url at the resize service and disable local generation for most sizes. It trades local CPU for CDN cost — usually the right trade at scale.

Performance Trap #3: The Database Storage Mode

Magento ships with a database media storage option (Stores → Configuration → Advanced → System → Storage Configuration for Media, or bin/magento config:set system/media_storage_configuration/media_storage 1). All media files are stored as BLOBs in media_storage_file, with metadata in media_storage_file_storage.

When is DB storage a good idea? Almost never for production — but it's a lifesaver for multi-node setups that can't share a filesystem. Instead of NFS, all nodes read/write media through MySQL, which is already replicated.

The price: every image request is a DB round-trip (mitigated by the cache/ dir), the media_storage_file table grows to hundreds of GB, and the DB backup gets brutally slow. The sync commands are also a classic source of pain:

bin/magento media:sync
Enter fullscreen mode Exit fullscreen mode

On a large catalog this can run for days, and it's a single-process command — you can't parallelize it out of the box. If you switch storage modes, plan a maintenance window and test the sync on a copy first.

Verdict: use DB storage only when you have no shared filesystem and no budget for object storage. In every other case, one of the options below wins.

Performance Trap #4: Object Storage (S3) — The Scalable Answer

The modern answer for scale is object storage: an S3-compatible bucket (AWS S3, DigitalOcean Spaces, MinIO, Cloudflare R2) behind your CDN. Magento has first-party S3 support (magento/module-remote-storage + the AWS S3 module), and the config is simple:

bin/magento setup:config:set \
  --remote-storage-driver="aws-s3" \
  --remote-storage-bucket="my-store-media" \
  --remote-storage-prefix="media/"
Enter fullscreen mode Exit fullscreen mode

With remote storage enabled, Magento lazily offloads files to the bucket when they're requested or written, and serves them from the bucket via the CDN. The local media/ directory stays small (just recent files and the cache), deploys become fast, and you get:precise metrics — S3 buckets give you object counts and transfer costs, which beats guessing about a local folder.

The tradeoffs to know before you switch:

  • First-request latency. Lazy sync means the first request for an old image triggers a fetch from S3. Pre-warm the bucket with a full media:sync during a maintenance window, or accept a one-time warmup cost.
  • Cache stampedes return. The cache/ resize directory is local by default, so multi-node stores without a shared cache directory will re-generate variants per node again. Fix by warming on build and letting the CDN absorb traffic.
  • Signed URLs for private media (e.g. PDF downloads behind login) need extra config — don't skip it, or you'll leak signed links.
  • Costs. S3 GET/PUT pricing is cheap at normal traffic levels, but a bad cache policy can run up a bill. 24/7 public CDN-plus-bucket access patterns are fine; direct bucket access from the origin with no CDN is not.

Performance Trap #5: The Media Gallery Table Itself

Even with perfect storage, the metadata layer can bite you. catalog_product_entity_media_gallery and its _value table grow one row per image per store view — and every Save on a product with a large gallery rewrites the whole gallery asset list. With the new Media Gallery (Adobe Stock integration), batch operations run through media_gallery_asset tables, which Magento fills with millions of rows on stores with large media folders.

Signs you're hitting this:

  • SELECT on catalog_product_entity_media_gallery_value shows up in slow query logs on product saves and category renders.
  • Admin Media Gallery pages take forever to load the folder tree.
  • media:sync crawls because every file becomes an asset row.

Practical fixes:

  1. Index the obvious joins if your install predates index fixes: (entity_id, store_id) and (value_id, store_id) composite indexes on the _value table.
  2. Don't keep years of orphaned media. The "Media Gallery" bulk delete tool exists for a reason — use it, then OPTIMIZE TABLE the gallery tables.
  3. Watch category/product saves in bulk. If a bulk update touches 10,000 products with galleries, break it into chunks of ~500. Each save rewrites gallery rows; chunking keeps row locks short and replication lag low.

A Decision Framework for Your Store

Store size Files (est.) Best storage Why
Small (< 5K SKUs) < 200K Local SSD + CDN Simplest, fast enough, no ops complexity
Medium (5–50K SKUs) 200K–1M Local SSD + CDN, or S3 for peace of mind Filesystem still fine if inodes are sized right
Large (50K+ SKUs, multi-node) 1M+ S3 (or object storage) + CDN Inode pressure, NFS pain, deploy speed
Multi-node, no budget Any DB storage as last resort Avoids NFS; accepts DB growth and slow sync

Whatever you pick, there's a non-negotiable baseline:

  1. Check inodes: df -i in your deploy runbook. Exhausting inodes is a full outage with a 5-minute fix that nobody finds for 2 hours.
  2. Pre-generate image cache on build (catalog:images:resize) so first-request generation races never hit production traffic.
  3. Put a CDN in front of media/ (this blog covered CDN config in depth before) — it converts origin image load from "per-visitor" to "per-variant-once".
  4. Monitor sync jobs. media:sync and catalog:images:resize in var/log with start/end timestamps; alert when they exceed your deploy window. A sync that never finishes is the first symptom of a storage architecture change.

Wrapping Up

Media storage doesn't show up in your profiler, but it decides how fast deploys run, how fast image-heavy pages render, and whether your server dies a confusing death at 50% disk usage. Start with the inode check — it's free and it catches the worst failure mode. Then decide deliberately: local SSD + CDN while you're small, object storage the moment files exceed a million or the second node joins the cluster, and database storage only when you have no other way to share media.

The storage decision is a one-way door for your ops setup — pick the one that scales with the catalog, not the one that's easiest today.

Top comments (0)