DEV Community

Cover image for Saving Millions of Images Series [Part Two]
Arthur Tkachenko
Arthur Tkachenko

Posted on

Saving Millions of Images Series [Part Two]

Saving assets

Ok, I agree with you that one of the easiest way is to work with S3 bucket. But there is more to talk about.

Lets’s talk about how to organize the storing of images. We can have a long discussion about how to store "paths to images."

A guy from a StackOverflow posted this: https://stackoverflow.com/questions/3748/storing-images-in-db-yea-or-nay/3751#3751

What about image names? can be something like this:

I would store the image hash in the database along with a
DATE / DATETIME / TIMESTAMP field indicating when the image was uploaded/processed and then place the image in a structure like this

Image description

Or

Image description

Ok, this method is more descriptive; I get it. But this structure is significant to host hundreds of thousands (depending on your file system boundaries) of images per day for several thousand years, this is how WordPress does it, and I think they got it right.

Duplicated images could be easily queried on the database, and you'd have to create symlinks.

I dont like to operate with user IDs unless you don't have that info available in your database, because of:

  • Disclosure of usernames in the URL
  • Usernames are volatile (you may be able to rename folders, but again...)
  • A user can hypothetically upload a large number of images

Don't forget about CDN! This scheme might work well with CDN. At least I don't see anything bad

Read those >= 10 articles about uploading images or check out these simple examples

Previous Article:
https://dev.to/atherdon/saving-millions-of-images-series-part-one-od1

Top comments (0)