Choosing an appropriate storage is one of the most important decisions while either creating an app, migration or simply creating something completely new and unique.
Nowadays, we have multiple options of storage types to choose from, but when deciding you will need to consider the purpose for it. For example, will it be for mobile applications, archiving, sharing content, who is going to use the application and so on…
In this article I am not hoping to give the answer to all doubts related to this, but share a little bit of my experience on the field and share some tips that have helped me when I need to design about it.
First let’s briefly describe the most common file storages types and some of their main features of each of them:
| File Storage | Block Storage | Object Storage |
|---|---|---|
| No Metadata | Limited Metadata | Metadata |
| High IO performance | Hierarchical System | Unique identifier |
| Write/Read efficiency | Limit in Scalability and performance. | Better for Read only applications and Distribution |
| Greater Storage efficiency | Protocols: NFS/CIFS, SMB, etc. | Protocols: HTTP, HTTPS, REST operation (GET, PUT, DELETE, etc.) |
And some other factor in data management should be identified for our type of data: Structured or Unstructured data. The majority of the data we use and access everyday is normally unstructured, such as videos, images, emails, spreadsheets, word processing files, etc.
This is a big and deep topic, however this is out of the scope of this blog, but because I know there are people like me who would like to go further on this, I found these articles, which I believe are a great start:
So, going back to our main topic, as a rule of thumb, if the data can be displayed in a row, columns and/or SQL DB without or minimal modification, it means that most likely you are working with structured data.
Please do not forget that normally unstructured data requires more storage than structured data, therefore choosing the right storage is crucial to keep a good performance for the apps and services.
Now that we've discussed different data types, the next question arises, how can we use this knowledge to choose the right storage when using AWS?
In order to connect the two points, I would like to describe a little bit of the storage types offered by AWS. If we go to the AWS official documentation page we will notice that they offered the following (by the time this blog was written):
I highly recommend go to the official documentation to understand a little bit more each of these storages types, or read some of these blogs written by other members of AWS Community Builders, for example check these:
- Amazon S3 by Adeline Makokha.
- Amazon EBS vs EFS vs S3: Picking the Best AWS Storage Option by Adit Modi.
- AWS Transfer Family – FTP for EFS and S3 by Pete Wilcock .
There are many more article explaining to this and different types of topics and services, please take a look in the channel
AWS Community Builders Channel.
So, I mapped the two points in the following table, which will give you a general idea of the type of data for which the storage is better suited for:
| Storage Type | Block Storage | File Storage | Object Storage | Volume Based Storage | Tape Storage |
|---|---|---|---|---|---|
| Amazon S3 | ⛔ | ⛔ | 😃 | ⛔ | ⛔ |
| AWS Backup | 😃 | 😃 | 😃 | 😃 | 😃 |
| Amazon EBS | 😃 | ⛔ | ⛔ | ⛔ | ⛔ |
| Amazon EFS | ⛔ | 😃 | ⛔ | ⛔ | ⛔ |
| AWS Elastic Disaster Recovery | NA | NA | NA | NA | NA |
| Amazon FSx | ⛔ | 😃 | ⛔ | ⛔ | ⛔ |
| Amazon S3 Glacier | ⛔ | ⛔ | 😃 | ⛔ | ⛔ |
| AWS Snow Family | NA | NA | NA | NA | NA |
| AWS Storage Gateway | ⛔ | 😃 | ⛔ | 😃 | 😃 |
Please use as a reference for table 1:
- 😃 => This is the storage type.
- ⛔ => It is not suitable for it.
And I also mapped the Storage features with AWS Storage offering, to give you a general idea and it can be used as a reference:
Disclaimer: Not everything is written in stone, so please make sure to use this tables as reference and general guidelines only.
| Storage Type | Metadata | High IO Performance | Distribution | Scalability | Data Modification |
|---|---|---|---|---|---|
| Amazon S3 | 😃🙌 | 😐 | 😃🙌 | 😃🙌 | 😃 |
| AWS Backup | 😃 | 😃 | 😃 | 😃 | 😃 |
| Amazon EBS | ⛔ | 😃🙌 | ⛔ | 😐 | 😃🙌 |
| Amazon EFS | 😐 | 😃 | 😃 | 😃 | 😃 |
| AWS Elastic Disaster Recovery | NA | NA | NA | NA | NA |
| Amazon FSx | 😐 | 😃🙌 *(Lustre) * | 😃 | 😃 | 😃 |
| Amazon S3 Glacier | 😃🙌 | ⛔ | 😃 | 😃🙌 | 😐 |
| AWS Snow Family | NA | NA | NA | NA | NA |
| AWS Storage Gateway | 😃 | 😃 | 😃 | 😃 | 😃 |
For anyone who is curious about it, this is the Link for the Lustre Project: https://www.lustre.org/.
Please use as reference for table 2:
- 😐 => Possible but with some caveats or there are some limitations or restrictions.
- 😃 => It will work but there might be other options.
- 🙌 => This is what it was built for.
- ⛔ => it is not suitable for it.
I hope you find this post useful and please if you have any comments, do not hesitate in letting me know.
Top comments (0)