Anuvindh for AWS Community Builders

Posted on Dec 18, 2022 • Updated on Sep 24, 2023

Everything you should know as a Cloud Guru for Storage

#devops #webdev #productivity #cloud

DAY 37 - Everything you should know as a Cloud Guru for AWS Storage - Day Thirty-Seven

100 days of Cloud on GitHub - Read On iCTPro.co.nz - Read on Dev.to

For who?

Data engineers , security engineers, AWS certification preparation, cloud engineers, devops engineers, Solutions Architects

Why shloud you learn

Demand for AWS skills: AWS is one of the most popular cloud computing platforms, and there is high demand for professionals with AWS skills. Learning about AWS storage services can help you to become proficient in using AWS, which can improve your job prospects and earning potential.
Variety of storage options: AWS offers a range of storage options to suit different needs and use cases. By learning about these options, you can better understand which storage solution is the best fit for your needs.
Cost savings: AWS storage services are designed to be scalable and cost-effective. By learning how to use these services, you can potentially save money on storage costs while still getting the performance and reliability you need.
Improved data management: AWS storage services can help you to better manage your data and ensure it is secure, available, and easy to access. This can be especially important for organizations that generate and handle large amounts of data.

In summary, learning about AWS storage services can help you to become more proficient in using AWS, understand the various storage options available, potentially save on storage costs, and improve your data management.

Types of Storage Services

Amazon S3
Amazon Glacier
Amazon EBS
Amazon Instance Storage
Amazon EFS
Amazon CloudFront
AWS Storage Gateway
AWS Snowball

Free HandsOn AWS Storage Lab - Link

Amazon S3 (Simple Storage Service

A block storage which offer high scalability, data availability, security, and performance. Can use for data lakes, cloud-native applications, and mobile apps.

info	Links
Introduction to Amazon S3	Video Link
Storage classes	Link, Link 2

Object based storage
Unlimited Storage
99.999999999% durability
Max file size for one file is 5 Terrabyte
Flat file structure
Regional Based Service
Data replicated for availability
You can encrypt the objects
Restrict bucket to just your VPC
Storage classes - S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, S3 One Zone-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, S3 Glacier Deep Archive, S3 Outposts

Instance store Volumes

info	Links
Amazon EC2 instance store	Link

Ephemeral storage
Donot store critical data
if instance is start or stop you will loose data
if rebooted the data will remain intact
If EC2 failed the data will be lost. (Start/Stop, Terminate)
No additional fees
IOPS are blazing fast
mainly used as cache or buffer
Not all Ec2 Support Instance store volume

Amazon EBS Volumes

info	Links
How to Encrypt an EBS Volume	Link

Persistence and block level storage.
Can be attached to Ec2 instance & in some regions we can multi attach the EBS Volume. More info on multi attach
Independent of EC2 instance
Logically attached to EC2 instance
Supports Snapshotting
can copy EBS snapshot from one region to other regions
EBS data is replicated to same AZ for Availability.
Types of EBS are SSD And HDD.
Ability to encrypt during rest and transit (AES256). Here is an awesome blog about this by STUART SCOTT from Cloud Academy

Amazon Elastic File System (EFS)

info	Links
Overview EFS	VideoLink
How to create EFS	Link

EFS , a file storage services
Can attach to multiple EC2 using mount point
Uses Hierarchy system.
Build under Gluster filesystem
Fully managed shared file system with low latency access
PB of Storage available
Uses NFS 4.1 and 4.0
replicated across multiple AZ for high availability
Highly scalable, regional services, available on most of the regions For more info

Amazon Cloud front (CDN)

info	Links
Intro	VideoLink
Workshop	Link

Content Delivery Network
Delivered as cache data from edge locations

Use Distributions to deliver content

Via Web distribution

Dynamic and Static
HTTP and HTTPS
Allow add, remove, update object
Live stream functionality support
Origins can be Ec2 or S3

Via RTMP Distribution

Distribute streaming media using RMTP protocol(Adobe flash media)
Origins can only from S3

Amazon CloudFront is a content delivery network (CDN) provided by Amazon Web Services (AWS). It is designed to deliver content, such as websites and applications, quickly and securely to users around the world. CloudFront uses a network of edge locations located in various parts of the world to deliver content to users with low latency. This means that users can access the content they need faster and with fewer interruptions. CloudFront also integrates with other AWS services, such as Amazon S3 and Amazon EC2, to provide a complete solution for delivering content over the internet.

Storage Gateway

info	Links
Features	link
Pricing	Link

Best way to transfer data from NAS, SAN & DAS to AWS
Secure and cost efficient
File ,Volume and Tape Configurations available
Consider Egress traffic charges while architecting
File gateways used to access files stored as objects from s3 . can be mounted as NFS to Corporate Environment. Using casing will reduce further latency and the Egress traffic charges.
Storage Volume gateway used for low latency access data from S3. Synchronously copied to S3.
Volume can be 1GB to 16 TB and MAX 512 TB.
Snapshots are stored inclemently in S3.
Cached Volume gateways, data provided by s3. local volume used for buffering and local cache for recently access data.
Volume can be 32TB, and storage capacity of 1024 TB
Tape gateway , back to s3 from on-prem ,take advantage of AWS glacier

AWS Snowball

info	Links
Specifications	Link
Shipping an AWS Snowball device	link

Securely transfer PB of data in and out of AWS
On perm to S3 or S3 to On perm
High speed data transfer
Data to Snowball is encrypted automatically
can be tracked also use SNS for tracking
HIPAA Compliant
Data removed with NIST standard after the transfer

AWS Snowball process
Import

Export

AWS Snow FAMILY

info	Links
Intro	Link

AWS Snowcone is a small, rugged, and secure device offering edge computing, data storage, and data transfer on-the-go, in austere environment with little or no connectivity.

These devices are designed for offline data processing and storage, and are particularly useful for transferring large amounts of data to or from AWS when dealing with data that is too large or too costly to transfer over the Internet.

AWS Snowball is a physical device that can hold up to 50 TB of data, while AWS Snowball Edge is a device that combines the capabilities of AWS Snowball with additional compute and storage capabilities. AWS Snowmobile is a shipping container that can hold up to 100 PB of data, and is designed for transferring extremely large amounts of data to AWS.

✅Connect with me on Twitter
🤝🏽Connect with me on Linkedin
🧑🏼‍🤝‍🧑🏻 Read more post on dev.to or iCTPro.co.nz
💻 Connect with me on GitHub