AWS S3 Overview
AWS S3 is used for backup and storage, disaster recovery, archive, and hybrid cloud storage. It is widely used for application hosting, media hosting, data lakes, big data analytics, software delivery, and static website hosting.
S3 stores objects(files) in buckets(directories). Buckets have globally unique name(across all reg and all accounts), and buckets are defined at the region level. There are naming restrictions such as no uppercase, underscore, no ip, not starts in prefix xn--, and not ends with suffix -s3alias.
Objects have keys, where key is the full path - key = prefix + object name. There is no concept of directories within buckets even if UI will trick you.
The max obj size is 5TB. If uploading > 5GB, must use "multi-part upload"(imp). Each object can have metadata (list of text key/value pairs), tags used for security, and version id.
A public URL is for public access, whereas another url (top right with open tag) is for creator itself (pre-signed url), where only creator can see the content of the object.
S3 Security
There are two types of security mechanisms in S3: user-based and resource-based.
In user-based security, IAM policies define which API calls should be allowed for a specific user from IAM.
In resource-based security, bucket policies define bucket wide rules from the S3 console and allow cross account access. Object ACLs provide finer grain control (can be disabled), and bucket ACLs are less common and can also be disabled.
Amazon Resource Name(ARN) in policy should be the bucket name.
Static Website Hosting in S3
S3 also supports static website hosting. We have to make sure the bucket policy allows public read, otherwise we will get 403 forbidden error.
To enable static website hosting, the process includes going to buckets → properties → edit static website hosting → enable → provide index.html → upload index.html file → and then use the link available under static website hosting.
Versioning in S3
Versioning enables at bucket level and protects unintended delete while allowing easy rollback to previous version. Suspending versioning does not delete the previous version.
Versioning helps to create version of files when we overwrite any file. For example, when uploading a new version of index.html, the old versions can still be viewed under “show versions”.
Deleting objects without versioning adds a delete marker by default, which can be seen after enabling show versions. If you delete the delete marker, you will get your object back.
Replication in S3
AWS provides cross region replication (CRR) and same region replication (SRR). Versioning must be enabled, and copying is asynchronous. Proper IAM permissions must be provided to S3.
CRR is used for compliance, lower latency, and replication across accounts, whereas SRR is used for log aggregation and live replication between production and test accounts.
Storage Classes, Lifecycle, and Encryption
S3 provides multiple storage classes, and durability and availability are key aspects in S3. Storage class can be selected while uploading objects and can also be changed later in properties.
Lifecycle rules can be configured under the management attribute of bucket to move objects after certain days.
S3 encryption is enabled using server-side encryption by default, while client-side encryption means the user encrypts the file before uploading it.
IAM Access Analyzer for S3 ensures only intended people have access to S3 buckets, such as detecting publicly accessible buckets or buckets shared with other AWS accounts.
Shared Responsibility Model
In AWS shared responsibility model, AWS is responsible for infrastructure, configuration, compliance validation, vulnerability analysis, durability, availability, and sustaining concurrent loss of data in two facilities.
The user is responsible for versioning, bucket policies, replication setup, logging, monitoring, storage class, and data encryption at rest and in transit.
AWS Storage and Hybrid Services
AWS provides different storage types including block storage (Amazon EBS, instance store), file storage (EFS), and object storage (Amazon S3, Glacier).
AWS Storage Gateway is a bridge between on-prem data and cloud data, acting as a hybrid storage service to allow on-prem to seamlessly use the AWS cloud. It is used in disaster recovery, backup, restore, and tiered storage services, with types such as file, volume, and tape.
Containers and Compute Services
Docker is used for containerization. AWS provides ECS (Elastic Container Service) to launch Docker containers, where we provision and maintain infrastructure while AWS manages container lifecycle.
Fargate allows launching Docker containers without provisioning infrastructure and is considered serverless.
ECR (Elastic Container Registry) is a private Docker registry used to store images that can be run by ECS or Fargate.
EKS (Elastic Kubernetes Service) allows launching managed Kubernetes clusters and can run on EC2 or Fargate.
Serverless Computing
Serverless means developers do not manage servers and only manage code. Initially it was FAAS, but now includes managed services like databases, messaging, and storage such as S3, DynamoDB, Fargate, Lambda, and API Gateway.
AWS Lambda
EC2 are virtual servers that are continuously running and limited by RAM and CPU, where scaling requires manual intervention.
Lambda provides virtual functions with no servers to manage, supports short executions, runs on-demand, and scales automatically. It follows pay per request and compute time model.
Lambda is event-driven and integrates with many programming languages, with monitoring through CloudWatch. It is also used in serverless CRON jobs using EventBridge triggers.
API and Batch Processing
Amazon API Gateway is a serverless API service used to create, publish, maintain, monitor, and secure APIs. It supports RESTful and WebSocket APIs.
AWS Batch is used for batch processing at scale, dynamically launching EC2 or Spot instances. Jobs are defined as Docker images and run on ECS.
Lambda vs Batch highlights that Lambda has time limits and is serverless, while Batch has no time limit and relies on EC2 and EBS.
Deployment and Developer Tools
AWS Lightsail provides virtual servers, storage, databases, and networking in one place with a simpler and cost-effective approach.
CloudFormation is used as Infrastructure as Code (IaC) to define AWS infrastructure declaratively and allows quick creation and deletion of resources.
AWS CDK allows defining infrastructure using programming languages like Python and Java and compiles into CloudFormation templates.
Elastic Beanstalk is a PaaS service where AWS manages infrastructure while users deploy application code.
CI/CD Services
AWS CodeCommit is used for storing code, CodeBuild compiles and tests code, CodeDeploy deploys applications, and CodePipeline orchestrates the workflow.
The workflow follows: CodeCommit → CodeBuild → CodeDeploy → Elastic Beanstalk.
AWS CodeArtifact is used for artifact management and dependency storage.
System Management
AWS System Manager helps manage EC2 and on-prem systems at scale.
Session Manager allows secure shell access without SSH keys or port 22.
Parameter Store provides secure storage for configuration and secrets with versioning and IAM-based access control.
Global Infrastructure and Networking
AWS Global Infrastructure consists of regions, availability zones, and edge locations.
Route 53 is a managed DNS service that translates domain names to IP addresses. It supports routing policies such as simple, weighted, latency-based, and failover.
Content Delivery and Performance
CloudFront is a CDN that caches content at edge locations.
S3 Transfer Acceleration increases upload speed using AWS edge locations.
AWS Global Accelerator improves application availability and performance using AWS global network.
CloudFront caches content, whereas Global Accelerator routes traffic without caching.
Edge and Hybrid Services
AWS Outposts provides AWS infrastructure on-premise with low latency and data residency benefits.
AWS Wavelength provides ultra low latency by using 5G networks and is used for applications like real-time gaming, AR/VR, smart cities, and connected devices
Top comments (0)