AMAZON S3 BRAIN DUMP

#aws #amazon #tutorial

Introduction

Amazon S3 is one of the main building blocks of AWS.
It’s advertised as “infinitely scaling” storage.
Many websites use Amazon S3 as a backbone.

Amazon S3 Overview - Buckets

Amazon S3 allows people to store objects (files) in “buckets”(directories)
Buckets must have a globally unique name
Buckets are defined at the region level
Naming convention . No uppercase . No underscore . 3-63 characters long

Amazon S3 Overview - Objects

Objects (files) have a key.
The key is the FULL path:
. S3://my-bucket/my_file.txt
. S3://my-bucket/my_folder1/another_folder/my_file.txt
The key is composed of prefix + object name
. S3://my-bucket/my_folder1/another_folder/my_file.txt
Object values are the content of the body:
. Max object size is 5TB (5000 GB)
. If uploading more than 5GB, must use “multi-part upload”
Metadata (list of text key / value pairs - system or user metadata)
Tags (unicode key / value pair - up to 10) - useful for security / lifecycle.
Version ID (if versioning is enabled)

Amazon S3 - Versioning

You can version your files in Amazon S3..
It is enabled at the bucket level.
Same key overwrite will increment the “version” : 1, 2, 3...
It is best practice to version your buckets.
. Protect against unintended deletes(ability to restore a version)
. Easy roll back to previous version
Any file that is not versioned prior to enable versioning will have version “null”
Suspending versioning does not delete the previous versions.

Amazon S3 Encryption for Objects

There are 4 methods of encrypting objects in S3 . SSE-S3: encrypts S3 objects using keys handled and managed by AWS . SSE-KMS: leverage AWS Key Management Service to manage encryption keys. . SSE-C: when you want to manage your own encrytion keys. . Client Side Encryption

SSE-S3

SSE-S3: encryption using keys handled and managed by Amazon S3.
Object is encrypted Server Side.
AES-256 encryption type.
Must set header: “x-amz-server-side-encryption”:”AES256”

SSE-KMS

SSE-KMS: encryption using keys handled and managed by KMS.
KMS Advantages: user control + audit trail.
Object is encrypted Server Side.
Must set header: “x-amz-server-side-encryption”:”aws:kms”

SSE-C

SSE-C: Server side encryption using data keys fully managed by the customer outside of AWS.
Amazon S3 does not store the encryption key you provide.
HTTPS must be used.
Encryption key must be provided in the HTTP headers, for every HTTP request made.

Client Side Encryption

Client library such as the Amazon S3 Encrytion Client.
Clients must encrypt data themselves before sending to S3.
Clients must decrypt data themselves when retrieving from S3.
Customer fully manages the keys and encryption cycle.

Encryption in transit (SSL/TLS)

Amazon S3 exposes:
. HTTP endpoint: non encrypted
. HTTPS endpoint: encryprion in flight
You are free to use the endpoint you want but HTTPS is recommended.
Most clients would use the HTTPS endpoint by default.
HTTPS is mandatory for SSE-C.
Encryption in flight is also called SSL/TLS.

S3 Security
. User based

IAM policies - which API calls should be allowed for a specific user from IAM console.

. Resource based
. Bucket policies - bucket wide rules from the S3 console - allows cross account.
. Object Access Control List (ACL) - finer grain
. Bucket Access Control List (ACL) - less common

S3 Bucket Policies
. JSON based policies
. Resources: buckets and objects
. Actions: set of API to Allow or Deny
. Effect: Allow/ Deny
. Principal: the account or user to apply the policy to.

{
“version” : “2022-10-18”
“Statement” : [
{
“Sid” : “PublicRead”,
“Effect” : “Allow”,
“Principal” : “*”,
“Action” : [
“S3: GetObject”
],
“Resource” : [
“arn:aws:S3:::examplebucket / * “
]
}
]
}

. Use S3 bucket for policy to:
. Grant public access to the bucket.
. Force objects to be encrypted at upload.
. Grant access to another account (cross Account)

Bucket Settings for Block Public Access

Block public access to buckets and objects granted through: . new access control lists (ACLs) . any access control lists (ACLs) . new public bucket or access point policies
Block public and cross-account access to buckets and objects through any public bucket or access point policies.
These settings were created to prevent company data leaks.
If you know your bucket should never be public, leave these on.

S3 Security - Other
. Networking
. Supports VPC Endpoints (for instances in VPC without www internet)
. Logging and Audit:
. S3 Access Logs can be stored in other S3 bucket.
. API calls can be logged in AWS cloudTrail.
. User Security:
. MFA Delete: MFA( multi factor authentication) can be required in versioned buckets to delete objects.
. Pre-signed URLs: URLs that are valid only fro a limited time (ex: premium video service for logged in users)

CORS - Explained

An origin is a scheme (protocol), host(domain) and port . E.g. : https://www.plentyofbananas.com (implied port is 443 for HTTPS, 80 for HTTP)
CORS means Cross-Origin Resource Sharing
Web Browser based mechanism to allow requests to other origins while visiting the main origin.
Same origin: http://plentyofbananas.com/v1 & http://plentyofbananas/v2
Different origins: http://www.plentyofbananas.com & http://plentyofstrawberries.com
The requests won’t be fulfilled unlessthe other origin allows for the requests, using CORS Headers ( ex: Access - Control - Allow - Origin)

S3 CORS

If a client does a cross-origin request on your S3 bucket, you need to enable the correct CORS headers.
You can allow for a specific origin or for * (all origins)