AMAZON S3 BRAIN DUMP

israel mvono — Tue, 18 Oct 2022 17:56:02 +0000

Introduction

Amazon S3 is one of the main building blocks of AWS.
It’s advertised as “infinitely scaling” storage.
Many websites use Amazon S3 as a backbone.

Amazon S3 Overview - Buckets

Amazon S3 allows people to store objects (files) in “buckets”(directories)
Buckets must have a globally unique name
Buckets are defined at the region level
Naming convention . No uppercase . No underscore . 3-63 characters long

Amazon S3 Overview - Objects

Objects (files) have a key.
The key is the FULL path:
. S3://my-bucket/my_file.txt
. S3://my-bucket/my_folder1/another_folder/my_file.txt
The key is composed of prefix + object name
. S3://my-bucket/my_folder1/another_folder/my_file.txt
Object values are the content of the body:
. Max object size is 5TB (5000 GB)
. If uploading more than 5GB, must use “multi-part upload”
Metadata (list of text key / value pairs - system or user metadata)
Tags (unicode key / value pair - up to 10) - useful for security / lifecycle.
Version ID (if versioning is enabled)

Amazon S3 - Versioning

You can version your files in Amazon S3..
It is enabled at the bucket level.
Same key overwrite will increment the “version” : 1, 2, 3...
It is best practice to version your buckets.
. Protect against unintended deletes(ability to restore a version)
. Easy roll back to previous version
Any file that is not versioned prior to enable versioning will have version “null”
Suspending versioning does not delete the previous versions.

Amazon S3 Encryption for Objects

There are 4 methods of encrypting objects in S3 . SSE-S3: encrypts S3 objects using keys handled and managed by AWS . SSE-KMS: leverage AWS Key Management Service to manage encryption keys. . SSE-C: when you want to manage your own encrytion keys. . Client Side Encryption

SSE-S3

SSE-S3: encryption using keys handled and managed by Amazon S3.
Object is encrypted Server Side.
AES-256 encryption type.
Must set header: “x-amz-server-side-encryption”:”AES256”

SSE-KMS

SSE-KMS: encryption using keys handled and managed by KMS.
KMS Advantages: user control + audit trail.
Object is encrypted Server Side.
Must set header: “x-amz-server-side-encryption”:”aws:kms”

SSE-C

SSE-C: Server side encryption using data keys fully managed by the customer outside of AWS.
Amazon S3 does not store the encryption key you provide.
HTTPS must be used.
Encryption key must be provided in the HTTP headers, for every HTTP request made.

Client Side Encryption

Client library such as the Amazon S3 Encrytion Client.
Clients must encrypt data themselves before sending to S3.
Clients must decrypt data themselves when retrieving from S3.
Customer fully manages the keys and encryption cycle.

Encryption in transit (SSL/TLS)

Amazon S3 exposes:
. HTTP endpoint: non encrypted
. HTTPS endpoint: encryprion in flight
You are free to use the endpoint you want but HTTPS is recommended.
Most clients would use the HTTPS endpoint by default.
HTTPS is mandatory for SSE-C.
Encryption in flight is also called SSL/TLS.

S3 Security
. User based

IAM policies - which API calls should be allowed for a specific user from IAM console.

. Resource based
. Bucket policies - bucket wide rules from the S3 console - allows cross account.
. Object Access Control List (ACL) - finer grain
. Bucket Access Control List (ACL) - less common

S3 Bucket Policies
. JSON based policies
. Resources: buckets and objects
. Actions: set of API to Allow or Deny
. Effect: Allow/ Deny
. Principal: the account or user to apply the policy to.

{
“version” : “2022-10-18”
“Statement” : [
{
“Sid” : “PublicRead”,
“Effect” : “Allow”,
“Principal” : “*”,
“Action” : [
“S3: GetObject”
],
“Resource” : [
“arn:aws:S3:::examplebucket / * “
]
}
]
}

. Use S3 bucket for policy to:
. Grant public access to the bucket.
. Force objects to be encrypted at upload.
. Grant access to another account (cross Account)

Bucket Settings for Block Public Access

Block public access to buckets and objects granted through: . new access control lists (ACLs) . any access control lists (ACLs) . new public bucket or access point policies
Block public and cross-account access to buckets and objects through any public bucket or access point policies.
These settings were created to prevent company data leaks.
If you know your bucket should never be public, leave these on.

S3 Security - Other
. Networking
. Supports VPC Endpoints (for instances in VPC without www internet)
. Logging and Audit:
. S3 Access Logs can be stored in other S3 bucket.
. API calls can be logged in AWS cloudTrail.
. User Security:
. MFA Delete: MFA( multi factor authentication) can be required in versioned buckets to delete objects.
. Pre-signed URLs: URLs that are valid only fro a limited time (ex: premium video service for logged in users)

CORS - Explained

An origin is a scheme (protocol), host(domain) and port . E.g. : https://www.plentyofbananas.com (implied port is 443 for HTTPS, 80 for HTTP)
CORS means Cross-Origin Resource Sharing
Web Browser based mechanism to allow requests to other origins while visiting the main origin.
Same origin: http://plentyofbananas.com/v1 & http://plentyofbananas/v2
Different origins: http://www.plentyofbananas.com & http://plentyofstrawberries.com
The requests won’t be fulfilled unlessthe other origin allows for the requests, using CORS Headers ( ex: Access - Control - Allow - Origin)

S3 CORS

If a client does a cross-origin request on your S3 bucket, you need to enable the correct CORS headers.
You can allow for a specific origin or for * (all origins)

AWS: Identity and Access Management[cheatsheet]

israel mvono — Thu, 04 Aug 2022 16:28:00 +0000

As I begin my journey studying towards being a devOps engineer, I would like to share what I learn throughout my journey. I am currently studying for the AWS CLOUD SOLUTIONS ARCHITECT ASSOCIATE EXAM
In this article, I will share a summary of IAM & AWS CLI

Summary
-IAM = Identity and Access Management

Users are people within your organization and they can be grouped[it’s best practice to always assign each member under a group(s). A user can be a member of more than one group.
Groups can only contain users but not other groups.
Policies define the permissions of the users.[best practice: use the least privilege principle; simply don’t give more permissions than the user needs]
Password policy:- you can set up a password policy that ensures higher security for your account.
Multi-Factor Authentication – MFA
. protects your root accounts and IAM users.
. MFA = password you know + security device you know
. If a password is stolen or hacked, the account is not compromised.
.You can use a virtual MFA device[google authenticator, authy} or a U2F Security Key{yubikey by yubico}
To access AWS, you have three options:
. AWS Management Console{protected by password + MFA}
. AWS Command Line Interface(CLI){protected by access keys}
. AWS Software Development Kit(SDK) – for code {protected by access keys}
-Access keys are generated through the AWS console
. Access Key ID ~= username
. Secret Access Key ~= password
AWS cloudshell is a CLI in the cloud.
IAM Credentials Report(account level): this is a report that lists all your account’s users and the status of their various credentials.
IAM Access Advisor(user-level): shows the service permissions granted to a user when those services were last accessed.

IAM Guidelines and Best Practices

Don’t use the root account except for AWS account setup
One physical user = One AWS user
Assign users to groups and assign permissions to groups
Create a strong password policy
Use and enforce the use of Multi-Factor Authentication(MFA)
Create and use roles for giving permissions to AWS services
Use Access Keys for Programmatic Access(CLI/SDK)
Audit permissions of your account with the IAM credentials report and IAM Access Advisor
Never share IAM users and Access Keys

DEV Community: israel mvono

AMAZON S3 BRAIN DUMP

AWS: Identity and Access Management[cheatsheet]