The best way to learn a skill is through deliberate practice!
S3 Game is an awesome game made by Vasily Pantyukhin which walks you through various features of S3 from beginner to fairly advanced.
What you will need:
- A browser (duh!)
- aws cli
CLICK HERE TO PLAY!
Each level consists of finding a "treasure" object and getting to the next level using the secret code in the "treasure".
Go ahead and play this game. Each level contains multiple hints with links to S3 API documentation.
Below is a walk through you can refer to if you are stuck on any level. Make sure that you try for a bit before seeing here :)
Walk through (Spoilers)
Level 1
Solution:
# List the objects in bucket
aws s3 ls s3://s3game-level1
Visit the treasure object
Level 2
Solution:
Just get the treasure2 object either using browser or aws s3 : https://s3game-level2.s3.us-east-2.amazonaws.com/treasure2
Make sure to save the Access and Secret key somewhere
Level 3
Create an AWS profile "tmp" with the access and secret key obtained from previous level. You will need to use this for almost every level from here on.
aws configure --profile tmp
> AWS Access Key ID
> AWS Secret Access Key
> Default region name - us-east-2
Solution:
aws s3 ls s3://s3game-level3 --profile tmp
aws s3 cp s3://s3game-level3/treasure3_has_no_secret_code . --profile tmp
cat treasure3_has_no_secret_code
# The object data itself does not have the secret code. It is present in the metadata
aws s3api head-object --bucket s3game-level3 --key treasure3_has_no_secret_code --profile tmp
Level 4
Solution:
# List the bucket ..
# get the file
aws s3 cp s3://s3game-level4-k73045aztqln/treasure4_also_has_no_secret_code . --profile tmp
# looks like the data is empty
aws s3api head-object --bucket s3game-level4-k73045aztqln --profile tmp --key treasure4_also_has_no_secret_code
# ... so is the metadata. Let's try object tags as mentioned in hint
aws s3api get-object-tagging --bucket s3game-level4-k73045aztqln --profile tmp --key treasure4_also_has_no_secret_code
Takeaway:
Object in S3, including its metadata, is immutable. When you edit object's metadata, you are actually overwriting the object with a copy of itself, with its metadata modified.
In contrast, tags are subresources. They are managed separately and can be modified without modifying the object itself.
Level 5
Solution
# We are listing all versions of objects as the hint says that the object is not present in its current version
aws s3api list-object-versions --bucket s3game-level5-8v95e5rv7z4i --profile tmp
# Get the specific version of treasure5_is_deleted using version-id from previous call
aws s3api get-object --bucket 's3game-level5-8v95e5rv7z4i' --key treasure5_is_deleted --version-id '344PQOyFqocF0TI66MbLynNNdQqHfBz3' --profile tmp treasure5_is_deleted
Takeaway
S3 versioning keeps multiple variants of an object in the same bucket.
If you enable versioning for a bucket, S3 automatically generates a unique version ID for the object being stored.
In one bucket you can have two objects with the same key, but different version IDs.
Check if versioning is enabled using:aws s3api get-bucket-versioning --bucket my-bucket
Level 6
Solution
# We need to get a gzipped object using S3 select
aws s3api list-objects --bucket s3game-level6-vjv45x1gux81 --profile tmp
# Now, do the select on s3select.csv.gz
aws s3api select-object-content --bucket s3game-level6-vjv45x1gux81 --key s3select.csv.gz --expression "SELECT s.Answer FROM s3object s WHERE Category = 'TREASURE'" --expression-type sql --input-serialization '{"CSV": {"FileHeaderInfo": "USE", "FieldDelimiter": ";"}, "CompressionType": "GZIP"}' --output-serialization '{"CSV": {}}' --profile tmp treasure6
Level 7
Solution
# Get 'somethingstrange' from the bucket
aws s3api get-object --bucket s3game-level7-zhovpo4j8588 --key somethingstrange --profile tmp t7
cat t7
# Visit the pre-signed URL in the file to get URL for next level
Takeaway
Pre-signed URLs are a powerful feature of S3. From the docs:
A presigned URL is generated by an AWS user who has access to the object. The generated URL is then given to the unauthorized user. The presigned URL can be entered in a browser or used by a program or HTML webpage. The credentials used by the presigned URL are those of the AWS user who generated the URL.
A presigned URL remains valid for a limited period of time which is specified when the URL is generated.
Level 8
Solution
# List objects
aws s3api list-objects --bucket s3game-level8-v6g8tp7ra2ld --profile tmp
# Visit the treasure file using the cloudfront URL
curl 'https://d2suiw06vujwz3.cloudfront.net/treasure8_CDN'
Takeaway
- You can easily create a Cloudfront distribution backed by an S3 bucket
- In such cases, you can restrict public access for the objects in the bucket as they can be accessed using Cloudfront
- Ensure that your bucket does not allow unprivileged users to list all objects in the bucket
Level 9
The bucket has a policy which checks that any request for "arn:aws:s3:::s3game-level9-781xtls2quvy/treasure9_referer" has the referer string "http://s3game.treasure"
Solution
All we need to do is curl with a --refer given in policy
curl 'https://s3game-level9-781xtls2quvy.s3.us-east-2.amazonaws.com/treasure9_referer' --refer 'http://s3game.treasure'
Level 10
Here, we need to fetch an object which is in Infrequent Access storage.
Solution
# We can use the powerful query param of aws cli to just fetch those objects which are in infrequent access storgae
aws s3api list-objects --bucket s3game-level10-gac6tf83erp6 --query 'Contents[?StorageClass == `STANDARD_IA`]' --profile tmp
Takeaway
Amazon S3 offers a range of storage classes designed for different use cases:
- Standard for general-purpose storage of frequently accessed data
- Intelligent-Tiering for data with unknown or changing access patterns
- Standard-Infrequent Access (Standard-IA) and One Zone-Infrequent Access (One Zone-IA) for long-lived, but less frequently accessed data
- Glacier and Glacier Deep Archive for long-term archive
Level 11
Here we need to fetch an object which has been encrypted using client side encryption and uploaded to S3. This means that we can not fetch the object unless we have the encryption key.
Solution
# The encryption key is given in the hint
aws s3 cp --sse-c 'AES256' --sse-c-key 'UkXp2s5v8y/B?E(H+MbPeShVmYq3t6w9' 's3://s3game-level11-djq30a807iyq/treasure11_encryption' --profile tmp .
# Alternate way using aws s3api. Here we also need to give md5 of encryption key as additional integrity check
aws s3api get-object --bucket s3game-level11-djq30a807iyq --key treasure11_encryption --sse-customer-key 'UkXp2s5v8y/B?E(H+MbPeShVmYq3t6w9' --sse-customer-algorithm AES256 --profile tmp treasure11_encryption
Takeaway
S3 supports both server side and client side encryption.
VICTORY! - Level 12
Congrats! You have cleared the game!
Top comments (1)
What an awesome and creative idea, great work! It was cool to walk through things and see you hit on everything I would mention to someone learning S3. Keep up the great work.