AWS Key Management Service (KMS) allows you to create/manage encryption keys that are used to encrypt/decrypt and sign/verify data. AWS internally uses KMS across many services for encryption, so even if you’re not familiar with it, there’s a high chance you’ve already used it while working with other services (like while creating an S3 bucket!).
In this post, we will explore how KMS uses envelope encryption, understand different KMS key types, and finally do hands-on with KMS using AWS CLI commands.
Let’s start with the story of Pandora to get a gist of envelope encryption.
Pandora And The Envelope Encryption
The story goes like this, Zeus (king of Olympus) gave Pandora a box and asked her to never open it at any cost. But curious Pandora got curious and opened it anyway.
Zeus could have given Pandora a locked box with a key and asked to not open it. In this scenario, Pandora might lose the key. Or she might open the locked box and claim she doesn’t know who did it.
Zeus can also try to lock the box using his own key and give only the box to Pandora. In this case, Zeus would be responsible for keeping the key safe. Also, someone might steal the key from Zeus and open the box.
To solve these problems, Zeus could have used envelope encryption. First, he locks the box using a unique key. Then, instead of giving her the raw key, he encrypts it using his lightning bolt weapon⚡(his strongest secret). Now, he gives Pandora both the locked box and the encrypted key. This ensures:
- Pandora can’t open the box without Zeus’ help (Zeus must decrypt the key first to unlock the box).
- If Pandora loses the key, no one else can use it.
- To unlock the box, Pandora must call Zeus to decrypt the key. Later, she can’t deny granting access by claiming she doesn’t know who opened the box.
Envelope Encryption
The above story is an example of envelope encryption, where the key used to encrypt the message (Pandora’s box of hope in our story) is encrypted itself (often using a more secret key ⚡). Then both the encrypted object (box) and encrypted key are stored together.
It is called envelope encryption because the key used for locking the box is wrapped (enveloped/encrypted) by a super-secure master key. Then both locked object and encrypted key are packed into a single structure (thus forming an envelope).
The purpose of encrypting a key with another key (multiple layers of keys) is to provide centralized key management and auditing while ensuring that the primary key is never exposed in an unencrypted form. Check out the article on FreeCodeCamp on how envelope encryption helps to manage encryption at scale.
KMS Key Types
Based on ownership, KMS offers three key types –
AWS Owned Keys (free) =>These keys provide default encryption for most of the services. You might have seen the SSE-S3 option while creating an S3 bucket. Everything is handled in the background and these keys are not visible anywhere. No visibility on audit trails or key rotation.
AWS Managed Keys (free) => These keys are also to be used by AWS services only (aws/s3 or aws/rds). However, customers can see key policies and CloudTrail events for AWS-managed keys. Everything else is managed by AWS.
Customer Managed Keys ($1/month) => You (customer) own this key and can view audit trails and rotate on demand (Careful! Each key rotation adds $1/month to the bill since all versions are retained). If you want to call KMS API to create a key and use it in your applications, this is the type that you need.
Based on cryptographic structure, a key can be symmetric or asymmetric. For the rest of the tutorial, we will focus on customer-managed symmetric KMS keys.
Envelope Encryption In Action – KMS Workflow
Fair warning before we proceed forward, you are going to encounter a lot of keys. Be prepared!
Step1: Key Creation
When a customer-managed KMS key is created, AWS uses HSM (hardware security module) hardware to create HSM Backing Key (HBK⚡). Since it’s not possible to keep HBK in memory of HSM forever, it must be exported to durable storage. But HBK cannot leave the HSM unencrypted. So, HBK is encrypted using HSM-managed domain keys to generate an EKT (Exported Key Token) and stored in a database. Then our newly created KMS key points to this EKT.
Yes, a KMS key (also called customer master key CMK) does not contain actual key material. Instead, it acts as a pointer to the EKT (which is an encrypted version of the HBK itself)!
Step2: Envelope Encryption
When the user (or services like S3) requests a data key (also called Data Encryption Key but we will call it data key because the term DEK is needed for another key – I promised too many keys ☺️) from KMS to encrypt an object, it also tells which KMS key to use (either by specifying key id or key alias). As we know, the KMS key serves as a pointer to the EKT. So EKT is taken to HSM and converted back to HBK (HSM Backing Key ⚡). After that, HBK is combined with Nonce (random number) to create a Derived Encryption Key (DEK).
Inside the HSM, a data key is also generated (also called Customer Data Key CDK or Data Encryption Key). This key will be used to lock the box (encrypt the object). Then the data key is encrypted using DEK (derived encryption key) and both the plaintext data key and encrypted data key are sent to the user (or service). For now, DEK will be discarded by HSM. Remember, the DEK encrypts the data key, which in turn encrypts the object.
Now user can encrypt a file/object using the plaintext data key and immediately discard it. Then store the encrypted file and encrypted data key together.
Step3: Envelope Decryption?
In storage, we have an encrypted object and encrypted data key. To decrypt the object, we again need the plaintext data key but we deleted it just after encrypting the file/object (even S3 would also have done the same).
To get the plaintext data key, we send the encrypted data key to KMS as part of the decrypt API. AWS HSM extracts the KMS key ID/alias and nonce from the encrypted data key. From the KMS key, EKT is traced and HBK is calculated.
With HBK and nonce, the system regenerates the DEK (which was originally used to encrypt the data key). Then DEK is used to decrypt the data key and the plaintext data key is returned to the user.
Finally, the plaintext data key can be utilized to decrypt the encrypted file/object.
KMS Hands-On Using AWS CLI – Less Than 4KB File Size
For files smaller than 4KB, no need to request a data key. Just call encrypt/decrypt API and KMS will take care of all other things internally.
- Create a symmetric KMS customer-managed key with Alias “learn”. Make sure to have encrypt and decrypt privileges for your AWS user.
- Open AWS CLI (top right in console), and create a file named myFile.txt.
Call KMS encrypt command
aws kms encrypt --key-id alias/learn --plaintext fileb://myFile.txt --output text --query CiphertextBlob --region ap-south-1 | base64 -d > myFile.encryptedNow you should be able to see a new encrypted file myFile.encrypted.
To decrypt the file, use the KMS decrypt command
aws kms decrypt --ciphertext-blob fileb://myFile.encrypted --output text --query Plaintext --region ap-south-1 | base64 –dIn CLI, you should be able to see the original text of myFile.txt.
KMS Hands-On Using AWS CLI – More Than 4KB File Size
For files bigger than 4KB, we can call GenerateDataKey API to get the data key and encrypted data key.
Get data key
aws kms generate-data-key --key-id alias/learn --key-spec AES_256 > data_key.jsonStore both data keys
jq -r '.Plaintext' data_key.json | base64 --decode > data_key_plaintext
jq -r '.CiphertextBlob' data_key.json | base64 -d | data_key_encryptedEncrypt the “test.txt” file and delete the plaintext data key
openssl enc -aes-256-cbc -pbkdf2 -salt -in test.txt -out test.encrypted -pass file:data_key_plaintext && rm data_key_plaintextFor decrypting the file, decrypt the encrypted data key
aws kms decrypt --ciphertext-blob fileb://data_key_encrypted --output text --query Plaintext --region ap-south-1 | base64 --decode > data_key_plaintextThen decrypt the locked file
openssl enc -d -aes-256-cbc -pbkdf2 -salt -in test.encrypted -out test.decrypted.txt -pass file:data_key_plaintext
KMS Use Cases – Project Ideas
Here are a few ideas to play with KMS:-
- Password manager using KMS and DynamoDB.
- JWT-based authentication – KMS can be used to sign and verify JWTs.
- Use KMS to implement passkeys using FIDO2.0.
Share your ideas in the comments!
Conclusion – KMS is Not Scary Anymore!
Hope this tutorial helped you to understand the ins and outs of AWS KMS. Let me know your feedback and thoughts, Thanks 🙂
Top comments (0)