DEV Community: Fran

AWS CLI SSO made easy

Fran — Tue, 16 Apr 2024 16:39:45 +0000

As the pretty self-explanatory title reads, this is AWS CLI IAM Identity Center (SSO) authentication made easy. Without walking you through unneeded details that may only confuse you.

For additional context, the post describes how to authenticate users with AWS IAM Identity Center to get credentials to run AWS Command Line Interface (CLI) commands via SSO token provider configuration, as recommended by AWS, in an easy way. With the SSO token provider configuration, your AWS SDK or tool can automatically retrieve refreshed authentication tokens.

Getting started

First of all, you have to configure your SSO session, 'linking' your CLI to the AWS IAM Identity Center instance login page.

aws configure sso

The above command triggers a wizard that guides you through configuring an sso-session and a profile. The important parameters you need to fill are:

sso-session = your_session_name
sso_start_url = https://my-sso-portal.awsapps.com/start (taken from the IAM Identity Center login portal UI)
profile = default NOTE: Use the default profile name so every time you login to your AWS SSO session the CLI commands are automatically run using your commonly-used (or only) profile role. (In my case, FullAccess)
sso_role_name = leave as suggested or give it a descriptive name. (In my case, FullAccess)

The other parameters can be left as default.

Typically, an this is where I couldn't get full clarity from AWS documentation, you would only have to configure one sso-session, as there is usually a single IAM Identity Center instance or AWS Organization you need to access to. If you had to access multiple AWS Organizations or IAM Identity Center portals, configuring additional SSO sessions would be needed.

After introducing all the required information in the configuration wizard, this is how my ~/.aws/config file looks:

You can now configure different profiles for the different accounts and/or roles you have access to within the IAM Identity Center instance (AWS Organization). I am going to configure two additional profiles.

aws configure sso

Running the above command again walks you through the same wizard as before. The main difference now is that you have already configured an sso-session, so such SSO session name would be taken as the default session within which we are going to create a new profile. You could also manually type your existing sso-session name.

This is how my ~/.aws/config file looks now:

You could specify a different sso-session if you wanted to access another AWS Organization or IAM Identity Center portal.
aws configure sso-session

Alright, let’s recap!

I now have one sso-session (to my AWS Organization or IAM Identity Center instance) and three profiles:

default - FullAccess to the 123456789000 account
read-only - ReadOnlyAccess role to the 123456789000 account
dev-account-admin - DevAccess role to the 123456789011 account

A profile can be thought as an AWS account+role tuple.

We can now proceed to using the AWS CLI.

Login to the sso-session

The first time, or whenever the token expires, you can either log in to your default profile and sso-session (if you had named a default profile)

aws sso login

or specify the sso-session in case you had configured multiple SSO sessions.

aws sso login --sso-session <session-name>

Running CLI commands

Once logged in to the sso-session, you can simply run the CLI commands as usual without having to re-authenticate until the token expires. Usually after 8 hours, when the token expires, you will have to run the aws sso login command again to refresh the token.

Example:

aws s3 ls

Notice that, if you don’t specify a profile, the command will be run assuming your default profile, if you had any configured. The above command lists the S3 buckets in the 123456789000 account.

If you want to run the CLI command in the scope of a different account or particular role, you need to specify the appropriate profile:

aws s3 ls --profile dev-account-admin

The above command will list the S3 buckets in the 123456789011 account.

Logout from the sso-session

You can logout from your current sso-session before the token expires running

aws sso logout

Wait, what profile am I using?

You can also check which profile you are using running

aws sts get-caller-identity

I hope this post has helped you to easily get going with the AWS CLI using the IAM Identity Center token provider credentials. In any case, if you want to understand this in more detail (or get lost in the weeds :P), you can refer to the AWS documentation.

Terraform modules and GitHub Actions to deploy secure cloud infrastructure

Fran — Tue, 16 Aug 2022 15:09:45 +0000

Countless organizations struggle with standardizing the provisioning of cloud resources, eventually resulting in a cloud infrastructure chaos. Resources are created either programmatically or manually from the cloud provider console, without a prior automated security configuration review. Hence, multiple vulnerabilities are introduced into cloud environments, ultimately generating an extra effort for Cloud Security teams, who will then have to work on the remediation steps.

This post aims to standardize the deployment of resources to AWS (although it can be extrapolated to any other cloud provider) in a secure and automated fashion by reusing pre-defined Terraform modules and GitHub Actions. Essentially, as popularly said, "shifting left", moving security sooner in the development process to prevent insecure resources from being deployed in first place.

Following this approach, all Cloud Security teams need to worry about is defining the appropriate security best-practices for the different cloud resource types, and writing up the Terraform modules with such security features pre-enforced. The GitHub Actions CI/CD will take care of the rest. If the resource is not compliant with the defined rules, it will not be deployed, period.

Refer to HashiCorp's documentation to learn more about creating reusable Terraform modules.

Enough theory, let's go into action!

Creating our reusable Terraform module

We will need a GitHub repository to store our Terraform modules. Let's call it aws-terraform-modules. For this post's purpose, let's pretend that we are creating a module for deploying AWS Systems Manager Parameter Store parameters. Within our repository, we will create a directory called ssm-parameters. Inside such directory, at a minimum, we will need two files: variables.tf and main.tf.

Here is an example of the variables.tf file:

variable "name" {
    type = string
    description = "Display name of the SSM Parameter Store parameter name."

    validation {
        condition = (length(var.name) >= 1 && length(var.name) <= 2048 && can(regex("^/(test|uat|prod)/", var.name)))
        error_message = "SSM parameter names must be between 1 (min) and 2048 (max) characters long and follow the naming convention (test|uat|prod)/."
    }
}

variable "description" {
    type = string
    description = "Description of the SSM Parameter Store parameter as viewed in the AWS console."
}

variable "value" {
    type = string
    description = "Value of the SSM Parameter Store parameter. If parameter type is SecureString, the value should be retrieved from a GitHub secret."
}

variable "key_id" {
    type = string
    description = "Customer managed KMS key id or arn used to encrypt the SSM Parameter Store parameter if type is SecureString."
    default = "alias/aws/ssm"
}

variable "tier" {
    type = string
    description = "Tier of the SSM Parameter Store parameter. Valid types are Standard, Advanced and Intelligent-Tiering."
    default = "Standard"

    validation {
        condition = (contains(["Standard", "Advanced", "Intelligent-Tiering"], var.tier))
        error_message = "Unsupported SSM parameter tier. Valid tiers: [Standard, Advanced, Intelligent-Tiering]."
    }
}

variable "tags" {
    type = map(string)
    description = "Key-value map of resource tags to be associated with the SSM Parameter Store parameter."

    validation {
      condition = (
                    contains(keys(var.tags), "owner") &&
                    contains(keys(var.tags), "env")
      )
      error_message = "Incomplete or invalid set of tags specified for the SSM Parameter Store parameter. Tags for every resource are required to have the following keys:\n\t- owner\n\t- env."
    }
}

The validation block can be used to specify custom conditions, based on our security standards, and produce error messages if the condition evaluates to false. Additionally, you can provide the default value for non required variables, as seen in the snippet above.

As you would expect, a main.tf file example is as follows:

resource "aws_ssm_parameter" "ssm_parameter" {
    name        = var.name
    value       = var.value
    description = var.description
    type        = "SecureString"
    key_id      = var.key_id
    tier        = var.tier
    tags        = var.tags
}

All we are doing there is calling the Terraform resources as specified in HashiCorp's documentation. In this case, the aws_ssm_parameter resource.

That's all we need to configure our reusable Terraform module. In our example, we are basically enforcing a naming convention and the use of specific tags, as well as only allowing the creation of encrypted parameters (using the "SecureString" type).

Let's now jump onto how to use it to deploy new AWS SSM parameters from GitHub Actions!

Using our Terraform module

Calling our reusable Terraform module from another GitHub repository where we maintain the actual AWS resources code is as simple as the following:

module "test-ssm-parameter" {
    source      = "github.com/<org-name>/aws-terraform-modules//ssm-parameters"
    name        = "/test/test-parameter"
    description = "CloudSec test SSM Parameter Store parameter."
    value       = "test-ssm-parameter-value"
    tags        = {
        owner    = "org-owner",
        env      = "test"
    }
}

Note that, in our snippet sample, we are exclusively specifying the values for the required variables. All other variables from our module not explicitly assigned will take their default value (type, key_id and tier). We could, of course, overwrite those optional variables, as long as the new values meet the pre-defined validation criteria.

Easy, right? Let's take a look at how we can automatically test that our SSM parameter configuration is valid!

Configuring our GitHub Workflows

The last piece of the puzzle is configuring our Terraform CI/CD pipeline. Assuming our organization follows a mainline development model, the CI/CD workflow would look similar to the following:

A terraform validate and terraform plan are triggered every time a developer opens a new Pull Request (PR) to the main branch, or when an open PR is updated. This step essentially verifies that our Terraform configuration is correct and compliant with the validation rules defined for our module's variables, and outlines the terraform plan. If the checks fail, we will see a detailed error of what's wrong so we can go and update our PR to fix it, triggering a new Workflow. And, of course, we won't be able to merge our PR and, thus, deploy our AWS resources, until our configuration is compliant.
If the plan looks as expected and all the validation checks pass successfully, we shall be able to merge our PR. As soon as the PR is merged, a terraform apply is kicked off, making the corresponding changes in our AWS cloud environment. This ensures that everything that is on our main branch is in sync with what is deployed to production.

So, how do translate those two steps into GitHub Workflows?

You will need a GitHub Personal Access Token (PAT) with rights to pull the code from the aws-terraform-modules repository and AWS access keys with the appropriate permissions to deploy the resources to AWS. We are storing such credentials in GitHub Secrets.

Knowing that GitHub Workflows are defined in the .github/workflows directory within your repository, here are the templates of the two Workflows that we will be creating:

terraform-plan.yml:

name: Terraform Plan
on:
  pull_request:
    branches:
      - $default-branch

env:
  TF_VERSION: 1.1
  TF_VAR_aws-access-key: ${{ secrets.AWS_ACCESS_KEY_ID }}
  TF_VAR_aws-secret-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  TF_VAR_terraform-state-bucket: 's3-terraform-state-files'
  TF_VAR_terraform-state-bucket-namespace: /
  TF_VAR_terraform-state-bucket-key: 'aws-env/terraform.tfstate'
  TF_VAR_terraform-state-bucket-aws-region: 'us-east-1'
  TF_VAR_terraform-dynamodb-state-locking-table-name: 'dynamodb-terraform-state-files'

jobs:
  terraform:
    name: Plan
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: GitHub Auth
        run:  
          git config --global url."https://oauth2:${GITHUB_TOKEN}@github.com/<org-name>".insteadOf "https://github.com/<org-name>" 
        env:  
          GITHUB_TOKEN: ${{ secrets.GITHUB_PAT }}

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1.2.1
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Terraform Get Update
        run: terraform get -update

      - name: Terraform Init
        run: |
          terraform init \
          -backend-config="dynamodb_table=${{ env.TF_VAR_terraform-dynamodb-state-locking-table-name }}" \
          -backend-config="access_key=${{ secrets.AWS_ACCESS_KEY_ID }}" \
          -backend-config="secret_key=${{ secrets.AWS_SECRET_ACCESS_KEY }}" \
          -backend-config="bucket=${{ env.TF_VAR_terraform-state-bucket }}" \
          -backend-config="key=${{ env.TF_VAR_terraform-state-bucket-key }}" \
          -backend-config="region=${{ env.TF_VAR_terraform-state-bucket-aws-region }}"

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        run: terraform plan

terraform-apply.yml:

name: Terraform Apply
on:
  push:
    branches:
      - $default-branch

env:
  TF_VERSION: 1.1
  TF_VAR_aws-access-key: ${{ secrets.AWS_ACCESS_KEY_ID }}
  TF_VAR_aws-secret-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  TF_VAR_terraform-state-bucket: 's3-terraform-state-files'
  TF_VAR_terraform-state-bucket-namespace: /
  TF_VAR_terraform-state-bucket-key: 'aws-env/terraform.tfstate'
  TF_VAR_terraform-state-bucket-aws-region: 'us-east-1'
  TF_VAR_terraform-dynamodb-state-locking-table-name: 'dynamodb-terraform-state-files'

jobs:
  terraform:
    name: Apply
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: GitHub Auth
        run:  
          git config --global url."https://oauth2:${GITHUB_TOKEN}@github.com/<org-name>".insteadOf "https://github.com/<org-name>" 
        env:  
          GITHUB_TOKEN: ${{ secrets.GITHUB_PAT }}

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1.2.1
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Terraform Get Update
        run: terraform get -update

      - name: Terraform Init
        run: |
          terraform init \
          -backend-config="dynamodb_table=${{ env.TF_VAR_terraform-dynamodb-state-locking-table-name }}" \
          -backend-config="access_key=${{ secrets.AWS_ACCESS_KEY_ID }}" \
          -backend-config="secret_key=${{ secrets.AWS_SECRET_ACCESS_KEY }}" \
          -backend-config="bucket=${{ env.TF_VAR_terraform-state-bucket }}" \
          -backend-config="key=${{ env.TF_VAR_terraform-state-bucket-key }}" \
          -backend-config="region=${{ env.TF_VAR_terraform-state-bucket-aws-region }}"

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        run: terraform plan

      - name: Terraform Apply
        run: terraform apply -auto-approve

Once we have the two GitHub Workflows configured in our repository, all the previously explained magic will happen automatically, exclusively allowing secure and compliant cloud infrastructure to be launched to production.

Say goodbye to fighting developers to adhere to cloud security configurations guidelines and remediating vulnerabilities after they are introduced in your cloud environment. In addition, you won't need to grant excessive privileges to your AWS developers any more, as everything will be deployed from the CI/CD pipeline ;)

Automate AWS access key rotation with GitHub Actions

Fran — Mon, 14 Feb 2022 16:23:00 +0000

From a security perspective, there is no need to explain why rotating your AWS access keys regularly is "key" ;), regardless of whether you are bound by laws and regulations. In any case, you can read a quick rationale in this AWS post .

On the other hand, if you have worked with AWS IAM for a while, you have probably struggled with access key rotation. This isn't a tough task at all. In fact, the AWS post above shows how easily we can rotate a user's AWS keys from the CLI. No additional complication doing so from the AWS console. As long as we have the required permissions, this is easy peasy. The problem comes when we have to automate this for all IAM users in our organization and share their new access keys in a secure fashion.

There are other solutions out there to solve this problem. From using AWS native services like CloudWatch and Lambda (where you still have the new-key-secure-share issue with the user), to custom scripts that you execute and rotate the access key on the user's machine. The latter meaning that we have to rely on the user to update their keys, something that my experience has taught me to avoid :)

In this post, I will explain how to automate key rotation for IAM users with access keys older than 90 days using GitHub Actions and a python script. I won't get into the weeds about how I have accomplished this for my organization's specific needs, but you'll get the idea ;)

First, like for any GitHub Workflow, we need to define our trigger. In our case, we want the workflow to run daily.

name: Daily AWS access key rotation check

on:
  #Every day at 5:55 AM UTC (23:55 CST) '55 5 * * *'
  schedule:
    - cron:  '55 5 * * *'

Then, we will need to define our elevated account's access keys as environment variables, which will be used to rotate every IAM user's access keys on its behalf. I have those variables stored as secrets in GitHub Secrets.

env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

And finally, we get to the fun part of the workflow, where we write the required steps to enable the GitHub MacOS runner to execute our python script smoothly. We need to checkout our code so the runner can access it, inject the secrets into our script, and install python and boto3.

jobs:
  AWS-key-rotation:
    name: Quarterly AWS access key rotation
    runs-on: macos-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v2

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.DEVOPS_TF_AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.DEVOPS_TF_AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - uses: actions/setup-python@v2
        with:
          python-version: 3

      - name: Install boto3
        run: |
          pip install boto3

      - name: Secret Injection MAILBOX credentials
        run: |
          sed -i "" 's/MAILBOX_EMAIL/${{ secrets.MAILBOX_EMAIL }}/' scripts/rotate-aws-keys.py
          sed -i "" 's/MAILBOX_PASSWORD/${{ secrets.MAILBOX_PASSWORD }}/' scripts/rotate-aws-keys.py
          cat scripts/rotate-aws-keys.py

      - name: Run AWS key rotation script
        run: |
          chmod +x scripts/rotate-aws-keys.py
          python3 scripts/rotate-aws-keys.py

The complete GitHub Workflow will look like this:

name: Daily AWS access key rotation check

on:
  #Every day at 5:55 AM UTC (23:55 CST) '55 5 * * *'
  schedule:
    - cron:  '55 5 * * *'

env:
  AWS_ACCESS_KEY_ID: ${{ secrets.DEVOPS_TF_AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.DEVOPS_TF_AWS_SECRET_ACCESS_KEY }}

jobs:
  AWS-key-rotation:
    name: Quarterly AWS access key rotation
    runs-on: macos-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v2

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - uses: actions/setup-python@v2
        with:
          python-version: 3

      - name: Install boto3
        run: |
          pip install boto3

      - name: Secret Injection MAILBOX credentials
        run: |
          sed -i "" 's/MAILBOX_EMAIL/${{ secrets.MAILBOX_EMAIL }}/' scripts/rotate-aws-keys.py
          sed -i "" 's/MAILBOX_PASSWORD/${{ secrets.MAILBOX_PASSWORD }}/' scripts/rotate-aws-keys.py
          cat scripts/rotate-aws-keys.py

      - name: Run AWS key rotation script
        run: |
          chmod +x scripts/rotate-aws-keys.py
          python3 scripts/rotate-aws-keys.py

That was straightforward, right? We already have a workflow that will be running our python script every day. Let's now tell the python script what to do!
The following libraries need to be imported.

import datetime
from datetime import date
import dateutil
from dateutil import parser
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import boto3
from botocore.exceptions import ClientError
iam_client = boto3.client('iam')

The following steps detail what the script does:

Iterate through all the IAM users in our AWS account. We need to use pagination because the default AWS API call only returns 100 items per page.
- Check the key age for each user with an active access key.
  - If the key has been active for 90 days, rotate it (delete the old key and create a new one).
  - If the key has been active for 83 days, send a one-week email reminder.
  - If the key has been active for 89 days, send a one-day email reminder.

try:
    marker = None
    paginator = iam_client.get_paginator('list_users')
    # Need to use a paginator because by default API call only returns 100 records
    for page in paginator.paginate(PaginationConfig={'PageSize': 100, 'StartingToken': marker}):
        print("Next Page : {} ".format(page['IsTruncated']))
        u = page['Users']
        for user in u:
            keys = iam_client.list_access_keys(UserName=user['UserName'])
            for key in keys['AccessKeyMetadata']:
                active_for = date.today() - key['CreateDate'].date()
                # With active keys older than 90 days
                if key['Status']=='Active' and active_for.days >= 90:
                    print (user['UserName'] + " - " + key['AccessKeyId'] + " - " + str(active_for.days) + " days old. Rotating.")
                    delete_key(key['AccessKeyId'], user['UserName'])
                    create_key(user['UserName'])
                # Send a notification email 7 days before rotation
                elif key['Status']=='Active' and active_for.days == 83:
                    send_email("MAILBOX_EMAIL", "MAILBOX_PASSWORD", "recipient_email", subject_1_week, body_1_week)
                    print ("Email sent to " + user['UserName'] + " warning of key rotation in a week.")
                # Send a notification email 1 day before rotation
                elif key['Status']=='Active' and active_for.days == 89:
                    send_email("MAILBOX_EMAIL", "MAILBOX_PASSWORD", "recipient_email", subject_1_day, body_1_day)
                    print ("Email sent to " + user['UserName'] + " warning of key rotation tomorrow.")

except ClientError as e:
    print("An error has occurred attempting to rotate user %s access keys." % user['UserName'])

Since I wanted to be a nice guy, I will be sending a reminder to the users a week before and a day before their key will expire (you don't have to, and can shorten your script by skipping those conditional statements). That means that I will be sending a few emails, thus I defined a send_email function for such task. In my case, I used Office365.

def send_email(sender, password, recipient, subject, body):
    mimemsg = MIMEMultipart()
    mimemsg['From']=sender
    mimemsg['To']=recipient
    mimemsg['Subject']=subject
    mimemsg.attach(MIMEText(body, 'html'))
    connection = smtplib.SMTP(host='smtp.office365.com', port=587)
    connection.starttls()
    connection.login(sender,password)
    connection.send_message(mimemsg)
    connection.quit()

At this point, we are only missing what the delete_key and create_key functions do.
The delete_key function couldn't literally be easier.

# Delete an specified access key for a user
def delete_key(access_key, username):
    try:
        iam_client.delete_access_key(UserName=username, AccessKeyId=access_key)
        print("%s has been deleted." % (access_key))
    except ClientError as e:
        print("The access key with id %s cannot be found" % access_key)

create_key isn't much harder. We simply need to collect the newly created AWS access key id and secret access key, as well as the IAM user email address (depending on your organization, you may retrieve it from the username, as a tag...), to share the rotated keys with the corresponding end users via email.

# Create a new AWS access key and share it with the user via email
def create_key(username):
    access_key_metadata = iam_client.create_access_key(UserName=username)['AccessKey']
    access_key = access_key_metadata['AccessKeyId']
    secret_key = access_key_metadata['SecretAccessKey']
    recipient_email = "end_user_email"
    subject = "AWS API Key rotation"
    body = """\
        <html>
        <head></head>
        <body>
            <p>Include your message body here, including your access_key and secret_key</p>
        </body>
        </html>
        """
    send_email("MAILBOX_EMAIL", "MAILBOX_PASSWORD", "recipient_email", subject, body)
    print("New access key (%s) created for %s" % (access_key, username))

As promised, I wasn't going to get into the weeds with org specifics. But, as you can imagine, you will need email account credentials and an encryption tool to send the updated keys securely with the user. In my scenario, I've used Virtru and Office365 email rules. You might also want to include some instructions in your email body explaining the end users how they can copy and paste the new access key in their ~/.aws/credentials file to avoid running into Access_Denied errors.

At this point:

You are complying with security regulations and best practices by periodically rotating AWS access keys
You aren't chasing your developers to rotate their keys (and praying that these keys don't get compromised in the meantime!)
Developers won't be running into permission issues attempting to rotate their keys themselves
It becomes the developers' responsibility to update their local credentials file with the shared keys

And that's all folks! Hope this solution helps you relieve your AWS access key 90 day rotation headache.

Maintaining a secure cloud environment isn't that complicated if you are determined to keep it secure :)