DEV Community: Josh Simpson

I'm deleting all of my AWS IAM keys

Josh Simpson — Sat, 18 Mar 2023 19:06:30 +0000

I’m done with AWS keys.

Keeping AWS keys around is like having a key to your house. You have to keep the keys safe, make sure you’ve got access to them any time you need them, and if somebody ever managed to steal a key or even a copy of that key, they could walk straight in whilst you’re out, and the next thing you know your sofa is gone, your freezer is blasting hot air, and your TV is running up thousands in bitcoin mining costs.

… or something like that.

Anyway, that’s why I moved all of my pipelines to OIDC.

What is OIDC?

OpenID Connect (OIDC) is an authentication protocol that allows users to authenticate with an identity provider (IDP) and obtain a token that can be used to access protected resources. OIDC is built on top of the OAuth 2.0 protocol and provides additional features such as user authentication and user information.

In the above analogy, OIDC is kind of like having a friend standing at the door of your house. That friend checks to make sure you’re really you before they let you in by checking with somebody you’ve both agreed you can trust.

When you’re building a pipeline, a lot of the time you have to give your tooling some pretty uncomfortable access. At the very least, it’ll likely be able to build and push to an artifact repository such as ECR or S3, and may even be able to start processes in your cloud environment. You don’t want malicious actors having access to these sorts of roles.

In this post I’m going to run through how to set up an IDP using Terraform to work with GitHub Actions, and a sample GitHub Action workflow that authenticates with our AWS environment using OIDC.

How to use OIDC for AWS in GitHub Actions

Assumptions I make about the reader

Expanding out on any topic could take us all the way to the absolute foundations of concepts, and neither of us want me to write that much, so I’m including some basic assumptions about you, the reader. If these assumptions are wrong then let me know, and maybe I’ll write about that next:

You’re familiar with Terraform
You’re familiar with GitHub Actions

Setting up an OIDC IDP

The Terraform block for setting up an IDP is actually pretty simple:

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [""]
}

But before we can run it, we do need to fill out the thumbprint_list variable. These thumbprints are how AWS verifies that an OIDC token has actually been provided by GitHub (as they’re actually the thumbprint of a certificate that the provider will advertise).

AWS provide a very handy guide for getting the thumbprint for a given url here - you can either follow these instructions using the url in the above terraform block OR you can use a little docker image I created just for you!*

As we’re talking about security in our build tooling and supply chains, I’m going to take a moment to remind folks to always double check before running / trusting a tool, especially if it’s in the security space! You can find the repository for the Docker image I’m about to show you here

If you have Docker installed, you can simply run:

docker run snufflufagus/oidc <URL>

# in this instance, it would be

docker run snufflufagus/oidc https://token.actions.githubusercontent.com

Once you have your thumbprint, throw it in your Terraform block (this thumbprint is correct at the time of publishing - 17th March, 2023).

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = ["6938fd4d98bab03faadb97b34396831e3780aea1"]
}

Finally:

terraform apply

If you want to check, you'll be able to find your new IDP in the IAM service in AWS, under Identity Providers.

Creating an IAM role to use our OIDC token with

Now that we’ve got our IDP set up, we need an IAM role to authenticate as. For the purposes of this tutorial, we’re going to push a file into an existing S3 bucket, but ultimately anything that you want to do can be achieved by swapping out the permissions in the IAM policy document we create.

First, we need a couple of data blocks:

data "aws_iam_policy_document" "assume_role_policy" {
  statement {
    effect = "Allow"
    actions = ["sts:AssumeRoleWithWebIdentity"]

    principals {
      type        = "Federated"
      identifiers = [aws_iam_openid_connect_provider.github.arn]
    }

    condition {
      test     = "StringLike"
      variable = "token.actions.githubusercontent.com:sub"
      values   = ["repo:<YOUR_GITHUB_USERNAME_OR_ORG_NAME_HERE>/<YOUR_REPO_NAME_HERE>:*"]
    }
  }
}

So what does this do?

The actions block tells AWS that some actors may assume the role. We don't want to make it so that anybody can do this, so we narrow it down using our principal and condition blocks:

principals - this block specifies that the thing trying to assume the role has to be via our IDP, which is a good start - but if we just left it at that, then anybody could use GitHub Actions to authenticate with this role. That's not good - so we use condition blocks to narrow it down.

condition - this particular block uses the StringLike condition (read more about conditions here) against the claims that our OIDC token will make. You find a full list of things you could test against in a GitHub OIDC token here, and change out sub for whatever value you'd like to test against.

In this block, we're specifying that only a specific GitHub repo can use this role, but the Action could come from any branch or Git ref (as dictated by the wildcard *). You could swap out the wildcard to specify a branch (which is useful if you do branch based deployment), or other Git compatible reference.

Now, by itself, this will do nothing - we need to attach it to a role!

resource "aws_iam_role" "deploy_role" {
  name = "deploy-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role_policy.json
}

resource "aws_iam_role_policy_attachment" "attach_deploy_policy" {
  role       = aws_iam_role.deploy_role.name
  policy_arn = aws_iam_policy.ecs_deploy_policy.arn
}

resource "aws_iam_policy" "ecs_deploy_policy" {
  name   = "deploy-policy"
  policy = data.aws_iam_policy_document.deploy_document.json
}

data "aws_iam_policy_document" "deploy_document" {
  statement {
    effect = "Allow"
    actions = [
      "s3:PutObject"
    ]
    resources = [
      "${aws_s3_bucket.test.arn}"
    ]
  }
}

resource "aws_s3_bucket" "test" {
  bucket = "test.man-yells-at.cloud"
  acl    = "private"

  versioning {
    enabled = true
  }
}

As this is just a demonstration, I've created an S3 bucket and given our role permissions to put objects in that bucket.

Now we've created a role, we've given it some permissions, and we've specified what can assume that role. Let's give it a whirl in a workflow!

Testing our new role

name: Test our new OIDC role!

on:
  push:
    branches: [main]
  workflow_dispatch:

permissions:
    id-token: write   # This is required for requesting the JWT
    contents: read    # This is required for actions/checkout

jobs:
  oidc-test:
    runs-on: ubuntu-latest
    steps:
      - name: Configure aws credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: eu-west-1

      - name: Sync the repo with S3
        id: sync-s3
        run:
          echo "testdata" > my-test-file.txt
          aws s3 cp my-test-file.txt s3://${{secrets.S3_BUCKET_NAME}}

We give the GitHub Action block the ARN of the role that we just created, and the region that we want to assume the role in, and then that session will persist through our workflow, which means in the next step we can create a file and copy it into S3!

It's important to note the permissions at the top of our flow - this tells Actions that this workflow is allowed to generate a token.

Finally, we're going to push that into our GitHub repository and see what happens. If you've been following along with my steps, you'll notice that I've used a couple of secrets for my AWS_ROLE_ARN and S3_BUCKET_ARN - you'll want to set those secrets up in GitHub before running this workflow!

Hopefully you've reached the end of this with a GitHub Action that's just done something in AWS without you having to give GitHub a set of credentials!

If you're looking to expand upon this, it's worth taking a look at what condition keys are available, and what claims you can use from GitHub's OIDC token for more secure / complex combinations and workflows.

It's also worth remembering that this isn't the pinnacle of security, it's just better in most cases. By implementing OIDC in my pipelines, I'm giving some degree of control over my security mechanisms to another entity, so I need to make sure I trust that entity before implementing this, I need to review that trust on a frequent basis, and I should absolutely combine it with other tools and mechanisms. There's a whole world of security tools within AWS and that's before even taking a look at the sea of vendors who all offer their own solutions.

I'll take my tin foil hat off now.

How I set up a Gatsby blog on AWS

Josh Simpson — Wed, 24 Aug 2022 00:00:00 +0000

Hi! In this post, I'm going to run through what I did to get my site https://man-yells-at.cloud onto the internet - the intent is to be a useful guide for anybody who's looking to do something similar, but as I'm writing this at the same time as I build everything, it'll likely include a bit of stream-of-consciousness on the process!

The best bit for you is that you get to skip the bits where I called AWS services foul things because I'd configured something wrong, and get right to the bits that work, with some explanations where things might have had me Googling or I had to backtrack on decisions.

The Stack

Gatsby

I won't be focusing too much on this in this post, but to get this built, I followed the Gatsby tutorial for a bit, threw a strop when I realised how much Gatsby loves GraphQL, and then begrudgingly came back when I realised that building much else was going to be a fair bit more work than I wanted to do.

Once I finished the guide I did a couple of modifications so that I could messily cram it all into a blog template theme built by Xiaoying Riley - you can find their templates here, which I'm going to hopefully be able to build and throw into AWS.

At a certain point, I realised that the routing wouldn't work without a plugin called gatsby-plugin-s3, which I gave the following config in my

gatsby-config

{
  resolve: `gatsby-plugin-s3`,
  options: {
    bucketName: "<BUCKET_NAME>",
    protocol: "https",
    hostname: "<DOMAIN_NAME>"
  }
}

AWS

Most of my interactions with AWS are through Terraform at the moment (and then a bit of ClickOps when I'm confused or trying to ascertain the state of a system).

Route53

Route53 is AWS' DNS service, which allows me to manage where my domain points to. By pointing my name servers at it, I can then automate setting up new records via Terraform.

One caveat to doing this is that currently the control planes for Route53 are located in AWS 'us-east-1' zone, which has seen a couple of major outages in the last year. If you have critical applications in production and DNS changes are part of your disaster recovery plan then make sure to consider this whilst building!

S3

AWS' Simple Storage Service serves as a suitable service to situate my very serious site. Whilst primarily advertised as a 'file storage' service, S3 is one of my favourite AWS services because of it's versatility. In this instance, we're going to use it as the place where we host our site, meaning we don't have to spin up and manage any pesky servers. I'm hoping this pleases the Serverless ~~cult~~ crowd.

Cloudfront

Cloudfront is a CDN that helps tie together the entire project by making sure my site is available as close to users as it can be in 'edge locations', and caching that site at those edge locations. This helps me by reducing latency for users, and reducing the amount of requests that need to fetch information from the S3 bucket itself, which is a massive cost saver.

The Guide

In broad steps, I'll be walking through:

Managing my externally purchased domain using AWS Route53
Hosting a Gatsby site in S3
Serving that site from Cloudfront

So let's get into it.

My setup

This is not prescriptive, but if you're following along then this is how I'm doing things:

Using Terraform to manage infrastructure, installed via tfenv
- Using a .tfvars file that I feed into my terraform apply for things like domain etc that might get repeated
Running everything in a new AWS Organizations sub account
The AWS CLI
Authenticating using Granted

Here's what my variables.tf file looks like:

variable domain_name {
  type = string
}

variable bucket_name {
  type = string
}

variable environment {
  type = string
}

I then fill out these records into a .tfvars file and use them as shown here

Managing my domain through AWS

Before setting up this site, I had purchased the domain https://man-yells-at.cloud through Namecheap (as they seem to always be decently priced, making any DNS changes isn't frustrating as hell, and they're familiar).

I could manually point domains at things later on in the guide, but I like being able to manage everything through Terraform, so I'm going to point my Namecheap domain at AWS. In order to do this, I'm going to create a little Terraform module to manage a Route53 Hosted Zone. I set up the provider and remote backend in a main.tf file, and then created a new file -

Route53

route53.tf

resource "aws_route53_zone" "main" {
  name = var.domain_name

  tags = {
    Name = var.bucket_name
    Environment = var.environment
  }
}

output "nameservers" {
  value = aws_route53_zone.main.name_servers
}

And running terraform apply. I included the output so that I'd get something that looks like this:

I then used these to change the nameservers settings in Namecheap. To check to make sure everything propagated and I used the correct nameservers, I created a TXT record by adding this block:

route53.tf

resource "aws_route53_record" "propagation_check" {
  zone_id = aws_route53_zone.main.zone_id
  name    = "test.${var.domain_name}"
  type    = "TXT"
  ttl     = 300
  records = ["teststring"]
}

And applying again. After a couple of minutes, I ran

dig -t txt test.man-yells-at.cloud

And saw the record I'd just set. So far, so good.

Creating an S3 bucket and uploading my site

I had to come back to this bit after a bit of a reminder as to how some types of site operate in S3. This was initially a bucket with absolutely no public read, where our Cloudfront distribution had the permissions necessary to serve the objects in it - a setup that is generally best practice as it means nobody is interacting directly with the bucket. Unfortunately this isn't feasible without breaking the way some routes work in Gatsby, so I'll write a different post about how to do that securely at some point!

With the above addendum in mind, I switched to making a publicly readable bucket.

S3 Bucket

resource "aws_s3_bucket" "site_bucket" {
  bucket = var.bucket_name

  tags = {
    Name = var.bucket_name
    Environment = var.environment
  }
}

resource "aws_s3_bucket_acl" "site_bucket_acl" {
  bucket = aws_s3_bucket.site_bucket.id
  acl    = "public-read"
}

resource "aws_s3_bucket_website_configuration" "site_config" {
  bucket = aws_s3_bucket.site_bucket.id

  index_document {
    suffix = "index.html"
  }

  error_document {
    key = "/404.html"
  }
}

resource "aws_s3_bucket_public_access_block" "public_access_block" {
  bucket = aws_s3_bucket.site_bucket.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

locals {
  s3_origin_id = "myS3Origin"
}

And another terraform apply.

Once the bucket and it's corresponding website configuration was set up, it was a simple matter of running the s3 deploy plugin in Gatsby

Getting this into Cloudfront

Okay so this got out of hand...

This bit is where it got frustrating. At first, as noted earlier - I had intended for the buckets to be locked down and only available via Cloudfront. It quickly became clear due to the way that routing works in Gatsby that this couldn't be the case, and so I slowly had to undo the pieces I had put in place to protect the bucket from direct access.

The most frustrating part of this was the subtle difference (when you're looking at code at least) that if you serve an S3 bucket on it's website endpoint, it changes from an s3_origin_config to a custom_origin_config which uses some different rules. It's a small gripe, but I am a drama queen and I will make a mountain out of this molehill.

Anyway, let's get building

Firstly, I knew I needed a certificate that I could attach to my Cloudfront distribution, so I made that first. Cloudfront only accepts certificates made in their us-east-1 zone, so remember this when you're building. I used a separate aliased Terraform provider, and considering how simple it makes things I used the AWS ACM module to set up the certificate - this handily does the minor lifting of validating the certificate using Route53 for me (but if you want to roll your own, Terraform gives you all the component pieces to do so pretty easily!):

Certificate

certificate.tf

provider "aws" {
  alias  = "us-east-1"
  region = "us-east-1"
}

module "acm" {
  source  = "terraform-aws-modules/acm/aws"
  version = "~> 3.0"

  domain_name  = var.domain_name
  zone_id      = aws_route53_zone.main.zone_id

  subject_alternative_names = [
    "*.${var.domain_name}",
  ]

  providers = {
    aws = aws.us-east-1
  }

  wait_for_validation = true
}

Applied that - this can take slightly longer if the DNS validation doesn't happen straight away - and I had my certificate!

Cloudfront

With the certificate made, I proceeded onto the Cloudfront distribution. This uses the S3 Website Endpoint as an origin (which doesn't accept HTTPS - this revelation took me another short while to figure out as I refreshed to multiple 504 pages). I made sure to include a redirect-to-https in the configuration so that we don't have anybody accessing the site over HTTP:

cloudfront.tf

resource "aws_cloudfront_distribution" "s3_distribution" {
  origin {
    domain_name = aws_s3_bucket.site_bucket.website_endpoint
    origin_id   = local.s3_origin_id

    custom_origin_config {
      http_port = 80
      https_port = 443
      origin_protocol_policy = "http-only"
      origin_ssl_protocols = ["SSLv3", "TLSv1.2", "TLSv1.1", "TLSv1"]
    }
  }

  enabled             = true
  is_ipv6_enabled     = true
  comment             = "Some comment"
  default_root_object = "index.html"

  aliases = [ var.domain_name ]

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD", "OPTIONS"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = local.s3_origin_id

    forwarded_values {
      query_string = false

      cookies {
        forward = "none"
      }
    }

    viewer_protocol_policy = "redirect-to-https"
    min_ttl                = 0
    default_ttl            = 3600
    max_ttl                = 86400
  }

  price_class = "PriceClass_200"

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  tags = {
    Environment = var.environment
  }

  viewer_certificate {
    acm_certificate_arn = module.acm.acm_certificate_arn
    ssl_support_method = "sni-only"
  }
}

Finally, I added one more block to my route53.tf:

route53.tf

resource "aws_route53_record" "site" {
  zone_id = aws_route53_zone.main.zone_id
  name    = var.domain_name
  type    = "A"

  alias {
    name = aws_cloudfront_distribution.s3_distribution.domain_name
    zone_id = aws_cloudfront_distribution.s3_distribution.hosted_zone_id
    evaluate_target_health = false
  }
}

Applying this all took the longest because Cloudfront distributions can take a little while (in their defence, they sort of have to take over the world).

Once the route had propagated... well, if you're reading this on https://man-yells-at.cloud then you're looking at it!

Conclusion

MVP is best

Some bits didn't work out the way I wanted, and that's okay! This is meant to be a rough side project that:

gives me the ability to post ✅
without having to worry about significant cost or security issues ✅
doesn't take long to set up and / or maintain ✅

It's hit the minimum of what I set out to do, and if it becomes infeasible to manage later down the line I can spend a bit of time moving the React bits into their own thing, and and making something a bit more fully formed later on. Maybe it's Lambdas!

Can we get more friendly?

Definitely the biggest issue I saw was between the Cloudfront -> S3 connection. Whilst typical setups are well documented, it took a fair bit of exploration and trial-and-error to get what I'd assume is a fairly common outlier up and running. Maybe it doesn't get documented a lot because there are friendlier platforms for Gatsby out there and I added complication by trying to put it on S3?

Stay tuned for more!

Getting this published once is all well and good, but I'm going to need to automate the deployment. In my next post, I'll build a secure pipeline that authenticates with AWS without needing an access key for deploying changes / new posts on my site.