A Story About CloudFront Root Objects, Chocolate, and Terraform Line-Ending Woes (and an unwanted replace)

Today I had to add a default root object to a cloudfront distribution. This was in order to resolve a vulnerability finding that was enforcing an AWS best practice (https://docs.aws.amazon.com/securityhub/latest/userguide/cloudfront-controls.html#cloudfront-1)

On my team we use Terraform to automate our AWS management, so I updated our terraform script to specify the root object:

resource "aws_cloudfront_distribution" "iatupdater-distribution" {
  ...
  default_root_object = "index.html"  
  ...
}

Initially I assumed I had to create and manually upload some dummy index.html to the origin bucket, then after looking in the Terraform docs for a bit found the aws_s3_object directive. This directive allows Terraform to declaratively upload files (objects) to an S3 bucket.

I created the dummy index file and placed it along with the rest of the TF files, so I only had to add the following block:

resource "aws_s3_object" "root-object" {
  bucket = aws_s3_bucket.iatupdater.id
  key = "index.html"
  source = "${path.module}/res/index.html"
}

"res/index.html" is the path to the html file to use as root object.

With all this in place I decided to try it out via terraform plan and if everything looked alright then a terraform plan would put everything in place. This change would be a simple upload and a quick (and safe) config change.

This is where things went weird.

To give some context, in our team we organize our Terraform files by feature. So running a plan/apply would go through all the relevant bits that are required to support a particular feature. This works fine for us since it allows to keep things together and consistent. Lower level and foundational constructs like VPNs or networking routes are handled on different Terraform files, the "feature" module consists of the higher level constructs that are specific to the application. Think things like S3 buckets, lambdas, queues, auto-scaling groups and so on.

So when I ran the plan it went through everything, including the encryption keys associated with the CloudFront distribution. Now, I didn't change the key for the update of the root object, however terraform plan was saying that it needed to be replaced which was not possible since the key was already in use. The weird thing was that on the diff shown by plan, the contents of the public key looked exactly the same.

Some research online let me to a few posts from other people that had encountered the same problem, and the suggested fixes ranged from adding a new line to the public key file to updating the aws provider in terraform.
https://stackoverflow.com/questions/74846676/updating-aws-cloudfront-public-key-encoded-key-with-the-same-value-forces-replac
https://github.com/hashicorp/terraform-provider-aws/issues/20081

I tried the suggested "fixes" and none worked. However the mention of adding new lines led me to start thinking about line endings. I am using a Windows workstation but when we run Terraform scripts through CI we use linux machines to run our Jenkins instances. So maybe the different line endings could be making Terraform (or AWS) think that the keys were different.

Next thing was to try the theory. In linux you can use the dos2unix command to easily change line endings from Windows style into Unix. Of course this doesn't exist in a Windows environment, nevertheless I tried the following to see what happened:

choco install dos2linux

bingo.

Now see if converting the file made a difference

terraform plan

This time Terraform didn't think it needed to recreate the key. So I was able to safely run terraform apply to create the index file and set it as the default root object of my CloudFront distribution.

To recap:

• Always configure your Cloudfront distributions with a root object
• Terraform can upload a file to an S3 bucket and set it as the root for the CDN
• When Terraform wants to recreate a public key with the same contents, it might be that its the line endings that are different (in particular when you operate in a multiple OS environment)
• In Windows you can use chocolatey to install dos2linux and change the line-endings of a file

Leaving this here in case someone runs into this same issue.