Reusing Providers in Terraform

(if you want to skip the BS,here's an example in my GitHub)

I recently came into a situation where I needed to import multiple S3 buckets from AWS. For the most part, importing the resources is pretty straightforward. Especially when compared to other service providers.

However, in this case the resources were all over the place as far as regions. This means that you couldn't necessarily use a default configuration to import and manage the resources. This is a weird limitation in the provider where it seems that the import can only search the region of the default provider unless told otherwise.

Anyways, since this is a thing I've ran into more than once, it felt important to document it somewhere.

Some Assumptions

I'm operating off the assumption that you have Terraform and (probably) the AWS CLIs installed.

I'm also operating on the assumption that you already have at least the base configuration of the AWS provider working with terraform. You'll need a secret key and all of that setup.

Main.tf

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# The us-east-1 instance
provider "aws" {
  region = "us-east-1"
}

# Create a bucket in us-east-1
resource "aws_s3_bucket" "east_1_example" {
  bucket = "east-1-example"

  tags = {
    Environment = "Dev"
  }
}

If you've worked in Terraform, a lot of this is straight forward.

First, we're establishing our required provider as "hashicorp/aws" so we can import that provider and the correct version.

The provider requires:

1. region
2. access_key
3. secret_key

Obviously, we NEVER HARDCODE THE ACCESS OR SECRET KEY INTO THE CONFIG so we've passed those in as environment variables and just set the provider's region to us-east-1. This means that all resources will be created in that region and, importantly, when importing resources you'll only be able to import resources from that region.

Alias

This is the thing that took me too long to figure out. In the old days, you used to configure a second provider name. At least I think that's how it worked. Honestly, it's not something I had to do a lot until recently. Now, we can split a provider using alias in the provider configuration.

Here's an updated version of the main.tf:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# The us-east-1 instance
provider "aws" {
  alias = "east-1"
  region = "us-east-1"
}

# Create a bucket in us-east-1
resource "aws_s3_bucket" "east_1_example" {
  provider = aws.east-1
  bucket = "east-1-example"

  tags = {
    Environment = "Dev"
  }
}

This version is pretty much the same. The big change is that we're adding the alias property to the provider configuration and setting it to east-1. Essentially, we're giving it another name.

To use that, we add the provider property to the storage bucket and specify we want to use the aws.east-1 provider. Again, this is masking (aliasing...wow) and telling the resource we want to use the aws provider with the name east-1.

In this configuration, we could have a provider with no alias and that would be the default and then we could work from there.

To that end, let's create a bucket in us-west-1 using an aliased provider.

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# The us-east-1 instance
provider "aws" {
  region = "us-east-1"
}

# The us-west-1 instance
provider "aws" {
  alias = "west-1"
  region = "us-west-1"
}

# Create a bucket in us-east-1
resource "aws_s3_bucket" "east_1_example" {
  bucket = "east-1-example"

  tags = {
    Environment = "Dev"
  }
}

# Create a bucket in us-west-1
resource "aws_s3_bucket" "west_1_bucket" {
  provider = aws.west-1
  bucket = "west-1-example"

  tags = {
    Environment = "Dev"
  }
}

This example will create a bucket in us-east-1 as that's our default and a second in us-west-1 using the aliased provider. In the case of an import, specifying the provider in the resource prior to import will make sure the terraform import command looks in the right region for the resource.

Summary

I don't know how many times I've run into this in the last few months. If you made it this far, that's great. I'm sure I'll be back in here in a few months trying to remember how I did this.

I hope you have a nice day.