DEV Community: John McMillan

tfenvy

John McMillan — Mon, 29 Jan 2024 17:52:40 +0000

Terraform / Git-Bash / Windows

For almost four years I'd been using a Macbook as my main work machine. One quality of life tool I'd gotten used to was tfenv, a lovely little tool for managing terraform versions & quickly switching between them.

I'm now using a Windows machine as my main work laptop having recently started a new job. I'm fairly pragmatic and don't really mind what OS my work laptop runs... at least until I found out that tfenv wasn't available on Windows/git-bash. Sad face.

I was jealous that I wouldn't be able to use tfenv any more... I wanted something like tfenv for git-bash... tfenvy was born.

If you're starting to think I'm only writing this article because I'm super proud of the punny name I came up with... you would be right, but still, this might be useful to you if you're using git-bash on Windows to manage terraform code.

'tfenvy' allows you to

List available versions of terraform (stable versions only).
List locally installed versions of terraform.
Download a specific version of terraform locally.
Activate a specific version of terraform to use.
Update to the latest version of terraform for windows.
Remove a version of terraform you no longer need.

I don't have an installer yet, but it's simple to setup:

Copy the script below and place it in a suitable PATH for you, making sure it's executable.
Update TF_DIR and replace YOURUSERNAME with your user name, and make sure you have a 'bin' folder, sorry - directory, there.

#!/usr/bin/sh

# A super basic attempt to recreate the benefits of choosing a tf version since tfenv isn't yet supported on windows.

# Set Variables:
TF_DOWNLOAD="https://releases.hashicorp.com/terraform/"
TF_DIR="/c/Users/YOURUSERNAME/bin"
NUMARGS=$#



####################################################################################################################
# FUNCTIONS - anything we might need to do more than once - write a function for.

#
# Function to find current ACTIVE version of terraform
#
list_active_tf () {
        CURRENT=`terraform -v 2> /dev/null | head -1 | cut -f 2 -d v`
        if [ -z $CURRENT ] ; then
                echo "TF not currently installed."
                exit 1
        else
                export CURRENT
        fi
}


#
# Function to list ALL available stable releases of terraform
#
list_online_versions () {
        curl -s $TF_DOWNLOAD | grep href | egrep -v "alpha|rc|beta" | sort -n | cut -f 3 -d /
}


#
# Function to find the LATEST stable release of terraform
#
list_latest_version (){
        LATEST=`list_online_versions | tail -1`
        export LATEST
}


#
# Function to compare semantic formatted versions (e.g. 1.2.3)
# Credit to: https://stackoverflow.com/questions/4023830/how-to-compare-two-strings-in-dot-separated-version-format-in-bash
#
vercomp () {
    if [[ $1 == $2 ]]
    then
        return 0
    fi
    local IFS=.
    local i ver1=($1) ver2=($2)
    # fill empty fields in ver1 with zeros
    for ((i=${#ver1[@]}; i<${#ver2[@]}; i++))
    do
        ver1[i]=0
    done
    for ((i=0; i<${#ver1[@]}; i++))
    do
        if [[ -z ${ver2[i]} ]]
        then
            # fill empty fields in ver2 with zeros
            ver2[i]=0
        fi
        if ((10#${ver1[i]} > 10#${ver2[i]}))
        then
            return 1
        fi
        if ((10#${ver1[i]} < 10#${ver2[i]}))
        then
            return 2
        fi
    done
    return 0
}


#
# Function to compare current version with latest available
#
compare_versions () {
        list_latest_version
        list_active_tf
        vercomp $LATEST $CURRENT
        if [ $? == "1" ] ; then
                echo "Update available. Latest=${LATEST}. Current=${CURRENT}."
        else
                echo "Latest version, ${LATEST}, already installed."
        fi
}


#
# Function to list installed versions of terraform
#
list_local_versions () {
        ls $TF_DIR | grep ^terraform-[0-9]*
}


#
# Function to activate the choosen version of terraform
#
activate_tf_version () {
        cp -p ${TF_DIR}/terraform-${WANTED}.exe ${TF_DIR}/terraform.exe
}


#
# Function to download a specific version of terraform
#
get_tf (){
        curl -s ${TF_DOWNLOAD}${WANTED}/terraform_${WANTED}_windows_amd64.zip -o /tmp/terraform_${WANTED}_windows_arm64.zip
        unzip -qp /tmp/terraform_${WANTED}_windows_arm64.zip > ${TF_DIR}/terraform-${WANTED}.exe
}

#
# Function to install the latest version of terraform
#
get_latest_tf () {
        list_latest_version
        WANTED=$LATEST
        get_tf
}


#
# Function to DELETE a specific version of terraform
#
remove_tf (){
        rm -i ${TF_DIR}/terraform-${WANTED}.exe
}


#
# Function to print usage message
#
print_usage () {
        echo "Usage:"
        echo "  -a <version>    Activate terraform <version>."
        echo "          e.g. `basename $0` -a 1.6.6"
        echo ""
        echo "  -d <version>    Download a specific terraform <version>."
        echo "          e.g. `basename $0` -d 1.0.1"
        echo ""
        echo "  -i      List all installed versions of terraform. "
        echo "          e.g. `basename $0` -i"
        echo ""
        echo "  -l      List all stable Terraform versions available from Hashicorp. "
        echo "          e.g. `basename $0` -l"
        echo ""
        echo "  -R <version>    Remove the specified terraform <version>."
        echo "          e.g. `basename $0` -R 1.0.0"
        echo ""
        echo "  -u      Update to latest stable version of terraform."
        echo "          e.g. `basename $0` -u"
        echo ""

}

# End of Functions section. Fun starts now!
####################################################################################################################


# Define Flags
while getopts 'a:ilud:R:' OPTION; do
        case "$OPTION" in
                a)
                        WANTED="$OPTARG"
                        activate_tf_version
                        ;;
                i)
                        list_local_versions
                        ;;
                l)
                        list_online_versions
                        ;;
                u)
                        get_latest_tf
                        activate_tf_version
                        ;;
                d)
                        WANTED="$OPTARG"
                        get_tf
                        ;;
                R)
                        WANTED="$OPTARG"
                        remove_tf
                        ;;
                *)
                        print_usage
                        exit 1
                        ;;
        esac
done
shift "$((OPTIND -1))"

if [ $NUMARGS == 0 ] ; then
        print_usage
        exit
fi

Generating dynamic subnets & AZs with Terraform's VPC module

John McMillan — Mon, 22 Jan 2024 13:14:35 +0000

The Challenge:

I often use the Terraform VPC module when creating environments in AWS. And I'm an advocate for writing reusable code where possible...

So it bugged me that my use of the VPC module relied on a hacky way to cater for setting up subnets in regions with differing numbers of Availability Zones.

So typically I'd done something like this:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.5.1"

  name            = var.vpc-name
  cidr            = var.vpc-cidr
  azs             = ["${var.region}a", "${var.region}b", "${var.region}c"]
  intra_subnets   = [cidrsubnet(var.vpc-cidr, 8, 0), cidrsubnet(var.vpc-cidr, 8, 1), cidrsubnet(var.vpc-cidr, 8, 2)]
  private_subnets = [cidrsubnet(var.vpc-cidr, 8, 10), cidrsubnet(var.vpc-cidr, 8, 11), cidrsubnet(var.vpc-cidr, 8, 12)]

}

As you can see, I'd define the region and vpc-cidr variables and then when it comes to defining things like intra_subnets and private_subnets I'd use the cidrsubnet function to carve up the CIDR into three subnets.

This approach is fine if you're only deploying to a region with three AZs.

But what if you want to deploy to ap-northeast-2 where there are 4 AZs, or us-east-1 where there are 6?

In those cases you'd need to update the code to add additional entries for each AZ - then consider what shifting up or down you had to do with the netnum argument in the cidrsubnet function.

So overall I had functioning code that could be reasonably quickly amended to cater for other regions. But it still wasn't very dynamic and I suspected it could be improved to be so.

Making it better:

Introducing the aws_availability_zones data source, along with the index function, and terraform locals, allowed me to modify the code so it caters for differing numbers of availability zones dynamically - meaning no changes to the code are necessary when deploying into different regions.

The code now looks like this:

data "aws_availability_zones" "my_azs" {
  state = "available"
}

locals {
  azs = data.aws_availability_zones.my_azs.names
  intra_subnets = [
    for az in local.azs : cidrsubnet(var.vpc_cidr, var.intra_subnet_size, index(local.azs, az))
  ]
  private_subnets = [
    for az in local.azs : cidrsubnet(var.vpc_cidr, var.private_subnet_size, (index(local.azs, az) + length(local.azs)))
  ]
}

module "vpc" {
  source               = "terraform-aws-modules/vpc/aws"
  version              = "5.5.1"
  name                 = var.vpc_name
  cidr                 = var.vpc_cidr
  azs                  = local.azs
  intra_subnets        = local.intra_subnets
  private_subnets      = local.private_subnets
}

This is better, and if you just needed to see that example and you understand what's happening, the rest of this blog probably isn't useful to you.

If you want to understand more about how the code above works, read on.

Lets look at each section, in turn:

data "aws_availability_zones" "my_azs" {
  state = "available"
}

This retrieves a list of all 'available' availability zones in the region you're deploying into.

This next section defining the locals is a little more complex, but it's not too bad when we break it down:

  azs = data.aws_availability_zones.my_azs.names

azs is straight forward enough, it simply the list of AZ's that our data resource discovered for us.

  intra_subnets = [
    for az in local.azs : cidrsubnet(var.vpc_cidr, var.intra_subnet_size, index(local.azs, az))
  ]

Defining intra_subnets relies on a 'for' loop.
This loop will iterate over each of the AZ's in our list and use that list to produce a subnet in each AZ regardless of the number of available AZs in the current region.

As an example, given the following variable definitions:
vpc_cidr = 10.0.0.0/16
intra_subnet_size = 8 (This was introduced as a variable so you can specify different subnet sizes according to your need.)
azs = [ eu-west-1a, eu-west-1b, eu-west-1c]

... the first pass of the 'for' loop would expand the cidrsubnet function like this:

cidrsubnet(10.0.0.0/16, 8, (index(local.azs, eu-west-1a)))

Hopefully the first half of the cidrsubnet function should be clear now, you can see that the first subnet will have a /24 netmask (16+8), and that the subnet will be somewhere in the wider 10.0.0.0/16 range. But what position in that range?

That's what index is being used for, to determine the value of netnum for the cidrsubnet function.

The index function, given a list, and an element in that list, will return the numerical position of that element in that list. e.g. eu-west-1a in the example above is position '0', eu-west-1b is position '1', & eu-west-1c is position '2'.

Therefore the index portion in the example above evaluates to "0" giving us:

cidrsubnet(10.0.0.0/16, 8, 0)

... meaning the first intra_subnet, for eu-west-1a, will be defined as 10.0.0.0/24.

When the loop iterates over eu-west-1b it expands as before, but this time because eu-west-1b is position 1 in the list the 'for' loop will evaulate to

cidrsubnet(10.0.0.0/16, 8, 1)

... meaning the second intra_subnet, for eu-west-1b, will be defined as 10.0.1.0/24... and so on.

Moving onto the private_subnets definition:

  private_subnets = [
    for az in local.azs : cidrsubnet(var.vpc_cidr, var.private_subnet_size, (index(local.azs, az) + length(local.azs)))

This works on exactly the same premise as before, but we need to prevent them overlapping with the intra_subnets. This is why the length function has been added.

This behaves as before, but the important thing to note here is that length(local.azs) will return a numerical value, equal to the length of the 'azs' list.
It's used here to add to the index numerical vale to provide an offset equal to the number of AZs.

Therefore with the same assumptions as before, for eu-west-1, this section will evaluate to:

cidrsubnet(10.0.0.0/16, 8, 3)
cidrsubnet(10.0.0.0/16, 8, 4)
cidrsubnet(10.0.0.0/16, 8, 5)

Giving private subnet ranges of:
10.0.3.0/24
10.0.4.0/24
10.0.5.0/24

Because we use length to count the number of AZ's in the list it will always offset by the right amount.

If you wanted to add a third set of subnets, for example public_subnets, you could use something like:

  public_subnets = [
    for az in local.azs : cidrsubnet(var.vpc_cidr, var.public_subnet_size, (index(local.azs, az) + length(local.azs)*2))

i.e. multiply the value of 'length' by 2 to give a third offset value.

Pulling it together:
So having defined the locals you now need to feed those into the VPC module. You can do that with something like:

module "vpc" {
  source               = "terraform-aws-modules/vpc/aws"
  version              = "5.5.1"
  name                 = var.vpc_name
  cidr                 = var.vpc_cidr
  azs                  = local.azs
  intra_subnets        = local.intra_subnets
  private_subnets      = local.private_subnets
}

The beauty of this is that regardless of whether we deploy to eu-west-1 or us-east-1, we've produced code that will deploy the right number of subnets. It will also offset by the correct amount in each region to prevent overlapping ranges.

Further reading:
Terraform Data Sources
Terraform locals