Terraform Modules

Modules are a way to organize Terraform code into re-usable chunks of IaC. Managing code for the complex infrastructure stack in a single go is not a good idea. This is for obvious reasons, the maintenance of the infrastructure itself may become a headache trying to figure out what caused what in lots of lines of IaC.

Thus, it makes sense to logically break down the infrastructure and wrap those logical parts into what is known as Modules. Modules represent a part of the infrastructure which is frequently (re)used together.

It is similar to libraries we use in various application programming languages. These libraries expose a set of functions that are packaged generically so that they can be used in different applications. Terraform Modules are a way to package coded infrastructure representation to be reused in multiple deployments.

Till now we have worked with an example where we have used a single Terraform root directory. If you are not aware of which example, refer to this commit and feel free to clone the same for reference.

Going beyond the root directory, when we use modules into our Terraform configurations — it gives rise to 2 types of modules — root modules and child modules.

Root module is the main Terraform configuration directory where other modules are imported. The modules being imported are known as child modules. As far as coding is concerned, all the basics of Terraform HCL still hold. After all, the module is nothing but a Terraform configuration with a few differences which shall be discussed in this post.

Modules are best understood by examples. We will refer to this commit while working through the examples. Let us assume that we require to create 2 VMs and place them in 2 private subnets of a VPC.

Generally, when we talk about configuring a VPC, there are quite a few things we need to take care of — VPC security group, public and private subnets, route tables, CIDR, Internet and NAT gateways, etc. If we go to configure this one by one, it would take quite some time to build the final Terraform configuration considering the development as well as testing time.

Here, Terraform Modules come to the rescue. Imagine a Terraform configuration package (Module) that takes care of all the above things — all you need to do is provide some basic details and let the module take care of the creation of all the above entities. AWS VPC module is one such module. Let us go ahead and use the same in our main.tf file as below.

module "vpc" {
 source = "terraform-aws-modules/vpc/aws"

 name = "my-vpc"
 cidr = "10.0.0.0/16"

 azs = ["us-west-1b", "us-west-1c"]
 private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
 public_subnets = ["10.0.101.0/24", "10.0.102.0/24"]

 tags = {
   Terraform = "true"
   Environment = "dev"
 }
}

Here, we are declaring a module block (locally) named “ vpc". Modules declared anywhere in Terraform configuration always take a single parameter - the name of the module, in this case, vpc. Within this module block, we declare a source argument, which is mandatory and helps Terraform identify the source of the same so that it can download it locally during the initialization phase.

We provide the required details like name, CIDR, availability zones, private and public subnet ranges. We also provide some tags for our reference. There are some required arguments and many more optional arguments provided in the documentation of this module which can be used to customize the same as per the requirement.

Running the above code alone will result in the creation of quite a few things. To know beforehand what will be created, run terraform plan, and see it for yourself. Don't forget to terraform init the root module again. 😉

As far as our current requirement is concerned, we want to place our virtual machines (EC2) instances into the private subnets of the VPC thus created. Let us move ahead and write our configuration to create 2 EC2 instances as below:

resource "aws_instance" "demo" {
 count = length(module.vpc.private_subnets)
 provider = aws.aws_west
 ami = var.ami
 instance_type = var.type
 vpc_security_group_ids = [module.vpc.default_security_group_id]
 subnet_id = tolist(module.vpc.private_subnets)[count.index]

 tags = {
   name = "Demo VM ${count.index}"
 }
}

We have already discussed the Terraform Syntax in 2 parts here. So without going into the details of it, let us focus on the crucial lines. count = length(module.vpc.private_subnets), here we are using the output generated by our VPC module in use. We have specified 2 private subnet ranges in our VPC module, so we are using the same to determine how many instances we have to ask Terraform to create.

vpc_security_group_ids = [module.vpc.default_security_group_id], here we tie the VPC security group ID with our instances. These IDs are one of the outputs of the given VPC module.

VPC module also returns a tuple of ID of private subnets created. We want one instance to be part of each private subnet. Thus we convert the returned tuple into a list and then based on the count.index assignment takes place in the below statement.

subnet_id = tolist(module.vpc.private_subnets)[count.index]

Now our configuration is ready. We have been able to express our desired infrastructure to Terraform in the form of code, now let us put it into action. Run terraform plan and observe the output. If you are following the example, it should tell you that 16 resources will be created! Where did they come from? The short answer is — from the VPC module we used. Take some time to read through the plan to understand what all is being created.

Feel free to terraform apply the same and verify below details:

1. A VPC is created with a security group

2. 2 private and 2 public subnets are also created

3. 2 EC2 instances are created, and each of them is part of 2 different private subnets

If everything is okay, then we have successfully been able to use a ready-made module for our requirements. Run terraform destroy to clean up all the resources.

About root and child

Modules have just opened up a huge ecosystem of resources that are already developed by community developers publicly or privately in the organizations. If you feel like developing and publishing a module all by yourself, you can do so as well! However, there are a few things that we need to consider before we begin that task.

There are small but important differences in a standalone Terraform config development and wrapping the same into modules. Instead of jumping to the bullet points below, take a look at the directory structure of your root configuration — especially the .terraform directory.

We knew till now that this folder contains the downloaded source for all the providers during the initialization phase. However, now there is another folder for “modules”. This folder was also created when we initialized the terraform directory after saving our module configuration in our main.tf file.

Module folder contains a JSON file modules.json which maintains the metadata, and then there is another subdirectory called " vpc". If you look at the contents of this, it looks like just another Terraform configuration - and it is! - with a few differences. Can you see " provider " configuration anywhere here? The answer to this question brings us to the first difference between root and child modules.

Inheritance of providers

Child modules inherit providers from the root module. It is not that you cannot write module-specific providers within the modules, but it is not considered a good practice. Declaring providers with their specific versions (whenever they are developed) causes backward compatibility issues during upgrades.

As a best practice, child modules should inherit provider configurations from the root module. In case if the module requires a specific change in the configuration of a root module provider — an alias should be created. Alias provider configurations can be passed explicitly to module blocks.

Encapsulation

The root module does not have access to the child module's data directly unless the child module exposes the same using output values. Modules can expose data points that can be used within our root configuration the way we did in our example.

Publishing a module

It is quite easy to publish your own Terraform modules and you can do it on multiple platforms you wish to publish. But for this post, we will stick to Terraform Registry. Yes, it is the same registry that you would like to refer to for documentation for other providers, modules, and resources.

As discussed before, writing a Terraform module is the same as writing any configuration code you want to accomplish using Terraform HCL. However, there are certain rules and best practices laid down by Terraform which should be followed while doing so. As I have said before, this blog series does not intend on reproducing the documentation for any tool so we would consider some of the practices below, to get you started.

1. Attributed to the Inheritance discussed previously, Module configuration should not contain any provider blocks.

2. All the configuration files should be placed in the root directory of the module.

3. Three files that should always exist in a module are — main.tf, variables.tf and outputs.tf. Variables and outputs should have descriptions. This is because, when publishing modules to Terraform Registry, documentation is automatically generated using this description.

4. There are certain patterns and techniques described in the documentation which refer to topics like conditional creation of modules, dependency inversion, data only modules, etc. which can be made use of depending on the requirement at hand.

5. Do not build modules for things that are already simpler. Modules are supposed to wrap a certain level of infrastructure abstraction and are not meant to be thin wrappers around small resources.

6. Name your module repository in this format — terraform-<provider>-<module_name>

Steps to publish a module on Terraform Registry. As an example, please take a look at this module which I published publicly for the sake of this blog. And here is the Github repository for the same.

1. Write a Terraform configuration, initialize it with Git repository.

2. Push this repository to Github. Terraform works with other VCS like Gitlab, Bitbucket, and Azure DevOps.

3. Log in to Terraform Cloud platform, and click on the Modules tab.

4. Click on the + Add module button. It will ask you to connect your VCS with Terraform Cloud.

5. Pick the VCS of your choice and follow along the steps to authorize Terraform Cloud to it. In this case, it is Github.

6. Once authorized successfully, Terraform Cloud will automatically detect repositories in the name format described in point# 6 above.

7. Select the repository and confirm selection and click on the Publish module. That’s it.

8. Navigate over to Terraform Registry, and search for your module by a given name.

As you can see, Terraform has automatically generated the documentation for this module. Feel free to try it out by including it in a different configuration. Whether you want your module to be public or private, it depends on how you set up your Github repository. If the VCS repository is public, the module will be public.

Originally published at http://letsdotech.dev on January 14, 2021.