Terraform Practice pt3: Modules Usage and Sources

Terraform modules are an incredibly important part of the Terraform.

Modules are the main way to package and reuse resource configurations within Terraform. Proper use of modules will make your code more readable and reusable. It also promotes consistency and best practices.

Terraform module is a collection of configuration files inside a folder. We have actually worked with a module in previous chapter. It was a root module configuration.

With modules we can organize our configurations. It may not seem that important with smaller size configurations, but modules become indispensable as infrastructure grows with time and grows in complexity.

How do we create a module and make sure that Terraform knows about it and loads it when necessary?
That's what we will learn next.

 

Code for this example: github

 

How to make a Terraform module

 

First we need to create a new folder within our terraform project directory, let's call it server. Inside this directory I will also create a new file called server.tf:

$ mkdir server
$ cd server
$ touch server.tf

 

This is the folder structure. Your editor may show .terraform/ folder in there as well, this is normal. Folders and files that start with a . are hidden on Unix systems like MacOS and Linux. This diagrams shows non-hidden files and folders:

.
├── main.tf
├── outputs.tf
├── server
│   └── server.tf
├── terraform.tf
├── terraform.tfstate
├── terraform.tfstate.backup
└── variables.tf

 

Let's create a configuration inside our new server.tf file:

# server/server.tf

variable "subnet_id" {}
variable "size" {
    default = "t2.micro"
}
variable "security_groups" {
  type = list(any)
}

data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"]
}

resource "aws_instance" "web_server" {
  ami = data.aws_ami.ubuntu.id
  instance_type = var.size
  subnet_id = var.subnet_id
  vpc_security_group_ids = var.security_groups

  tags = {
    Name = "Web Server from module"
    Terraform = "true"
  }
}

output "public_ip" {
  value = aws_instance.web_server.public_ip
}
output "public_dns" {
  value = aws_instance.web_server.public_dns
}

 

Now we will use this new module in our main configuration. Modules are invoked using the module block type. Within this block we must specify the location of the module so that Terraform knows where to look. The name of the module is only for use to reference it in our code. The important part is the directory that stores module's files. We are free to call it whatever we want. I will name mine my_server_module for illustration purposes.
Add this code to the main.tf file:

# main.tf

# ... some code above

module "my_server_module" {
  source          = "./server"
  subnet_id       = aws_subnet.public_subnet.id
  security_groups = [aws_security_group.public_sg.id]
}

 

Before we proceed we need to run terraform init command. This command is required every time we add new modules and providers:

$ terraform init

 

Next we validate our code to check for any syntax errors:

$ terraform validate
# Output
Success! The configuration is valid.

 

We can also check if the new module was added by running the providers command:

$ terraform providers
# Output
Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/aws] >= 2.7.0
└── module.my_server_module
    └── provider[registry.terraform.io/hashicorp/aws]

 

Let's see what the plan command will show us:

$ terraform plan
# Output
Plan: 19 to add, 0 to change, 0 to destroy.

Which is correct since we added only 1 new resource: web_server.

 

I will deploy my current configuration to check that it is working:

$ terraform apply
# Output
Apply complete! Resources: 19 added, 0 changed, 0 destroyed.

Outputs:

security_group_private = "sg-08af5494603e89b47"
security_group_public = "sg-070da46a3a465887f"

 

I can also check if the module configuration works by checking the state. This command will output a list of all deployed resources:

$ terraform state list
# Output
data.aws_availability_zones.available
data.aws_region.current
aws_eip.nat_gateway_eip
aws_internet_gateway.internet_gateway
aws_nat_gateway.nat_gateway
aws_route_table.private_route_table
aws_route_table.public_route_table
aws_route_table_association.private
aws_route_table_association.public
aws_security_group.private_sg
aws_security_group.public_sg
aws_security_group_rule.private_in
aws_security_group_rule.private_out
aws_security_group_rule.public_http_in
aws_security_group_rule.public_https_in
aws_security_group_rule.public_out
aws_security_group_rule.public_ssh_in
aws_subnet.private_subnet
aws_subnet.public_subnet
aws_vpc.vpc
module.my_server_module.data.aws_ami.ubuntu
module.my_server_module.aws_instance.web_server

Last 2 entries are from my module. Awesome!

 

There is also a way to see a lot of individual settings for a particular resource using state show <resource> command like this:

$ terraform state show module.my_server_module.aws_instance.web_server

You should get a long list of properties for this resource.

 

How to reuse modules

 

So how can this module be useful? Let's illustrate it with deploying another instance using this module, but this time we will put in a private subnet.

# main.tf

# ... some code above

module "another_server_from_a_module" {
  # location of module directory
  source          = "./server"
  subnet_id       = aws_subnet.private_subnet.id
  security_groups = [aws_security_group.private_sg.id]
}

We need to run terraform init again to add this module. You can verify that this new resource shows up both in plan, apply, and state.

 

Terraform Module sources

Modules are just Terraform configuration files located outside out current working directory.

Modules can be added using a variety of sources. This is a list of some of the supportws ways to specify module source in Terraform module block:

• Local paths
• Terraform registry
• GitHub/Bitbucket or other generic Git/Mercurial repos
• HTTP URLs
• S3 Bucket (AWS) or GCS Bucket (GCP)

 

Terraform Module from local path

This is what we used earlier in this article. The convention is to create a modules/ directory within our current working directory and place module directories there. Let's do that while we are at it.
Create a new directory for modules and place our server module there. Update project structure can be found below:

.
├── main.tf
├── modules
│   └── server
│       └── server.tf
├── outputs.tf
├── terraform.tf
├── terraform.tfstate
├── terraform.tfstate.backup
└── variables.tf

2 directories, 7 files

 

To import a module from local path you just need to specify the path to that module and initialize terraform to load it into our configuration:

# main.tf

# ... some code above

module "server_from_local_module" {
  # local module
  source          = "./modules/server"
  subnet_id       = aws_subnet.private_subnet.id
  security_groups = [aws_security_group.private_sg.id]
}

Don't forget to update sources for our 2 existing modules to reflect new module location.
Run terraform init to update modules.

 

Terraform Public Module Registry

Terraform conveniently provides a public registry of modules. Checkout https://registry.terraform.io/ to what is available there.

I will select an autoscaling module for this example. This module has been used more than 3 million times and it seems to be quite popular.

To add this module we need to go to module's page and copy the code.

 

Now we need add a bit of our own code to it to make it work with our configuration. I added a data block to specify the AMI for the autoscaling:

# main.tf

# ... some code above

data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"]
}

module "autoscaling_from_registry" {
  source  = "terraform-aws-modules/autoscaling/aws"
  version = "6.5.0"

  name = "demo_module_asg"

  vpc_zone_identifier = [aws_subnet.private_subnet.id]
  min_size = 0
  max_size = 1
  desired_capacity = 1

  image_id = data.aws_ami.ubuntu.id
  instance_type = "t2.micro"

  tags = {
    Name = "Web servers from asg module"
    Terraform = "true"
  }
}

Now can run init to make Terraform load this module and add it to our configuration. Verify the code with terraform validate and you do a dry-run with terraform plan.

 

Terraform Modules from GitHub repo

Terraform has support for Github repos and it will automatically recognize that the link points to a github repo. Notice that we do not specify version attribute in this case.

We will add the same autoscaling group module, however this time the source will point to github:

# main.tf

# ... some code above

module "autoscaling_from_github" {
  source  = "github.com/terraform-aws-modules/terraform-aws-autoscaling"

  name = "demo_module_asg"

  vpc_zone_identifier = [aws_subnet.private_subnet.id]
  min_size = 0
  max_size = 1
  desired_capacity = 1

  image_id = data.aws_ami.ubuntu.id
  instance_type = "t2.micro"

  tags = {
    Name = "Web servers from asg module"
    Terraform = "true"
  }
}

 

Conclusion

In this article we got our first taste of Terraform modules.

Thank you for reading! See you in the next chapter!