Query data sources using state file in Terraform - 1

Terraform data sources help you fetch data dynamically from APIs or other Terraform state backends. Examples include AMIs from a cloud provider or Terraform outputs from other configurations.

In this 1st article, I am going to show you using data sources from Terraform, how easy it is to make your configuration more flexible and dynamic while still referencing any dependent resource attributes.

First, you are going to create an AWS VPC and security groups.

Next, you will use the aws_availability_zones data source to make your configuration deployable across any region.

In my 2nd article, You will then deploy application infrastructure defined by a separate Terraform configuration and use the terraform_remote_state data source to query information about your VPC.

Finally, you will use the aws_ami data source to configure the correct AMI for the current region.

Please visit my GitHub Repository for Terraform articles on various topics being updated on constant basis.

Let’s get started!

Objectives:

1. Create infrastructure for VPC Block

2. Change to the VPC directory and Run terraform init to initialize Terraform.

3. Update VPC region

4. Create infrastructure for VPC to apply the configuration

Pre-requisites:

• AWS user account with admin access, not a root account.
• Cloud9 IDE with AWS CLI.

Resources Used:

Terraform module which creates VPC resources on AWS

Steps for implementation to this project:

1. Create infrastructure for VPC Block

• Let’s create the following organizational structure as shown below.

• Create a directory - terraform-data-sources-vpc

• Create 4 files - terraform.tf, main.tf, variables.tf, outputs.tf file.

• Create a terraform.tf file.

# terraform-data-sources-vpc/terraform.tf
# PROVIDERS BLOCK
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.23"
    }
  }
  required_version = ">= 1.2.0"
}

• Create a main.tf file.

# terraform-data-sources-vpc/main.tf
# VPC BLOCK
provider "aws" {
  region = var.aws_region
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.14.0"

  cidr = var.vpc_cidr_block

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
  private_subnets = slice(var.private_subnet_cidr_blocks, 0, 2)
  public_subnets  = slice(var.public_subnet_cidr_blocks, 0, 2)

  enable_nat_gateway = true
  enable_vpn_gateway = false

  map_public_ip_on_launch = false
}

module "app_security_group" {
  source  = "terraform-aws-modules/security-group/aws//modules/web"
  version = "4.9.0"

  name        = "web-server-sg"
  description = "Security group for web-servers with HTTP ports open within VPC"
  vpc_id      = module.vpc.vpc_id

  ingress_cidr_blocks = module.vpc.public_subnets_cidr_blocks
}

module "lb_security_group" {
  source  = "terraform-aws-modules/security-group/aws//modules/web"
  version = "4.9.0"

  name        = "lb-sg-project-alpha-dev"
  description = "Security group for load balancer with HTTP ports open within VPC"
  vpc_id      = module.vpc.vpc_id

  ingress_cidr_blocks = ["0.0.0.0/0"]
}

• Create a variables.tf file.

# terraform-data-sources-vpc/variables.tf
# VARIABLES BLOCK
variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "vpc_cidr_block" {
  description = "CIDR block for VPC"
  type        = string
  default     = "10.0.0.0/16"
}

variable "public_subnet_cidr_blocks" {
  description = "Available cidr blocks for public subnets"
  type        = list(string)
  default = [
    "10.0.1.0/24",
    "10.0.2.0/24",
    "10.0.3.0/24",
    "10.0.4.0/24",
    "10.0.5.0/24",
    "10.0.6.0/24",
    "10.0.7.0/24",
    "10.0.8.0/24"
  ]
}

variable "private_subnet_cidr_blocks" {
  description = "Available cidr blocks for private subnets"
  type        = list(string)
  default = [
    "10.0.101.0/24",
    "10.0.102.0/24",
    "10.0.103.0/24",
    "10.0.104.0/24",
    "10.0.105.0/24",
    "10.0.106.0/24",
    "10.0.107.0/24",
    "10.0.108.0/24"
  ]
}

variable "public_subnet_count" {
  description = "Number of public subnets"
  type        = number
  default     = 2
}

variable "private_subnet_count" {
  description = "Number of private subnets"
  type        = number
  default     = 2
}

• Create an outputs.tf file.

# terraform-data-sources-vpc/outputs.tf
# OUTPUTS BLOCK
output "lb_security_group_ids" {
  description = "Security group IDs for load balancer"
  value       = [module.lb_security_group.security_group_id]
}

output "app_security_group_ids" {
  description = "Security group IDs for application servers"
  value       = [module.app_security_group.security_group_id]
}

output "public_subnet_ids" {
  description = "Public subnet IDs"
  value       = module.vpc.public_subnets
}

output "private_subnet_ids" {
  description = "Private subnet IDs"
  value       = module.vpc.private_subnets
}

2. Change to the VPC directory and run terraform init

cd terraform-data-sources-vpc

• Run terraform init to initialize Terraform.

3. Update VPC region

• In the VPC configuration, an azs argument is a hard-coded list of availability zones in the us-east-1 region which sets the Availability Zones.

# terraform-data-sources-vpc/main.tf

module "vpc" {

###...
  azs   = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
###...

• Use the aws_availability_zones data source to load the available AZs for the current region.

• Add the following to terraform-data-sources-vpc/main.tf.

• In this case, the state argument gets only the availability zones that are currently available.
  

# terraform-data-sources-vpc/main.tf

data "aws_availability_zones" "available" {
  state = "available"

  filter {
    name   = "zone-type"
    values = ["availability-zone"]
  }
}

• Update the VPC configuration to use this data source to set the list of availability zones.

# terraform-data-sources-vpc/main.tf

module "vpc" {

###...

/*
azs             = ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1e"]
*/

azs             = data.aws_availability_zones.available.names
###...

• Configure the VPC block to output the region, which the application will require as an input.

• Add a data source to terraform-data-sources-vpc/main.tf to access region information.
  

# terraform-data-sources-vpc/main.tf

data "aws_region" "current" { }

• Add an output for the region to terraform-data-sources-vpc/outputs.tf.

# terraform-data-sources-vpc/outputs.tf

output "aws_region" {
  description = "AWS region"
  value       = data.aws_region.current.name
}

4. Create infrastructure for VPC to apply the configuration

• Run terraform apply to apply the configuration setting the value of aws_region to us-west-1 and type yes when prompted.

terraform apply -var aws_region=us-west-1

What we have done so far

We have successfully created and configured the VPC Infrastructure.