Provisioning Multiple AWS RDS Instances with Terraform

In this post, we'll create multiple RDS instances using Terraform, a cloud provider-agnostic Infrastructure as Code (IAC) tool, primarily used by DevOps teams to automate infrastructure tasks. For a quick introduction to AWS RDS, you can refer to the blog by K21Academy here.

Prerequisites

To complete this tutorial, ensure that the AWS CLI client is installed and configured with the access and secret keys. If you haven't done this yet, follow the instructions in the previous post here to complete this stage.

1. Create Development Folder

Create the folder named RDS and within it, create a file named main.tf.

$ mkdir RDS
$ touch RDS/main.tf

Directory structure:

- RDS/
  - main.tf

2. Provider Configuration

Open main.tf in your favorite editor (if you use vim, say hi in the comment😊) and add the following code:

provider "aws" {
    region = ""
    profile = ""
}

Explanation:

• Terraform, being cloud provider agnostic, requires a provider block. The AWS provider authenticates Terraform with your AWS account and exposes the necessary resources for management.

• region: Specifies the AWS region where resources will be provisioned or managed.

• profile: Specifies the AWS profile name, especially useful if multiple profiles are defined. If not, set it to default.

After filling:

provider "aws" {
    region = "us-west-2"
    profile = "default"
}

3. Instance Template

Below is the Terraform template for creating an RDS instance:

resource "aws_db_instance" "db_instance1" {
  engine                 = "postgres"       # Specify the database engine (PostgreSQL)
  engine_version         = "16.1"           # Specify the PostgreSQL version
  multi_az               = false            # Set to true for Multi-AZ deployment
  identifier             = "xxxxxxxxxx" # Unique identifier for the first DB instance
  username               = "xxxxxxxxxx"     # Username for the master DB user of the first instance
  password               = "xxxxxxxxxx"     # Password for the master DB user of the first instance
  instance_class         = "db.m5d.xlarge"  # Instance type for the first DB instance
  allocated_storage      = xxx              # Allocated storage in GB for the instance
  db_subnet_group_name   = aws_db_subnet_group.database_subnet_group.name
  vpc_security_group_ids = [aws_security_group.database_security_group.id]
  availability_zone      = data.aws_availability_zones.available_zones.names[0]
  db_name                = "xxxxxxxxx"
  skip_final_snapshot    = true
  publicly_accessible    = true             # Allow access from the public internet
}

4. Define Dependencies:

The aws_db_instance is a Terraform block that produces an RDS instance resource. This RDS instance is also a database instance, which is an isolated database environment in the cloud. A DB instance can contain multiple user-created databases depending on preference.

• engine: The engine field specifies the type of database instance to be provisioned. Common examples are MySQL, PostgreSQL, and Amazon Aurora. We are experimenting with the PostgreSQL engine.

• engine_version: This defines the version of the Postgres database to be used. Note that this version must be within the ranges recognized by AWS. Therefore, check your web console to be sure.

• multi_az: Defines if instance the should be provisioned in multiple AWS available zone. This is useful in cases where our customers are in different geographical location, and we will have planned to deployed database to be queried by each customer group in the same location with them. This is mostly useful in a microservice architecture, but our design is a simple one, as a result, we have explicitly set it value to false.

• identifier: This is the field that specifies a unique identifier for the db instance.

• username: Defines a username for the db instance.

• password: Password to be used to access the database instance after it is provisioned. I recommend that you use at least a 12-character-long password. Use a free password generator here.

• instance_class: Defines the class of the database instance to be provisioned. For a list of all classes with great insight into usage, visit here.

• allocated_storage: Size of the db storage capacity.

• db_subnet_group_name: Defines the IP address ranges group to add the database instance to. The Terraform block for creating a subnet is aws_db_subnet_group. Preceding steps are dependent steps to achieve the aim. Follow the steps in the code below to define a subnet.
  

#step 1: use data source to get all availability zones in region
data "aws_availability_zones" "available_zones" {}

#step 2: create a default subnet in the first AZ if one does not exist
resource "aws_default_subnet" "subnet_az1" {
  availability_zone = data.aws_availability_zones.available_zones.names[0]
}

#step 3: create a default subnet in the second AZ if one does not exist
resource "aws_default_subnet" "subnet_az2" {
  availability_zone = data.aws_availability_zones.available_zones.names[1]
}

#step 4: create the subnet group for the rds instance
resource "aws_db_subnet_group" "database_subnet_group" {
  name        = "database-subnets"
  subnet_ids  = [aws_default_subnet.subnet_az1.id, aws_default_subnet.subnet_az2.id]
  description = "subnets for database instance"

  tags = {
    Name = "database-subnets"
  }
}

• vpc_security_group_ids: VPC security group controls the traffic that is allowed to reach and leave the resources within the VPC. This field specifies the ID of the predefined VPC security group to be used. Follow the code below to define one:

#Step 1: create default vpc if one does not exist
resource "aws_default_vpc" "default_vpc" {
  tags = {
    Name = "default vpc"
  }
}

# create security group for the database
resource "aws_security_group" "database_security_group" {
  name        = "database security group"
  description = "enable postgres/aurora access on port 5432"
  vpc_id      = aws_default_vpc.default_vpc.id

  ingress {
    description = "postgre/aurora access"
    from_port   = 5432
    to_port     = 5432
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"] # Modify this to allow access from specific IPs
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = -1
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "database security group"
  }
}

• availability_zone: Availability zones (AZs) are isolated data centers located within specific regions in which public cloud services originate. Remember that you have defined a region to provision your instances in the provider block. Typically, a region contains at least 2 available zones; this field allows you to specify which of the zones to use. To get all available zones, you use a data block in Terraform as shown below:

# use data source to get all availability zones in region
data "aws_availability_zones" "available_zones" {}

Select the first AZ as shown below:

data.aws_availability_zones.available_zones.names[1]

• db_name: Takes a unique name for the db instance.

• skip_final_snapshot: Determines whether a final DB snapshot is created before the DB cluster is deleted. If true is specified, no DB snapshot is created.

• publicly_accessible: This is a true or false field. If true, it means that your database instance will be accessible from the public internet. However, note that this is also dependent on the defined ingress cidr_block. If it is not set to be accessible from anywhere i.e., 0.0.0.0/0, then only the specified CIDR will have access to the database instance.

5. Create multiple instances

You can create multiple instances by copying and pasting the aws_db_instance block template and making the necessary fields unique to each instance. Alternatively, a more sophisticated approach would involve specifying an array of objects holding unique fields for each instance and iterating through them to create instances dynamically.

You can download an extended template for creating multiple databases here.

6. Running the code

Change your directory into the folder where the multiple_postgres_db.tf file is. Update all the x-redacted field values. Then perform the operation below.

$ terrafrom init
$ terraform plan
$ terraform apply

After running the files successfully, you get the 3 db instance host endpoint output on your command. To connect to the DB instances, assuming you have psql, the Postgres terminal client is installed. Run the following command.

$ psql -U <username> -h <db host endpoint as outputted as applying the .tf script> -p 5432 <db_name>

Thank you 🙏🤗🙌