AWS Networking with Terraform: VPC Interface Endpoint to connect to SQS over AWS PrivateLink

In our previous post, we explored the intricacies of AWS networking by exploring the VPC Transit Gateway, a powerful tool for interconnecting multiple VPCs seamlessly.

Building upon that foundation, we now turn our attention to another essential component of AWS networking: VPC Interface Endpoints. Specifically, we'll focus on leveraging VPC Interface Endpoints to connect to an essential AWS service - the Simple Queue Service (SQS).

A VPC endpoint enables customers to privately connect to supported AWS services and VPC endpoint services powered by AWS PrivateLink. Amazon VPC instances do not require public IP addresses to communicate with resources of the service. Traffic between an Amazon VPC and a service does not leave the Amazon network.

There are two types of VPC endpoints:

1) interface endpoints
2) gateway endpoints

Interface endpoints

Interface endpoints enable connectivity to services over AWS PrivateLink. An interface endpoint is a collection of one or more elastic network interfaces with a private IP address that serves as an entry point for traffic destined to a supported service. Interface endpoints support many AWS managed services.

Architecture Diagram:

Before we embark on the implementation journey, let's outline the architecture we'll be working with:

Step 1: Create a VPC

Create a VPC with a public and private subnet. Please refer to my github repo in resources section below.

Step 2: Set up IAM Role

Create an IAM role with a tailored policy and instance profile to facilitate secure access to SQS.

####################################################
# Create the IAM role for EC2 assumption
####################################################
resource "aws_iam_role" "ec2_sqs_role" {
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Action" : [
          "sts:AssumeRole"
        ],
        "Principal" : {
          "Service" : [
            "ec2.amazonaws.com"
          ]
        }
      },
    ]
  })
  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-ec2-iam-role"
  })
}

####################################################
# Create the IAM policy to allow all sqs.* actions
####################################################
resource "aws_iam_policy" "ec2_sqs_policy" {
  name = "ec2-iam-sqs-policy"
  path = "/"
  policy = jsonencode(
    {
      "Version" : "2012-10-17",
      "Statement" : [
        {
          "Action" : [
            "sqs:*"
          ],
          "Effect" : "Allow",
          "Resource" : "*"
        },
      ]
    }
  )
  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-ec2-sqs-policy"
  })

}

####################################################
# Attach IAM policy to the role
####################################################
resource "aws_iam_policy_attachment" "ec2_sqs_role_policy" {
  policy_arn = aws_iam_policy.ec2_sqs_policy.arn
  roles      = [aws_iam_role.ec2_sqs_role.name]
  name       = "${var.naming_prefix}-ec2-sqs-policy-att"
}

####################################################
# Create an EC2 instance profile with the role
####################################################
resource "aws_iam_instance_profile" "ec2_sqs_instance_profile" {
  role = aws_iam_role.ec2_sqs_role.name
  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-ec2-sqs-instance-profile"
  })
}

Step 3: Deploy Bastion and Private Host

Instantiate a bastion host and a private host, private host equipped with the requisite instance profile. Also amend security group to connect private host from bastion host.

####################################################
# Create EC2 Server Instances
####################################################

module "vpc_a_bastion_host" {
  source           = "./modules/web"
  instance_type    = var.instance_type
  instance_key     = var.instance_key
  subnet_id        = module.vpc_a.public_subnets[0]
  vpc_id           = module.vpc_a.vpc_id
  ec2_name         = "Bastion Host A"
  sg_ingress_ports = var.sg_ingress_public
  common_tags      = local.common_tags
  naming_prefix    = local.naming_prefix
}

module "vpc_a_private_host" {
  source           = "./modules/web"
  instance_type    = var.instance_type
  instance_key     = var.instance_key
  subnet_id        = module.vpc_a.private_subnets[0]
  vpc_id           = module.vpc_a.vpc_id
  ec2_name         = "Private Host A"
  sg_ingress_ports = var.sg_ingress_private
  common_tags      = local.common_tags
  naming_prefix    = local.naming_prefix
  instance_profile = module.iam.instance_profile_name
}

####################################################
# Amend Private Host SG to allow traffic from Bastion Host SG
####################################################
resource "aws_security_group_rule" "public_in_ssh" {
  type                     = "ingress"
  from_port                = 22
  to_port                  = 22
  protocol                 = "tcp"
  security_group_id        = module.vpc_a_private_host.security_group_id
  source_security_group_id = module.vpc_a_bastion_host.security_group_id
}

Step 4: Create SQS Queue

Establish the SQS queue to facilitate seamless messaging within the AWS ecosystem.

####################################################
# Create an SQS Queue
####################################################
resource "aws_sqs_queue" "sqs" {
  name = "SQS-Queue-CT"
}

Step 5: Configure VPC Interface Endpoint

Create a VPC Interface Endpoint in the private subnet along with security group to enable secure and efficient communication with SQS via the AWS private network.

####################################################
# Create Security Group and Interface Endpoint
####################################################
resource "aws_security_group" "security_group_ie" {
  vpc_id = module.vpc_a.vpc_id
  ingress {
    description = "HTTPS for Endpoint Interface"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [var.vpc_cidr_block_a]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = merge(local.common_tags, {
    Name = "${local.naming_prefix}-sg-interface-endpoint"
  })
}

resource "aws_vpc_endpoint" "sqs_vpc_ep_interface" {
  vpc_id              = module.vpc_a.vpc_id
  vpc_endpoint_type   = "Interface"
  service_name        = "com.amazonaws.${var.aws_region}.sqs"
  subnet_ids          = [module.vpc_a.private_subnets[0]]
  private_dns_enabled = true
  security_group_ids  = [aws_security_group.security_group_ie.id]

  tags = merge(local.common_tags, {
    Name = "${local.naming_prefix}-interface-endpoint"
  })
}

Steps to Run Terraform

Follow these steps to execute the Terraform configuration:

terraform init
terraform plan 
terraform apply -auto-approve

Upon successful completion, Terraform will provide relevant outputs.

Apply complete! Resources: 19 added, 0 changed, 0 destroyed.

Outputs:

sqs_url = "https://sqs.us-east-1.amazonaws.com/197317184204/SQS-Queue-CT"
vpc_a_bastion_host_IP = "54.89.142.51"
vpc_interface_endpoint_dns_entry = [
  tolist([
    {
      "dns_name" = "vpce-0724f06ffa900d501-tlboufch.sqs.us-east-1.vpce.amazonaws.com"
      "hosted_zone_id" = "Z7HUB22UULQXV"
    },
    {
      "dns_name" = "vpce-0724f06ffa900d501-tlboufch-us-east-1a.sqs.us-east-1.vpce.amazonaws.com"
      "hosted_zone_id" = "Z7HUB22UULQXV"
    },
    {
      "dns_name" = "sqs.us-east-1.amazonaws.com"
      "hosted_zone_id" = "Z04430242JIZLP89ADN4P"
    },
  ]),
]

Testing the outcome

VPC Interface Endpoint:

Listing SQS queues via Interface Endpoint

Send a message to SQS:

[ec2-user@ip-10-1-2-238 ~]$ aws sqs send-message --queue-url https://sqs.us-east-1.amazonaws.com/197317184204/SQS-Queue-CT --message-body "AWS VPC Interface Endpoint via Terraform" --region us-east-1 --endpoint-url https://sqs.us-east-1.amazonaws.com
{
    "MD5OfMessageBody": "866fda719233ddb5c4cee060c8f35cf1", 
    "MessageId": "cbb41ab2-4c9d-4251-8b04-39520d148eb3"
}

Message in SQS:

Receive Message from SQS:

[ec2-user@ip-10-1-2-238 ~]$ aws sqs receive-message --queue-url https://sqs.us-east-1.amazonaws.com/197317184204/SQS-Queue-CT --region us-east-1 --endpoint-url https://sqs.us-east-1.amazonaws.com
{
    "Messages": [
        {
            "Body": "AWS VPC Interface Endpoint via Terraform", 
            "ReceiptHandle": "AQEBpulyP6NtbDrTpcYsaFEhq8EGEBtoJZNfuOg0ypGSJ4T5NmaIXuYUoEttEIygMO3vgJne/5bnXLbjU7XkGxwz68dv4sh9HIsX30JRyjaS9mjFpnyqR3vs4XdS9jo6mahmQMdk2td3ei8Wzf/NL1Y/lwYCBkfzGEutMAWH3Zddf093c1cWndMhLQiCJQ0MdTWrRVgCy7vL2VZZcYseOHnSYxMdYCP5TQlJFjYRbJWA5NW+PhPGDH7CEHgIMrUgTHqYrTGaUyF96J2lREJCxY828Wgzv+QB+o/fAgZhQl7J84DeInSwvi01LBmC/3ns/I5XN/opKxB6owWQxXdY7RAu3gJ7dyCO/MgKPcBVj1tsVIvf4uAq/ki31PbLwiA9oyN0YWGtjGubwsf/J7MNCTBB7A==", 
            "MD5OfBody": "866fda719233ddb5c4cee060c8f35cf1", 
            "MessageId": "cbb41ab2-4c9d-4251-8b04-39520d148eb3"
        }
    ]
}

Cleanup

Remember to stop AWS components to avoid large bills.

terraform destroy -auto-approve

By leveraging capabilities of VPC Interface Endpoints, we've simplified the process of connecting to AWS services securely. With our architecture in place, communication with SQS traverses the AWS private network, enhancing reliability and security.

In our forthcoming module, we'll elevate our AWS networking prowess by exploring VPC Gateway Endpoints, further enhancing our connectivity options within the AWS ecosystem.

Resources:

Github Link: https://github.com/chinmayto/terraform-aws-networking-vpc-interface-endpoint
VPC Endpoints: https://docs.aws.amazon.com/whitepapers/latest/aws-privatelink/what-are-vpc-endpoints.html