Implementing S3 Gateway VPC Endpoints with Terraform

Recently, in the process of optimizing the security and cost of our service infrastructure, we revisited the way applications placed in private subnets access S3. Until now, access was via the Internet (NAT Gateway). However, by introducing a Gateway VPC Endpoint, we achieved a more secure and cost-effective private access.

Directory Structure

The directory structure for Terraform is modularized, with configurations separated for each environment (dev, stg, prod). The modules directory contains the Terraform modules (like VPC Endpoint, Route Table, etc.) used in the project.

-- terraform-project/
   -- environments/
      -- dev/
         -- backend.tf
         -- main.tf
      -- stg/
         -- backend.tf
         -- main.tf
      -- prod/
         -- backend.tf
         -- main.tf
   -- modules/
      -- vpc_endpoint/
         -- main.tf
         -- variables.tf
         -- outputs.tf
         -- provider.tf
         -- README.md
      -- route_table/
         -- main.tf
         -- variables.tf
         -- outputs.tf
         -- provider.tf
         -- README.md
      -- ...other…
   -- docs/
      -- architecrture.drowio
      -- architecrture.png

Implementation

Endpoint Configuration

First, we describe the VPC Endpoint configuration in modules/vpc_endpoint/main.tf.

data "aws_ssm_parameter" "app_vpc_id" {
  name = "/vpc/id/app"
}

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = data.aws_ssm_parameter.app_vpc_id.value
  service_name      = "com.amazonaws.ap-northeast-1.s3"
  route_table_ids   = [var.route_table_app_endpoint_id]
  vpc_endpoint_type = "Gateway"

  policy = <<POLICY
{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Action": "*",
      "Effect": "Allow",
      "Resource": "*",
      "Principal": "*"
    }
  ]
}
POLICY

  tags = {
    Name = "app-vpce-s3"
  }
}

• Retrieving VPC ID
  For vpc_id, specify the ID of the VPC where the application accessing S3 is placed. In this case, the VPC ID is fetched from the SSM Parameter Store, as the corresponding VPC was manually created before the introduction of Terraform.

• Specifying the Route Table
  route_table_ids specifies the ID of the route table to be associated with the created VPC Endpoint. This route table is already managed by Terraform, so its ID is read via a variable.

modules/vpc_endpoint/variables.tf

Copy code
variable "route_table_app_endpoint_id" {
  description = "The ID of the route table"
  type        = string
}

• Endpoint Policy Endpoint policies control access to S3 via the endpoint. With policies, you can allow access only to specific buckets or specific actions. Below is an example allowing only the GET action to a specific S3 bucket (my-allowed-bucket):

policy = <<POLICY
{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Action": "s3:GetObject",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::my-allowed-bucket/*",
      "Principal": "*"
    },
    {
      "Action": "s3:*",
      "Effect": "Deny",
      "Resource": "arn:aws:s3:::my-allowed-bucket/*",
      "Principal": "*",
      "Condition": {
        "StringNotEquals": {
          "s3:action": "s3:GetObject"
        }
      }
    }
  ]
}
POLICY

The s3:GetObject action for the my-allowed-bucket bucket is permitted, allowing reading of objects within this bucket. At the same time, all actions other than s3:GetObject for the same bucket are denied.

• Tagging The tag Name = "app-vpce-s3" helps in identifying and filtering resources. Proper naming conventions and tagging are vital, especially in large environments or when managing multiple endpoints. This enables connecting to S3 through a private access route while avoiding costs and overhead associated with NAT Gateway.