Terraform: Steps to create complete AWS S3 module

Terraform module is used to simplify process of organizing and reusing terraform code. Essentially module is a container for multiple resources logically connected and which are used together. This blog post will explain what are steps for creating Terraform module for managing an AWS S3 bucket.

When to use module?

• You need to reuse infrastructure code across different projects
• You want to keep Terraform code organized and maintainable
• You want to encapsulate the logic for a group of resources or single resource
• You want to minimize duplication
• You aim to follow best practices

Step 1: Setup module structure

I always try to follow this structure, especially if module is around one core service as AWS S3 in this case:

module/
  |-- main.tf         # Core resource definitions
  |-- variables.tf    # Input variables
  |-- outputs.tf      # Output values
  |-- providers.tf    # Required providers

Step 2: Define resources in main.tf

• The main.tf file defines the core resources for this module. Please note that I have practice to put all module related resources in main.tf file since I have tendency to keep modules as short as possible, however I've seen a lot of modules where multiple resources are needed and then each file is for specific resource.
• For our use case, S3 bucket example, main.tf will include:
• S3 bucket creation with optional lifecycle rules to ignore specific changes.
• Public access block which ensures that bucket is protected from public access.
• Versioning for data recovery.
• Server-side encryption which configures encryption using KMS key.
• Bucket Policy which is optional and applies only if enable_s3_bucket_policy is set to true.

Here is the content of main.tf:

resource "aws_s3_bucket" "this" {
  bucket = var.bucket_name

  lifecycle {
    ignore_changes = [
      tags_all,
    ]
  }
}

resource "aws_s3_bucket_public_access_block" "this" {
  bucket = aws_s3_bucket.this.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_s3_bucket_versioning" "this" {
  bucket = aws_s3_bucket.this.id
  versioning_configuration {
    status = var.bucket_versioning
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
  bucket = aws_s3_bucket.this.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = var.kms_arn
    }
  }
}

resource "aws_s3_bucket_policy" "this" {

  count = var.enable_s3_bucket_policy ? 1 : 0

  bucket = aws_s3_bucket.this.id
  policy = var.s3_bucket_policy
}

It’s important to note that you should determine which parts of the module need to be configurable and use Terraform variables for those. For properties that remain consistent across all module calls, you can define them directly within the module.
For example resource for managing S3 bucket-level public access is set to true since I know all my buckets need to have blocked public access, otherwise you could is variable here and then set to true or false depending what you need.
Additionally if you aren't 100% sure whether you might want to disable S3 versioning for specific buckets, you can use default value as true when defining variable and then in module call override that value with false.
There will be use cases when you need more flexibility for creating specific resources as aws_s3_bucket_policy in this case. The count here make sense since we want bucket policy to be optional based on the variable value. Without it you lose ability to skip the policy if the resource is already defined in module.

Step 3: Input Variables (variables.tf)

The variables.tf file defines the inputs for the module. This allows users to customize the module's behavior without modifying its internal logic. For this module, we’ve defined the following variables:

variable "bucket_name" {
  description = "The name of the bucket"
  type        = string
}

variable "bucket_versioning" {
  description = "Enable or disable bucket versioning"
  type        = bool
  default     = true
}

variable "enable_s3_bucket_policy" {
  description = "Whether to enable the S3 bucket policy"
  type        = bool
  default     = false
}

variable "s3_bucket_policy" {
  description = "value of the s3 bucket policy"
  type        = string
  default     = ""
}

variable "kms_arn" {
  description = "The ARN of the KMS key to use for server-side encryption"
  type        = string
}

Step 4: Required Providers (providers.tf)

terraform {
  required_version = "~> 1.9.7"
  required_providers {
    aws = {
      source  = "aws"
      version = ">= 4.37"
    }
  }
}

Step 5: Output Values (outputs.tf)

The outputs.tf file defines the values that the module will expose. This allows users to retrieve important information, such as the bucket's name and ARN.

output "bucket_name" {
  description = "The name of the bucket"
  value       = aws_s3_bucket.this.id
}

output "bucket_arn" {
  description = "The ARN of the bucket"
  value       = aws_s3_bucket.this.arn
}

Step 6: Using the Module

Once the module is created, you can use it in your root configuration. Here’s an example of how to call this module:

module "test_bucket" {
  source = "../modules/s3"

  bucket_name = "${local.env}-test-${random_password.random_hash.result}"
  kms_arn     = aws_kms_key.test_key.arn
}

As we have ability to optionally create bucket policy if needed a module call could look like this:

module "test_bucket" {
  source                  = "../modules/s3"
  bucket_name = "${local.env}-test-${random_password.random_hash.result}"
  kms_arn                 = aws_kms_key.this.arn
  enable_s3_bucket_policy = true
  s3_bucket_policy        = data.aws_iam_policy_document.allow_test_bucket_access.json
}

NOTE to double check the source of the module or path where is it located but always try to follow practice of creating directory modules and put all your modules in there.

Conclusion

Creating a Terraform module involves organizing your code into a well-structured format and defining reusable components. In this example, we created a complete module for managing S3 buckets, including advanced features like versioning, encryption, and policies. By following these steps, you can create robust and reusable modules for your infrastructure needs.