Day 70 - Terraform Modules

modules are where Terraform starts to feel like real engineering. Below is a practical, step-by-step workflow follow to design, build, test and publish reusable Terraform modules.

1) Decide module responsibility
1. Pick a single, focused purpose (networking, ec2, rds, alb, etc.).
2. Keep modules small and opinionated enough to enforce best practices, but configurable via variables.

Goal: one module = one responsibility.

2) Create module directory structure

Create a folder for the module and a small, consistent layout:

modules/
  my-server/
    README.md
    main.tf
    variables.tf
    outputs.tf
    examples/
      simple/
        main.tf

examples/simple is for a runnable example that shows how to consume the module.

3) Write module files (what to include)
• main.tf: resource definitions (no hardcoded values).
• variables.tf: declare all inputs, provide clear descriptions and sensible defaults where appropriate. Mark sensitive variables if needed.
• outputs.tf: expose IDs/ARNs/important values consumers will need.
• README.md: usage, inputs, outputs, example, constraints and notes.

Keep tasks idempotent and avoid side effects.

4) Make the module configurable (variables best practices)
• Give descriptive names and description for every variable.
• Use types (string, number, list(string), map(string), object({...})) for clarity.
• Provide reasonable defaults for optional values; require explicit for critical ones (e.g., vpc_id).

Example pattern (conceptually):
variable "instance_type" { type = string; default = "t3.micro"; }

5) Expose useful outputs

Only output what consumers need: resource IDs, ARNs, connection info. Keep outputs minimal and stable. Example: instance_id, subnet_id, security_group_id.

6) Add an example (and test locally)

Create modules/my-server/examples/simple/main.tf that calls your module with realistic variable values. This is how people (and CI) will sanity-check the module.

Run:

cd modules/my-server/examples/simple
terraform init
terraform validate
terraform plan -out plan.tfplan
terraform apply plan.tfplan
# check resources, then:
terraform destroy -auto-approve

7) Format and validate

Before committing:

terraform fmt -recursive
terraform validate

Install and run terraform-docs to auto-generate README sections:

terraform-docs md . > README.md

(Keep a human-written intro + generated inputs/outputs.)

8) Version control and collaboration
• Keep modules in Git (e.g., git repo with modules/ or individual repos per module).
• Use semantic versioning for published modules (v1.0.0).
• Tag releases (git tag v1.0.0 and push tags) so consumers can reference git::https://...//?ref=v1.0.0.

9) CI: run plan & tests on PR

Set up CI that:
1. Runs terraform init and terraform validate for module and examples.
2. Runs terraform fmt -check.
3. Optionally runs integration tests:
• Terratest (Go) for real cloud checks (recommended for modules that create real infra).
• Or lightweight smoke tests: terraform apply against a short-lived test workspace and terraform destroy.

Keep credentials secure (CI secrets, temporary accounts).

10) Documentation & discoverability
• Write a clear README: purpose, usage, inputs, outputs, example, constraints.
• Add use-cases and recommended defaults.
• Add CHANGELOG.md for breaking changes.

11) Publishing and consumption
• Consume locally:

module "server" {
  source = "../modules/my-server"
  ...
}

• Consume from Git:

module "server" {
  source = "git::https://github.com/you/terraform-modules.git//my-server?ref=v1.0.0"
}

• Publish to Terraform Registry (public or private) if you want organization-wide reuse.

12) Maintenance & versioning policy
• Keep modules backward compatible when possible.
• For breaking changes bump major version; provide migration notes.
• Use terraform state mv guidance in docs if consumers need to migrate resources after structural changes.

13) Testing checklist before release
• terraform fmt passed
• terraform validate passed
• Example apply/destroy succeeded
• README autogenerated/updated (terraform-docs)
• CI checks green
• Module documented with inputs/outputs and examples
• Release tag created

14) Example workflow: make a network module then use it

1. Implement modules/network with VPC/subnets/route tables.
2. Add example for modules/network/examples/simple and verify apply/destroy.
3. In root project main.tf reference module via source = "../modules/network".
4. Run root terraform init → plan → apply.

Comments

[Oguzhan Bassari]

Great post, Deborah! This is incredibly relevant. I just finished deploying my V1.0 serverless project using Terraform, and I can already see how my main.tf file is starting to get complex. Your points about reusability and organization are exactly what I'll need for V2.0.

For now, my immediate focus is on expanding the content library for the platform rather than refactoring the infrastructure code. This brings up a practical question I've been thinking about: In your experience, when is the best time to 'pause' and refactor a working V1.0 project into modules? Is it better to wait until you feel a specific pain point (like a bug or slow deployment), or do you recommend proactively refactoring immediately after a successful launch?