After building out the first version of my Azure infrastructure with Terraform, I ran into the problem every team eventually runs into. I needed the same infrastructure in three different places, a payments environment, an analytics environment, and a platform environment, and my first instinct was to just copy the same main.tf into three folders and tweak a few values.
That works until it doesn't. The moment you need to change one thing, add a tag, fix a rule, adjust a size, you're now updating it in three places by hand. Forget one, and you've got three environments quietly drifting apart from each other. This post is how I refactored that same infrastructure into a single reusable module that all three environments call, what broke along the way, and the two real recovery situations I had to work through when things genuinely went wrong.
The shape of it
Instead of one flat project, this became two things. A module, which holds the actual resource definitions written once. And three environments, which each call that module with their own values.
The module lives in modules/infrastructure/ with a main.tf, variables.tf, and outputs.tf, same idea as before just without hardcoding anything specific to one environment. Each environment folder under environments/ gets its own providers.tf, main.tf, and terraform.tfvars. The environment's main.tf doesn't define any resources itself anymore, it just calls the module and passes in values.
Here's the flow, end to end:
You run terraform apply inside an environment folder. That folder's providers.tf and terraform.tfvars feed into a module call, which reaches into modules/infrastructure where the resource group, virtual network, subnet, NSG, public IP, network interface, and VM are actually defined. Terraform hands the final plan to Azure ARM, and that's what creates the real infrastructure.
One backend, three isolated state files
The backend gets created manually, same as before, since Terraform can't create the backend it depends on:
az group create --name backend-rg --location eastus
az storage account create --name backendstorageaccs --resource-group backend-rg --location eastus --sku Standard_LRS --kind StorageV2
az storage container create --name backend-container --account-name backendstorageaccs --auth-mode login
The important decision here was not sharing one state file across all three environments. If payments, analytics, and platform all wrote to the same tfstate, a mistake in one could accidentally touch the other two. So each environment points at the same storage account and container, but a different key.
backend "azurerm" {
resource_group_name = "backend-rg"
storage_account_name = "backendstorageaccs"
container_name = "backend-container"
key = "payments.tfstate"
}
Small note on structure. I originally planned separate backend.tf and provider.tf files per environment, but ended up combining both into one providers.tf instead. Functionally identical, Terraform doesn't care which file a block lives in, it reads every .tf file in the folder as one combined config. Just a naming choice.
The design question that actually mattered
Before writing the module's variables, I got stuck on something that seemed small but wasn't. Some variables had default values, some didn't, and I couldn't articulate why.
The answer turned out to be a real principle, not just a style preference. Any variable that's environment specific, security related, or identity related should never have a default. If admin_username silently defaults to someone else's name because a new engineer forgot to set it, that new engineer can't log into their own server, and the person whose name it defaulted to now has access to a server they didn't know existed. If project_name silently defaults, someone on the analytics team could end up deploying into the payments resource group. If environment silently defaults, dev work could land in what's meant to be staging.
Compare that to something like location or vm_size. If those default to the wrong value, you notice immediately, the resource shows up in the wrong region or the VM is a bit underpowered, and you just fix it. No security incident, no confusion about who owns what.
So the rule I landed on: infrastructure should fail loudly with a clear error rather than succeed silently with the wrong value. Anything where a wrong guess could hurt someone gets no default, full stop.
A copy-paste bug, twice
Copying the module call from one environment to the next introduced two separate bugs, both worth mentioning because they're the kind of thing that's easy to miss and easy to fix once you see them.
First, a line assigning the wrong variable:
backend_storage_account_name = var.backend_container_name
Right variable name on the left, wrong one on the right. Small thing to catch, easy to miss when everything else validates fine.
Second, one environment's module call was literally named differently from the other two:
module "enviroment" {
against
module "infrastructure" {
in the other two. Both work, Terraform lets you name a module call whatever you want locally, but having one out of three be inconsistent is exactly the kind of thing that makes a shared codebase harder to reason about later. Renamed it to match, reran terraform init, and it fell in line.
Applying all three
Once the module and all three environments were consistent, applying each one followed the same pattern: terraform validate, terraform plan, terraform apply. Ten resources each, matching what the earlier single-project version had, just parameterized this time.
Each environment ended up SSH accessible on its own public IP once applied.
And in the portal, all three environments plus the backend show up as separate resource groups, each with their own network interface, NSG, public IP, VM, disk, and virtual network.
When the network actually dropped
This is the part worth reading carefully if you're doing anything with a remote backend, because it will happen to you eventually.
Midway through applying the analytics environment, my network dropped for a few minutes right as Terraform finished creating the VM and tried to write the final state back to the storage account. The apply itself succeeded, every resource was really created, but the state write failed with a DNS lookup error, since my machine briefly couldn't resolve the storage account's hostname.
Because it couldn't reach the backend, Terraform did the safe thing. It saved the correct, complete state to a local file called errored.tfstate and told me exactly what to do next. It also tried to release its lock on the remote state and failed at that too, since the same connection was down, leaving an orphaned lock behind.
Every apply after that failed with "state blob is already locked," which looked alarming but wasn't actually a conflict with anyone else, just a lock nobody could release because the connection that was supposed to release it was gone. Recovery was three commands:
terraform force-unlock <lock-id>
terraform state push errored.tfstate
terraform plan
The force-unlock cleared the stuck lock. Pushing errored.tfstate uploaded the correct, complete state that Terraform had already built locally. The follow-up plan came back with "No changes, your infrastructure matches the configuration," confirming everything reconciled correctly. Nothing was lost, nothing had to be rebuilt, and the VM that got created during the outage was sitting there working the whole time.
Tearing it all down
terraform destroy on each environment removed exactly what that environment's module call had created, in the correct dependency order. One of the three hit a second, smaller network hiccup, this time a connection reset while deleting the VM's managed disk mid-teardown. No stuck lock this time, just a straightforward second terraform destroy picked up exactly where the first one left off and finished the job.
One thing worth knowing: terraform destroy only removes what Terraform manages. It doesn't touch the backend itself, since that was created manually and Terraform was never given authority over it. After destroying all three environments, the three state files were still sitting in the container, just now describing empty environments. Removing the backend for good meant one more manual step:
az group delete --name backend-rg
That took the resource group, the storage account, the container, and every state file inside it down in one shot.
What this actually bought me
Going from one flat project to a module plus three environments wasn't just about avoiding copy and paste. It meant a single change to modules/infrastructure/main.tf would ripple out to payments, analytics, and platform automatically, instead of needing to be applied three separate times by hand and risking one getting missed. It also meant each environment's blast radius was contained, its own resource group, its own state file, its own lock, so a mistake in one couldn't reach into another.
If you want to see the actual code, including the module and all three environments:
https://github.com/highpee1991/terraform-enterprise
If you're building something similar and want to talk through the module structure or the state recovery, drop a comment.
Top comments (0)