A colleague opened a small pull request removing one decommissioned service from a list of ten. The plan came back with one resource to destroy, which is what everyone expected, and eighteen more to replace, which nobody read carefully because the summary line was at the bottom of two hundred lines of diff. We caught it in review by luck. Had it applied, nine unrelated services would have had their instances and their attached disks rebuilt on a Tuesday morning.
The cause is a property of count that is obvious once you have been bitten and invisible before that. With count, Terraform addresses resources positionally: aws_instance.svc[0], [1], [2]. The identity of a resource is its index in the list. Remove the second element and everything after it shifts down one. Terraform does not see "one item deleted." It sees that the resource at index 1 used to be the payments worker and is now the reporting worker, and the correct action for a changed name and AMI is destroy and create. Nine times over.
for_each addresses by key instead. aws_instance.svc["payments-worker"] is stable no matter what happens to the rest of the map, and removing one entry plans exactly one destroy. We migrated the modules that manage anything stateful first, using terraform state mv in a scripted, reviewed change so the migration itself produced a no-op plan. That took a full day for four modules and was worth every hour of it.
We also changed how we read plans, which mattered more. terraform plan now runs with -out and the pipeline parses the JSON, then fails the job if any resource with a prevent_destroy lifecycle rule or a stateful tag appears with a replace action. A human is not a reliable diff reader at two hundred lines, and the summary count sits where your eye has stopped looking.
The general lesson is that in infrastructure code, resource identity is the thing you are actually managing, not the resource. If identity is derived from position, then editing a list edits your infrastructure in ways the diff does not show you.
– Sergey Shinder
Top comments (0)