Friday, 6:10pm. The terraform plan says "12 to add, 3 to change, 1 to destroy". You read it, nothing looks odd. Apply. Green bar. You get up for a coffee and when you come back the alerts channel has 40 new messages. The load balancer is not responding and half the pods will not start. The plan was green. What happened?
The deploy that "could not fail"
The change was small: clean up an old subnet in one cloud and move a few workloads to another region. Terraform calculated the plan against its tfstate and saw no problem — because from where Terraform looks, there was none. The state knows which resources it created. It does not know who else, outside its world, depends on them.
And there is the trap: the resource you destroyed was the plan's "1 to destroy". A subnet that, according to the state, had nothing hanging off it. Except it did: a load balancer created six months earlier, by hand, from the console, in a hurry. That LB referenced the subnet. Terraform never knew, because that LB was never in its state.
Had the LB been in the same region and account, the cloud would most likely have blocked the delete with a dependency error — and the apply cuts out halfway, with the state half applied. The silent case, the one that "goes green and falls over later", is when the dependency crosses a boundary the integrity check does not look at: another region, another account, another cloud. There the delete goes through and the breakage shows up downstream.
The dependency nobody saw
Clouds are full of implicit dependencies that do not surface until you step on them:
- A subnet a load balancer or an instance is still using.
- A gateway a route table points at — delete it and internet egress dies without warning.
- A security group referenced by another group's rules.
- A public IP attached to a resource you believed was free.
None of these is a "Terraform bug". They are dependencies that live in the cloud's real graph, not in the .tf file. The plan is honest: it tells you what it will do with what it manages. The problem is everything else — what was created by hand, what another team manages, what lives in another account or another cloud.
Why multi-cloud makes it worse
Seeing the whole graph is already hard in a single cloud. With OCI + Azure + AWS at once, you multiply the problem:
- Three consoles, three mental models (Compartment vs Resource Group vs account), three ways of naming the same thing.
- Cross-cloud dependencies (a DNS record in one pointing at an LB in another) are invisible to every
plan, because each state only knows its own cloud. - Whoever approves the change rarely has all three consoles open at once to check by hand.
The result is that at 6pm on a Friday nobody has a single view of "what depends on what". And without that view, validating a destructive change is guesswork.
The pattern underneath: validating after instead of before
The deploy failed because dependency validation happened at the worst possible moment: at runtime, after the apply, when the resource is already gone. By then the rollback is manual, with the clock running and half the system down.
The right question is not "is the plan green?". It is "what breaks if I touch this?" — and that question has to be asked BEFORE the apply, not after.
The way out: pre-flight dependency checks
The idea is old and comes from aviation: before takeoff, a checklist confirms that everything you depend on is where it should be. Applied to infrastructure:
- Before deleting a subnet, ask: is there an LB, an instance or a NIC referencing it? If so, stop.
- Before deleting a gateway, look at which route tables use it.
- Run the check against the cloud's REAL graph (what exists), not against the state (what Terraform believes exists).
- And run it across clouds, not one at a time.
This does not replace Terraform — it complements it. Terraform applies the change; the pre-flight tells you whether that change is safe to apply. It is the difference between "12 to add, 1 to destroy" and "1 to destroy — careful, that resource has a load balancer hanging off it in another region".
See the graph before you touch it
Control Cloud was born from this pain: it draws your OCI, Azure and other cloud resources on a single map, with dependencies visible, and runs the pre-flight checks before you apply a destructive change. At 6pm on a Friday you do not want to guess — you want to see.
If the green deploy that fell over anyway has ever happened to you, the exercise worth doing is looking at your infrastructure as a graph before the next apply: read-only permissions are enough to see what each resource depends on. Sometimes seeing it is all it takes to understand it.
Top comments (0)