DEV Community

Cover image for Five Terraform state mistakes I keep seeing after 13 years (and the fixes)
Suresh Babu Narayanan
Suresh Babu Narayanan

Posted on

Five Terraform state mistakes I keep seeing after 13 years (and the fixes)

Terraform's state file is the most important file most teams never think about - right up until the day it thinks about them.

I've spent thirteen years in infrastructure, the last several deep in Terraform across AWS and Azure, and the same five state mistakes keep appearing. Not in beginner projects - in real companies, with real change boards, usually discovered at the worst possible moment. Every one of them is boring to fix before it happens and expensive to fix after.

Here they are, with the fixes I now bake into every project on day one.

Mistake 1 - Local state on someone's laptop

The default terraform.tfstate sitting in the project folder works perfectly for exactly one person, for exactly as long as their laptop lives. Then a second engineer joins, or the laptop dies, and you discover that the state file is the infrastructure's memory. Lose it, and Terraform looks at your cloud account like it's never met it before.

I've seen this exact scenario play out early in a project, before anyone had gotten around to setting up a proper backend. State living locally, on one engineer's laptop, "for now." Then a re-clone, or a laptop swap, or just someone else needing to run a plan - and the moment of "wait, where's the state file?" that makes everyone's stomach drop slightly. Nothing was permanently lost in the end, but it's the kind of near-miss that makes you migrate to remote state on every project since, on day one, no exceptions.

The fix: remote state in S3, from the first commit, with versioning and encryption on. The bucket is a five-minute bootstrap:

resource "aws_s3_bucket" "tf_state" {
  bucket = var.state_bucket_name
  lifecycle {
    prevent_destroy = true # deleting your state bucket is a very bad day
  }
}

resource "aws_s3_bucket_versioning" "tf_state" {
  bucket = aws_s3_bucket.tf_state.id
  versioning_configuration {
    status = "Enabled"
  }
}
Enter fullscreen mode Exit fullscreen mode

That prevent_destroy line has saved more weekends than any monitoring tool I've deployed.

Mistake 2 - No state locking

Two engineers run terraform apply at the same time. Both read the same state, both write different endings to it. The result is a state file that matches neither reality nor anyone's intentions - Terraform's version of a git merge conflict, except the conflict is your production VPC.

The fix used to be a DynamoDB table, and half the tutorials online still say so. But since Terraform 1.11, the S3 backend can lock natively:

terraform {
  backend "s3" {
    bucket       = "yourco-tfstate"
    key          = "prod/terraform.tfstate"
    region       = "ap-southeast-2"
    use_lockfile = true   # S3-native locking - no DynamoDB table needed
    encrypt      = true
  }
}
Enter fullscreen mode Exit fullscreen mode

One less resource to bootstrap, pay for, and explain to auditors. If you're pinned below 1.11, keep DynamoDB - but put a version upgrade on the roadmap, because this is genuinely simpler.

![How S3-native locking resolves the race - the second apply waits, nothing corrupts]

The second apply doesn't corrupt anything or silently overwrite - it just waits its turn.

Mistake 3 - One giant state file for everything

Dev, staging, prod, the DNS zones, that one Lambda from 2023 - all in a single state. Every plan takes minutes and touches everything. Every mistake has maximum blast radius. Every junior engineer's experiment holds a loaded reference to prod.

I've worked in environments where dev, staging, and prod infrastructure lived far too close together - sometimes sharing state, sometimes just sharing enough blast radius that a plan in one environment made everyone nervous about the others. Plans that should take seconds stretch out uncomfortably long when Terraform has to reason about everything at once, and every review turns into "wait, what else does this touch?" It's a slow tax that nobody notices being introduced and everybody complains about once it's there.

The fix: one state file per environment, minimum. Same code, different backends:

envs/
├── dev/    → key = "dev/terraform.tfstate"    (2 AZs, NAT off - cheap)
└── prod/   → key = "prod/terraform.tfstate"   (3 AZs, NAT on)
Enter fullscreen mode Exit fullscreen mode

The objection is always "but then dev and prod can drift apart." They can - and that's a code review problem, which is a much better problem than "dev corrupted prod's state," which is a résumé problem.

Mistake 4 - Treating state as too sacred to touch (or not sacred enough)

Two opposite failure modes, same root cause: nobody on the team actually knows the state manipulation commands.

The not sacred enough version: someone opens terraform.tfstate in an editor and hand-edits JSON. I have watched this happen. It works right up until it catastrophically doesn't.

The too sacred version: a resource needs renaming, and because nobody trusts themselves with state surgery, the team does a destroy-and-recreate of a production database instead. The scheduled-downtime email goes out for what should have been a zero-downtime metadata change.

The fix: learn three tools, in this order -

# renaming/refactoring a resource without touching the real thing:
terraform state mv aws_instance.web aws_instance.web_server

# adopting an existing resource into Terraform (1.5+, plan-reviewable):
import {
  to = aws_s3_bucket.legacy
  id = "my-existing-bucket"
}

# removing something from state WITHOUT destroying it (1.7+):
removed {
  from = aws_instance.snowflake
  lifecycle { destroy = false }
}
Enter fullscreen mode Exit fullscreen mode

The import and removed blocks are the underrated ones - they turn state surgery into a reviewable pull request instead of a hero typing commands into prod at 6pm.

I've had to reach for terraform import and manual state inspection more than once - usually to bring an existing resource under management without Terraform deciding the "safe" move was to destroy and recreate it. That instinct - trusting state mv and import over letting Terraform tear something down "because state says so" - is something you only really learn by almost letting the destroy happen once and catching it in the plan output first.

Mistake 5 - Secrets in state, state in Git

Here's the uncomfortable truth many teams learn during their first security review: Terraform state stores resource attributes in plaintext - including database passwords, and anything a provider returns. Which means the day someone commits terraform.tfstate to the repo "just as a backup," those secrets are in Git history forever.

The fix is layered:

  1. .gitignore the state files on day one - *.tfstate and *.tfstate.* (backup files too)
  2. Encrypt the state bucket (KMS server-side encryption in the bootstrap above)
  3. Prefer ephemeral values and secret-manager references over raw secret strings in configuration, so fewer secrets land in state at all
  4. Treat state bucket access as production access - IAM-restricted, logged

If you're checking one thing after reading this article, make it git log --all -- "*.tfstate" on your oldest repo. I hope it comes back empty. It often doesn't.

Patterns you'll probably recognise, even if the stories above don't quite match

State mistakes tend to rhyme across companies more than people expect. A few more I'd bet you've seen at least one of:

  • terraform.tfstate committed to Git "just this once" - and now it's in history forever
  • Two people running apply in the same environment at the same time, no lock, no warning until the diff looks wrong
  • A renamed resource treated as delete-then-create instead of a state mv, because nobody knew the command existed
  • Local state surviving purely because nobody's laptop has died yet - not a policy, just luck
  • State quietly holding secrets nobody remembered were sensitive, until a security review found them

If even one of these lands, you already know exactly which of the five fixes above to reach for first.

The pattern behind all five

None of these mistakes are exotic. They're all the same mistake wearing five costumes: treating state as an implementation detail instead of what it actually is - the single source of truth that makes Terraform work at all.

The teams that avoid the 2am state incidents aren't smarter. They just made five boring decisions on day one: remote backend, locking, per-env isolation, learned state surgery, and secrets hygiene. Thirty minutes of setup, total.

I packaged the thirty minutes

Because I kept setting up the same foundations on every new engagement, I turned my day-one layout into a small kit: bootstrap for a versioned/encrypted/locked state bucket, per-environment structure with S3-native locking, a multi-AZ VPC module, provider-level default_tags so untagged resources are impossible, and pre-commit hooks (fmt, validate, tflint) so bad code never reaches CI.

The free version of the knowledge is this article - you can build all of it yourself from what's above. If you'd rather skip the assembly, the kit is here: Terraform AWS Starter Kit - and the repo layout is documented in the listing so you know exactly what you're getting.

Either way: go check where your state lives. Today, ideally.


I'm a DevOps engineer with 13 years across AWS, Azure, Kubernetes, Terraform and Ansible. Previously: I built a self-healing pipeline with Prometheus and 50 lines of Python. I also maintain CronPort, a free cron expression converter for crontab, Kubernetes, GitHub Actions, AWS EventBridge and Terraform.

Top comments (0)