Terraform state with native s3 locking

Terraform has many popular mechanisms for storing its state file. While I have grown quite fond of HCP Terraform, there are occasional scenarios where I want to use S3 backend to hold my Terraform state files.

Best practice when sharing a state file among multiple parties is to have a locking mechanism, to avoid concurrent writes from corrupting the Terraform state. This prevents 2 parties from clobbering each other if they happen to be working on the same infrastructure at the same time.

Starting in Terraform v1.10 the S3 backend features S3 native state locking. Prior to this feature state file lock setups required access to a DynamoDB table - which can be completely foregone now!

How?

Lets start by creating a S3 bucket that has the required settings:

Note: It is common to create the S3 state file bucket outside of Terraform. However, you can create the bucket with Terraform, its just generally not advised. Example HCL below.

$ BUCKET_NAME="<your bucket name>"
$ REGION="us-east-1"

$ aws s3api create-bucket \
    --bucket $BUCKET_NAME \
    --region $REGION

$ aws s3api put-bucket-versioning \
    --bucket $BUCKET_NAME \
    --versioning-configuration Status=Enabled

Using the bucket

Create a new backend.tf file (or add this block to an existing .tf file). Make sure you fill in the bucket name!

Below we use the parameter use_lockfile which is an experimental feature that enforces using the s3 buckets lock file

terraform {
  backend "s3" {
    bucket       = <bucket name>
    key          = "backend/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
  }
}

Wrapping up

For a deep dive on this feature I recommend you read through the initial PR by @bschaatsbergen who wrote this great feature

Optional: Create bucket using Terraform

resource "aws_s3_bucket" "backend" {
  bucket = var.bucket_name
}

resource "aws_s3_bucket_versioning" "backend" {
  bucket = aws_s3_bucket.backend.id
  versioning_configuration {
    status = "Enabled"
  }
}

Edited 12/19 to remove ObjectLock which is not required. Kudos to @philippe_robitaille_f3908

Comments

[Philippe Robitaille]

You don't need to enable the Object Lock on your S3 bucket. The new feature rely on a file created using same name as state file (ex: terraform.tflock) to check if a lock exist. So It uses Amazon S3’s recently introduced conditional writes feature to implement a locking mechanism.

[drewmullen]

updated and cited you - thanks!