DEV Community

Python-T Point
Python-T Point

Posted on • Originally published at pythontpoint.in

☁️ How to store Terraform state in S3 with encryption

#devops #cloud #tutorial #kubernetes

📊 Opening Surprise — Why It Matters

store Terraform state in S3 with encryption

Approximately 30% of Terraform state files are stored without encryption, according to a HashiCorp survey. That statistic matters because an unencrypted state file contains raw provider credentials, resource IDs, and network topology—all of which are prime targets for attackers.

📑 Table of Contents

  • 📊 Opening Surprise — Why It Matters
  • 💾 Backend Configuration — How to Store Terraform state in S3 with encryption
  • 🔧 S3 Bucket Settings — What Properties to enable
  • 🔐 Encryption Mechanisms — Why Server‑Side Encryption is required
  • 🗝️ KMS Key Management — How to Reference a CMK
  • 📜 State Locking — How DynamoDB prevents concurrent updates
  • 🛡️ Access Controls — How IAM policies protect the state bucket
  • 🟩 Final Thoughts
  • ❓ Frequently Asked Questions
  • Can I use SSE‑S3 instead of SSE‑KMS for Terraform state?
  • What happens if the DynamoDB lock table is deleted?
  • Do I need to version the S3 bucket if I already have state locking?
  • 📚 References & Further Reading

💾 Backend Configuration — How to Store Terraform state in S3 with encryption

This section shows the exact Terraform backend block that stores state in an S3 bucket and enables server‑side encryption.

# backend.tf
terraform { backend "s3" { bucket = "my-terraform-state-prod" key = "global/terraform.tfstate" region = "us-east-1" encrypt = true dynamodb_table = "tf-state-lock" }
}
Enter fullscreen mode Exit fullscreen mode

What this does: (Also read: ☁️ Importing existing S3 buckets into Terraform state made easy with terraform import existing s3 bucket)

  • bucket: name of the S3 bucket that will hold the state file.
  • key: logical path inside the bucket; useful for separating environments.
  • region: AWS region where the bucket resides.
  • encrypt: tells Terraform to require server‑side encryption on the object.
  • dynamodb_table: enables state locking via a DynamoDB table (covered later).

Terraform itself does not perform the encryption; it merely demands that the S3 service apply server‑side encryption. The actual encryption algorithm is chosen when the bucket is created.

🔧 S3 Bucket Settings — What Properties to enable

Before Terraform can write to the bucket, the bucket must exist and have encryption and versioning turned on.

$ aws s3api create-bucket -bucket my-terraform-state-prod -region us-east-1
{ "Location": "/my-terraform-state-prod"
}



$ aws s3api put-bucket-versioning -bucket my-terraform-state-prod -versioning-configuration Status=Enabled
{ "ResponseMetadata": { "HTTPStatusCode": 200, "RequestId": "A1B2C3D4E5F6G7H8" }
}



$ aws s3api put-bucket-encryption -bucket my-terraform-state-prod -server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"aws:kms","KMSMasterKeyID":"arn:aws:kms:us-east-1:123456789012:key/abcd1234-ef56-7890-abcd-1234567890ab"}}]}'
{ "ResponseMetadata": { "HTTPStatusCode": 200, "RequestId": "I9J0K1L2M3N4O5P6" }
}
Enter fullscreen mode Exit fullscreen mode

Enabling versioning ensures that any accidental overwrite can be recovered, while Server‑Side Encryption with KMS (SSE‑KMS) provides key‑level access control and audit trails.

Key point: The bucket must already enforce encryption; otherwise Terraform's encrypt = true flag will reject the operation.

🔐 Encryption Mechanisms — Why Server‑Side Encryption is required

This section explains the two primary server‑side encryption options for S3 and why KMS is preferred for Terraform state. (Also read: ☁️ Terraform vs CloudFormation for Lambda deployments — which one should you use?)

Feature SSE‑S3 (AES‑256) SSE‑KMS
Key management AWS manages a single master key for the bucket. Customer‑Managed Keys (CMK) in AWS KMS, with fine‑grained IAM policies.
Auditability No per‑object audit logs. All encrypt/decrypt actions are logged in CloudTrail.
Performance impact Minimal latency. Additional kms:Decrypt call per object read/write.
Compliance Meets basic encryption requirements. Meets PCI‑DSS, HIPAA, and many government standards.

Because Terraform state can contain secrets, compliance‑driven organizations usually mandate SSE‑KMS. The extra API call is negligible compared to the security benefit. (More onPythonTPoint tutorials)

🗝️ KMS Key Management — How to Reference a CMK

When you create a CMK you must grant the Terraform execution role permission to use it.

# kms-key-policy.json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowTerraformAccess", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:role/TerraformExecRole" }, "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:GenerateDataKey", "kms:DescribeKey" ], "Resource": "*" } ]
}
Enter fullscreen mode Exit fullscreen mode

What this does:

  • Principal: the IAM role that runs terraform apply.
  • Action: the minimal set of KMS operations required to encrypt/decrypt the state file.

After saving the policy, attach it to the CMK: (Also read: ⚙️ Terraform create AWS EC2 instance with Python environment)

$ aws kms put-key-policy -key-id abcd1234-ef56-7890-abcd-1234567890ab -policy-name default -policy file://kms-key-policy.json
{ "ResponseMetadata": { "HTTPStatusCode": 200, "RequestId": "Q1W2E3R4T5Y6U7I8" }
}
Enter fullscreen mode Exit fullscreen mode

According to the AWS documentation, the kms:Decrypt permission is required for any client that reads encrypted objects from S3.

Key point: Using a CMK lets you rotate keys without touching the Terraform configuration; the backend continues to work as long as the alias stays the same.

📜 State Locking — How DynamoDB prevents concurrent updates

This section shows how to create a DynamoDB table for state locking and why the lock prevents race conditions.

$ aws dynamodb create-table \ -table-name tf-state-lock \ -attribute-definitions AttributeName=LockID,AttributeType=S \ -key-schema AttributeName=LockID,KeyType=HASH \ -provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5
{ "TableDescription": { "TableName": "tf-state-lock", "TableStatus": "CREATING", "CreationDateTime": "-09-01T12:34:56.000Z", "ProvisionedThroughput": { "ReadCapacityUnits": 5, "WriteCapacityUnits": 5, "NumberOfDecreasesToday": 0, "NumberOfIncreasesToday": 0 }, "TableArn": "arn:aws:dynamodb:us-east-1:123456789012:table/tf-state-lock", "TableId": "abcd1234-ef56-7890-abcd-1234567890ab", "KeySchema": [ { "AttributeName": "LockID", "KeyType": "HASH" } ], "AttributeDefinitions": [ { "AttributeName": "LockID", "AttributeType": "S" } ] }
}
Enter fullscreen mode Exit fullscreen mode

The table contains a single item with a fixed LockID value (e.g., terraform). When Terraform starts a plan, it attempts a conditional write of this item. If another process already holds the lock, the conditional write fails and Terraform aborts, preventing simultaneous state modifications.

Key point: State locking eliminates the classic “last write wins” problem that can corrupt the state file in collaborative environments.

🛡️ Access Controls — How IAM policies protect the state bucket

This section defines an IAM policy that restricts bucket access to the Terraform execution role and enforces encryption.

# terraform-s3-policy.json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowReadWriteWithEncryption", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:role/TerraformExecRole" }, "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::my-terraform-state-prod/*", "Condition": { "StringEquals": { "s3:x-amz-server-side-encryption": "aws:kms" } } }, { "Sid": "DenyUnencryptedUploads", "Effect": "Deny", "Principal": "*", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::my-terraform-state-prod/*", "Condition": { "StringNotEquals": { "s3:x-amz-server-side-encryption": "aws:kms" } } } ]
}
Enter fullscreen mode Exit fullscreen mode

What this does:

  • AllowReadWriteWithEncryption: grants the Terraform role permission to read/write only when the request includes the KMS encryption header.
  • DenyUnencryptedUploads: blocks any attempt to upload objects without SSE‑KMS, ensuring compliance even if a user forgets the encrypt = true flag.

Attach the policy to the role:

$ aws iam put-role-policy -role-name TerraformExecRole -policy-name TerraformS3Policy -policy-document file://terraform-s3-policy.json
{ "ResponseMetadata": { "HTTPStatusCode": 200, "RequestId": "Z9X8C7V6B5N4M3L2" }
}
Enter fullscreen mode Exit fullscreen mode

This policy enforces encryption at the bucket level, making the encrypt = true flag a safety net rather than the sole line of defense.

Encrypting state is not optional; it is the baseline security control for any production Terraform workflow.

🟩 Final Thoughts

Storing Terraform state in S3 with encryption combines three AWS services—S3, KMS, and DynamoDB—to provide durability, confidentiality, and concurrency safety. The bucket holds the immutable state file, KMS guarantees that only authorized roles can decrypt it, and DynamoDB ensures that only one Terraform process modifies the file at a time.

When these pieces are wired together, the workflow scales to dozens of developers without sacrificing security. The configuration shown here can be copied into any CI pipeline; the only required change is the bucket name and the ARN of the KMS key that matches your organization’s key‑rotation policy.

❓ Frequently Asked Questions

Can I use SSE‑S3 instead of SSE‑KMS for Terraform state?

Yes, but SSE‑S3 only provides a single AWS‑managed key per bucket and lacks fine‑grained IAM control or CloudTrail audit logs. For compliance‑heavy environments, SSE‑KMS is the recommended choice.

What happens if the DynamoDB lock table is deleted?

Terraform will lose its ability to acquire a lock and will fall back to an optimistic concurrency model, which can lead to state corruption when multiple users run apply simultaneously. Re‑create the table with the same name and attributes to restore locking.

Do I need to version the S3 bucket if I already have state locking?

Versioning protects against accidental deletions or overwrites that occur outside of Terraform (e.g., manual S3 console actions). It complements locking but is not a substitute; both should be enabled for robust protection.

💡 Want to practise this hands-on? DigitalOcean gives new accounts $200 free credit for 60 days — enough to spin up a full Linux/Docker/Kubernetes environment at no cost.

📚 Recommended reading: Best DevOps & cloud books on Amazon — from Linux fundamentals to Kubernetes in production, curated for working engineers.

📚 References & Further Reading

Top comments (0)