DEV Community: Lucas Tonelli

Storing and managing secrets cheaply and safely

Lucas Tonelli — Tue, 27 Oct 2026 13:00:00 +0000

Every application has its secrets (database passwords, API keys, tokens, etc.) and they need to live somewhere that isn't a plaintext file committed to Git (please, don't). For those familiar with AWS, the first thought that comes to mind is to go with Secrets Manager to manage them, and it works great, except for the price per secret per month. Later down the road, it quietly turns into a line item nobody remembers approving once you multiply it across services, environments and regions. The good news is: for most secrets, there's a cheaper place to put them that's just as safe.

Storing secrets with SSM Parameter Store

Parameter Store is part of AWS Systems Manager and it holds configuration values for your applications. For anything sensitive, you use the SecureString type, which encrypts the value at rest using AWS KMS; for everything else you may either use String or StringList. The best part is that the standard tier is free and you can get a decent amount of storage and other benefits without paying a cent.

Writing a secret is a one-liner and if you don't point it at a specific key, it uses the AWS managed key aws/ssm that Systems Manager creates for you, which costs nothing to have:

aws ssm put-parameter \
--name "/myapp/prod/my-secrets" \
--value "super-secret-value" \
--type SecureString

Reading it back requires --with-decryption, otherwise you'll just get the ciphertext staring back at you:

aws ssm get-parameter \
--name "/myapp/prod/my-secrets" \
--with-decryption \
--query Parameter.Value \
--output text

A nice bonus is that the path-like naming isn't just for looks. You can scope IAM permissions to a prefix like /myapp/prod/*, so a service only ever sees the secrets it's supposed to and nothing from other paths.

Comparing with AWS Secrets Manager

Both services encrypt with KMS, so from a security standpoint they're basically the same. What you're actually paying for with Secrets Manager is a set of features Parameter Store doesn't have:

Lambda rotations included in the price;
Native integration with RDS, Redshift and DocumentDB, so rotating database credentials is mostly wired up already (managed rotation);
Cross-region replication and easier cross-account sharing.

Breaking it down, if you have credentials that auto-rotate through Secrets Manager, leave them there. Rebuilding rotation yourself to save a few cents might not be a good use of your time. However, a large chunk of secrets might never rotate at all - static API keys, third-party tokens, webhook signing secrets and so on. Those are often wasting money with Secrets Manager.

For those, Parameter Store solves the problem for free. And if you ever bump into the size limit or need parameter policies, advanced parameters are still way cheaper than Secrets Manager, in case someone tries to guilt you about it. However, just like every other AWS' product, Secrets Manager and Parameter Store both have quotas and limits, so check and compare before committing.

Watch out for the KMS decryption bill

Now, there's always a caveat (another one). Parameter Store doesn't charge you to store a SecureString, but the decryption and other requests, aren't free. Despite having a free tier, every single get-parameter --with-decryption triggers a KMS Decrypt call, and if you take a look at the pricing, it's nothing, right?!

Now, picture a Lambda that pulls fifteen separate secrets on every request, receiving traffic all day, plus a couple of CI pipelines fetching the same values on every build. Suddenly you're making KMS requests like you're trying to break a record. Also, if those parameters are on the advanced tier, you're also paying just to store them separately.

So, if applicable, group related secrets into a single SecureString parameter as a JSON string. One parameter, one decrypt call per fetch, one thing to manage.

aws ssm put-parameter \
--name "/myapp/prod/my-secrets" \
--type SecureString \
--value '{"db_password":"...","api_key":"...","jwt_secret":"..."}'

Then you fetch it once and let the application pull out whatever it needs:

RAW=$(aws ssm get-parameter \
--name "/myapp/prod/my-secrets" \
--with-decryption \
--query Parameter.Value \
--output text)

echo "$RAW" | jq -r '.db_password'

Also, since I don't want you creating new problems: both Parameter Store tiers have a storage cap, so don't try to cram every secret you own into one blob - group them by what's actually fetched together, usually per service or per environment. Adding to that, since the whole value is one parameter, changing a single field means rewriting the entire JSON, so anything that needs independent rotation might be better off on its own. It's a trade-off, not a silver bullet.

Wrap-up

Reach for SSM Parameter Store as your cheap-and-safe default and group the values into a JSON (beware of the storage limit) so KMS doesn't nickel-and-dime you on every request. Keep Secrets Manager for the handful of secrets where rotation and native integrations actually earn their price. Same security, a fraction of the bill.

This article was created with the help of AI because, while I'm good at being a software engineer (I hope so), I suck at being a writer. On the other hand, the silly jokes and wordplay are completely mine because I'm really good at those. Also, the content comes from things I've done or learned, has been verified and is edited to be straight to the point, since AI loves to stall, but I hate it.

Optimizing AWS S3 costs

Lucas Tonelli — Tue, 20 Oct 2026 13:00:00 +0000

It's easy to create an S3 bucket, let objects pile up and watch everything sit comfortably in the Standard class whether it's accessed a thousand times a day or has been its fourth abandonment anniversary. However, AWS gives you the tools to figure out where your money is actually going (they'll surely charge for it lol) and when used properly, they'll show you where to start cutting costs.

Getting the big picture with S3 Storage Lens

S3 Storage Lens is an organization-wide dashboard for your storage. It aggregates usage and activity across all your accounts, regions and buckets into a single place. So instead of clicking through buckets one by one to answer questions such as "Which buckets hold most of my data?" you get a unified view with filters to precisely help you with that.

The good thing is that every account gets a default dashboard with free metrics, no setup required. The free tier gives you several usage metrics with 14 days of retention, which is already enough to spot some offenders. For example:

Buckets with a huge amount of noncurrent object versions (versioning enabled, lifecycle rules forgotten);
Incomplete multipart upload bytes sitting there for months;
Buckets that are 100% Standard storage even though nothing suggests they need to be.

If you want to go deeper, there are the advanced metrics and recommendations: activity metrics (requests, bytes downloaded, etc.), advanced cost optimization and data protection metrics, prefix-level aggregation, CloudWatch publishing and 15 months of retention. These are paid though, and we'll get back to that in a bit.

Understanding access patterns with S3 Storage Class Analysis

Storage Lens gives you the big picture while Storage Class Analysis tells you your storage's access patterns. You enable it per bucket (optionally filtered by prefix or tags) and it starts watching your objects' access patterns: how much data is stored, how much is retrieved, and how that changes through time.

After observing for at least 30 days, it gives you a recommendation on when objects become "infrequently accessed", which is your cue to create a lifecycle rule transitioning them to Standard-IA. You can also export the results daily as CSV to a bucket and slice them however you like (QuickSight, a spreadsheet, whatever makes you happy).

It's worth noting that the analysis only makes recommendations for transitions from Standard to Standard-IA, and it's also a paid feature, charged per million objects monitored. If your access patterns are unpredictable and you just want to stop thinking about the problem, S3 Intelligent-Tiering might be a simpler answer than analyzing anything at all.

Don't let these tools become your new costs

Both Storage Lens' advanced metrics and Storage Class Analysis charge you per million objects monitored, per month. Turn them on across your accounts indiscriminately, where you might have billions of small objects (such as build outputs), and congratulations, you've just blown your AWS bill.

The way to avoid that is to do the heavy lifting with the free tier first:

Start with the free Storage Lens metrics. They offer pretty decent data for you to build a shortlist of offenders (try using the Pareto principle here);
Enable the paid tools only on the shortlist and estimate the costs before turning them on (you may do that with AWS Pricing Calculator). Turn on Storage Class Analysis for specific buckets, and use prefix or tag filters to narrow it down further instead of monitoring every single object. Use the same logic for Storage Lens advanced metrics: scope the dashboard to the accounts or buckets you're actually studying;
Turn them off when you're done. Once you have gathered enough information to define what you need to do and your lifecycle rules are in place, disable the analysis. It's a diagnostic tool, not an Amazon Prime subscription. The same goes for Storage Lens.

Wrap-up

Use free Storage Lens metrics (advanced ones, if needed) to find the suspects, Storage Class Analysis on the few buckets that justify it, then apply lifecycle rules to act on what you learned and switch the paid monitoring off. That's it, see you!

Caching your API at the edge with fine-grained purging controls

Lucas Tonelli — Tue, 13 Oct 2026 13:00:00 +0000

Caching API responses at the edge sounds great until you hit the invalidation problem. A TTL that's too short and your origin keeps getting hammered, too long and your users see stale data for who knows how long. If what you're looking for is to cache aggressively and purge surgically, the moment something changes and only what changed, there's a solution for that. Spoiler alert: the usual AWS suspects don't really compete here.

Putting your API behind Fastly

The setup is the usual reverse proxy dance: you create a Fastly service, point it to your API as the origin (or "host"), and move your DNS to Fastly. From that moment on, every request goes through their points of presence (POP) and cacheable responses get stored close to your users.

Now that caching is set up, what about the "fine-grained purging controls"? This is where the Surrogate-Key header comes in. Your origin can tag any response with one or more keys, space-separated, and Fastly will index the cached object under all of them:

HTTP/1.1 200 OK
Content-Type: application/json
Surrogate-Key: products product-42 category-shoes
// Surrogate-Control works as well
Cache-Control: max-age=86400

Think of it as labeling your cache. That response for GET /products/42 is now reachable by "everything product-related", "this specific product" and "everything in the shoes category" at the same time. Also, Fastly strips the header before sending the response to the client, so your users never see your labeling scheme. One key can be attached to thousands of objects, and one object can carry multiple keys, allowing for surgical purging later on.

Purging by keys

When product 42 gets updated, you don't need to guess which URLs contain it, or nuke the entire cache out of despair. You just purge the key:

curl -X POST "https://api.fastly.com/service/$SERVICE_ID/purge/product-42" \
  -H "Fastly-Key: $FASTLY_API_TOKEN"

Every cached object tagged with product-42 is invalidated across the whole network, usually in a few milliseconds. The listing page, the detail endpoint, that search response that happened to include it, all gone in one call while everything else stays warm.

There's a catch, though (of course). URL-based purges on Fastly work out of the box without authentication, so anyone who knows your endpoints could purge them and turn your cache into swiss cheese (rude, but possible). But don't worry, the fix is adding the header http.Fastly-Purge-Requires-Auth: "1" change at your service configuration.

With authenticated purging enabled, every purge request must carry a valid API token with purge permissions, otherwise it gets a 401 and goes home empty-handed.

Customizing further with VCL scripts

If the defaults aren't enough, Fastly lets you write custom VCL (Varnish Configuration Language) scripts that run at the edge, on every request and you can invoke them in hooks of the request lifecycle: vcl_recv when the request arrives, vcl_fetch when the origin responds, vcl_deliver before sending it back, and so on.

That opens a lot of possibilities. For instance, you can normalize query strings so ?a=1&b=2 and ?b=2&a=1 hit the same cache object, vary the cache by an API version header or even generate surrogate keys at the edge when your origin can't be bothered:

# Script injected in the "hash" hook to cache products based on page number
if (req.url.path ~ "^/products") {
    set req.hash += querystring.filter_except(querystring.sort(req.url, true), "page");
} else {
    set req.hash += querystring.remove(req.url);
}

set req.hash += req.http.host;
set req.hash += req.vcl.generation;

return (hash);

You can also cache things Fastly wouldn't cache by default (like POST bodies for search endpoints, if you know what you're doing) or strip cookies that would otherwise bust your cache. Basically, if the behavior you want isn't a checkbox, it's probably a few lines of VCL away.

What about CloudFront and API Gateway caching?

If your stack is on AWS, the natural instinct is to reach for CloudFront or the API Gateway cache. They'll cache your responses just fine, but the control story is a different beast.

CloudFront has no equivalent to surrogate keys. Its Invalidations are path-based, so to invalidate "everything related to product 42" you either track every URL that contains it (good luck with query strings) or throw a /* wildcard and wipe way more than you needed. On top of that, invalidations take minutes to propagate instead of milliseconds, and past the free monthly quota, each path costs money. Purging as part of your write path, the way you'd casually do with Fastly, just isn't how it was designed.

API Gateway is even more restrictive: the cache is a fixed-size instance you pay for hourly, and invalidation is either flushing the entire stage cache or letting clients send Cache-Control: max-age=0 to invalidate a single entry, which you then have to control with an IAM policy so random clients can't do it. No tags, no keys, no grouping, no edge programmability. It works for "cache this GET for 5 minutes", and that's about it.

None of this means these services are bad, they're just built for different use cases. If precise, event-driven invalidation is a requirement, they'll fight you the whole way. It's also worth mentioning that Fastly offers a free tier for their services, so you'll likely be able to set up a few projects without spending anything.

Wrap-up

Fine-grained purging is what turns edge caching from a risky optimization into something you can lean on: tag responses with Surrogate-Key, purge those keys from your write path the instant data changes, lock purging down with Fastly-Purge-Requires-Auth, and reach for VCL when the defaults fall short. Your origin gets quieter, your users get fast and fresh responses, and you stop choosing between the two.

Easily secure your non-prod environments from external users for free

Lucas Tonelli — Tue, 06 Oct 2026 13:00:00 +0000

If you have a private domain sitting on the public internet, chances are anyone can reach it. Search engines index it, curious people poke at it, and bots scan it all day long. The usual fixes are a VPN (complex and annoying to maintain), IP allowlists (break the moment someone works from a coffee shop), or basic auth (everyone shares the same password forever). However, the good thing is that there's another way.

Cloudflare Access

The idea is simple, your DNS record for the environment must be proxied through Cloudflare (the orange cloud), so every request passes through their edge. Then you create a self-hosted Cloudflare Access application pointing to that domain, and attach a policy to it, for example, allowing only emails ending in @mycompany.com or a specific list of addresses.

From that point on, anyone hitting the gated address gets redirected to a login page, with an input to enter an email. Once the email is submitted, Cloudflare sends a one-time PIN to it, if it matches the conditions in the policy, then the user gets a session cookie and goes through. If the email doesn't match, the user never touches your servers.

Code example

You can click this together on the Cloudflare dashboard, but you can also do it through Terraform with two resources:

resource "cloudflare_zero_trust_access_application" "development" {
  account_id                 = local.cloudflare_account_id
  name                       = "development.${var.project_name}"
  domain                     = "development.${var.apex_domain_name}"
  type                       = "self_hosted"
  session_duration           = "12h"
  http_only_cookie_attribute = true
  app_launcher_visible       = false
  policies = [{
    id         = cloudflare_zero_trust_access_policy.default.id
    precedence = 1
  }]
}

resource "cloudflare_zero_trust_access_policy" "default" {
  account_id       = local.cloudflare_account_id
  name             = "Default"
  decision         = "allow"
  session_duration = "24h"
  include = [{
    email = {
      email = local.default_email
    }
  }]
}

Worth noting

session_duration controls how long the cookie is valid, so people are not logging in on every request;
http_only_cookie_attribute keeps the session cookie away from JavaScript;
app_launcher_visible hides the app from Cloudflare's app launcher page, since this is just an internal environment; The policy is separate from the application, so you can reuse the same default policy across multiple apps. Do you need to protect staging and QA too? Duplicate the application resource (or use for_each) pointing to each subdomain, and reference the same policy.

Costs

This is the best part, it's free. Cloudflare's Zero Trust free tier covers up to 50 users, which is plenty for protecting internal environments of most teams.

There is only one catch (of course there would be a catch). The free Universal SSL certificate covers the root domain and one level of subdomain. So development.myproject.com, potatochips.myproject.com and whatever.myproject.com are all fine. But if you go one level deeper, like api.development.myproject.com, that's a second-level subdomain and the free certificate won't cover it, so you'd need Cloudflare's paid Advanced Certificate Manager for that.

Wrap-up

For such a simple setup and a proxied DNS record, your non-prod environments stop being public. No VPN, no shared passwords, no drastic changes, just an email check at the edge. It's one of those rare cases where it seems too good to be true, but it actually is pretty good.

Reduce SQS costs with these three strategies

Lucas Tonelli — Tue, 29 Sep 2026 13:00:00 +0000

SQS is a very popular and cheap service from AWS which seems pretty inoffensive to your monthly bill. However, it charges per request, not per message, and that detail is where most of the waste hides. Things like racking up old and unused queues with live consumers, using FIFO indiscriminately and being careless about polling will surely make AWS very happy and your pockets lighter. The good news is: there are a few ways which will help you save some money.

Enable Long Polling

By default, SQS uses short polling: your consumer calls ReceiveMessage, SQS answers immediately (even with an empty response), and your consumer calls again. Every one of those calls are billed, including the empty ones. A worker polling in a tight loop can easily generate millions of requests per month just to hear "nothing new".

With long polling, SQS holds the connection open for up to 20 seconds and only responds when there's a message (or when the wait time expires). One call every 20 seconds instead of dozens per second - same behavior for your application, but a fraction of the requests on your bill. Also, enabling it is a one-liner at the queue level:

aws sqs set-queue-attributes \
  --queue-url $QUEUE_URL \
  --attributes ReceiveMessageWaitTimeSeconds=20

Or per request, by passing it on the ReceiveMessage API request. There might be no downsides for most workloads, but be sure to check if it fits your use case.

Prefer standard queues to FIFO

FIFO queues are convenient, exactly-once processing and strict ordering out of the box, but that convenience costs more per request than standard queues, and it also caps your throughput (you'll also have to pay more to increase it). The thing is, for a lot of systems you can get the same guarantees in the application layer.

Handling duplicates

Standard queues offer at-least-once delivery, so the same message may arrive more than once. Instead of paying FIFO to deduplicate for you, make your consumer idempotent. Give each message a unique ID and record it when processed (a unique constraint in your database or a key in Redis). If the same ID shows up again, skip it (or do something else, I don't know).

Ensuring ordering

Standard queues also don't guarantee ordering, but in practice you might not need the global ordering from the queue. To circumvent that issue, add a timestamp (or version number) to each message at the producer, generated by the application. Then, when consuming, compare it against the last message processed: if the incoming message is older, skip it. An out-of-order message becomes a no-op instead of a bug.

Migrating from FIFO to standard

This is the sad part, where there's no configuration, nothing simple or even magical to do that. To go back to using standard queues you need to create another queue and migrate your application to the newly created one. If you have a lot, like an enormous amount of FIFO queues, and you need to migrate them, I feel for you, I really do.

Delete unused queues

This one sounds obvious, but it's pretty easy to forget a queue created for a feature that was deprecated, an experiment, an old environment and, along with it, a consumer somewhere still polling it 24/7. Since empty receives are billed like any other request, you're literally paying for API calls that will never return anything.

A quick way to find these zombies is to look at the NumberOfEmptyReceives metric in CloudWatch. A queue with a high count of empty receives and zero messages sent is a strong candidate for deletion.

Wrap-up

None of these strategies require redesigning your architecture. Long polling is a configuration change, idempotency and timestamps are small additions to your application code, and cleaning up unused queues is just housekeeping. So, keep up with these tips, and make every SQS request count.

Host your async app without spending a dime using AWS Lambda

Lucas Tonelli — Tue, 22 Sep 2026 13:00:00 +0000

If you have an async app, an internal tool, side project or PoC, there's no reason to pay for a machine that's awake all day for a job that takes so little time. That's where AWS Lambda comes in, charging you only when your function gets executed, with a generous free tier. So how do you take what you have, written with FastAPI, Express, Django, Fastify or whatever, and drop it into Lambda without a lot of refactoring?

AWS Lambda Web Adapter

AWS maintains an open source project called the Lambda Web Adapter which is a small proxy that sits in front of your app. Lambda invokes the adapter, the adapter translates the event into a plain HTTP request, forwards it to your app and translates the response back into whatever shape Lambda expects. Your app has no idea it's running inside Lambda. It just sees an HTTP request on a port, like it always did.

This means you don't have to refactor anything (not this time at least), you just add the adapter (copying the binary into your image if you're using a container or adding a Lambda Layer) and point it to your app's port. Here's a minimal Dockerfile example for a NodeJS app:

FROM public.ecr.aws/lambda/nodejs:24 AS base

# This is required to make the adapter work as expected
ENTRYPOINT []

# Copy the Lambda Web Adapter binary from its distributed image
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:1.0.1 /lambda-adapter /opt/extensions/lambda-adapter

# Tells the adapter which port to listen
ENV AWS_LWA_PORT=3000

COPY package.json ${LAMBDA_TASK_ROOT}/package.json
COPY package-lock.json ${LAMBDA_TASK_ROOT}/package-lock.json

RUN npm ci --production

COPY . .

CMD ["node", "app.js"]

The adapter handles that translation transparently, including support for streaming responses if your app relies on them. For instance, this very website is running inside a Lambda function (yup).

Packaging your app

I've given an example using Docker images, but there are a few other options for you to package your app, and they all fit different needs.

ZIP

The simplest option. You zip your code and dependencies, upload it directly (through the console or CLI), and Lambda runs it on its own managed runtime. There's no image to build or registry to manage, but you're capped at 250 MB unzipped (50 MB zipped), and native dependencies need to match Lambda's underlying OS.

S3

Practically the same as the ZIP approach, but it can accept packages greater than 50 MB zipped. You push the zip to a bucket and point Lambda at that object instead of uploading it inline. Same 250 MB limit applies.

Docker

The option to reach for once you outgrow the other limits, need OS-level dependencies that aren't in the standard runtimes, or just want your Lambda environment to look like everything else you already run in containers. Also, the images can go up to 10 GB.

Once the application is packaged and the Lambda function is deployed, you need something in front of it that speaks HTTPS.

Getting an endpoint: Lambda Function URL

Function URLs are the fastest option. Every Lambda function can have a dedicated HTTPS endpoint enabled with a single setting, no extra resources to create. You get a generated URL and requests are forwarded straight to your function. If you just need an endpoint for a webhook, an internal service, or a quick prototype, this is usually all you need.

Getting an endpoint: AWS HTTP API Gateway

HTTP API Gateway makes sense once you need more than a bare endpoint: custom domains, request throttling, built-in authorizers, etc. It's a small extra piece of infrastructure to manage, but it's also cheap and pay-per-request, so it doesn't break the "free unless you're actually getting traffic" (or a DDoS attack lol) model.

Worth knowing before you commit

Lambda has a hard execution limit, so this isn't a fit for long-lived connections;
Cold starts are real, especially with container images, but they shouldn't be such a problem in a function with more frequent traffic. So, keep the image lean and consider using provisioned concurrency if it matters. However, this is basically spinning up a server and paying for the time it's up;
A single Lambda function reserved concurrency serves a single request, so plan your concurrency accordingly, to avoid being surprised by throttling;

Wrap-up

For various async apps out there, some idle, others with spiky traffic or without persistent connections, this setup gets you a real production app with an endpoint to receive requests that'll cost you nothing until someone actually uses it! So, what are you going to build with it?!

Avoid these mistakes when using Terraform with S3 backend

Lucas Tonelli — Tue, 15 Sep 2026 13:00:00 +0000

Using an S3 backend for Terraform is a popular choice because it's cheap, reliable and widely understood. However, it's pretty easy to configure the bare minimum and move on, forgetting three small things that, in the long run, will very likely make someone's life miserable.

Table for locking

Without a lock table, two engineers (or two CI/CD pipelines) can run terraform apply at the same time, both reading the same state, and then both writing conflicting results back. It can corrupt your state file silently, leaving your real infrastructure, and the recorded state, permanently out of sync and you rethinking your life choices.

Adding a DynamoDB lock table

resource "aws_dynamodb_table" "tf_lock" {
  name         = "terraform-state-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

terraform {
  backend "s3" {
    bucket         = "my-tf-state"
    key            = "terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock"
    encrypt        = true
  }
}

Versioning S3 bucket

Your state file is not code, it's a live record of everything Terraform believes about your infrastructure (that's a beautiful quote). Without versioning, a bad apply, a manual edit or an accidental terraform state rm overwrites the state with no way back. Enabling versioning keeps a full history of every state revision, turning recovery from a crisis into a five-minute rollback instead of a life of suffering.

Enabling versioning on the state bucket

resource "aws_s3_bucket" "tf_state" {
  bucket = "my-tf-state"
}

resource "aws_s3_bucket_versioning" "tf_state" {
  bucket = aws_s3_bucket.tf_state.id

  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_lifecycle_configuration" "tf_state" {
  bucket = aws_s3_bucket.tf_state.id

  rule {
    id     = "expire-old-versions"
    status = "Enabled"
    noncurrent_version_expiration {
      noncurrent_days = 90
    }
  }
}

It's worth having a lifecycle rule to keep storage costs in check, in cases where there are a lot of changes.

Enabling cross-region bucket replication

While it's rare for an AWS region to go down, when it does, your S3 bucket goes with it. If your state file is only in one region, you can't run Terraform at all during the outage, even to fix the very infrastructure that's affected. Cross-region replication keeps a live copy of every state version in a second region so your operations don't have a single geographic point of failure. Also, for the paranoid ones, if you want to add to the already humongous durability of S3 objects, having another region is a way to do that.

Replicating state to a second region

resource "aws_s3_bucket" "tf_state_replica" {
  bucket   = "my-tf-state-replica"
  provider = aws.us_west_2 # Secondary region
}

resource "aws_s3_bucket_versioning" "tf_state_replica" {
  bucket   = aws_s3_bucket.tf_state_replica.id
  provider = aws.us_west_2

  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_replication_configuration" "tf_state" {
  bucket = aws_s3_bucket.tf_state.id
  role   = aws_iam_role.replication.arn

  rule {
    id     = "replicate-state"
    status = "Enabled"
    destination {
      bucket             = aws_s3_bucket.tf_state_replica.arn
      storage_class      = "STANDARD"
      replica_kms_key_id = aws_kms_key.replica.arn
    }
  }
}

Replication requires versioning enabled on both source and destination buckets.

Wrap-up

These three settings are pretty quick to set up and they help mitigating a class of incidents that make software engineers wake up at 3 am. So, ensure you're locking, versioning and replicating your state - sleep tight.