Karl Schriek

Posted on Jun 30 Edited on Aug 21 Originally published at snapcd.io

Splitting a Terraform Monolith into Smaller States

#terraform #cloud #cicd #infrastructureascode

If your Terraform plans are slow, your blast radius is too wide, or multiple teams are stepping on each other's changes, it's time to split your monolith. See The Problem with Large Terraform States for how to diagnose whether you've reached that point.

This guide walks through breaking a monolithic Terraform root into smaller, independent roots — each with its own state — by hand, and what happens to the dependencies between them afterward. Every step here is mechanical, but there are ten of them and the second half runs against real infrastructure.

If you'd rather not do it by hand, a companion tool automates the whole procedure — see Automating a Terraform Monolith Split with Demonolith. This guide is the reference for what that tool is doing on your behalf, and for the cases where you want to drive it yourself.

Doing it by hand

The split is ten steps. Step 1 sets up the working session; after that it divides cleanly in two. Steps 2 to 5 refactor the code offline, where the only cost of a mistake is a broken plan you can fix before it runs. Steps 6 to 10 migrate the state — the state surgery, the credentials and inputs, the one-shot proof, the push, and the cutover — where a missed detail becomes a resource destroyed and recreated in production. Work through them in order.

1. Pin the working session

Before you carve anything, get the monolith into a known-good state and write down what made it one. In a single shell, terraform init the monolith with every -backend-config flag it needs, and terraform plan it to a clean, zero-surprise run.

Then record the ambient inputs, because every later step runs in this same shell and nothing captures them for you:

the backend flags and their values,
every TF_VAR_* set in the environment,
every -var / -var-file argument,
the provider credentials the session carries (AWS_PROFILE, ARM_*, and so on).

A -var that appears nowhere else in your config lives only in these notes — Terraform state does not record the inputs that produced it, so a value you pass on the command line and nowhere else is gone the moment you close the shell.

2. Find the seams

Group your resources by lifecycle and ownership, and decide per resource which future module owns it. Common boundaries:

Networking — VPCs, subnets, route tables, NAT gateways. Changes rarely, underpins everything.
DNS — Zones, records. Usually owned by a platform team.
Compute — Kubernetes clusters, VM scale sets, container services. Changes more often, depends on networking.
Application infrastructure — Databases, caches, queues, storage accounts. Owned by application teams.
Monitoring — Dashboards, alerts, log sinks. Changes frequently, depends on everything but nothing depends on it.

A useful test: if two resources would never be changed in the same PR by the same person, they probably belong in different states.

Then find every reference that will cross a boundary. Grep for each resource address among its consumers — and remember the ones that hide inside templatefile(...), jsonencode(...), index expressions, depends_on lists, provider configs, and locals. A reference you miss here is a broken root later.

networking          dns
    │                 ▲
    ▼                 │
  compute ──────────►─┘
    │
    ▼
application
    │
    ▼
monitoring

The values that cross these boundaries are the wiring surface of the split. Typical examples:

Networking → Compute: vpc_id, private_subnet_ids
Compute → DNS: load_balancer_ip
Compute → Application: cluster_endpoint, cluster_ca_certificate
Application → Monitoring: database_id, cache_name

Check for cycles: if module A needs an output of module B and B needs an output of A, no valid apply order exists. Catch it now — move one of the cross-referencing resources to the other side, or extract the shared resource into a third module — because you find out late otherwise, after the code and state are already carved.

3. Refactor the code

For each new root, create a directory and move the assigned resources into it, along with everything they require: the required_providers block (conventionally into each root's root.tf), the provider configs they use, every variable and local the moved blocks reference (following local-to-local chains), and the source directories of any local child modules. Then three things happen at each boundary.

On the producer side, expose cross-boundary values as output blocks:

# networking/outputs.tf
output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

On the consumer side, declare those values as variable blocks:

# compute/variables.tf
variable "private_subnet_ids" {
  type = list(string)
}

In the consumer's resource definitions, rewrite the hard references to use the new variable:

# Before (monolith) — direct reference
resource "aws_eks_cluster" "main" {
  vpc_config {
    subnet_ids = aws_subnet.private[*].id
  }
}

# After (split) — variable reference
resource "aws_eks_cluster" "main" {
  vpc_config {
    subnet_ids = var.private_subnet_ids
  }
}

A depends_on that pointed at a resource now in another root should be removed — the ordering dependency is carried by the input/output wiring instead. Copy each data source into every root that reads it; data sources are stateless reads, so they get duplicated rather than moved. This step is a hundred small edits, and any reference you don't rewrite is a root that won't plan.

4. Copy the backend over

Each new root needs its own backend config with its own state location — take the monolith's location and postfix it per module (prod/terraform.tfstate → prod/terraform-networking.tfstate). Carry over every non-secret setting the monolith's init resolved, whether it came from HCL or from a -backend-config flag in your step-1 notes.

Keep the secret-shaped settings out of the files entirely — they're handled in step 7, along with the input values, because both are credentials-and-values work rather than code. Keep the backend block in its own backend.tf rather than in root.tf: step 8 needs to set it aside whole, and separating it spares you surgery on the terraform block. Give every root a .gitignore covering .terraform/, *.tfstate*, .env, and the tfvars file from step 7, so none of this can be committed by accident.

5. Review the refactor

This is the last step before you touch any state, and there's no automated diff to lean on, so the gate is a careful read: go through every refactored file against the monolith source, resource by resource, and convince yourself nothing was dropped, misfiled, or mis-rewritten. Repeat this review after every change to the source before the migration is done — a late edit to the monolith invalidates a review you already did. Everything up to here has been offline code work, reversible with git; from the next step on you're moving real state.

6. Carve the state

Terraform's state mv command moves resources from one state to another without destroying and recreating them. Work on local copies — never against the live backend during the migration.

# Pull the monolith state to a local working copy, and back it up before any surgery
cd monolith
terraform state pull > monolith.tfstate
cp monolith.tfstate monolith.backup.tfstate

# One move per managed resource, into its module's state file.
# A module.<name> address moves the whole subtree; data sources carry no state.
terraform state mv \
  -state=monolith.tfstate \
  -state-out=../networking.tfstate \
  aws_vpc.main aws_vpc.main

terraform state mv \
  -state=monolith.tfstate \
  -state-out=../networking.tfstate \
  aws_subnet.private aws_subnet.private

# ...repeat for every resource of every module...

Keep notes of every address you moved and where — that list is your receipt. Whatever remains in monolith.tfstate after all the moves is the remainder module's state: exactly the resources you didn't move out. This is the first step that touches real state; a mistyped address here is the one mistake that ruins a day.

7. Copy the credentials and input values over

With the code refactored and the state carved, each root still needs two things before it will plan: the backend credentials to reach its state, and the same variable values the monolith resolved.

Credentials. Put the secret-shaped backend settings from your step-1 notes into a per-root .env (chmod 600), in the engine's official variables (TF_HTTP_USERNAME, AWS_ACCESS_KEY_ID, ARM_ACCESS_KEY, …), and source it before each init — never write them into HCL.

Input values. A split-out root won't plan correctly unless it resolves the same variable values the monolith did — and the monolith resolved them silently, from several sources at once. Per root, list the variables its moved blocks declare, then reproduce each value by replaying the engine's precedence from the bottom up: TF_VAR_* env, overridden by terraform.tfvars, then *.auto.tfvars in lexical order, then -var-file files in argument order, then -var flags — all read off your step-1 notes. Write the winning values into a per-root *.auto.tfvars so every later plan loads them with no flags. Values the code declares as defaults already moved with it in step 3; only the values the monolith resolved from outside need an entry here. This is the piece that's easiest to get subtly wrong, because you're reconstructing something the engine never showed you.

8. Prove the split offline

Every new root must plan to zero changes — no create, no destroy, and no in-place update either, since a changed value that forces no replacement is still a wrong value. This is the proof that the split is operationally inert — and it's the only check that you carved the code and state correctly. Do it offline, against the local state copies, before anything touches a backend.

The catch: a split-out module planned in isolation has its upstream-sourced variables unset, because the runtime wiring doesn't exist yet at the Terraform level. Walk the modules in topological order — every producer before its consumers — and thread each producer's planned outputs into its consumers as -var values by hand. You are playing the role a control plane would play at runtime.

cd networking

# Offline proof: the engine refuses to plan a declared-but-uninitialized backend,
# so hold the backend config aside and let the local state copy rule.
mv backend.tf backend.tf.hold
cp ../networking.tfstate terraform.tfstate

terraform init -backend=false
terraform plan -out demono.tfplan   # root values load from the step-7 tfvars

# The only acceptable answer contains no "create", no "delete" — and no "update":
terraform show -json demono.tfplan | jq '[.resource_changes[].change.actions[]] | unique'

# Extract this root's planned outputs to hand to its consumers:
terraform show -json demono.tfplan | jq '.planned_values.outputs'

mv backend.tf.hold backend.tf; rm terraform.tfstate

Feed the extracted outputs as -var arguments into each consumer's plan, in topological order. If any module shows creates or destroys, something upstream is wrong — a resource missed in the state move, a reference rewritten incorrectly, a variable type that doesn't match, or an input value reconstructed wrong in step 7. By hand, you run this proof once, on the day you migrate. That's the whole safety net.

9. Push the state to the real backends

Only now do you touch the real backends. Per root, source its .env, terraform init against its new backend, confirm the target is empty, and terraform state push its state file from step 6 — never forced. The monolith's own state is never written; retiring it is a deliberate human step, later.

10. Adopt

Prove the migration against reality. Per root, a fresh terraform init must find the state you pushed in the new backend, and a refresh-on plan must show zero changes. A plan that wants to create everything means the init didn't find the state you pushed — stop and fix the backend config, don't apply.

Prove every root, then retire the monolith — its pipelines and its old state — only once they all come up clean.

The graph doesn't disappear — it moves outside

The monolith had one thing going for it: every dependency between resources lived in a single graph, and Terraform walked it for you. Compute couldn't plan against a subnet that didn't exist yet, because the subnet was a node in the same state. Splitting the monolith breaks that graph apart. The edges that crossed a seam are now output/variable pairs across separate states, and nothing walks them for you anymore.

So the dependencies you spent ten steps carefully preserving still exist — they've just moved out of Terraform's reach. On every deploy, each producer's outputs have to reach its consumers' inputs, in the right order, and a change upstream has to know which downstream roots to re-plan. Inside the monolith that was automatic. Outside it, it's a job that needs an owner: extracted and threaded by hand each time, wired into CI glue, or handed to a control plane that models the cross-root graph and enforces it on every apply.

That choice — how the graph gets managed once it lives outside any single state — is the real subject of running split roots, and it's where a tool like Snap CD earns its place. It's beyond the scope of the split itself; Modular Deployments picks up the thread from here.

Tips

Split incrementally. Move one logical group at a time. Don't try to split everything in one go.
Start with the layer that changes least. Networking is usually the best first candidate — it has many dependents but few dependencies.
Keep shared modules small. If a Terraform module (in the module {} sense) is used by multiple states, keep it focused. A module that provisions "everything for an app" is just a monolith in disguise.
Test with terraform plan after every move. A clean plan (no changes) on both the source and destination states confirms the migration was correct.

DEV Community