DEV Community

Pranit Raje
Pranit Raje

Posted on

How to Refactor a Terraform Monolith into Modules Without Downtime

Part 2: Splitting a 1,600-line imported main.tf into seven modules with HashiCorp Agent Skills, the Terraform MCP Server and moved blocks, then retiring the CloudFormation stack that started it all


At the end of Part 1 I made a confession: I had a managed application, not a well-structured one. Everything was in Terraform state, the app never went down, and the code was still one enormous main.tf that I would not wish on a colleague.

Here is what that file looked like by the time I started the refactor:

  • Resource names like stack_subnets_3, iam_roles_33 and ingress_rules_in_stack_groups_1_5. They describe the query that found the resource, not what the resource does.
  • Hardcoded IDs and ARNs everywhere. Terraform did not know the subnets belonged to the VPC. The strings just happened to match.
  • 45 import blocks still sitting next to the resources they imported.
  • Nothing I could reuse for a second environment.

Part 1 closed with about 40 resources in state. While planning this refactor I went back and imported the few I had left out, mostly extra routes and security group rules, which brought the total to 45 and the file to around 1,600 lines. Everything below works from that complete set.

This post is the follow-up I promised: refactoring that file into proper modules, quickly, with AI tooling grounded in HashiCorp's own guidance, without touching a single real resource.

A note on what this is

Same caveat as Part 1. This is a demo I built while working at HashiCorp to show what HCP Terraform, the Terraform MCP Server and HashiCorp's Agent Skills can do together. It is not production-ready. Your environment, naming conventions and appetite for risk will differ from mine. Take the learnings, not the artefacts.

Here is the whole journey in one picture.
Overview of the refactor workflow: monolith, AI assistant with skills and MCP server, seven steps with an approval gate, then plan, Sentinel and CloudFormation retirement


The one number that matters

The refactor had a single acceptance test:

0 to add, 0 to change, 0 to destroy

Every resource keeps its real-world identity. Only its address in Terraform state and the layout of the code change. If the plan shows anything beyond moves, the refactor is wrong. No judgement call, no "looks about right".

The two Sentinel policies from Part 1 were still attached to the workspace, which made that goal enforceable rather than aspirational. no-destructive-changes fails the run on any delete or replace, and flag-resource-updates flags any in-place update. A move-only plan trips neither. So the policies became an independent check on the AI's work, which is exactly where I want them. I would rather not rely on my own reading of a 45-resource plan.


The toolkit

HashiCorp Agent Skills. These are packaged instructions that teach an AI assistant how HashiCorp recommends doing something. I used two from the official repo:

  • refactor-module turns monolithic configurations into modules following HashiCorp's module design principles. It covers module boundaries, inputs and outputs, and moved blocks.
  • terraform-style-guide encodes HashiCorp's conventions for naming, file layout, typed and described variables and outputs, and terraform fmt.

The Terraform MCP Server. Same role as in Part 1. My config pins hashicorp/aws = 6.67.0, and whenever the assistant needed to know whether an argument existed, was computed, or conflicted with another, it checked the registry docs for that exact version instead of guessing.

HCP Terraform. The state lives here, so the authoritative plan runs here. A local terraform plan is no use for this job, which I will come back to.

The Part 1 lesson holds: a skill gives the assistant the conventions, and the MCP server gives it the facts. You want both.


The prompt

I did not type "make this nicer". I gave the assistant a process, constraints and a definition of done. Condensed, the shape was:

# Process
1. Inventory every resource and import block: address, Name tag, references.
2. Propose a module layout and the full old-address -> new-address map.
   STOP and wait for my approval.
3. Rename logical names by role, not discovery order.
4. Preserve state with moved blocks. Remove the import blocks.
5. Replace hardcoded cross-references with real references.
6. Add depends_on only where a reference can't express the ordering,
   each with a one-line justification.
7. Clean up and apply the style guide.
8. Validate: fmt, init, validate, plan.

# Constraints
- Never run apply, import, state mv, state rm or destroy.
- Don't change any argument value that would alter real infrastructure.
- Every argument must exist in the pinned aws 6.67.0 schema. Never guess.
- Don't add resources that aren't in the current config. List them as open items.
- If something is ambiguous, stop and ask.
Enter fullscreen mode Exit fullscreen mode

The constraints did as much work as the steps. Telling an assistant what not to do is how it stays inside the lines.

The sample prompts for the import and clean-up phases are in the demo repo. Adapt them rather than copying them, because your module boundaries will not be mine.

The approval gate

Step 2 is the one I would keep if I could keep only one. Before a single line of code was written, the assistant showed me the proposed module layout and a 45-row table mapping every old address to its new one. I reviewed it and approved it, and only then did it generate code.

A wrong module boundary or a misnamed resource takes ten seconds to fix in a table. Finding it across 28 files is a miserable afternoon.


What came out

The monolith became seven local modules, with no cycles. Data moves only through inputs and outputs.

.
├── main.tf               # root composition: seven module calls
├── moved.tf              # 45 moved blocks
├── outputs.tf            # key IDs and ARNs
├── providers.tf
├── variables.tf          # region, stack_name, environment_name, cluster_name, dynamodb_table_name
└── modules/
    ├── network/          # VPC, IGW, 4 subnets, EIP, NAT, route tables, routes, associations
    ├── security_groups/  # 3 security groups, 8 ingress rules
    ├── dynamodb/         # application table
    ├── eks/              # IAM roles, launch template, cluster, node group, OIDC provider
    ├── irsa/             # ALB controller and app roles and policies
    ├── alb/              # target group, load balancer, listener
    └── ecr/              # container repository
Enter fullscreen mode Exit fullscreen mode

Every module has the same four files (main.tf, variables.tf, outputs.tf, versions.tf), straight from the style guide. network and dynamodb sit at the bottom, security_groups and eks build on them, and irsa and alb sit on top.

The root main.tf went from roughly 1,600 lines to 47, and it is now pure composition:

module "security_groups" {
  source = "./modules/security_groups"
  region = var.region
  vpc_id = module.network.vpc_id
}

module "eks" {
  source                    = "./modules/eks"
  region                    = var.region
  cluster_name              = var.cluster_name
  cluster_security_group_id = module.security_groups.cluster_security_group_id
  cluster_subnet_ids        = module.network.cluster_subnet_ids
  private_node_subnet_ids   = module.network.private_node_subnet_ids
  dynamodb_table_arn        = module.dynamodb.table_arn
}
Enter fullscreen mode Exit fullscreen mode

You can read the architecture off that file now. Before, you had to infer it from a wall of strings.


Names that mean something

Names now come from each resource's role (its Name tag, description or availability zone), not from the order the query found it. A single instance of a type is just this. Where there are several, the name says which is which. A few of the 45:

Old address New address
aws_vpc.stack_vpcs_0 module.network.aws_vpc.this
aws_subnet.stack_subnets_3 module.network.aws_subnet.public_a
aws_route.routes_in_stack_route_tables_1_1 module.network.aws_route.private_nat
aws_vpc_security_group_ingress_rule.ingress_rules_in_stack_groups_2_0 module.security_groups.aws_vpc_security_group_ingress_rule.alb_http_public
aws_iam_role.iam_roles_33 module.irsa.aws_iam_role.alb_controller
aws_eks_node_group.stack_node_groups_0 module.eks.aws_eks_node_group.primary

I kept resources as separate blocks instead of forcing for_each. For a demo that is clearer and less risky. You may well want for_each for the subnets in a real environment.


moved blocks are what make zero downtime possible

A moved block tells Terraform that a resource has a new address but is still the same real object. Terraform rewrites the state entry instead of destroying and recreating:

moved {
  from = aws_vpc.stack_vpcs_0
  to   = module.network.aws_vpc.this
}

moved {
  from = aws_iam_role.iam_roles_33
  to   = module.irsa.aws_iam_role.alb_controller
}
Enter fullscreen mode Exit fullscreen mode

The refactor produced 45 of them in a root-level moved.tf, one per resource. HashiCorp's guide on refactoring with moved blocks covers the mechanics.

What happened to the import blocks

I deleted all 45. Every resource is already in state, so the imports have done their job. There is also a hard rule here: import blocks cannot live inside child modules, and one left pointing at an old address would fight with the moves. The assistant listed the removed import IDs in its summary, so there is still a record of what was adopted.


Replacing hardcoded values with references

This is the part I care about most, because it turns a snapshot of your infrastructure into a description of it. Some examples from this repo:

Resource Argument Before After
aws_route.private_nat route_table_id literal rtb-... aws_route_table.private.id
aws_iam_role_policy.node_dynamodb Resource ARNs literal table ARN var.dynamodb_table_arn
aws_iam_openid_connect_provider.this url literal issuer URL aws_eks_cluster.this.identity[0].oidc[0].issuer
IRSA trust policies OIDC condition keys literal issuer host trimprefix(var.oidc_issuer_url, "https://")
aws_subnet.* kubernetes.io/cluster/... tag hardcoded cluster name var.cluster_name

The rule I gave the assistant: a reference must resolve to exactly the value that is there now. If it would not, keep the literal and flag it. A "tidy" reference that quietly changes a value is how a zero-change refactor becomes a one-change refactor.

The subnet tags are the interesting case. The cluster depends on the subnets, so having the subnet tags reference the cluster's name would create a dependency cycle. The fix is a shared cluster_name variable that both sides read. The assistant spotted the cycle and picked the variable without being told.

Genuinely external constants (AWS service principals, managed policy ARNs, CIDR blocks) stayed as literals. Not everything needs to be a reference.


depends_on, but only where it earns its place

The classic refactoring mistake is sprinkling depends_on everywhere "to be safe". I asked for it only where no attribute reference can express an ordering requirement, with a one-line comment on each. It ended up in two places:

resource "aws_nat_gateway" "this" {
  # A public NAT gateway requires the internet gateway to be attached first.
  depends_on = [aws_internet_gateway.this]
  # ...
}

resource "aws_eks_node_group" "primary" {
  # The node role policy must be present before managed nodes start.
  depends_on = [aws_iam_role_policy.node_dynamodb]
  # ...
}
Enter fullscreen mode Exit fullscreen mode

The more interesting trick was routing the ordering through module outputs instead of module-level depends_on:

# modules/network/outputs.tf
output "private_node_subnet_ids" {
  value = [aws_subnet.private_b.id, aws_subnet.private_a.id]
  # Ensure node consumers wait for private egress and subnet associations.
  depends_on = [aws_route.private_nat, aws_route_table_association.private_a, aws_route_table_association.private_b]
}
Enter fullscreen mode Exit fullscreen mode

Anything that consumes that output inherits the ordering. The node group cannot start before the private NAT route exists (nodes need egress to join the cluster), and the internet-facing ALB waits for the public default route. No module-level depends_on needed.

Just as useful was what it rejected. The EIP-to-internet-gateway dependency was already covered by the NAT gateway one. The IRSA roles to OIDC provider link was already implicit through the trust policy. A depends_on you do not need is still a depends_on someone has to understand later.


Clean-up

With the style guide applied, the assistant also removed noise, but only where it caused no plan diff: redundant provider = aws lines, arguments explicitly set to null, timeouts {} blocks containing only nulls, and environment-specific values moved into typed, described variables. It also added a root outputs.tf exposing the identifiers you actually want from outside: VPC ID, cluster name and ARN, ALB DNS name, ECR URL, DynamoDB ARN and the IRSA role ARNs.


Proving it

terraform fmt -recursive
terraform init
terraform validate
Enter fullscreen mode Exit fullscreen mode

All three passed. Then I ran a local terraform plan, and it said 45 to add.

That is not a bug and it is not the answer. My local directory has no state, because the real state lives in HCP Terraform, so from where my laptop sits all 45 resources are brand new. This is exactly why the authoritative check has to be a speculative plan in the workspace, the one that holds the state.

So I ran that:

Plan: 0 to add, 0 to change, 0 to destroy.
Enter fullscreen mode Exit fullscreen mode

Every resource showed up as a move. Both Sentinel policies passed with nothing to flag. The app stayed up throughout, because nothing real was touched. Applying that plan only rewrites addresses in state.

HCP Terraform plan showing resources moved to their new module addresses with 0 to add, 0 to change, 0 to destroy


Retiring the CloudFormation stack without deleting anything

One loose end. The original app came from a CloudFormation stack, and that stack still believes it owns every resource. Delete it naively and CloudFormation will cheerfully delete your running infrastructure, which Terraform now manages. Two systems must never both think they own the same resource.

The safe way is DeletionPolicy: Retain. In the demo template each of the 39 resources has the line commented out:

  VPC:
    Type: AWS::EC2::VPC
    # DeletionPolicy: Retain
    Properties:
      CidrBlock: !Ref VpcCIDR
Enter fullscreen mode Exit fullscreen mode

The sequence:

  1. Uncomment DeletionPolicy: Retain on every resource.
  2. Update the stack with the modified template. Do not skip this. The policy only takes effect once the stack has been updated to include it. The update records Retain for every resource and changes nothing about the resources themselves.
  3. Delete the stack. CloudFormation removes its own record and leaves every resource where it is, because each one is marked Retain.
  4. Run a plan in HCP Terraform and confirm it still shows no changes.

Do this last. Delete the stack only after the refactor is merged and the plan is clean, so Terraform has already been managing these resources successfully. And check your own template for nested stacks or custom resources, which may need individual handling.


What I would tell you before you try this

Define "done" as a plan result. "0 add, 0 change, 0 destroy" is unambiguous. "Looks cleaner" is not.

Put an approval gate before code generation. Reviewing a mapping table is far cheaper than reviewing 28 files.

Tell the AI what not to do. No state mv, no guessing arguments, no new resources. The constraints kept it inside the lines.

Let policy as code be your second reviewer. Sentinel does not care how plausible the assistant sounds.

Ground it in the registry. Skills carry the conventions. The MCP server carries the facts for your exact provider version.

Verify against real state. A local plan against no state is noise. Use a speculative plan in the workspace that holds the state.

Keep the moved blocks in your commit history. They document what happened and let other workspaces migrate their state the same way.


Closing thought

In Part 1 the argument was that Search and Import changes the economics of adopting brownfield infrastructure, and policy as code changes the confidence. This part adds a third piece: AI tooling, when it is grounded in HashiCorp's skills and the registry docs, makes the cleanup affordable too. Cleanup is the step most teams quietly skip, and it is the step that decides whether you end up with Infrastructure as Code or just a very large state file.

The sequence that worked:

  1. Import to get management.
  2. Refactor to get Infrastructure as Code, with moved blocks and a 0/0/0 plan.
  3. Retire the old tooling with DeletionPolicy: Retain.

None of it needed downtime.

Same disclaimer as before: there is room to do all of this better, and your environment will want something different. If you find a better way, I would like to hear it.


Try it yourself

Setup used: Terraform 1.14+, hashicorp/aws 6.67.0, HCP Terraform with Sentinel policy sets, the Terraform MCP Server, and the refactor-module and terraform-style-guide Agent Skills.

If you try this on your own app, I would like to hear where your plan stopped being clean.

Top comments (0)