DEV Community: Pranit Raje

How to Refactor a Terraform Monolith into Modules Without Downtime

Pranit Raje — Fri, 23 Oct 2026 17:20:06 +0000

Part 2: Splitting a 1,600-line imported main.tf into seven modules with HashiCorp Agent Skills, the Terraform MCP Server and moved blocks, then retiring the CloudFormation stack that started it all

At the end of Part 1 I made a confession: I had a managed application, not a well-structured one. Everything was in Terraform state, the app never went down, and the code was still one enormous main.tf that I would not wish on a colleague.

Here is what that file looked like by the time I started the refactor:

Resource names like stack_subnets_3, iam_roles_33 and ingress_rules_in_stack_groups_1_5. They describe the query that found the resource, not what the resource does.
Hardcoded IDs and ARNs everywhere. Terraform did not know the subnets belonged to the VPC. The strings just happened to match.
45 import blocks still sitting next to the resources they imported.
Nothing I could reuse for a second environment.

Part 1 closed with about 40 resources in state. While planning this refactor I went back and imported the few I had left out, mostly extra routes and security group rules, which brought the total to 45 and the file to around 1,600 lines. Everything below works from that complete set.

This post is the follow-up I promised: refactoring that file into proper modules, quickly, with AI tooling grounded in HashiCorp's own guidance, without touching a single real resource.

A note on what this is

Same caveat as Part 1. This is a demo I built while working at HashiCorp to show what HCP Terraform, the Terraform MCP Server and HashiCorp's Agent Skills can do together. It is not production-ready. Your environment, naming conventions and appetite for risk will differ from mine. Take the learnings, not the artefacts.

Here is the whole journey in one picture.

The one number that matters

The refactor had a single acceptance test:

0 to add, 0 to change, 0 to destroy

Every resource keeps its real-world identity. Only its address in Terraform state and the layout of the code change. If the plan shows anything beyond moves, the refactor is wrong. No judgement call, no "looks about right".

The two Sentinel policies from Part 1 were still attached to the workspace, which made that goal enforceable rather than aspirational. no-destructive-changes fails the run on any delete or replace, and flag-resource-updates flags any in-place update. A move-only plan trips neither. So the policies became an independent check on the AI's work, which is exactly where I want them. I would rather not rely on my own reading of a 45-resource plan.

The toolkit

HashiCorp Agent Skills. These are packaged instructions that teach an AI assistant how HashiCorp recommends doing something. I used two from the official repo:

refactor-module turns monolithic configurations into modules following HashiCorp's module design principles. It covers module boundaries, inputs and outputs, and moved blocks.
terraform-style-guide encodes HashiCorp's conventions for naming, file layout, typed and described variables and outputs, and terraform fmt.

The Terraform MCP Server. Same role as in Part 1. My config pins hashicorp/aws = 6.67.0, and whenever the assistant needed to know whether an argument existed, was computed, or conflicted with another, it checked the registry docs for that exact version instead of guessing.

HCP Terraform. The state lives here, so the authoritative plan runs here. A local terraform plan is no use for this job, which I will come back to.

The Part 1 lesson holds: a skill gives the assistant the conventions, and the MCP server gives it the facts. You want both.

The prompt

I did not type "make this nicer". I gave the assistant a process, constraints and a definition of done. Condensed, the shape was:

# Process
1. Inventory every resource and import block: address, Name tag, references.
2. Propose a module layout and the full old-address -> new-address map.
   STOP and wait for my approval.
3. Rename logical names by role, not discovery order.
4. Preserve state with moved blocks. Remove the import blocks.
5. Replace hardcoded cross-references with real references.
6. Add depends_on only where a reference can't express the ordering,
   each with a one-line justification.
7. Clean up and apply the style guide.
8. Validate: fmt, init, validate, plan.

# Constraints
- Never run apply, import, state mv, state rm or destroy.
- Don't change any argument value that would alter real infrastructure.
- Every argument must exist in the pinned aws 6.67.0 schema. Never guess.
- Don't add resources that aren't in the current config. List them as open items.
- If something is ambiguous, stop and ask.

The constraints did as much work as the steps. Telling an assistant what not to do is how it stays inside the lines.

The sample prompts for the import and clean-up phases are in the demo repo. Adapt them rather than copying them, because your module boundaries will not be mine.

The approval gate

Step 2 is the one I would keep if I could keep only one. Before a single line of code was written, the assistant showed me the proposed module layout and a 45-row table mapping every old address to its new one. I reviewed it and approved it, and only then did it generate code.

A wrong module boundary or a misnamed resource takes ten seconds to fix in a table. Finding it across 28 files is a miserable afternoon.

What came out

The monolith became seven local modules, with no cycles. Data moves only through inputs and outputs.

.
├── main.tf               # root composition: seven module calls
├── moved.tf              # 45 moved blocks
├── outputs.tf            # key IDs and ARNs
├── providers.tf
├── variables.tf          # region, stack_name, environment_name, cluster_name, dynamodb_table_name
└── modules/
    ├── network/          # VPC, IGW, 4 subnets, EIP, NAT, route tables, routes, associations
    ├── security_groups/  # 3 security groups, 8 ingress rules
    ├── dynamodb/         # application table
    ├── eks/              # IAM roles, launch template, cluster, node group, OIDC provider
    ├── irsa/             # ALB controller and app roles and policies
    ├── alb/              # target group, load balancer, listener
    └── ecr/              # container repository

Every module has the same four files (main.tf, variables.tf, outputs.tf, versions.tf), straight from the style guide. network and dynamodb sit at the bottom, security_groups and eks build on them, and irsa and alb sit on top.

The root main.tf went from roughly 1,600 lines to 47, and it is now pure composition:

module "security_groups" {
  source = "./modules/security_groups"
  region = var.region
  vpc_id = module.network.vpc_id
}

module "eks" {
  source                    = "./modules/eks"
  region                    = var.region
  cluster_name              = var.cluster_name
  cluster_security_group_id = module.security_groups.cluster_security_group_id
  cluster_subnet_ids        = module.network.cluster_subnet_ids
  private_node_subnet_ids   = module.network.private_node_subnet_ids
  dynamodb_table_arn        = module.dynamodb.table_arn
}

You can read the architecture off that file now. Before, you had to infer it from a wall of strings.

Names that mean something

Names now come from each resource's role (its Name tag, description or availability zone), not from the order the query found it. A single instance of a type is just this. Where there are several, the name says which is which. A few of the 45:

Old address	New address
`aws_vpc.stack_vpcs_0`	`module.network.aws_vpc.this`
`aws_subnet.stack_subnets_3`	`module.network.aws_subnet.public_a`
`aws_route.routes_in_stack_route_tables_1_1`	`module.network.aws_route.private_nat`
`aws_vpc_security_group_ingress_rule.ingress_rules_in_stack_groups_2_0`	`module.security_groups.aws_vpc_security_group_ingress_rule.alb_http_public`
`aws_iam_role.iam_roles_33`	`module.irsa.aws_iam_role.alb_controller`
`aws_eks_node_group.stack_node_groups_0`	`module.eks.aws_eks_node_group.primary`

I kept resources as separate blocks instead of forcing for_each. For a demo that is clearer and less risky. You may well want for_each for the subnets in a real environment.

`moved` blocks are what make zero downtime possible

A moved block tells Terraform that a resource has a new address but is still the same real object. Terraform rewrites the state entry instead of destroying and recreating:

moved {
  from = aws_vpc.stack_vpcs_0
  to   = module.network.aws_vpc.this
}

moved {
  from = aws_iam_role.iam_roles_33
  to   = module.irsa.aws_iam_role.alb_controller
}

The refactor produced 45 of them in a root-level moved.tf, one per resource. HashiCorp's guide on refactoring with moved blocks covers the mechanics.

What happened to the `import` blocks

I deleted all 45. Every resource is already in state, so the imports have done their job. There is also a hard rule here: import blocks cannot live inside child modules, and one left pointing at an old address would fight with the moves. The assistant listed the removed import IDs in its summary, so there is still a record of what was adopted.

Replacing hardcoded values with references

This is the part I care about most, because it turns a snapshot of your infrastructure into a description of it. Some examples from this repo:

Resource	Argument	Before	After
`aws_route.private_nat`	`route_table_id`	literal `rtb-...`	`aws_route_table.private.id`
`aws_iam_role_policy.node_dynamodb`	`Resource` ARNs	literal table ARN	`var.dynamodb_table_arn`
`aws_iam_openid_connect_provider.this`	`url`	literal issuer URL	`aws_eks_cluster.this.identity[0].oidc[0].issuer`
IRSA trust policies	OIDC condition keys	literal issuer host	`trimprefix(var.oidc_issuer_url, "https://")`
`aws_subnet.*`	`kubernetes.io/cluster/...` tag	hardcoded cluster name	`var.cluster_name`

The rule I gave the assistant: a reference must resolve to exactly the value that is there now. If it would not, keep the literal and flag it. A "tidy" reference that quietly changes a value is how a zero-change refactor becomes a one-change refactor.

The subnet tags are the interesting case. The cluster depends on the subnets, so having the subnet tags reference the cluster's name would create a dependency cycle. The fix is a shared cluster_name variable that both sides read. The assistant spotted the cycle and picked the variable without being told.

Genuinely external constants (AWS service principals, managed policy ARNs, CIDR blocks) stayed as literals. Not everything needs to be a reference.

`depends_on`, but only where it earns its place

The classic refactoring mistake is sprinkling depends_on everywhere "to be safe". I asked for it only where no attribute reference can express an ordering requirement, with a one-line comment on each. It ended up in two places:

resource "aws_nat_gateway" "this" {
  # A public NAT gateway requires the internet gateway to be attached first.
  depends_on = [aws_internet_gateway.this]
  # ...
}

resource "aws_eks_node_group" "primary" {
  # The node role policy must be present before managed nodes start.
  depends_on = [aws_iam_role_policy.node_dynamodb]
  # ...
}

The more interesting trick was routing the ordering through module outputs instead of module-level depends_on:

# modules/network/outputs.tf
output "private_node_subnet_ids" {
  value = [aws_subnet.private_b.id, aws_subnet.private_a.id]
  # Ensure node consumers wait for private egress and subnet associations.
  depends_on = [aws_route.private_nat, aws_route_table_association.private_a, aws_route_table_association.private_b]
}

Anything that consumes that output inherits the ordering. The node group cannot start before the private NAT route exists (nodes need egress to join the cluster), and the internet-facing ALB waits for the public default route. No module-level depends_on needed.

Just as useful was what it rejected. The EIP-to-internet-gateway dependency was already covered by the NAT gateway one. The IRSA roles to OIDC provider link was already implicit through the trust policy. A depends_on you do not need is still a depends_on someone has to understand later.

Clean-up

With the style guide applied, the assistant also removed noise, but only where it caused no plan diff: redundant provider = aws lines, arguments explicitly set to null, timeouts {} blocks containing only nulls, and environment-specific values moved into typed, described variables. It also added a root outputs.tf exposing the identifiers you actually want from outside: VPC ID, cluster name and ARN, ALB DNS name, ECR URL, DynamoDB ARN and the IRSA role ARNs.

Proving it

terraform fmt -recursive
terraform init
terraform validate

All three passed. Then I ran a local terraform plan, and it said 45 to add.

That is not a bug and it is not the answer. My local directory has no state, because the real state lives in HCP Terraform, so from where my laptop sits all 45 resources are brand new. This is exactly why the authoritative check has to be a speculative plan in the workspace, the one that holds the state.

So I ran that:

Plan: 0 to add, 0 to change, 0 to destroy.

Every resource showed up as a move. Both Sentinel policies passed with nothing to flag. The app stayed up throughout, because nothing real was touched. Applying that plan only rewrites addresses in state.

Retiring the CloudFormation stack without deleting anything

One loose end. The original app came from a CloudFormation stack, and that stack still believes it owns every resource. Delete it naively and CloudFormation will cheerfully delete your running infrastructure, which Terraform now manages. Two systems must never both think they own the same resource.

The safe way is DeletionPolicy: Retain. In the demo template each of the 39 resources has the line commented out:

  VPC:
    Type: AWS::EC2::VPC
    # DeletionPolicy: Retain
    Properties:
      CidrBlock: !Ref VpcCIDR

The sequence:

Uncomment DeletionPolicy: Retain on every resource.
Update the stack with the modified template. Do not skip this. The policy only takes effect once the stack has been updated to include it. The update records Retain for every resource and changes nothing about the resources themselves.
Delete the stack. CloudFormation removes its own record and leaves every resource where it is, because each one is marked Retain.
Run a plan in HCP Terraform and confirm it still shows no changes.

Do this last. Delete the stack only after the refactor is merged and the plan is clean, so Terraform has already been managing these resources successfully. And check your own template for nested stacks or custom resources, which may need individual handling.

What I would tell you before you try this

Define "done" as a plan result. "0 add, 0 change, 0 destroy" is unambiguous. "Looks cleaner" is not.

Put an approval gate before code generation. Reviewing a mapping table is far cheaper than reviewing 28 files.

Tell the AI what not to do. No state mv, no guessing arguments, no new resources. The constraints kept it inside the lines.

Let policy as code be your second reviewer. Sentinel does not care how plausible the assistant sounds.

Ground it in the registry. Skills carry the conventions. The MCP server carries the facts for your exact provider version.

Verify against real state. A local plan against no state is noise. Use a speculative plan in the workspace that holds the state.

Keep the moved blocks in your commit history. They document what happened and let other workspaces migrate their state the same way.

Closing thought

In Part 1 the argument was that Search and Import changes the economics of adopting brownfield infrastructure, and policy as code changes the confidence. This part adds a third piece: AI tooling, when it is grounded in HashiCorp's skills and the registry docs, makes the cleanup affordable too. Cleanup is the step most teams quietly skip, and it is the step that decides whether you end up with Infrastructure as Code or just a very large state file.

The sequence that worked:

Import to get management.
Refactor to get Infrastructure as Code, with moved blocks and a 0/0/0 plan.
Retire the old tooling with DeletionPolicy: Retain.

None of it needed downtime.

Same disclaimer as before: there is room to do all of this better, and your environment will want something different. If you find a better way, I would like to hear it.

Try it yourself

Part 1: How to Import Existing AWS Infrastructure into Terraform Without Downtime
Demo repo from Part 1 (query file, prompts, app): github.com/pranit-hashi/aws-brownfield-terraform-import
HashiCorp Agent Skills for Terraform: github.com/hashicorp/agent-skills
Terraform MCP Server: developer.hashicorp.com/terraform/mcp-server
Refactoring with moved blocks: developer.hashicorp.com/terraform/language/modules/develop/refactoring
Terraform style guide: developer.hashicorp.com/terraform/language/style

Setup used: Terraform 1.14+, hashicorp/aws 6.67.0, HCP Terraform with Sentinel policy sets, the Terraform MCP Server, and the refactor-module and terraform-style-guide Agent Skills.

If you try this on your own app, I would like to hear where your plan stopped being clean.

How to Import Existing AWS Infrastructure into Terraform Without Downtime

Pranit Raje — Wed, 07 Oct 2026 07:11:11 +0000

Bringing a live three-tier app under Terraform control with Terraform Search, HCP Terraform's Search and Import, Terraform MCP Server, HashiCorp Agent Skills and Sentinel policy that stopped a plan from replacing my EKS cluster

Update: Part 2 is out. Once your app is imported, here is how to refactor the monolith into modules without downtime.

Most organisations have at least one of these: an application that is genuinely important, genuinely in production, and genuinely not in code. Nobody planned it that way. Someone spun it up in the console, a team started depending on it, then a customer did, and now there is a quiet understanding that nobody touches it unless something is on fire.

This is the real IaC adoption problem. Greenfield is easy. The hard part is the brownfield stuff that already works and therefore cannot be rebuilt just so it can be described properly.

Terraform has had an answer for years — terraform import, and later the declarative import block. Both work. Both are a grind at any real scale. One import block per resource, hand-written config to match whatever is actually deployed, then plan, read the diff, fix a field, plan again. For a forty-resource app that is a tense sprint, because every plan against a live system is a small act of faith.

So when Terraform shipped Search (list blocks in .tfquery.hcl) and HCP Terraform shipped Search and Import, I wanted to know whether the economics had actually changed. Not on a toy S3 bucket — on something with the shape of a real app: load balancer in front, compute in the middle, database at the back, and all the networking and IAM plumbing holding it together.

A note on what this is

This was a demo I built for a few clients while working at HashiCorp, to show what HCP Terraform, Search and Import, the Terraform MCP Server, and HashiCorp's official Agent Skills can do together. It is not production-ready, and there is plenty of room to improve it.

Every environment is different. Account layouts, tagging discipline, naming conventions, and blast-radius tolerance all vary. Take the learnings here, not the artefacts, and adapt them — you will likely find better approaches for your own setup. Search and Import is also still an experimental feature as of this date, which is worth factoring into how much you lean on it.

The workflow

Everything below is one box in this picture.

Four moving parts:

Terraform Search does discovery. I declare what I am looking for instead of clicking through the console.
The Terraform MCP Server and HashiCorp's Terraform Agent Skills do the authoring. The assistant writes the query file and later fixes the generated config, but every argument comes from the official registry docs rather than from memory.
HCP Terraform's Search and Import turns discovered resources into a starter configuration plus matching import blocks, in bulk.
Sentinel policies sit on the workspace so no run can quietly turn an import into a rebuild.

That fourth one is the reason this was safe enough to attempt at all.

The application

Deliberately boring, because the interesting part is the import workflow.

Tier	AWS services
Front end	Application Load Balancer, target group, HTTP listener
Web / app	EKS cluster and managed node group, launch template, IAM roles, IRSA OIDC provider, ECR repository
Data	DynamoDB table
Foundation	VPC, 4 subnets across 2 AZs, internet gateway, NAT gateway, EIP, 2 route tables, routes, subnet associations, 3 security groups and their ingress rules

A small Flask app. User fills in a form, the app writes it to DynamoDB, a list view reads it back.

That screenshot matters more than it looks like it should. Through everything that follows, this page stayed up. That was the actual success criterion — not "did apply exit zero" but "did the thing I imported keep serving users while I imported it".

I provisioned the stack with a CloudFormation template, purely to create the brownfield condition. It could equally have been a console session, a Python script, or a runbook full of aws CLI commands. The only property I needed was that Terraform had never seen it and had no state for it.

The one convenience CFN gave me is that it tags everything with aws:cloudformation:stack-name, which made a clean discovery handle. If your brownfield app has no such tag, you will lean harder on names, VPC IDs, and parent-resource scoping — which I had to do for plenty of resources anyway.

Step 1: Discovery as code

Terraform Search works through list blocks. Each one says "find resources of this type matching this filter". You run terraform query, and Terraform returns what it found, including the identity attributes needed to import each resource.

Small feature, but it changes the character of the work. Discovery stops being archaeology you do by hand and becomes a file you can commit, review, and re-run.

Before you write anything, check that your resource types are actually supported. List resource coverage is per-provider and per-version, and it is growing fast, so anything written down about it goes stale quickly. The reference I use every time is this auto-generated report, which tracks supported list resources and refreshes as providers ship updates. Cheaper than discovering mid-build that the resource you care about has no list support yet. You can also query your own provider schema directly:

terraform providers schema -json | \
  jq '.provider_schemas | to_entries | map({provider: .key, lists: (.value.list_resource_schemas // {} | keys)})'

Writing the query file is fiddly, though. Twenty-three resource types, each with its own filter semantics, and the argument names are not guessable. aws_vpc takes an EC2-style filter { name = "tag:..." }. aws_launch_template takes launch_template_names. aws_eks_node_group takes cluster_name. aws_iam_role takes nothing and returns every non-service-linked role in the account.

Exactly the kind of work where an AI assistant helps, and exactly the kind where it will confidently invent an argument that does not exist. So I constrained it with two things: the terraform-search-import Agent Skill (Search conventions and workflow) and the Terraform MCP Server (live access to registry docs). Then I made the lookup non-optional:

# Process
1. **Inventory**: Parse `three-tier-app.yaml` and list every resource
   (logical ID + `Type`).
2. **Map each resource to a list resource**:
   a. Map each CloudFormation type to its Terraform resource type in `hashicorp/aws`.
   b. Use the Terraform MCP server to check whether the latest `hashicorp/aws`
      provider offers a **list resource** for that type. Check the registry docs;
      don't rely on memory.
   c. If, and only if, `aws` has no list resource for it, check `hashicorp/awscc`.
   d. If neither provider supports it, don't invent one. Record it as unsupported.
3. **Filter by tag**: If a list resource can't filter by tag, use the narrowest
   documented alternative (e.g., VPC ID, name prefix, parent resource).
   Add an HCL comment that explains why you used it.

# Constraints
- Every list resource and argument must exist in the registry docs you looked up.
  Never guess.

Three instructions did most of the work.

"Don't rely on memory." On an earlier attempt, before I was this explicit, one filter argument came back from what was effectively a general web search. It looked plausible. It was wrong.

"Don't invent one. Record it as unsupported." List resource coverage is expanding fast but incomplete. In the end there was one gap: AWS::EC2::VPCGatewayAttachment has no aws list resource, so it fell back to awscc.

"Add an HCL comment that explains why." The most valuable line in the prompt. The query file now documents its own compromises.

Clean, tag-filtered discovery:

list "aws_vpc" "stack_vpcs" {
  provider         = aws
  include_resource = true

  config {
    filter {
      name   = "tag:aws:cloudformation:stack-name"
      values = [var.stack_name]
    }
  }
}

And the compromises, which are the more honest half of the file:

# Route resources cannot be tagged; discover routes beneath the tagged tables.
list "aws_route" "routes_in_stack_route_tables" {
  for_each         = toset([for table in list.aws_route_table.stack_route_tables.data : table.state.id])
  provider         = aws
  include_resource = true

  config {
    route_table_id = each.value
  }
}

# The EKS cluster list resource has no filter arguments and lists clusters region-wide.
list "aws_eks_cluster" "eks_clusters" {
  provider         = aws
  include_resource = true
}

If you take one piece of syntax from this post, take the for_each in that first block. Child resources — routes, route table associations, security group rules, listeners — generally cannot be tagged, so you cannot find them by tag. But you can find their tagged parents and pivot: feed the parents' IDs into a for_each on the child's list block. list.aws_route_table.stack_route_tables.data is the output of one query used as input to the next. Discovery composes.

That pivot is also why include_resource = true appears on every block. By default a query returns only resource identities. Set it to true and the provider returns full resource objects, which is what makes table.state.id readable by the next query. Without it the for_each has nothing to pivot on. The docs note it may affect performance, so it is worth being deliberate about rather than setting everywhere out of habit — though for a few dozen resources I never noticed the cost.

The resources with no filters at all (EKS clusters, load balancers, DynamoDB tables, ECR repositories, IAM roles) return everything in the region or account. Fine in a demo account. Noisy in a busy shared one — a real limitation, and one more reason the phased approach below is worth the extra runs.

Twenty-three list blocks. One terraform query. Every resource found, with its import identity attached.

Step 2: From discovery to configuration

With the query file validated, I moved to HCP Terraform's Search and Import. Point it at the query, it runs discovery, you tick the resources you want, and it generates a starter configuration.

Have your resource IDs ready before you open this screen

This is the prerequisite nobody tells you about, and it caught me out on my first attempt.

Search and Import returns everything your query matched — resources already managed by Terraform alongside the unmanaged ones you are trying to adopt. It is a discovery tool, not a filter. Selecting the right rows is your job.

Say you want to import your unmanaged security groups. Your list block matches on the VPC or a tag, and you get back a list of every security group in scope. Some are already in state. Some belong to a different team. Some are the ones you actually want. To tell them apart on that screen you need the exact security group IDs of your targets in hand — sg-0bd9f4d46a38xxxxx, not "the one for the EKS nodes".

Two things follow from that:

Build your inventory first. Before you touch the UI, have a list of the resources you intend to import and their unique identifiers. Console, aws CLI, a tagging report, your CMDB — however you get there, get there before you start selecting. Trying to assemble that list while staring at a few hundred discovered rows is how you import something that belongs to somebody else.

Know which identifier each resource type uses, because it is not consistent. A security group is identified by id, an EKS cluster by name, a load balancer or IAM policy by arn, an EKS node group by cluster_name plus node_group_name. The identity schema is documented per resource in the Terraform Registry — check it there rather than assuming, which is another spot where having the MCP Server wired in paid off. You can see the shape of it in the generated import blocks:

# Security group — identified by id
identity = {
  account_id = "1243XXXXXXXX"
  id         = "sg-0bd9f4d46a38XXXXX"
  region     = "ap-southeast-1"
}

# EKS node group — composite identity, no id at all
identity = {
  account_id      = "1243XXXXXXXX"
  cluster_name    = "demo-eks-cluster"
  node_group_name = "demo-eks-nodegroup"
  region          = "ap-southeast-1"
}

This gets worse the busier your account is. In a demo account with one stack, the discovered list is short and obvious. In a shared production account where list "aws_iam_role" returns every role you own, the identifiers are the only thing standing between you and adopting a resource another team depends on.

The same thing works from the CLI:

terraform init
terraform query                                  # discover
terraform query -generate-config-out=main.tf     # generate resource + import blocks

For each resource you get a resource block populated from the live API response, plus an import block keyed on identity:

import {
  to = aws_vpc.stack_vpcs_0
  identity = {
    account_id = "1243XXXXXXXX"
    id         = "vpc-037c4dd02c90XXXXX"
    region     = "ap-southeast-1"
  }
}

resource "aws_vpc" "stack_vpcs_0" {
  cidr_block                           = "10.0.0.0/16"
  enable_dns_hostnames                 = true
  enable_dns_support                   = true
  enable_network_address_usage_metrics = false
  instance_tenancy                     = "default"
  region                               = "ap-southeast-1"
  tags = {
    Name = "demo-vpc"
  }
}

Previously I would have written that import block by hand, looked up the ID, hand-written the resource block, and iterated on plan until the diff was empty — forty times. Here, forty resources' worth of scaffolding arrived at once with correct identities.

That is the real win, and I want to state it plainly: Search and Import does not give you production-ready Terraform. It gives you a very good head start on the mechanical half of the job.

Step 3: "Starter configuration" means exactly that

HCP Terraform is upfront about it. The feature is experimental, and the console tells you to review and modify this configuration before applying. Having worked through it, that label is precise.

The reason is structural. The generator populates config from everything the API returned, but a provider schema mixes three kinds of fields: things you set, things the API computes, and things that are mutually exclusive with other things. The API response does not distinguish between them. So you get attributes that cannot legally be set, pairs of conflicting arguments, and a lot of zero-valued noise.

terraform validate on the raw file produced a long list of errors. The patterns repeat, so they are worth naming:

Read-only attributes written as inputs. tags_all is the classic — computed from tags plus provider default tags.
= null and zero-value noise. disk_size = 0, read_capacity = 0 on a PAY_PER_REQUEST DynamoDB table, empty timeouts blocks.
Mutually exclusive arguments both set. The biggest category. launch_template.id and launch_template.name. max_unavailable and max_unavailable_percentage. subnets and subnet_mapping on an LB.
Values outside valid ranges. order = 0 on a listener action, where the provider needs 1 or higher.
Inline vs standalone conflicts. Subtle and important: the generator emitted inline ingress/egress blocks on aws_security_group and standalone aws_vpc_security_group_ingress_rule resources for the same rules, because the query discovered both. Those cannot coexist. Same story with inline route blocks vs aws_route.
Redundant provider = aws lines, which validate rejects inside import blocks.

Rather than fix 1,300 lines by hand, I pointed the same MCP-grounded setup at the generated file:

# Objective
Find and fix broken or invalid code in `main.tf`. Keep the intended
infrastructure exactly as it is.

# Process
1. **Baseline**: Run `terraform validate`. Record every error and warning.
2. **Documentation review**: Use the Terraform MCP server to check that every
   argument is valid in the schema, that conflicting arguments aren't set
   together (`ConflictsWith` / `ExactlyOneOf`), and that read-only attributes
   are not being set.
3. **Classify** each finding as Breaking or Warning, based on a specific doc
   page or validate error. Never guess.
4. **Fix**: smallest change that works. If a fix could force a resource to be
   replaced, or the right value is unclear, stop and ask.
5. **Re-validate** and repeat until validate passes or the rest needs my input.

The two instructions I would not drop: "keep the intended infrastructure exactly as it is" and "if a fix could force a replacement, stop and ask." An assistant optimising for a clean validate will happily delete an argument to make an error go away. During a brownfield import, deleting an argument is how you end up with a plan that wants to replace your production node group. Correctness here is not "validate passes" — it is "validate passes and the plan is a pure import".

terraform validate only checks schema and syntax. It cannot tell you whether your plan is about to replace a live resource. A clean validate is the start of the review, not the end of it.

So I needed something that could tell me that, on every run, before anything was applied.

Step 4: Guardrails first, imports second

I want to be emphatic about the ordering, because it is the most transferable idea here.

I attached the policies before I imported anything.

The failure mode of a brownfield import is not subtle, but it is quiet. You misread one attribute. That attribute turns out to be ForceNew. Your plan, which you believed was an import, now contains a replace. Reading a forty-resource plan at the end of a long day, you can absolutely miss that — and the blast radius is your running application.

That is a textbook policy-as-code problem: a specific, machine-checkable invariant that must hold on every run.

Policy one: no destructive changes (hard-mandatory)

An import run must not destroy or replace anything. In plan terms, three action sets are forbidden: ["delete"], ["delete","create"], and ["create","delete"]. Everything else — imports, no-ops, updates — is fine as far as this policy is concerned.

import "tfplan/v2" as tfplan
import "tfrun"

# Skip enforcement for explicit destroy runs.
is_destroy_run = tfrun.is_destroy

destructive_actions = [
    ["delete"],
    ["delete", "create"],
    ["create", "delete"],
]

destructive_changes = filter tfplan.resource_changes as _, rc {
    not is_destroy_run and any destructive_actions as da {
        actions_equal(rc.change.actions, da)
    }
}

main = rule {
    is_destroy_run or violation_count is 0
}

Two details worth copying. The tfrun.is_destroy escape hatch means terraform destroy still works when you genuinely mean it — a guardrail that blocks legitimate operations gets switched off, and a switched-off guardrail protects nothing. And the policy prints the address, type, and action of every offending resource, so a failure tells you why without hunting.

This runs at hard-mandatory. No override. I cannot talk myself past it at 7pm.

Policy two: flag in-place updates (soft-mandatory)

A correct import produces a plan full of imports and nothing else. If the plan also wants to update something, that is not dangerous but it is always informative — it means my generated config does not exactly match what is deployed. Either the generator got a field slightly wrong, or there is real drift.

I do not want to block that. I want to be told and then decide.

updated_resources = filter tfplan.resource_changes as _, rc {
    any update_actions as ua {
        actions_equal(rc.change.actions, ua)
    }
}

main = rule {
    update_count is 0
}

policy "no-destructive-changes" {
  source            = "./policies/no-destructive-changes.sentinel"
  enforcement_level = "hard-mandatory"
}

policy "flag-resource-updates" {
  source            = "./policies/flag-resource-updates.sentinel"
  enforcement_level = "soft-mandatory"
}

Two policies, about 120 lines of Sentinel. That is the entire safety apparatus.

Step 5: Import in phases

The temptation, once you have a working query file and generated config, is to apply all forty resources in one run. I did not, and I would push back on anyone who wanted to. I imported in dependency order, validating each layer before starting the next:

Networking — VPC, subnets, gateways, route tables, routes, associations
IAM — roles, inline policies, customer-managed policies, OIDC provider
Security groups — and their ingress rules
Compute — launch template, EKS cluster, node group
Load balancing — target group, ALB, listener
Data — DynamoDB table
Registry — ECR repository

Networking:

IAM:

Compute:

Why this is worth the extra runs:

Blast radius is one layer. If something looks wrong in the security group phase, there are three resources to inspect, not forty across seven services.

Dependencies resolve naturally. The EKS cluster references subnet IDs, security group IDs, and a role ARN. By the time I reach compute, those are already in state and verified.

Plans stay readable. This is the one I care about most. A plan with five imports is a plan I will actually read. A plan with forty is a plan I will scroll past while telling myself it looks fine. The policies exist because I do not fully trust myself on that — but the best way to use a guardrail is to not need it, and small plans are how you get there.

The full sequence in the workspace history, each entry one layer, reviewed and applied on its own:

Where the guardrails earned their keep

They fired twice. Both times they were right.

The one that would have cost me my EKS cluster

The starter configuration for my cluster came back with this:

resource "aws_eks_cluster" "eks_clusters_0" {
  bootstrap_self_managed_addons = false
  # ...
}

One boolean, generated for me, with a value that looks like a sensible default. But bootstrap_self_managed_addons is ForceNew on aws_eks_cluster, because it describes a decision made at creation time. My live cluster was created with it true. A mismatch on that field does not produce an update — it produces a replacement.

So a plan I thought was importing my EKS cluster was proposing to destroy and recreate the cluster that was, at that moment, serving the app in the screenshot at the top of this post.

The hard-mandatory policy caught it and failed the run:

This is the whole argument, so it is worth sitting with. I did review the config. terraform validate passed cleanly — false is a perfectly valid value for that argument, the right type, in the right place. Nothing about the line was wrong in isolation; it just did not match reality. The only way to catch it by eye was to already know that this specific boolean is ForceNew, and then to spot the # forces replacement marker in several hundred lines of plan output for a resource with a dozen nested blocks.

A policy reading the plan's action list needs none of that context. It does not need to know what bootstrap_self_managed_addons means. It needs to know that ["delete","create"] is not allowed during an import. That is the strength of policy as code — it checks a structural invariant instead of relying on my attention span.

The fix was a one-character change: flip it to true to match the cluster as deployed, re-run, and the plan became a clean import. Found by a guardrail instead of by an incident.

The one that was just drift

During the security group phase, the soft-mandatory policy spoke up. A few in-place update actions — not destructive, just minor differences between my generated config and the live rules.

Exactly the behaviour I wanted. The run paused, told me which resources it intended to modify, and waited. I checked each one, confirmed they were benign, overrode deliberately, and applied.

The difference in handling between these two cases is the point of having enforcement levels. hard-mandatory for "this must never happen", soft-mandatory for "a human needs to look at this". Collapse both into a blocking rule and the import becomes unworkable. Collapse both into a warning and the EKS replacement goes through.

The result

Every component of the app is now in Terraform state and managed through HCP Terraform. Around 40 resources across 23 types, including the one that needed awscc. About 1,300 lines of validated HCL, with the import blocks as a precise record of what was adopted and when.

And the app never went down. Not because I was careful, though I tried to be, but because the system was set up so that carelessness would be caught.

What I had at the end is a managed application. What I did not have is well-structured code.

Next: a 1,600-line `main.tf` is not where this ends

Let me be honest about what came out of this, because "I imported it successfully" papers over a lot.

The file is a monolith. Every ID is hardcoded — vpc-037c4dd02c90XXXXX appears a dozen times, security group IDs are inlined into the EKS cluster's vpc_config, role ARNs are string literals. Resource names are generator artefacts (stack_subnets_0, iam_roles_32, ingress_rules_in_stack_groups_1_5) that describe the query that found them, not what they do. The region repeats on nearly every resource.

This is a reasonable place for an import to end. It is an unreasonable place to stay. You cannot reuse any of it for a second environment, and the hardcoded IDs mean Terraform does not actually know the subnets belong to the VPC — it just happens to have the right strings in the right places.

So the next step is refactoring, and I used HashiCorp's Agent Skills again for it, since they encode HashiCorp's own module design best practices:

Decompose into network, iam, security, eks, alb, data, registry
Replace every hardcoded ID with a real reference, so the dependency graph reflects actual dependencies
Promote region, environment name, and sizing to variables
Name resources after their purpose, not their discovery order
Define proper inputs and outputs at each module boundary

The ID-to-reference conversion is the bulk of the work and the highest-value part, because it is what turns a snapshot of your infrastructure into a description of it. That turned out to be a post of its own, and it is now live: Part 2: How to Refactor a Terraform Monolith into Modules Without Downtime. It covers splitting this file into seven modules with HashiCorp's Agent Skills and the Terraform MCP Server, preserving state with moved blocks to get a 0 to add, 0 to change, 0 to destroy plan, and retiring the CloudFormation stack safely with DeletionPolicy: Retain.

The framing I have settled on: import gets you management; refactoring gets you Infrastructure as Code. Two different milestones, and worth being clear-eyed about which one you have reached.

What I would tell you before you try this

Discovery is now a reviewable artefact. Moving from "paste IDs into a spreadsheet" to "a committed .tfquery.hcl file" is a bigger shift than it looks. When someone adds a resource next month, I re-run the query instead of repeating the archaeology.

Check support before you plan the work, not during it. List resource coverage varies by provider and version. The supported-resources report takes a minute to check and tells you whether this workflow is even available for the resources you care about.

Bring your resource IDs to the party. Search and Import shows you managed and unmanaged resources together. Picking the right ones out of that list requires knowing their exact identifiers beforehand — and knowing which identifier each resource type uses, because it varies. Do that inventory before you open the UI.

Learn the parent-pivot pattern. Most child resources cannot be tagged. for_each over a parent list block's output is how you reach them. That one pattern unlocked routes, associations, SG rules, and listeners.

Ground your AI assistant in the registry, or don't use it. An Agent Skill for conventions plus the MCP Server for version-specific docs is what made AI assistance trustworthy rather than a source of plausible-looking errors. "Check the docs, never guess, record what you couldn't cover" belongs in your prompt verbatim.

Read "starter configuration" literally. The generator writes back everything the API returned — computed attributes, conflicting arguments, zero-value noise. Budget real time for cleanup. Still dramatically faster than hand-writing forty import blocks.

A clean terraform validate is not safety. The one I would underline. The gap between "valid configuration" and "safe plan" is exactly where brownfield imports go wrong, and exactly the gap policy as code fills.

Attach policies before the first import, not after the first scare. Mine were in place before any resource entered state, which is why the EKS near-miss is an anecdote rather than an incident report.

Use both enforcement levels. Hard for invariants that must never break, soft for things a human must consciously approve. One without the other gives you either a workflow you cannot complete or a warning nobody reads.

Import in layers. Smaller plans get read properly, dependencies resolve in order, failures stay localised. The extra runs cost minutes.

Closing thought

The capability to import brownfield infrastructure has existed for years. What has been missing is not capability but economics and confidence.

Search changes the economics — discovery becomes declarative and generation becomes bulk, so adopting forty resources is an afternoon rather than a sprint. Policy as code changes the confidence — you no longer have to be certain you read the plan correctly, because a machine checks the one invariant that matters on every run. MCP-grounded AI assistance makes the authoring in between fast without making it unreliable.

The infrastructure that most needs to be in code is the infrastructure people are most afraid to touch. That fear is rational. It is also, I think, now addressable.

Try it yourself

Part 2: How to Refactor a Terraform Monolith into Modules Without Downtime
Demo repo (CloudFormation template, query file, prompts, app): github.com/pranit-hashi/aws-brownfield-terraform-import
Sentinel policies used as guardrails: github.com/pranit-hashi/import-sentinel-policies
HashiCorp Agent Skills for Terraform — highly recommended reading before you use them: github.com/hashicorp/agent-skills/tree/main/plugins/terraform
Terraform MCP Server: developer.hashicorp.com/terraform/mcp-server
Which resources support list/import? An auto-updating report that tracks provider-level support, and the first thing I check before planning an import: list-resources-report.json

Setup used: Terraform 1.14+, hashicorp/aws 6.67.0 and hashicorp/awscc 1.104.0, HCP Terraform with Sentinel policy sets, the Terraform MCP Server, and the terraform-search-import Agent Skill.

If you try this on your own brownfield app, I would like to hear which resource type caught you out. My money is on something in IAM.

How to create a CodePipeline with source from another AWS account?

Pranit Raje — Sun, 22 Mar 2020 15:22:30 +0000

When using AWS services, more often than not, we need multiple AWS accounts for various reasons. If you work on AWS DevOps tools, you must have come across AWS CodePipeline which is a pipeline automation tool from AWS. While working with CodePipeline in multi-account scenario, I'm sure you all have faced many issues like Access Denied errors which is the most puzzling and frustrating error IMHO because it doesn't always say where we're lacking permissions. In multi-account scenario for CodePipeline, if you're deploying your application in another AWS account, AWS has this excellent documentation which I'd highly recommend you to go through. But what if you have a unique requirement when the source of your pipeline itself is in another AWS account and the deployment is in the same account as of CodePipeline?

Recently, I came across this use-case and tried to find some starting guide to create a CodePipeline having source from another AWS account but couldn't find any useful resources. Hence, I decided to write this blog to help others who might run into such use-case.

This blog will walk you through an example of deploying from Amazon ECR to Amazon ECS with the help of CodePipeline. Here, I'm having a use-case in which,

Account-A has already-built Docker image hosted in ECR (Source stage)
Account-B has S3 bucket (Source stage) containing imagedefinitions.json and ECS (Deploy stage).

NOTE: All AWS resources in this example exist in the same AWS region.

Our requirement is such that we have a team of developers in Account-A who will build their Docker images locally, push it in ECR repository in Account-A and this Docker image should be deployed automatically to the resources (ECS) present in Account-B whenever our developers make changes to their Docker images and push it in ECR repository of Account-A. And, we want our CodePipeline to be set up in Account-B itself where our deployment is done. Sounds challenging right? No worries. Let's get started...

Prerequisites:

Two active AWS accounts.
A working Amazon ECS cluster up and running with all other related resources e.g., Task Definition, ECR repository, ECS Service, any Docker image in one of the AWS account.

Target Architecture:

The above diagram shows the following workflow:

Initially, the developers will build the Docker image and push it to their central ECR repository in Account A.
Account A will have a EventBridge Rule created listening to our ECR repository in Account A. As soon as a successful Docker image is pushed to our ECR repository, an EventBridge Rule will be triggered in Account A.
Default EventBridge Bus in Account B will be configured as a target to our EventBridge rule in Account A.
We will have another EventBridge rule created in Account B which will listen to the push events from the ECR repository in Account A.
The EventBridge rule in Account B will trigger our CodePipeline.
CodePipeline will assume a cross-account IAM role created in Account A to list and describe the latest Docker image pushed into ECR repository.
CodePipeline will read the Docker image and pass it to the next stage of CodePipeline.
The imagedefinitions.zip file from S3 bucket will contain the information about the container name and the image URI of our Docker image stored in Account A’s ECR repository which will be used in ECS Standard deployment.

Now, let's see how we can build such pipeline from AWS Management Console in a step-by-step manner,

Step 1: IN ACCOUNT-B -> Set up a normal CodePipeline:

Firstly, create a standard pipeline having all the components of pipeline (ECR,S3,ECS) residing in same Account-B normally the way you would. We are creating this pipeline in the same account first just to get a backbone structure for our cross-account pipeline which we will modify according to our requirements in later steps.
In this pipeline, Source stage will have two actions i.e. ECR and S3 where ECR will be your repository with image tag and S3 bucket will be having zip file of the JSON file named imagedefinitions.json. This JSON file is required for the ECS deployment as it contains container name and the ECR image URI with its image tag. Refer to this official documentation to know more about this file. Do remember to zip the file when S3 is used in the source stage.

NOTE: This S3 bucket should be versioned.
When you include ECS (Standard deployment) as a deployment provider in your pipeline, make sure to point S3 action's output as an input to ECS because ECR action's output will contain information only about the Docker image and not the container name which is expected by CodePipeline job worker for ECS.
Once you have working pipeline in Account-B, we will modify it for cross-account deployments having source from Account-A.

Step 2: IN ACCOUNT-B -> Create custom AWS KMS key:

For any cross-account deployments, CodePipeline require custom KMS key allowing another AWS account to access encrypted CodePipeline artifacts stored in CodePipeline’s S3 bucket (artifact store).
Create custom AWS KMS key allowing Account-A to access CodePipeline artifacts residing in Account-B’s S3 bucket.

While creating KMS policy via console, when you reach to the 5th step Review and edit key policy, enter the below policy by replacing appropriate ARN at the commented places.

{
    "Id": "key-consolepolicy",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<ACCOUNT-B_ID>:root"  //MAKE CHANGES HERE
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow access for Key Administrators",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<ACCOUNT-B_ID>:user/<YOUR_IAM_USER>"   //MAKE CHANGES HERE
            },
           "Action": [
                "kms:Create*",
                "kms:Describe*",
                "kms:Enable*",
                "kms:List*",
                "kms:Put*",
                "kms:Update*",
                "kms:Revoke*",
                "kms:Disable*",
                "kms:Get*",
                "kms:Delete*",
                "kms:TagResource",
                "kms:UntagResource",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::<ACCOUNT-B_ID>:user/<YOUR_IAM_USER>",   //MAKE CHANGES HERE
                    "arn:aws:iam::<ACCOUNT-B_ID>:role/service-role/AWSCodePipelineServiceRole-<AWS_REGION>-<CODEPIPELINE_NAME>",  //REPLACE WITH CODEPIPELINE SERVICE ROLE ARN
                    "arn:aws:iam::<ACCOUNT-A_ID>:root"  //MAKE CHANGES HERE
                ]
            },
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow attachment of persistent resources",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::<ACCOUNT-B_ID>:user/<YOUR_IAM_USER>",   //MAKE CHANGES HERE
                    "arn:aws:iam::<ACCOUNT-B_ID>:role/service-role/AWSCodePipelineServiceRole-<AWS_REGION>-<CODEPIPELINE_NAME>",  //REPLACE WITH CODEPIPELINE SERVICE ROLE ARN
                    "arn:aws:iam::<ACCOUNT-A_ID>:root"  //MAKE CHANGES HERE
                ]
            },
            "Action": [
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:RevokeGrant"
            ],
            "Resource": "*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        }
    ]
}

The above KMS key policy will allow your IAM user, CodePipeline service role and Account-A to access this KMS key with which our artifacts will be encrypted.

Step 3: IN ACCOUNT-A -> Create a custom IAM role for cross-account access:

Create a new IAM role named CrossAccount-A_Role. You can name this role anything you want as long as it follows the naming conventions in IAM. Consider giving the role a name that clearly states its purpose.
On the Create role page, choose Another AWS account option from Select type of trusted entity section. Enter the AWS Account ID of Account-B in the Specify accounts that can use this role section. Click Next: Permissions.
Attach two AWS managed policies named AmazonEC2ContainerRegistryReadOnly and AmazonS3FullAccess to this IAM Role.
Create another custom inline IAM policy named cross-account-KMS for this same IAM role to access Account-B’s KMS key which we created in Step 2. Enter the below IAM policy by replacing ARN of KMS key in Account-B at the commented place for this inline policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kms:DescribeKey",
                "kms:GenerateDataKey*",
                "kms:Encrypt",
                "kms:ReEncrypt*",
                "kms:Decrypt"
            ],
            "Resource": [
                "<ARN_OF_KMS_KEY_IN_ACCOUNT-B>"  //MAKE CHANGE HERE
            ]
        }
    ]
}

Step 4: IN ACCOUNT-B -> Create a policy for the S3 bucket that grants access to Account A:

Choose your CodePipeline’s S3 bucket i.e. artifact store (not the source S3 bucket) and edit its Bucket policy according to the following policy by replacing appropriate ARN at the commented places.
The below S3 bucket policy will allow Account-A to access our pipeline's artifacts.

{
    "Version": "2012-10-17",
    "Id": "SSEAndSSLPolicy",
    "Statement": [
    {
        "Sid": "DenyUnEncryptedObjectUploads",
        "Effect": "Deny",
        "Principal": "*",
        "Action": "s3:PutObject",
        "Resource": "arn:aws:s3:::<CODEPIPELINE'S_S3_BUCKET_NAME>/*",   //MAKE CHANGE HERE
        "Condition": {
                "StringNotEquals": {
                    "s3:x-amz-server-side-encryption": "aws:kms"
                }
            }
    },
    {
        "Sid": "DenyInsecureConnections",
        "Effect": "Deny",
        "Principal": "*",
        "Action": "s3:*",
        "Resource": "arn:aws:s3:::<CODEPIPELINE'S_S3_BUCKET_NAME>/*",   //MAKE CHANGE HERE
        "Condition": {
            "Bool": {
                    "aws:SecureTransport": false
                }
            }
    },
    {
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::<ACCOUNT-A_ID>:root"   //MAKE CHANGE HERE
            },
        "Action": [
                "s3:Get*",
                "s3:Put*"
            ],
        "Resource": "arn:aws:s3:::<CODEPIPELINE'S_S3_BUCKET_NAME>/*"   //MAKE CHANGE HERE
    },
    {
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::<ACCOUNT-A_ID>:root"   //MAKE CHANGE HERE
                },
        "Action": "s3:ListBucket",
        "Resource": "arn:aws:s3:::<CODEPIPELINE'S_S3_BUCKET_NAME>"   //MAKE CHANGE HERE
        }
    ]
}

Step 5: IN ACCOUNT-B -> Create an inline policy for service role of CodePipeline:

As we want CodePipeline service role to assume the cross-account IAM role we created in Step 3, we should add an extra inline IAM policy to the CodePipeline service role.
Enter the below IAM policy by replacing Account-A ID at the commented place for this inline policy.

{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Action": "sts:AssumeRole",
        "Resource": [
            "arn:aws:iam::<ACCOUNT-A_ID>:role/*"   //MAKE CHANGE HERE
        ]
    }
}

Step 6: IN ACCOUNT-A -> Apply ECR Repository Policy:

To allow CodePipeline in Account-B to pull ECR images residing in Account-A, ECR repository should allow Account-B to pull those images from its repository. For this, we will apply resource-based policy to our ECR repository.
You can use the below ECR repository policy by replacing appropriate ARN at the commented places.

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Sid": "Cross-account-policy",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::<ACCOUNT-B_ID>:root",   //MAKE CHANGE HERE
          "arn:aws:iam::<ACCOUNT-A_ID>:role/CrossAccount-A_Role"   //ARN OF CROSS-ACCOUNT IAM ROLE CREATED IN ACCOUNT-A AT STEP 3   
        ]
      },
      "Action": [
        "ecr:BatchCheckLayerAvailability",
        "ecr:BatchGetImage",
        "ecr:CompleteLayerUpload",
        "ecr:DescribeImages",
        "ecr:DescribeRepositories",
        "ecr:GetDownloadUrlForLayer",
        "ecr:GetLifecyclePolicy",
        "ecr:GetLifecyclePolicyPreview",
        "ecr:GetRepositoryPolicy",
        "ecr:ListImages",
        "ecr:UploadLayerPart"
      ]
    }
  ]
}

Step 7: IN ACCOUNT-B -> Modify your pipeline’s configuration file:

Now that we have most of the required components of our cross-account pipeline ready, we will modify the configuration file of our pipeline that we created in Step 1 by changing the source to our ECR repository in Account-A and we will also add KMS related details.
To do that, run get-pipeline command aws codepipeline get-pipeline --name <CODEPIPELINE_NAME> --region <AWS_REGION> > pipeline.json from AWS CLI.

Open this pipeline.json file in your favourite text editor (vim?) and add ‘encryptionKey’ in ‘artifactStore’ section.

 "artifactStore": {
    "type": "S3",
    "location": "codepipeline-<AWS_REGION>-XXXXXXXXXX",  //MAKE CHANGE HERE
    "encryptionKey": {
        "id": "arn:aws:kms:<AWS_REGION>:<ACCOUNT-B_ID>:key/XXXXXXXXXX-XXXX-XXX-XXXX-XXXXXXXXXX",    //MAKE CHANGE HERE    
        "type": "KMS"
        }
     }

Then, modify your ECR action of Source stage where you will change the values of ECR repository name and its image tag which is in Account-A.

In the same ECR action, include the "roleArn": "arn:aws:iam::<ACCOUNT-A_ID>:role/CrossAccount-A_Role" which we created in Step 3 so that this cross-account IAM role from Account-A will be used for ECR action. Following is the snippet for our cross-account source action in pipeline's JSON file.

NOTE: Replace appropriate values at commented places below

{
    "name": "Source",
    "actionTypeId": {
      "category": "Source",
      "owner": "AWS",
      "provider": "ECR",
      "version": "1"
     },
    "runOrder": 1,
    "configuration": {
      "ImageTag": "latest",    //MAKE CHANGE HERE IN CASE OF CUSTOM TAG
      "RepositoryName": "XXXXX"    //MAKE CHANGE HERE
     },
    "outputArtifacts": [
      {
        "name": "SourceArtifact"
      }
     ],
    "inputArtifacts": [],
    "region": "<AWS_REGION>",  //MAKE CHANGE HERE
    "namespace": "SourceVariables",
    "roleArn": "arn:aws:iam::<ACCOUNT-A_ID>:role/CrossAccount-A_Role"  //MENTION CROSS-ACCOUNT IAM ROLE    
}

NOTE: Above is just a snippet of our ECR action of Source stage and not the entire JSON configuration of pipeline.

Make sure to perform the 4th step of this documentation to remove the unwanted metadata section from configuration file.
Once you make the above changes, update the pipeline from update-pipeline CLI command aws codepipeline update-pipeline --region <AWS_REGION> --cli-input-json file://pipeline.json

After you update your pipeline, don't forget to update your imagedefintions.json file with the correct imageURI from Account-A if you haven't done so. It should look like below,

[
  {
    "name": "<CONTAINER_NAME>",  //MAKE CHANGES HERE
    "imageUri": "<ACCOUNT-A_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com/<REPOSITORY_NAME>:<IMAGE_TAG>"  //MAKE CHANGES HERE    
  }
]

Once updated the imagedefinitions.json file, zip it and upload it again to your source S3 bucket.

As you might be aware that CloudWatch Event (CWE) Rules are the resources in the background which trigger your pipeline whenever there are any changes to your source present in AWS. Since, one of our source action, i.e. ECR is existing in another AWS account, CodePipeline cannot create this CWE rule automatically for you for ECR as it did for S3 action. Hence, we will now create CWE rule for ECR repository with the help of CloudWatch Event Bus.

Step 8: IN ACCOUNT-B -> Edit default Event Bus to allow account A:

Go to CloudWatch console and select your default Event Bus.
Click on Add permission to add your Account-A ID and check the Everybody(*) option.

Step 9: IN ACCOUNT-A -> Create CWE Rule for your ECR repository:

Follow this official documentation till Step 5 to create CWE rule for your ECR repository as a source for your CWE rule.
As a target for this CWE rule, select Event bus in another AWS account. Enter your Account-B ID and select Create a new role for this specific resource option for IAM role which will automatically create IAM role for this CWE rule.

Step 10: IN ACCOUNT-B -> Create CWE Rule with CodePipeline as a target:

Create a new CWE rule with Event Source exactly same as you made in Account-A. Edit this event pattern by adding account section with the value as Account-A’s ID. This will listen to any changes occurring to your ECR repository present in Account-A with the help of Event Bus. Refer to this official documentation to know how to do this exactly. Once you create Event pattern, it should look like below,

{
  "detail-type": [
    "ECR Image Action"
  ],
  "account": [
    "ACCOUNT-A_ID"  //MAKE CHANGES HERE
  ],
  "source": [
    "aws.ecr"
  ],
  "detail": {
    "action-type": [
      "PUSH"
    ],
    "image-tag": [
      "latest"  //MAKE CHANGE HERE IN CASE OF CUSTOM TAG
    ],
    "repository-name": [
      "<REPOSITORY_NAME>"  //MAKE CHANGES HERE
    ],
    "result": [
      "SUCCESS"
    ]
  }
}

Put your CodePipeline ARN (arn:aws:codepipeline:<AWS_REGION>:<ACCOUNT-B_ID>:<PIPELINE_NAME>) as a target for this CWE rule.

Hooray!! Now, we have successfully set up CodePipeline in Account-B with ECR being in Account-A, S3 bucket containing only zipped imagedefinitions.json file for ECS deployment and ECS service in Account-B. Whenever you make any changes to any of your source, your pipeline will be triggered in a CI/CD manner.

Automation & Scale:

Following the step-by-step instructions from the above section can be useful if we want to understand what we are doing to implement the architecture. However, once we understand that and when we want to automate the whole target architecture, it can be deployed in an automated way using AWS CloudFormation which is an Infrastructure as Code (IaC) tool from AWS. To automate the deployment in multiple AWS accounts, make sure to use the CloudFormation templates from GitHub repository. In the repository, we have three CloudFormation templates which should be deployed sequentially as noted below:

In Account B, create a new CloudFormation stack using AccB-pipeline-1.yaml template.
In Account A, create a new CloudFormation stack using AccA-pipeline.yaml template.
In Account B, run following AWS CLI command from Account B to allow Account A to send ECR push events to Event Bus in Account B. aws events put-permission --action events:PutEvents --statement-id MySid --principal <ACCOUNT_A_ID>
In Account B, update the CloudFormation stack created in Step 1 using AccB-pipeline-2.yaml template. In this template, we will use one additional parameter named CrossAccountRoleName which will be the name of cross-account role created in Step 2. The IAM role name will be available from the Outputs section of CloudFormation console. Along with that, we will also put the ECR repository name created in Account A in Step 2 which is available in the Outputs section of CloudFormation console.

In Account B, make sure to update imagedefintions.json file with the correct imageURI from Account A. It may look like below,

[
{
"name": "<CONTAINER_NAME>",  //MAKE CHANGES HERE
"imageUri": "<ACCOUNT-A_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com/<REPOSITORY_NAME>:<IMAGE_TAG>"  //MAKE CHANGES HERE    
}
]

In Account A, push already built Docker image to ECR repository in Account A.
In Account B, notice how CodePipeline will get triggered automatically as soon as any Docker image is pushed to ECR repository in Account A. This Docker image will be deployed to ECS cluster in Account B in CI/CD manner.

Thanks for reading this blog! I hope you will find this useful. Feel free to like, share and comment on this article with your constructive feedback. If you find this blog helpful or if you're stuck at any of the above step, do let me know via comments section and I'd be more than happy to help!

Have an AWSome Day! 😎

DEV Community: Pranit Raje

How to Refactor a Terraform Monolith into Modules Without Downtime

Part 2: Splitting a 1,600-line imported main.tf into seven modules with HashiCorp Agent Skills, the Terraform MCP Server and moved blocks, then retiring the CloudFormation stack that started it all

A note on what this is

The one number that matters

The toolkit

The prompt

The approval gate

What came out

Names that mean something

moved blocks are what make zero downtime possible

What happened to the import blocks

Replacing hardcoded values with references

depends_on, but only where it earns its place

Clean-up

Proving it

Retiring the CloudFormation stack without deleting anything

What I would tell you before you try this

Closing thought

Try it yourself

How to Import Existing AWS Infrastructure into Terraform Without Downtime

Bringing a live three-tier app under Terraform control with Terraform Search, HCP Terraform's Search and Import, Terraform MCP Server, HashiCorp Agent Skills and Sentinel policy that stopped a plan from replacing my EKS cluster

A note on what this is

The workflow

The application

Step 1: Discovery as code

Step 2: From discovery to configuration

Have your resource IDs ready before you open this screen

Step 3: "Starter configuration" means exactly that

Step 4: Guardrails first, imports second

Policy one: no destructive changes (hard-mandatory)

Policy two: flag in-place updates (soft-mandatory)

Step 5: Import in phases

Where the guardrails earned their keep

The one that would have cost me my EKS cluster

The one that was just drift

The result

Next: a 1,600-line main.tf is not where this ends

What I would tell you before you try this

Closing thought

Try it yourself

How to create a CodePipeline with source from another AWS account?

Prerequisites:

Target Architecture:

Step 1: IN ACCOUNT-B -> Set up a normal CodePipeline:

Step 2: IN ACCOUNT-B -> Create custom AWS KMS key:

Step 3: IN ACCOUNT-A -> Create a custom IAM role for cross-account access:

Step 4: IN ACCOUNT-B -> Create a policy for the S3 bucket that grants access to Account A:

Step 5: IN ACCOUNT-B -> Create an inline policy for service role of CodePipeline:

Step 6: IN ACCOUNT-A -> Apply ECR Repository Policy:

Step 7: IN ACCOUNT-B -> Modify your pipeline’s configuration file:

Step 8: IN ACCOUNT-B -> Edit default Event Bus to allow account A:

Step 9: IN ACCOUNT-A -> Create CWE Rule for your ECR repository:

Step 10: IN ACCOUNT-B -> Create CWE Rule with CodePipeline as a target:

Automation & Scale:

`moved` blocks are what make zero downtime possible

What happened to the `import` blocks

`depends_on`, but only where it earns its place

Next: a 1,600-line `main.tf` is not where this ends