DEV Community: Raghvendra Pandey

Terraform Modules: Writing, Testing, and Reusing Infrastructure Code

Raghvendra Pandey — Sat, 20 Jun 2026 11:11:19 +0000

Every Terraform codebase eventually hits the same inflection point: the root module grows unwieldy, the same patterns repeat across environments, and someone asks "can we just reuse the VPC setup from the payment team?" Modules are Terraform's answer to reuse and encapsulation. Done well, they let you compose infrastructure from tested, versioned building blocks. Done poorly, they create abstraction layers that hide problems, make debugging harder, and need to be broken apart six months later.

This guide covers what modules actually are, how to structure them, the patterns that work at scale, and how to test them — including when you probably shouldn't extract a module at all.

What a module is

In Terraform, every directory of .tf files is a module. The directory you run terraform apply from is the root module. Any other directory of .tf files that you reference with a module block is a child module.

That's it. There's no special module declaration or module keyword inside the module directory. A module is just a collection of Terraform resources that accepts inputs via variable blocks and exposes outputs via output blocks.

# Calling a module from a root module
module "vpc" {
source  = "./modules/vpc"      # local path
# or:
source  = "terraform-aws-modules/vpc/aws"  # Terraform Registry
version = "~> 5.0"

name             = "production"
cidr             = "10.0.0.0/16"
azs              = ["us-east-1a", "us-east-1b", "us-east-1c"]
private_subnets  = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
public_subnets   = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
}

Module structure

A well-structured module has four files at minimum:

modules/vpc/
├── main.tf       # resource definitions
├── variables.tf  # input variable declarations
├── outputs.tf    # output value declarations
└── versions.tf   # required_providers and terraform version constraint

Optional but common additions:

modules/vpc/
├── main.tf
├── variables.tf
├── outputs.tf
├── versions.tf
├── README.md     # documents inputs, outputs, usage examples
└── examples/
└── complete/ # a working example that calls the module

variables.tf

Variables are the public interface of a module. Every variable should have a description and a type. Use default for optional variables; omit it for required ones.

variable "name" {
description = "Name prefix for all resources created by this module."
type        = string
}

variable "cidr_block" {
description = "CIDR block for the VPC."
type        = string
default     = "10.0.0.0/16"
validation {
condition     = can(cidrhost(var.cidr_block, 0))
error_message = "Must be a valid CIDR block."
}
}

variable "enable_nat_gateway" {
description = "Whether to create a NAT Gateway in each availability zone."
type        = bool
default     = true
}

variable "tags" {
description = "Tags to apply to all resources."
type        = map(string)
default     = {}
}

The validation block runs before any resource creation and gives users a useful error immediately instead of an obscure provider error later.

outputs.tf

Outputs expose values that callers need. Be generous with outputs — it's easier to add an output than to break consumers who discover they needed one.

output "vpc_id" {
description = "The ID of the VPC."
value       = aws_vpc.main.id
}

output "private_subnet_ids" {
description = "List of IDs of private subnets."
value       = aws_subnet.private[*].id
}

output "public_subnet_ids" {
description = "List of IDs of public subnets."
value       = aws_subnet.public[*].id
}

output "nat_gateway_ids" {
description = "List of NAT Gateway IDs."
value       = aws_nat_gateway.main[*].id
}

versions.tf

Constrain the Terraform version and provider versions to prevent incompatible configurations:

terraform {
required_version = ">= 1.5.0"

required_providers {
aws = {
source  = "hashicorp/aws"
version = ">= 5.0, = 5.0, 5.0` for modules — it prevents a major version bump from breaking users who haven't explicitly upgraded.

## Local modules vs registry modules

Local modules (relative paths like `./modules/vpc`) are for internal organizational abstractions. They live in your repo, version-controlled alongside the infrastructure that uses them. Advantages: fast iteration, no publishing step, easy to understand the full codebase as a unit. Disadvantages: can't be shared across repos without copy-pasting.

Registry modules are published to the Terraform Registry (public) or a private registry (Terraform Cloud, Artifactory, etc.). They're pinned by version and fetched by `terraform init`. The key attributes of a good registry module:

- Semantic versioning with a clear changelog
- Well-documented inputs and outputs
- Working examples in `examples/`
- Automated tests
- Maintained and responding to issues

The community modules at `terraform-aws-modules` on GitHub are the gold standard. The `terraform-aws-modules/vpc/aws` module, for example, manages hundreds of resource configurations across a very large variable surface. It's an excellent reference for how to structure a complex module.

## Version pinning

Always pin module versions in production. Unpinned modules pull the latest version on `terraform init -upgrade`, which can introduce breaking changes silently.

Good — pinned to a specific minor version

module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 5.1" # allows 5.1.x, not 5.2.x or 6.x

...

}

Bad — no version constraint

module "vpc" {
source = "terraform-aws-modules/vpc/aws"

version omitted — gets latest on terraform init -upgrade

}


Use `~> X.Y` (allows patch updates within the minor version) for registry modules. Only upgrade to a new minor or major version intentionally.

## Module composition patterns

### Flat composition

The simplest and most common pattern: the root module calls several child modules, each responsible for one infrastructure concern.

module "vpc" {
source = "./modules/vpc"
name = var.environment
cidr = "10.0.0.0/16"
}

module "eks" {
source = "./modules/eks"
cluster_name = "${var.environment}-cluster"
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnet_ids
}

module "rds" {
source = "./modules/rds"
name = "${var.environment}-db"
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnet_ids
}


Outputs from one module become inputs to another. This dependency chain is explicit — Terraform builds a dependency graph and applies modules in the right order.

### Wrapper modules

A wrapper module adds organizational defaults on top of a community module:

modules/vpc-standard/main.tf

Internal module that wraps the community VPC module

with company-standard settings

module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 5.1"

name = var.name
cidr = var.cidr

Company standard: always enable DNS, always tag

enable_dns_hostnames = true
enable_dns_support = true
enable_nat_gateway = true
single_nat_gateway = var.environment != "production"

tags = merge(
var.tags,
{
ManagedBy = "terraform"
Environment = var.environment
Team = var.team
}
)
}


Consumers call the wrapper instead of the community module directly. When the company standard changes (add a required tag, change a default), update the wrapper and all users get the change.

## When to extract a module — and when not to

A common mistake: extracting modules too early. If a set of resources is only used once, extracting it to a module adds abstraction overhead without the reuse benefit. The rule of thumb: extract a module when you actually need to reuse it in a second place, not in anticipation of possible reuse.

Good candidates for extraction:

- The same VPC pattern deployed in multiple environments (dev, staging, prod)
- A standard RDS setup used by multiple application teams
- A "microservice" pattern (ECS task + ALB target group + CloudWatch alarms) used for every service
- Anything with complex interdependencies that benefit from hiding implementation details

Poor candidates:

- A one-off resource with unique configuration (extract is pure overhead)
- Resources that need to be modified differently in each environment (module abstraction fights the customization)
- Very small sets of resources (2–3 resources don't need a module)

## Testing modules with Terratest

Terratest is a Go library for writing automated tests for infrastructure code. A Terratest test deploys real infrastructure, runs assertions against it, and tears it down:

// modules/vpc/tests/vpc_test.go
package test

import (
"testing"
"github.com/gruntwork-io/terratest/modules/terraform"
"github.com/gruntwork-io/terratest/modules/aws"
"github.com/stretchr/testify/assert"
)

func TestVpcModule(t *testing.T) {
t.Parallel()

opts := &terraform.Options{
TerraformDir: "../examples/complete",
Vars: map[string]interface{}{
"name": "test-vpc",
"region": "us-east-1",
},
}

defer terraform.Destroy(t, opts)
terraform.InitAndApply(t, opts)

vpcId := terraform.Output(t, opts, "vpc_id")
assert.NotEmpty(t, vpcId)

// Verify the VPC actually exists in AWS
vpc := aws.GetVpcById(t, vpcId, "us-east-1")
assert.Equal(t, "10.0.0.0/16", aws.GetVpcCidrBlock(t, vpcId, "us-east-1"))
assert.True(t, *vpc.EnableDnsHostnames)
}


Terratest tests are slow (they deploy real infrastructure) and cost money. Reserve them for modules that will be shared and relied upon by multiple teams. For most internal modules, a combination of `terraform validate`, `terraform plan` output review in CI, and manual testing is sufficient.

For lightweight unit testing without deploying infrastructure, `terraform test` (built into Terraform 1.6+) lets you write test files that exercise module logic with mock providers:

modules/vpc/tests/input_validation.tftest.hcl

run "invalid_cidr_block" {
command = plan

variables {
name = "test"
cidr_block = "not-a-cidr"
}

expect_failures = [var.cidr_block]
}


## Module documentation

`terraform-docs` is a tool that reads your module's `variables.tf` and `outputs.tf` and generates a markdown table of inputs and outputs. Running it in CI keeps documentation in sync with the code:

Generate docs and update README.md

terraform-docs markdown table --output-file README.md ./modules/vpc


The generated section looks like:

Inputs

Name	Description	Type	Default	Required
name	Name prefix for all resources	`string`	n/a	yes
cidr_block	CIDR block for the VPC	`string`	`"10.0.0.0/16"`	no

Outputs

Name	Description
vpc_id	The ID of the VPC
private_subnet_ids	List of private subnet IDs




When a module is consumed by other teams, this documentation is the contract. Good documentation reduces questions and prevents misuse.

To visualize the resources a Terraform module creates and how they relate — useful during module design or review — paste the module's HCL into [InfraSketch](/) to see the architecture diagram. Module calls in a root module appear as labeled groups, with each resource inside the module shown individually.

## Related articles

- [Terraform State Explained: What It Is, How It Works, and Why It Breaks](/blog/terraform-state-explained.html)
- [Terraform Visualization: 5 Ways to See What Your Code Actually Builds](/blog/terraform-visualization-best-practices.html)
- [Terraform vs CDK vs Pulumi: Choosing Your IaC Tool](/blog/iac-tool-comparison.html)

Kubernetes Resource Management: Requests, Limits, and QoS Classes

Raghvendra Pandey — Sat, 20 Jun 2026 11:10:46 +0000

Liquid syntax error: Unknown tag 'endraw'

GitOps Explained: How It Works, ArgoCD vs Flux, and When to Use It

Raghvendra Pandey — Sat, 20 Jun 2026 11:10:44 +0000

GitOps is a way of operating infrastructure and applications where Git is the single source of truth for what should be running, and an automated agent continuously reconciles the actual state of the system to match it. The term was coined by Weaveworks in 2017, but the underlying idea — declarative desired state, version control, automated reconciliation — predates it. What GitOps added was a specific set of practices and a set of tools that made the pattern practical for Kubernetes.

This guide covers what GitOps actually means (not just the marketing definition), how the reconciliation loop works, how ArgoCD and Flux compare, and where GitOps genuinely helps versus where it adds unnecessary complexity.

The four GitOps principles

The OpenGitOps project (a CNCF working group) defines GitOps around four principles:

1. Declarative. The desired state of the entire system is expressed declaratively. This means Kubernetes YAML manifests, Helm chart values, Kustomize overlays — not imperative scripts. "Apply these manifests" rather than "run these commands." The advantage: the desired state can be stored, diffed, and audited.

2. Versioned and immutable. The desired state is stored in a versioning system (Git) that enforces immutability of history. Every change has a timestamp, an author, and a hash. Rollback means reverting a commit. Audit means reading the commit log. You always know what changed, when, and who approved it.

3. Pulled automatically. Approved changes are automatically applied to the system. The GitOps agent inside the cluster watches the Git repository and pulls changes — the cluster initiates the connection, not external systems. This is a security advantage: you don't need to give CI/CD systems credentials to access the cluster. The cluster's outbound network needs to reach Git; nothing external needs inbound access to the API server.

4. Continuously reconciled. Software agents continuously compare actual state with desired state and automatically correct any divergence. If someone manually changes a Kubernetes resource (kubectl edit), the GitOps agent detects the drift and reverts it. This is the enforcement mechanism.

How the reconciliation loop works

A GitOps controller runs inside the cluster. It watches one or more Git repositories (or OCI registries) and continuously runs a control loop:

Fetch the latest desired state from the Git repository
Read the actual state of the cluster from the Kubernetes API
Compute the diff
Apply changes to make actual state match desired state
Wait for the reconciliation interval (typically 1–5 minutes) and repeat

This pull-based model contrasts with the push-based model used by traditional CI/CD pipelines. In push-based deployments, a pipeline runs after a merge and pushes changes to the cluster via kubectl apply or Helm. In pull-based GitOps, the cluster continuously checks whether it matches Git and corrects itself.

# Push-based CI/CD (traditional)
# Pipeline runs:
kubectl apply -f manifests/
# Cluster state = whatever kubectl apply did

# GitOps (pull-based)
# GitOps controller runs in cluster, continuously:
# 1. Reads desired state from git
# 2. Compares to actual cluster state
# 3. Applies the diff
# Cluster state = what's in Git (enforced, not just applied once)

The Git repository structure

GitOps works best when the application code repository and the deployment configuration repository are separate. This "app repo + config repo" pattern is the most common structure:

App repo: source code, Dockerfile, application tests. CI pipeline builds images, tags them with the git commit SHA, pushes to a registry, then opens a PR against the config repo to update the image tag.
Config repo: Kubernetes manifests, Helm values, Kustomize overlays. This is what the GitOps controller watches. Changes here trigger deployments; changes to the app repo trigger a CI pipeline that eventually updates the config repo.

# Config repo structure (example)
clusters/
production/
namespaces/
api/
deployment.yaml
service.yaml
hpa.yaml
database/
statefulset.yaml
service.yaml
staging/
namespaces/
api/
deployment.yaml  # different image tag, lower resource requests
database/
statefulset.yaml

Some teams keep everything in one repo using directory structure or branch strategy to separate environments. Others use separate repos per environment or per cluster. There's no single right structure — the key is that everything that defines cluster state lives in version control.

ArgoCD

ArgoCD is the most widely adopted GitOps tool. It runs as a set of controllers in the cluster and exposes a web UI and CLI. The core concept is an Application resource that maps a Git repository path to a Kubernetes namespace:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: api-production
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/mycompany/config-repo
targetRevision: main
path: clusters/production/namespaces/api
destination:
server: https://kubernetes.default.svc
namespace: api
syncPolicy:
automated:
prune: true      # delete resources removed from git
selfHeal: true   # revert manual changes
syncOptions:
- CreateNamespace=true

ArgoCD's web UI shows the sync status of every Application — green (in sync), yellow (out of sync), red (degraded). Out-of-sync resources show exactly what differs between the cluster and Git. This visual diff is one of ArgoCD's most valued features for operators who want to understand deployment state at a glance.

ArgoCD uses an app of apps pattern for managing many Applications: a root Application points to a directory of Application manifests, which ArgoCD then deploys. This bootstraps an entire cluster from a single ArgoCD Application.

ArgoCD's approach to Helm and Kustomize

ArgoCD natively supports Helm charts and Kustomize overlays. You can point an Application at a Helm chart in a Git repo or a remote Helm repository, and specify values files or inline value overrides. Kustomize overlays let you maintain a base configuration and layer environment-specific patches on top.

Flux

Flux takes a more composable, Kubernetes-native approach. Instead of a single Application CRD, Flux uses several specialized CRDs that each handle a specific concern:

GitRepository: defines a Git source (URL, branch, interval, credentials)
HelmRepository: defines a Helm chart source
OCIRepository: defines an OCI registry source
Kustomization: applies Kustomize overlays from a source
HelmRelease: deploys a Helm chart from a source with specified values
ImageRepository: watches a container registry for new image tags
ImageUpdateAutomation: automatically updates image tags in Git when new images are pushed

# Flux GitRepository
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: config-repo
namespace: flux-system
spec:
interval: 1m
url: https://github.com/mycompany/config-repo
ref:
branch: main

# Flux Kustomization
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: api-production
namespace: flux-system
spec:
interval: 5m
sourceRef:
kind: GitRepository
name: config-repo
path: ./clusters/production/namespaces/api
prune: true
targetNamespace: api

Flux doesn't have a built-in web UI in the same way ArgoCD does. It's managed primarily via CLI (flux) and manifests. This makes it feel more "GitOps-native" to some practitioners — Flux's own configuration lives entirely in Git, managed by Flux itself.

ArgoCD vs Flux: which to choose

Both tools are mature, widely used, and maintained by strong communities. The choice usually comes down to UI preferences and composition model:

Choose ArgoCD if: your team values a visual dashboard for deployment status, you want a unified view across many clusters, or you're adopting GitOps for the first time and want a guided UI to understand what's happening.

Choose Flux if: you prefer a Kubernetes-native composable model, you want to manage Flux's own configuration via GitOps, or you need the image automation features (automatic image tag updates) that Flux provides out of the box.

Both support multi-cluster deployments, RBAC, SSO integration, Helm, and Kustomize. Both are CNCF graduated projects. Switching between them is possible but non-trivial — once a team is invested in ArgoCD's Application CRDs or Flux's source controller patterns, migrating means rewriting deployment configuration.

When GitOps makes sense

GitOps genuinely helps in these situations:

Multi-environment management. When you have dev, staging, and production environments and want changes to flow through them consistently. GitOps makes the progression explicit — a PR to update staging, review, then a PR to update production.

Regulatory compliance and audit requirements. Every change is a Git commit with an author and timestamp. Audit trails are the commit log. Change approval is the PR review process. This maps naturally to SOC 2, ISO 27001, and similar requirements.

Preventing configuration drift. Without GitOps, someone inevitably makes a kubectl change that isn't reflected anywhere, and six months later nobody knows why that resource is configured differently from what the manifests say. Continuous reconciliation closes this gap.

Large teams. When multiple teams deploy to shared clusters, GitOps provides a structured review process via PRs rather than direct cluster access. Cluster operators review and merge config changes; they don't need to trust every team to run kubectl correctly.

When GitOps adds overhead without value

GitOps isn't always the right choice:

Small teams or solo projects. A two-person team doesn't need the overhead of maintaining separate config repos and GitOps controllers. A simple CI pipeline that runs helm upgrade after merge is easier to understand and maintain.

Stateful workloads with complex upgrade procedures. Database migrations, multi-step upgrades, or operations that require ordering guarantees don't fit well into declarative reconciliation. GitOps handles stateless workloads well; it's awkward for operations that need imperative steps.

Early-stage applications with rapidly changing infrastructure. If your deployment configuration changes multiple times per day, the PR review process becomes a bottleneck. GitOps adds value when the review gate is valuable; it adds friction when fast iteration matters more.

Bootstrapping a cluster with GitOps

A common pattern: use Terraform to provision the cluster (EKS, GKE, AKS), then bootstrap GitOps with a single apply command that hands off further configuration to the GitOps controller.

# Bootstrap Flux onto a new cluster
flux bootstrap github \
--owner=mycompany \
--repository=config-repo \
--branch=main \
--path=clusters/production \
--personal=false

# After this, everything in clusters/production/ is managed by Flux
# Including Flux's own configuration

After bootstrap, even adding new namespaces, installing cluster addons (cert-manager, external-dns, ingress-nginx), or configuring RBAC happens through Git commits rather than direct cluster commands.

Docker Networking Explained: Bridge, Host, Overlay, and DNS

Raghvendra Pandey — Sat, 20 Jun 2026 11:10:13 +0000

Docker networking confuses people because the defaults are unintuitive and the mental model isn't obvious from the CLI. A container can reach the internet by default, but two containers on the same host can't reach each other unless they're on the same network. Port publishing doesn't work the way most people expect on Linux hosts. Container-to-container DNS just works in Docker Compose but not when you run containers manually. Understanding why requires knowing a bit about how Docker uses Linux networking primitives.

This guide covers each Docker network driver, how DNS works inside containers, port publishing mechanics, and how Docker Compose handles multi-container networking.

Network drivers overview

Docker's networking system is pluggable — different drivers implement different networking behaviors. The built-in drivers are:

bridge — the default for containers on a single host
host — container shares the host's network namespace
overlay — multi-host networking for Docker Swarm or Compose with multiple hosts
macvlan — assigns a MAC address to each container so it appears as a physical device on the network
none — disables networking entirely
ipvlan — similar to macvlan but operates at Layer 3

Bridge networking: the default

When you run a container without specifying a network, Docker attaches it to the docker0 bridge network. This is a virtual Ethernet bridge — a software switch that forwards packets between container network interfaces and the host's physical interface.

Each container gets a virtual Ethernet pair (veth): one end is placed inside the container's network namespace as eth0, the other end is attached to docker0 on the host. The container gets an IP from Docker's private subnet (typically 172.17.0.0/16).

# View the default bridge network
docker network inspect bridge

# Containers on the default bridge can reach the internet
# via NAT rules that docker manages in iptables/nftables

# But they CANNOT reach each other by name — only by IP
docker run -d --name app1 nginx
docker run -it --rm busybox ping app1  # This FAILS
docker run -it --rm busybox ping 172.17.0.2  # This works

The default bridge network has two important limitations: containers can't resolve each other by name, and all containers on it can communicate with each other by IP (no isolation). Both problems are solved by user-defined bridge networks.

User-defined bridge networks

Creating a custom bridge network gives you DNS-based service discovery and network isolation:

# Create a custom bridge network
docker network create my-app-network

# Containers on the same user-defined network can resolve each other by name
docker run -d --name database --network my-app-network postgres
docker run -d --name api --network my-app-network my-api-image

# From inside the api container:
# ping database   ← works
# curl database:5432  ← works

Docker's embedded DNS server (at 127.0.0.11 inside containers) handles name resolution for user-defined networks. It maps container names and network aliases to their current IPs. When a container is restarted and gets a new IP, the DNS entry updates automatically.

Containers can also be connected to multiple networks simultaneously:

docker network connect frontend-network api
docker network connect backend-network api
# api can now communicate with containers on both networks

Host networking

With --network host, the container shares the host's network namespace. It doesn't get its own IP — it uses the host's IP directly. Port bindings are not needed because the container's ports are the host's ports.

docker run --network host nginx
# nginx is now listening on the HOST's port 80, not a container port

This eliminates the overhead of network address translation and can provide meaningfully better throughput for high-performance networking workloads. However, it also means:

No port isolation — if two containers try to bind the same port, one fails
No network namespace isolation — the container can see all host interfaces
Port publishing (-p) flags are ignored; they have no effect with host networking

Host networking is useful for monitoring agents (that need to see all network traffic on the host), high-performance proxies, and cases where the NAT overhead of bridge networking is measurably impactful. It doesn't work on macOS or Windows Docker Desktop because those run Docker inside a Linux VM — the "host" network is the VM's network, not your Mac's.

Port publishing

Port publishing (-p host_port:container_port) maps a port on the host to a port in the container's network namespace. Docker does this by adding iptables rules that DNAT (Destination NAT) incoming traffic on the host port to the container's IP and port.

docker run -p 8080:80 nginx
# Traffic arriving at host:8080 is forwarded to the container's port 80

docker run -p 127.0.0.1:8080:80 nginx
# Bind only to localhost — not accessible from other hosts

docker run -p 80 nginx
# Docker picks a random ephemeral host port; docker port  80 shows it

A subtle point: -p 8080:80 binds on all interfaces (0.0.0.0:8080) by default. This means if your firewall is open, the port is accessible from outside the host. Always specify the bind address for sensitive ports: -p 127.0.0.1:8080:80.

Docker's iptables rules can interact unexpectedly with firewall software like ufw. Docker modifies iptables directly, and ufw's FORWARD chain rules don't apply to Docker's DOCKER-USER chain. A container with a published port may be accessible from outside the host even if ufw blocks that port. The fix is to add rules to the DOCKER-USER chain, not to ufw's INPUT or FORWARD chains.

Overlay networking for multi-host setups

Overlay networks span multiple Docker hosts. They're the networking layer for Docker Swarm services and can also be used standalone with Docker Compose when running across multiple hosts.

An overlay network uses VXLAN encapsulation: packets between containers on different hosts are wrapped in UDP packets and sent over the physical network. From the container's perspective, other containers are on the same flat network regardless of which host they're running on.

# Overlay networks require Swarm mode to be initialized
docker swarm init

docker network create \
--driver overlay \
--attachable \
my-overlay-network

# Services on this network can reach each other by service name
# regardless of which Swarm node they're running on

Each overlay network gets a private subnet (default /24). Docker maintains a distributed key-value store (using Raft consensus within Swarm) to coordinate IP assignments and routing across nodes.

The --attachable flag allows standalone containers (not Swarm services) to be connected to the overlay network. Without it, only Swarm services can join.

DNS inside containers

Every container has /etc/resolv.conf with a nameserver entry. For user-defined networks, this points to Docker's embedded DNS at 127.0.0.11. For the default bridge network or host networking, it points to the host's DNS resolver.

# Inside a container on a user-defined network:
cat /etc/resolv.conf
# nameserver 127.0.0.11
# options ndots:0

# Verify DNS is working
nslookup database    # resolves to the database container's IP
nslookup google.com  # external DNS also resolves via 127.0.0.11 → forwarded to host resolver

Docker's embedded DNS serves several record types:

A records for container names and network aliases
A records for service names in Swarm (round-robin across all task IPs)
SRV records for Swarm services (used by DNS-based load balancers)

Network aliases allow multiple containers to share the same DNS name, creating a simple round-robin load balancer:

docker run -d --network my-net --network-alias web nginx
docker run -d --network my-net --network-alias web nginx
# DNS for "web" now round-robins between both containers

Docker Compose networking

Docker Compose automatically creates a user-defined bridge network for each project and connects all services to it. Services can reach each other by service name.

services:
api:
image: my-api
# No port publishing needed for internal communication
depends_on:
- database

database:
image: postgres
environment:
POSTGRES_PASSWORD: secret

nginx:
image: nginx
ports:
- "80:80"  # Only nginx needs to publish to the host
depends_on:
- api

In this setup: api reaches database at hostname database:5432. nginx reaches api at api:8080. Only nginx is accessible from outside Docker.

Compose also supports multiple networks for isolation:

services:
frontend:
networks: [frontend]

api:
networks: [frontend, backend]  # bridge between networks

database:
networks: [backend]

networks:
frontend:
backend:

frontend can reach api. api can reach database. frontend cannot reach database directly — it has no network in common.

External networks

Compose can connect services to a network created outside the Compose project:

networks:
shared-monitoring:
external: true
name: monitoring-network

This is useful for a monitoring stack (Prometheus, Grafana) in one Compose project that needs to scrape targets from other projects.

Macvlan and ipvlan

Macvlan assigns each container a unique MAC address and connects it directly to a physical network interface. The container appears as a separate physical device on the network and can get an IP from a DHCP server or a static assignment within the physical subnet.

docker network create \
--driver macvlan \
--subnet 192.168.1.0/24 \
--gateway 192.168.1.1 \
--opt parent=eth0 \
my-macvlan-network

docker run --network my-macvlan-network --ip 192.168.1.10 nginx

Macvlan is useful for containers that need to be reachable at a fixed IP on the physical network without port publishing — legacy applications that require specific IP addresses, network monitoring tools, or applications that need to be accessible from VLAN-isolated segments.

The limitation: macvlan interfaces can't communicate with their parent interface by default. A container on a macvlan network can talk to other hosts on the physical network, but not to the Docker host itself. This is a Linux kernel limitation with macvlan.

Debugging container networking

Container can't reach external hosts. Check that IP forwarding is enabled on the host (sysctl net.ipv4.ip_forward should be 1). Check iptables for a MASQUERADE rule in the POSTROUTING chain: iptables -t nat -L POSTROUTING. If Docker was recently reinstalled, iptables rules may have been lost.

Two containers can't reach each other. Check they're on the same user-defined network. The default bridge network doesn't do DNS. Run docker network inspect to see which containers are connected.

Published port isn't accessible from outside the host. Check the host firewall. Note that Docker may have bypassed ufw by writing directly to iptables — check iptables -L DOCKER. Also verify the container is actually listening on the port: docker exec ss -tlnp.

DNS resolution fails inside container. Verify you're on a user-defined network (not the default bridge). Check /etc/resolv.conf inside the container. Try nslookup 127.0.0.11 to test if Docker's DNS is reachable. Also check that the --dns flag isn't overriding the default nameserver.

Visualizing how Docker Compose services connect to each other — which services share networks and which are isolated — helps catch misconfigured networking before deploying. InfraSketch parses Docker Compose files and maps services and their network relationships as a diagram.

AWS IAM Explained: Users, Roles, Policies, and Least Privilege

Raghvendra Pandey — Sat, 20 Jun 2026 11:10:11 +0000

AWS IAM (Identity and Access Management) is the authorization layer for everything in AWS. Every API call — whether from the console, CLI, an EC2 instance, or a Lambda function — is evaluated against IAM policy. Get it wrong and you either lock out legitimate services or leave your infrastructure open to lateral movement. Despite its importance, IAM is where most teams accumulate the most technical debt, usually in the form of overly broad policies that were "temporary" and became permanent.

This guide covers how IAM actually works — the principal model, how policies are evaluated, role assumption, and the patterns used most in production infrastructure.

Principals: who is making the request

Every IAM interaction starts with a principal — the entity making the API call. AWS has four types of principals:

IAM users are long-lived identities with permanent credentials. They authenticate with a username/password for the console or an access key pair for the CLI/API. Creating IAM users for human access is legacy practice now — most organizations use SSO (AWS IAM Identity Center, formerly SSO) to federate human identities from an identity provider like Okta or Azure AD, then assume roles within AWS accounts. IAM users still make sense for service accounts that need permanent, programmatic API access, though roles are preferred even there when possible.

IAM roles are identities that can be assumed. They don't have permanent credentials — instead, when a principal assumes a role, STS (Security Token Service) issues temporary credentials that expire (typically 1–12 hours). Roles are the preferred mechanism for granting AWS services access to other AWS services, for cross-account access, and for federated human access.

IAM groups are collections of users. A group has policies attached to it; all users in the group inherit those permissions. Groups don't support role assumption — they're only for organizing users and applying policies in bulk.

AWS service principals represent AWS services (EC2, Lambda, ECS, etc.) acting on your behalf. When you create an EC2 instance profile or a Lambda execution role, you're granting an AWS service the ability to assume that role. The service principal looks like ec2.amazonaws.com or lambda.amazonaws.com.

Policies: what the principal is allowed to do

Policies are JSON documents that specify what actions are allowed or denied on which resources under what conditions. IAM has six types of policies:

Identity-based policies are attached directly to users, groups, or roles. They specify what that principal is allowed (or denied) to do.

Resource-based policies are attached to resources (S3 buckets, SQS queues, KMS keys, Lambda functions, etc.) rather than to principals. They specify who is allowed to do what with that resource. S3 bucket policies are the most common example. Resource-based policies support cross-account access without role assumption — you can grant a principal in another account direct access to an S3 bucket.

Permission boundaries cap the maximum permissions an IAM user or role can have. Even if an identity-based policy grants s3:*, if the permission boundary only allows s3:GetObject, only GetObject is allowed. This is useful for delegating IAM management — you can let teams create their own roles as long as those roles don't exceed the boundary.

Service Control Policies (SCPs) apply to entire AWS Organizations accounts. An SCP can prevent all accounts in an OU from using specific regions or services regardless of what IAM policies in those accounts say. SCPs are a ceiling, not a floor — they don't grant permissions, they only restrict them.

Session policies are optional policies passed at role assumption time via sts:AssumeRole or sts:GetFederationToken. They further restrict the permissions for that specific session without modifying the role itself.

Endpoint policies restrict what specific VPC endpoints can access.

How IAM evaluates a request

When an API call arrives, IAM evaluates it through a multi-step decision tree. Understanding this order is essential for diagnosing access issues:

Explicit deny check: If any applicable policy (identity, resource, SCP, permission boundary) has an explicit Deny for the action, the request is denied immediately. Explicit denies always win.
SCP check: If the account is in an Organization and an SCP doesn't allow the action, the request is denied regardless of IAM policies in the account.
Resource-based policy check: If a resource-based policy explicitly allows the request (and there's no explicit deny), it's allowed. This is how cross-account access without role assumption works — the resource policy grants access to a principal in another account directly.
Identity-based policy check: If an identity-based policy attached to the principal allows the action, it's allowed.
Permission boundary check: If a permission boundary is set, the intersection of what the identity-based policy allows and what the boundary allows is the effective permission set.
Default deny: If none of the above granted access, the request is denied. AWS defaults to deny-everything.

The practical takeaway: cross-account access needs to be allowed in both the identity-based policy on the source side AND the resource-based policy on the target side. A common mistake is adding a resource-based policy without giving the source principal permission to assume the role or access the resource.

Policy anatomy

An IAM policy document is JSON with a Statement array. Each statement has an Effect (Allow or Deny), Action (which API operations), Resource (which ARNs), and optionally a Condition block:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-company-artifacts/*"
},
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::my-company-artifacts",
"Condition": {
"StringLike": {
"s3:prefix": ["releases/*"]
}
}
}
]
}

The Condition block can restrict access by source IP, time of day, MFA status, required tags, and many other attributes. Conditions don't grant permissions — they further restrict when an Allow applies.

The Resource field is where least privilege is enforced. Instead of "Resource": "*", use the actual ARN of the specific resource. For S3, arn:aws:s3:::bucket-name grants access to the bucket itself (for ListBucket), while arn:aws:s3:::bucket-name/* grants access to objects inside it (for GetObject, PutObject).

Trust policies and role assumption

Every IAM role has two components: an identity-based policy (what the role can do) and a trust policy (who can assume the role). The trust policy is a resource-based policy on the role itself.

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}

This trust policy allows EC2 instances to assume the role. For human cross-account access, the principal would be an IAM role or user ARN from another account. For OIDC-based access (GitHub Actions, EKS pods via IRSA), the principal is an OIDC provider with a condition that matches the federated identity claim.

Common patterns with Terraform

EC2 instance profile

An instance profile is the mechanism that attaches an IAM role to an EC2 instance. When EC2 starts an instance with an instance profile, the instance metadata endpoint (169.254.169.254 or IMDSv2 equivalent) provides temporary credentials for the attached role.

resource "aws_iam_role" "app_instance" {
name = "app-instance-role"

assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect    = "Allow"
Principal = { Service = "ec2.amazonaws.com" }
Action    = "sts:AssumeRole"
}]
})
}

resource "aws_iam_role_policy" "app_instance" {
role = aws_iam_role.app_instance.id

policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect   = "Allow"
Action   = ["s3:GetObject"]
Resource = "arn:aws:s3:::${var.config_bucket}/*"
},
{
Effect   = "Allow"
Action   = ["secretsmanager:GetSecretValue"]
Resource = "arn:aws:secretsmanager:${var.region}:${var.account_id}:secret:app/*"
}
]
})
}

resource "aws_iam_instance_profile" "app" {
name = "app-instance-profile"
role = aws_iam_role.app_instance.name
}

Lambda execution role

Lambda functions need a role with at minimum the AWSLambdaBasicExecutionRole managed policy (for CloudWatch Logs). Add only the additional permissions the function actually needs:

resource "aws_iam_role" "lambda_execution" {
name = "my-function-execution-role"

assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect    = "Allow"
Principal = { Service = "lambda.amazonaws.com" }
Action    = "sts:AssumeRole"
}]
})
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
role       = aws_iam_role.lambda_execution.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "lambda_specific" {
role = aws_iam_role.lambda_execution.id

policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect   = "Allow"
Action   = ["dynamodb:GetItem", "dynamodb:PutItem"]
Resource = aws_dynamodb_table.data.arn
}]
})
}

IRSA — IAM Roles for Service Accounts (EKS)

IRSA allows Kubernetes pods to assume IAM roles without sharing credentials at the node level. The trust policy uses the cluster's OIDC provider:

data "aws_iam_openid_connect_provider" "eks" {
url = aws_eks_cluster.main.identity[0].oidc[0].issuer
}

resource "aws_iam_role" "pod_role" {
name = "my-app-pod-role"

assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = {
Federated = data.aws_iam_openid_connect_provider.eks.arn
}
Action = "sts:AssumeRoleWithWebIdentity"
Condition = {
StringEquals = {
"${trimprefix(data.aws_iam_openid_connect_provider.eks.url, "https://")}:sub" =
"system:serviceaccount:${var.namespace}:${var.service_account_name}"
}
}
}]
})
}

The Kubernetes ServiceAccount is annotated with the role ARN. When a pod runs with that ServiceAccount, the EKS Pod Identity webhook injects environment variables that direct the AWS SDK to use WebIdentity tokens for authentication.

Cross-account access

Cross-account access requires granting permission on both sides. In the target account, create a role with a trust policy that allows the source account:

# In the target account (account B)
resource "aws_iam_role" "cross_account" {
name = "cross-account-access-role"

assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect    = "Allow"
Principal = { AWS = "arn:aws:iam::${var.source_account_id}:root" }
Action    = "sts:AssumeRole"
Condition = {
StringEquals = {
"sts:ExternalId" = var.external_id  # prevents confused deputy attacks
}
}
}]
})
}

# In the source account (account A), grant the permission to assume
resource "aws_iam_role_policy" "assume_cross_account" {
role = aws_iam_role.deployer.id

policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect   = "Allow"
Action   = "sts:AssumeRole"
Resource = "arn:aws:iam::${var.target_account_id}:role/cross-account-access-role"
}]
})
}

Implementing least privilege

The principle of least privilege means granting only the permissions needed to perform a specific function and nothing more. In practice, it means:

Scope actions precisely: Use s3:GetObject not s3:*. Use ec2:DescribeInstances not ec2:*.
Scope resources precisely: Use the specific resource ARN rather than *. For DynamoDB: arn:aws:dynamodb:region:account:table/specific-table, not *.
Use conditions: Add conditions to restrict when permissions apply — source IP, MFA requirement, resource tags, etc.
Avoid wildcard patterns in sensitive services: iam:*, sts:*, and kms:* are especially dangerous. Wildcard access to IAM means the principal can grant itself any permission.

AWS's IAM Access Analyzer and the IAM Policy Simulator help audit existing policies. Access Analyzer also has a feature that generates least-privilege policies from CloudTrail logs — it observes what an identity actually calls over a period and generates a policy based on observed usage. This is the most practical approach for brownfield permissions: run the role in production, observe for 30–90 days, then generate a tight policy from actual usage.

Common mistakes

AdministratorAccess on service roles. The policy arn:aws:iam::aws:policy/AdministratorAccess grants full access to everything in the account. Attaching this to a Lambda function or EC2 instance because "it needs to do a lot of things" is a security catastrophe waiting to happen. A compromised instance or function with AdministratorAccess can exfiltrate data, create new users, disable logging, or delete everything.

Storing long-lived access keys in code or environment variables. Access keys associated with IAM users don't expire. If they leak (in a git commit, an S3 bucket, logs), they're valid indefinitely until rotated. Use IAM roles and instance metadata instead. For CI/CD, use OIDC federation (GitHub Actions OIDC, GitLab OIDC) rather than static keys.

Shared IAM users across services or teams. One IAM user shared by multiple services means you can't audit which service made which API call, can't rotate credentials for one service without affecting others, and can't revoke access for one service without affecting all. Each service gets its own role.

Permission boundaries not used for delegated IAM. If you let application teams create their own IAM roles, they can escalate their own privileges. A permission boundary constrains what roles they can create — the team can create roles, but those roles can't exceed what the permission boundary allows.

Debugging access issues

When an IAM error appears (AccessDeniedException or is not authorized to perform: on resource:), the debug workflow is:

Identify the principal in the error — it's in the message as "User: arn:aws:iam::..."
Use the IAM Policy Simulator in the console to test whether that principal has the required permission
Check for explicit denies — SCPs, permission boundaries, resource policies
Check that resource-based policies allow the cross-account action if applicable
Enable CloudTrail data events if not already enabled — access denials are logged there with full context

IAM is complex enough that AWS's documentation for specific services often lists exactly which IAM permissions are needed. The IAM actions reference at docs.aws.amazon.com/service-authorization/latest/reference/ is the authoritative source for every service, action, resource type, and condition key.

Terraform State Explained: What It Is, How It Works, and Why It Breaks

Raghvendra Pandey — Sun, 07 Jun 2026 00:25:56 +0000

Terraform state is one of those concepts that every Terraform user encounters early, trips over repeatedly, and eventually develops strong opinions about. If you've ever seen Error: state lock, found your terraform.tfstate committed to git by accident, or watched a colleague manually delete a resource from AWS only to have Terraform recreate it, you've already met the state file.

This article explains what Terraform state actually is, how Terraform uses it, what goes wrong, and how teams structure their state to stay sane at scale.

What is Terraform state?

Terraform needs to map the resources defined in your .tf files to the real resources that exist in your cloud provider. That mapping is stored in the state file — by default a JSON file called terraform.tfstate in your working directory.

When you run terraform apply, Terraform:

Reads your .tf configuration files
Reads the current state file to understand what already exists
Calls the cloud provider APIs to get the current real-world state
Computes a diff between desired state (config) and actual state (provider)
Applies the changes and updates the state file

Without state, Terraform would have no way to know that the aws_instance.web in your config corresponds to instance i-0abc123def456 in AWS. It would try to create a new one every time you run apply.

State is also how Terraform tracks metadata that isn't visible in the config — resource IDs, ARNs, IP addresses assigned by the cloud provider, dependency ordering, and provider version constraints. Open any terraform.tfstate file and you'll see a JSON structure with a resources array where each entry contains both your config attributes and the provider-returned attributes.

{
"version": 4,
"terraform_version": "1.7.0",
"resources": [
{
"mode": "managed",
"type": "aws_vpc",
"name": "main",
"provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
"instances": [
{
"schema_version": 1,
"attributes": {
"id": "vpc-0a1b2c3d4e5f6",
"cidr_block": "10.0.0.0/16",
"arn": "arn:aws:ec2:us-east-1:123456789012:vpc/vpc-0a1b2c3d4e5f6",
"enable_dns_hostnames": true,
...
}
}
]
}
]
}

Local vs remote state

By default, Terraform writes state to a local file. This works fine when you're learning or building something solo. It breaks down immediately in a team setting because:

Two engineers can't safely run Terraform at the same time
The state file needs to be shared somehow — and sharing via git is dangerous (state contains sensitive values like passwords and access keys)
If the file is lost or corrupted, Terraform loses track of all managed resources

Remote state solves this. Instead of writing to a local file, Terraform stores state in a remote backend — S3, Azure Blob Storage, GCS, Terraform Cloud, or others. The backend configuration goes in your Terraform code:

terraform {
backend "s3" {
bucket         = "my-company-terraform-state"
key            = "production/vpc/terraform.tfstate"
region         = "us-east-1"
dynamodb_table = "terraform-state-lock"
encrypt        = true
}
}

The dynamodb_table entry is not optional if you have multiple people or pipelines running Terraform. It's the locking mechanism that prevents two simultaneous applies from corrupting state.

State locking

When Terraform starts an operation that modifies state (plan, apply, destroy), it acquires a lock on the state file. Any other Terraform operation that tries to acquire the same lock will wait or fail, depending on whether you pass -lock-timeout.

Locking is implemented differently per backend:

S3 + DynamoDB: Terraform writes a lock record to a DynamoDB table. The record contains the operation, who started it, and a unique lock ID.
Terraform Cloud / HCP Terraform: Locking is built into the backend — no separate database needed.
Azure Blob Storage: Uses blob lease mechanism, which Azure provides natively.
Local: Creates a .terraform.tfstate.lock.info file.

The most common locking failure is a stale lock — a lock that wasn't released because Terraform was killed mid-run (Ctrl-C, CI timeout, power loss). When this happens, subsequent runs fail with something like:

Error: Error locking state: Error acquiring the state lock:
ConditionalCheckFailedException: The conditional request failed

Lock Info:
ID:        abc123-def456
Path:      production/vpc/terraform.tfstate
Operation: OperationTypeApply
Who:       ci-runner@build-server
Version:   1.7.0
Created:   2026-06-01 09:23:11 UTC

To clear a stale lock, you need to confirm the original operation actually stopped (check your CI logs), then run terraform force-unlock. Never force-unlock while another operation is running — you'll get state corruption.

State drift

Drift happens when the real-world state of a resource no longer matches what's in the Terraform state file. This is usually caused by manual changes — someone logs into the AWS console and modifies a security group rule, or uses the AWS CLI to change an instance type, or another tool (CloudFormation, a script, another Terraform root) touches the same resource.

From Terraform's perspective, nothing changed — its state file still says the old configuration. The next time you run terraform plan, it might show no changes (if the drift is in an attribute Terraform isn't managing) or show unexpected changes (if Terraform wants to revert the manual change).

To detect drift explicitly, use terraform plan -refresh-only. This refreshes Terraform's view of actual resource attributes without proposing any config changes. If there's drift, the plan will show what changed in the real world. You can then either:

Accept the drift: terraform apply -refresh-only updates the state file to match reality
Revert the drift: Run a normal terraform apply to push your config back over the manual change

Teams that want to detect drift continuously can run terraform plan -refresh-only on a schedule (hourly, nightly) and alert if the plan is non-empty. This is sometimes called "drift detection" and it's available natively in Terraform Cloud and HCP Terraform.

Importing existing resources

When you have infrastructure that predates your Terraform codebase — resources created manually, from another tool, or by another team — you need to import them into Terraform state before Terraform can manage them.

The old way: terraform import .

# Import an existing VPC
terraform import aws_vpc.main vpc-0a1b2c3d4e5f6

# Import an existing S3 bucket
terraform import aws_s3_bucket.assets my-company-assets-bucket

This writes the resource into state, but it doesn't write the corresponding .tf config. You have to write that yourself, then run terraform plan to verify it produces no changes (meaning your config matches what was imported).

Since Terraform 1.5, you can use import blocks in your config instead:

import {
to = aws_vpc.main
id = "vpc-0a1b2c3d4e5f6"
}

resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
# ... other attributes
}

The 1.5+ import blocks also support terraform plan --generate-config-out=generated.tf, which writes the HCL config for you based on what Terraform reads from the provider. It's not perfect — some attributes require manual adjustment — but it significantly reduces the manual work of importing large amounts of existing infrastructure.

State file structure and workspaces

Every Terraform root module has exactly one state file per workspace. Workspaces let you maintain multiple independent state files from the same configuration:

terraform workspace new staging
terraform workspace new production
terraform workspace list
# * staging
#   production
#   default

When using S3 as a backend, workspaces are stored as separate files under a env:// path prefix by default:

s3://my-bucket/terraform.tfstate              # default workspace
s3://my-bucket/env:/staging/terraform.tfstate  # staging workspace
s3://my-bucket/env:/production/terraform.tfstate

Workspaces are useful for managing identical infrastructure across environments (dev, staging, prod) from the same Terraform configuration, using terraform.workspace to parameterize values. The downside is that a single mistake in the wrong workspace can affect production. Many teams prefer separate root modules per environment (separate directories, separate state files) rather than workspaces, because the blast radius of a mistake is confined to one directory.

Structuring state for large codebases

As infrastructure grows, keeping everything in a single Terraform root module with one state file becomes a liability. A large state file means every plan/apply operation reads and writes the entire state, locking is coarser (one lock for all infrastructure), and a corrupted state file is a catastrophe.

The standard approach is to split infrastructure into multiple Terraform root modules, each with its own state file. Common split strategies:

By layer: networking, compute, databases, IAM, DNS — each in its own root. Lower layers are applied first; higher layers reference outputs via terraform_remote_state data sources.
By environment: separate roots for dev, staging, production. The same module code, different variable files and state buckets.
By team: each team owns their infrastructure layer. Cross-team dependencies are exposed via outputs or shared data sources.

Terragrunt is a wrapper around Terraform that automates multi-root state management. It lets you define a hierarchy of terragrunt.hcl files where each directory represents a root module with its own state key, and dependencies are declared explicitly. Running terragrunt run-all apply from the repo root applies all roots in dependency order.

What's actually in the state file (and why it's sensitive)

Terraform state stores every attribute of every managed resource, including values that Terraform marks as sensitive in the provider schema. That means:

Database passwords (RDS master passwords, Redis auth tokens)
IAM access keys if you manage them with Terraform
TLS private keys if you create certificates with tls_private_key
Any secret values passed to resources

This is why committing terraform.tfstate to git is a security problem, regardless of whether the repo is private. Use remote state with server-side encryption at rest. For S3, the encrypt = true backend option enables SSE-S3. Most teams pair this with a strict .gitignore that excludes all *.tfstate and *.tfstate.backup files as a defense in depth measure.

Recovering from state problems

State problems are stressful because they can affect live infrastructure. Here's a quick reference for the most common scenarios:

State file accidentally deleted. If you have backups (S3 versioning, Terraform Cloud history), restore from the most recent backup. If not, you'll need to re-import every resource — tedious but possible. Enable S3 versioning on your state bucket before you need it.

State file corrupted (invalid JSON). Terraform keeps a terraform.tfstate.backup of the previous successful state. Restore it: cp terraform.tfstate.backup terraform.tfstate. With remote state on S3 with versioning, use aws s3api list-object-versions to find and restore a previous version.

Resource exists in state but not in cloud. Someone manually deleted the resource. Run terraform state rm to remove it from state, then apply to let Terraform recreate it. Or skip the rm and let the next apply fail with a provider error and recreate from scratch.

Resource exists in cloud but not in state. You need to import it: terraform import.

Two state files got merged or need splitting. Use terraform state mv and terraform state pull/push to move resources between state files. This is advanced — read the Terraform docs carefully and take a backup first.

State commands reference

The terraform state subcommand group is the primary interface for inspecting and modifying state without running apply:

terraform state list — list all resources tracked in state
terraform state show — show all attributes of a specific resource from state
terraform state rm — remove a resource from state (doesn't destroy the real resource)
terraform state mv — rename or move a resource within state (useful when refactoring module structure)
terraform state pull — download and print remote state as JSON
terraform state push — upload a local state file to the remote backend (dangerous — use with care)
terraform force-unlock — release a stuck state lock

Understanding Terraform state isn't just academic — it directly affects how safely your team can make infrastructure changes. Investing in remote state with versioning and locking early pays off every time someone needs to recover from a mistake or audit what changed and when.

If you want to see what your Terraform code is actually building, InfraSketch generates architecture diagrams directly from .tf files or terraform show -json plan output — useful for reviewing what state will look like after the next apply.

Kubernetes Networking Explained: Pods, Services, Ingress, and Network Policies

Raghvendra Pandey — Sun, 07 Jun 2026 00:25:24 +0000

Kubernetes networking is one of the most misunderstood parts of running containerized workloads. A pod can reach another pod by IP — but why does that stop working after a deployment? A service exists and resolves in DNS — but traffic isn't arriving at the application. An Ingress resource is configured — but requests return 502. These puzzles are common and they stem from the same root: Kubernetes networking has several distinct layers, each solving a different problem, and it's easy to conflate them.

This article walks through how Kubernetes networking actually works at each layer — from pod networking to services to Ingress to network policy — so the next time something breaks, you have a mental model to reason from.

The fundamental promise: flat pod networking

Kubernetes makes one core promise about networking: every pod can communicate directly with every other pod in the cluster without NAT. Every pod gets a real IP address from the cluster's pod CIDR range, and those IPs are routable between pods regardless of which node they're running on.

This is not something Kubernetes itself implements. It's a contract that every Kubernetes-conformant CNI (Container Network Interface) plugin must fulfill. When you install Calico, Cilium, Flannel, Weave, or any other CNI, you're installing the component that actually creates this flat network. The mechanism varies — Flannel uses VXLAN overlays, Calico can use BGP for direct routing, Cilium uses eBPF — but the result is the same: pod-to-pod communication without NAT.

Here's what a pod's network namespace looks like:

$ kubectl exec -it my-pod -- ip addr
1: lo:  ...
3: eth0@if12:  ...
inet 10.244.1.15/24 brd 10.244.1.255 scope global eth0

$ kubectl exec -it my-pod -- ip route
default via 10.244.1.1 dev eth0
10.244.0.0/16 via 10.244.1.1 dev eth0

The pod has an IP (10.244.1.15) on a /24 subnet. The node this pod runs on has an IP from the same range — or a different /24 within the same /16. Traffic from this pod to 10.244.2.8 (a pod on another node) goes through the CNI-managed overlay or routed network.

Why you shouldn't use pod IPs directly

If pods can communicate directly, why not just use pod IPs? Two reasons.

First, pod IPs are ephemeral. When a pod is killed and rescheduled — whether from a node failure, a rolling deployment, or the scheduler moving it — it gets a new IP. Any hardcoded reference to the old IP breaks immediately.

Second, horizontal scaling creates multiple pod instances. If you have three replicas of your API service, you need something that routes traffic across all three, not just the first one.

Services solve both problems.

Services: stable DNS names and load balancing

A Kubernetes Service is a stable virtual endpoint that fronts a set of pods. It has a cluster-scoped DNS name (my-service.my-namespace.svc.cluster.local) and a virtual IP (ClusterIP) that doesn't change. The Service uses a label selector to determine which pods are in its backend pool:

apiVersion: v1
kind: Service
metadata:
name: api
namespace: backend
spec:
selector:
app: api
tier: backend
ports:
- port: 80
targetPort: 8080

Kubernetes continuously watches pods with labels app=api, tier=backend. When pods matching the selector are created, their IP:port is added to an Endpoints object. When they're deleted, they're removed. The Service IP stays the same; the backing Endpoints change as pods come and go.

Traffic routing to services is handled by kube-proxy (or Cilium/eBPF in modern clusters). kube-proxy watches for Endpoints changes and programs iptables rules (or IPVS rules) that intercept traffic to the ClusterIP and distribute it across the current pod set.

ClusterIP, NodePort, and LoadBalancer

Services have a type field that controls how they're exposed:

ClusterIP (default): The service is accessible only within the cluster. Other pods can reach it at api.backend.svc.cluster.local:80. External traffic cannot reach it. This is the right type for internal services — databases, caches, backend APIs called only by other services.

NodePort: Opens a port on every node (in the 30000–32767 range by default) and routes traffic to the service. You can reach the service at :. NodePort services are rarely used in production for external traffic because they require knowing node IPs and using non-standard ports. They're sometimes used for development or for specific cases where an external load balancer handles the stable IP layer.

LoadBalancer: Provisions an external load balancer from the cloud provider (an AWS ALB/NLB, a GCP load balancer, an Azure LB). The load balancer gets a stable external IP and routes to the NodePort service underneath. This is the straightforward way to expose a single service externally, but it creates one load balancer per service — expensive at scale.

ExternalName and headless services

Two less-common service types worth knowing:

ExternalName: Returns a CNAME record to an external DNS name. Useful for giving a cluster-internal alias to an external database or API: db.backend.svc.cluster.local resolves to my-db.rds.amazonaws.com. Services can reference external dependencies by internal name without embedding external DNS names in application code.

Headless services (clusterIP: None): Instead of returning the virtual ClusterIP, DNS returns the actual pod IPs. Each DNS query returns the IPs of all pods matching the selector. This is used for stateful applications (StatefulSets) where clients need to connect to specific pod instances, not a load-balanced virtual IP. MongoDB, Cassandra, and other distributed databases use headless services so each pod is individually addressable.

DNS in Kubernetes

Kubernetes runs CoreDNS (or kube-dns in older clusters) as a cluster DNS resolver. Every pod is configured with a resolv.conf that points to CoreDNS:

$ kubectl exec -it my-pod -- cat /etc/resolv.conf
nameserver 10.96.0.10
search backend.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

The search entries mean that short service names like api are automatically expanded. A lookup for api from within the backend namespace first tries api.backend.svc.cluster.local, finds the service, and returns. A lookup from a different namespace would need api.backend or the full FQDN.

The ndots:5 setting means names with fewer than 5 dots are treated as relative and go through the search path before being treated as absolute. This can cause unexpected DNS latency if every external hostname lookup triggers multiple failed cluster-internal lookups first. Setting ndots: 2 or using fully-qualified hostnames (with a trailing dot) for external lookups can reduce DNS lookup latency in high-throughput services.

Ingress: HTTP routing at the cluster edge

LoadBalancer services expose one service per load balancer. In practice, a production cluster might have 20 HTTP services that should all be accessible under a single external IP. Ingress handles this with HTTP-level routing — one external load balancer, multiple services, routing by hostname and path.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: api.mycompany.com
http:
paths:
- path: /v1
pathType: Prefix
backend:
service:
name: api-v1
port:
number: 80
- path: /v2
pathType: Prefix
backend:
service:
name: api-v2
port:
number: 80
- host: admin.mycompany.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: admin
port:
number: 80

An Ingress object is just configuration. The actual traffic routing is done by an Ingress controller — a deployment running in your cluster that watches Ingress resources and programs an actual reverse proxy accordingly. Common controllers:

nginx-ingress-controller: The most widely deployed. Configures nginx based on Ingress resources. Mature, well-documented, broad feature support via annotations.
AWS Load Balancer Controller: For EKS, provisions AWS ALBs instead of nginx. Routes HTTP/HTTPS at the ALB layer, with support for target groups, WAF integration, and certificate management via ACM.
Traefik: Auto-discovers Kubernetes services, has a dashboard, good support for middleware (rate limiting, auth, header rewriting).
Istio Gateway: Part of the Istio service mesh. More complex but enables traffic management, mTLS, and observability beyond what a simple Ingress controller provides.

The Ingress spec is intentionally minimal — just host/path rules and backend services. Advanced features (SSL termination, authentication, rate limiting, sticky sessions) are handled via controller-specific annotations. This means Ingress resources are somewhat controller-specific despite being a standard API object.

For clusters on Kubernetes 1.19+, ingressClassName replaces the older kubernetes.io/ingress.class annotation and allows multiple Ingress controllers to coexist in the same cluster, each handling different Ingress objects.

Network Policies: restricting traffic between pods

By default, Kubernetes networking is fully permissive. Any pod can reach any other pod, any service, and any external IP. In a shared cluster, this means a compromised pod can attempt to reach databases, other teams' services, or cluster control plane components.

NetworkPolicy resources define firewall rules for pod-to-pod and pod-to-external traffic:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-network-policy
namespace: backend
spec:
podSelector:
matchLabels:
app: api
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
- namespaceSelector:
matchLabels:
name: monitoring
ports:
- protocol: TCP
port: 8080
egress:
- to:
- podSelector:
matchLabels:
app: database
ports:
- protocol: TCP
port: 5432
- to:  # allow DNS
- namespaceSelector: {}
ports:
- protocol: UDP
port: 53

This policy says: pods labeled app=api in the backend namespace accept ingress on port 8080 only from pods labeled app=frontend or from any pod in namespaces labeled name=monitoring. They can only egress to pods labeled app=database on port 5432, plus DNS (port 53 UDP to any namespace).

Two things to know about NetworkPolicy:

NetworkPolicy requires a compatible CNI. Not every CNI plugin enforces NetworkPolicy rules. Flannel does not enforce NetworkPolicy without a companion policy controller. Calico, Cilium, and Weave enforce NetworkPolicy natively. If your cluster's CNI doesn't support it, NetworkPolicy objects exist in the API but have no effect — silently.

Default-deny is not automatic. A NetworkPolicy that selects a pod switches that pod from the default-allow to the deny-unless-explicitly-allowed model. But you have to write the NetworkPolicy for it to apply. A common pattern is a default-deny policy that selects all pods in a namespace, then explicit allow policies for specific traffic:

# Default deny all ingress and egress for all pods in namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: backend
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress

Common networking problems and how to debug them

Pod can't reach a service by DNS name. Check that CoreDNS is running: kubectl get pods -n kube-system -l k8s-app=kube-dns. Check the service exists: kubectl get svc. Check the Endpoints object has pods: kubectl get endpoints my-service. If Endpoints is empty, the service's selector isn't matching any pods — check pod labels match the service selector exactly.

Service exists but traffic isn't reaching pods. Check the Endpoints: kubectl describe endpoints my-service. If there are endpoints, the problem is between kube-proxy and the pods. Run kubectl exec -it debug-pod -- curl http://: to check direct pod connectivity. Also check that the pod's containerPort matches the service's targetPort.

Ingress returns 502. The Ingress controller reached the backend service, but the service is returning an error or not listening. Check that the backend service exists and its Endpoints are populated. Check pod logs for errors. A 502 at the Ingress layer almost always means the backend is reachable but failing, not an Ingress configuration problem.

NetworkPolicy broke everything. If you applied a NetworkPolicy and traffic stopped working, check what DNS looks like — often the first symptom is DNS failures because the default-deny policy blocks UDP/53 egress. Make sure your network policy allows egress to the DNS port, or tests will fail mysteriously on service name resolution before you get to the actual policy issue you were trying to configure.

Kubernetes manifests describe all of these components — pods, services, Ingress, NetworkPolicy — in text form. Visualizing how they connect can be useful when debugging or reviewing changes to a cluster's configuration. InfraSketch parses Kubernetes YAML and generates diagrams that show which services expose which pods and how Ingress rules route between them.

Terraform vs CDK vs Pulumi: Choosing Your Infrastructure-as-Code Tool

Raghvendra Pandey — Sun, 07 Jun 2026 00:25:22 +0000

The IaC landscape split into two philosophies about a decade ago and hasn't fully resolved the argument since. On one side: declarative configuration languages designed specifically for infrastructure (Terraform HCL, CloudFormation YAML, Bicep). On the other: general-purpose programming languages brought to infrastructure (AWS CDK, Pulumi). Both approaches have won in production at major organizations. Neither is clearly superior.

This comparison covers Terraform, AWS CDK, and Pulumi in depth — how they work, where they excel, where they struggle, and which makes sense for different team situations. It isn't a beginner introduction to any of these tools; if you're choosing between them for a real project, this assumes you've at least skimmed each one.

The core philosophical difference

Terraform's HCL is a purpose-built configuration language. It's not Turing-complete (no arbitrary loops, no recursion, limited conditionals). This is by design: HashiCorp's position is that infrastructure definitions should be readable, predictable, and safe to generate tooling around. When you read a .tf file, you can understand what it creates without executing anything.

CDK and Pulumi take the opposite position: the limitations of configuration languages are a tax on productive engineers. Why invent a domain-specific language when TypeScript already exists? Real programming languages have proper abstractions, test frameworks, package managers, IDE support, and a billion engineers who already know them. Infrastructure should be no different from application code.

Both positions have merit. The choice between them often comes down to who's writing the infrastructure more than which approach is technically superior.

Terraform

Terraform is the default choice for infrastructure-as-code in 2026. It works with every major cloud provider and hundreds of minor ones. The Terraform Registry has thousands of modules — reusable packages for common patterns like VPCs, EKS clusters, and RDS databases. That ecosystem is its most durable advantage.

HCL syntax is approachable. You describe what you want to exist:

resource "aws_vpc" "main" {
cidr_block           = "10.0.0.0/16"
enable_dns_hostnames = true
}

resource "aws_subnet" "private" {
count             = length(var.availability_zones)
vpc_id            = aws_vpc.main.id
cidr_block        = cidrsubnet(var.cidr_block, 4, count.index)
availability_zone = var.availability_zones[count.index]
}

Terraform figures out the execution order based on the references between resources. You don't specify steps — you specify the end state.

Where Terraform struggles:

Dynamic resource generation: count and for_each are usable but awkward. Generating 20 similar resources with slight variations requires careful HCL gymnastics. CDK and Pulumi can use a for loop.
Abstraction limitations: You can create modules, but you can't do real object-oriented composition. CDK constructs and Pulumi component resources are substantially more powerful for building internal platforms.
Testing: Testing Terraform is harder than it should be. terraform validate checks syntax; terratest (Go) or pytest-terraform can do integration tests, but unit testing logic is awkward because HCL isn't executable without a provider.
State management complexity: As covered in depth in the Terraform state guide, managing state at scale requires careful structure and tooling.

When to choose Terraform: You need multi-cloud support, your team is a mix of application and platform engineers (not all software engineers), or you're inheriting existing Terraform infrastructure. Also: when you want the broadest hiring pool and the most community resources.

AWS CDK

CDK (Cloud Development Kit) is AWS's answer to the "write real code" camp. You write TypeScript, Python, Java, Go, or C# that constructs a tree of CDK constructs. When you run cdk synth, it compiles to CloudFormation JSON or YAML, which AWS then deploys.

The synthesis step is important: CDK is an abstraction on top of CloudFormation, not a deployment tool in itself. Under the hood, your resources become CloudFormation stacks with CloudFormation change sets. This means CDK inherits CloudFormation's deployment semantics, including its quirks — stack drift, resource replacement behavior, and the 500-resource-per-stack limit.

import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ecs from 'aws-cdk-lib/aws-ecs';

const vpc = new ec2.Vpc(this, 'AppVpc', {
maxAzs: 3,
natGateways: 1
});

const cluster = new ecs.Cluster(this, 'AppCluster', {
vpc,
containerInsights: true
});

// L2 constructs handle security groups, IAM roles,
// log groups, and other boilerplate automatically
const service = new ecs.FargateService(this, 'AppService', {
cluster,
taskDefinition: taskDef,
desiredCount: 3
});

CDK's L2 constructs are its killer feature. An L2 construct is an opinionated abstraction over one or more CloudFormation resources that handles the boilerplate. ecs.FargateService doesn't just create a AWS::ECS::Service — it creates the service, the task definition's execution role with the right policies, the security group, and the CloudWatch log group. Getting the same result in raw CloudFormation or Terraform requires 4–6 separate resources and knowing which policies to attach.

Where CDK struggles:

AWS-only: CDK is designed for AWS CloudFormation. Using non-AWS resources requires CDK for Terraform (CDKTF), which is a separate project with its own quirks, or mixing in other tools.
CloudFormation limits: That 500-resource limit per stack bites real production systems. Large applications need careful stack decomposition. CloudFormation deployments also tend to be slower than Terraform — change sets take time.
Synthesis surprises: A single CDK construct can expand to dozens of CloudFormation resources. If you're used to explicit resource definitions, the synthesized output can be surprising — and the diff on a CDK update is sometimes much larger than expected.
CDK versioning pain: CDK 1.x to 2.x was a painful migration. The framework moves fast and breaking changes happen.

When to choose CDK: Your team is primarily software engineers who find HCL limiting, you're AWS-only, and you want L2 construct abstractions to reduce boilerplate. Also: when you're building an internal infrastructure platform that other teams consume as a library — CDK constructs compose like packages.

Pulumi

Pulumi is similar in philosophy to CDK (real programming languages for infrastructure) but different in execution. Pulumi doesn't compile to CloudFormation — it has its own deployment engine that calls provider APIs directly, similar to Terraform. It supports TypeScript, Python, Go, C#, Java, and YAML.

import * as aws from "@pulumi/aws";

const vpc = new aws.ec2.Vpc("main", {
cidrBlock: "10.0.0.0/16",
enableDnsHostnames: true,
});

const privateSubnets = availabilityZones.map((az, i) =>
new aws.ec2.Subnet(`private-${az}`, {
vpcId: vpc.id,
cidrBlock: `10.0.${i}.0/24`,
availabilityZone: az,
})
);

This is the key difference from CDK: new aws.ec2.Vpc(...) isn't creating a CloudFormation resource. It's registering a resource with the Pulumi engine, which maintains its own state (either in Pulumi Cloud or in a self-managed backend like S3). The deployment runs directly against the AWS API, like Terraform.

Pulumi's multi-cloud support is genuine — it has providers for AWS, Azure, GCP, Kubernetes, and many others, all within the same program. You can write a TypeScript program that creates an AWS VPC, deploys an Azure SQL database, and configures a Cloudflare DNS record, all in a single Pulumi stack.

Where Pulumi struggles:

Community and ecosystem size: Pulumi's ecosystem is smaller than Terraform's. The Registry has fewer published modules. When you hit an edge case, there are fewer Stack Overflow answers and community resources.
Language complexity can become a liability: The same expressiveness that makes Pulumi powerful also means infrastructure code can become complex. A Pulumi program that makes API calls, fetches secrets, and uses complex TypeScript generics is harder to review than equivalent Terraform HCL.
State backend choices: Pulumi's hosted backend (Pulumi Cloud) is polished but adds a vendor dependency. The self-managed S3 backend works but lacks features like the run history and policy-as-code that Pulumi Cloud provides.

When to choose Pulumi: Your team writes TypeScript or Python daily and finds HCL genuinely limiting. You need multi-cloud in a single program. You're building Kubernetes-native infrastructure where the Pulumi Kubernetes provider is a natural fit alongside AWS/GCP resources.

Side-by-side comparison

Dimension	Terraform	AWS CDK	Pulumi
Language	HCL (DSL)	TS, Python, Java, Go, C#	TS, Python, Go, C#, Java, YAML
Deployment engine	Direct provider API	CloudFormation	Direct provider API
Multi-cloud	Yes (native)	AWS only (CDKTF for others)	Yes (native)
State management	Self-managed or Terraform Cloud	CloudFormation (AWS-managed)	Self-managed or Pulumi Cloud
Abstractions	Modules	Constructs (L1/L2/L3)	Component resources
Testing	Limited unit testing	Jest + CDK assertions	Jest/pytest + Pulumi mocking
Ecosystem	Largest	Large (AWS-focused)	Growing
Learning curve	Low (for HCL)	Medium (requires knowing CDK patterns)	Low (if already know the language)

Hybrid approaches

Real production environments often use more than one tool. Some common combinations:

Terraform for infrastructure, CDK for application stacks. Platform teams manage VPCs, RDS clusters, and shared IAM with Terraform. Application teams deploy their ECS services and Lambda functions with CDK. The CDK stacks reference Terraform outputs via SSM parameters or remote state. This splits ownership cleanly and lets each team use the tool that fits their skillset.

Terraform for everything except Kubernetes. Terraform creates the EKS cluster. Pulumi or Helm manages Kubernetes resources. Terraform's Kubernetes provider exists but is awkward for complex K8s workloads — the HCL representation of a Kubernetes deployment is verbose and hard to read. Pulumi's Kubernetes provider or plain Helm charts are usually a better fit for the K8s layer.

CDK for core AWS, Terraform for third-party services. CDK shines for AWS resources with its L2 constructs. Terraform shines for the 1000+ non-AWS providers (Datadog, PagerDuty, Cloudflare, GitHub, Vault). Many teams use CDK for their AWS infrastructure and Terraform for everything else.

Migration considerations

If you're migrating existing infrastructure to a different IaC tool, the transition cost is often higher than it looks. Consider:

State migration: Moving from Terraform to Pulumi or CDK means re-importing every resource or doing a careful blue-green migration. There are tools to convert Terraform state to Pulumi (pulumi convert --from terraform) but they require significant manual cleanup.

Team knowledge: If your team knows Terraform, migrating to CDK doesn't make everyone immediately productive. The learning curve is real, especially the CDK mental model (constructs, stacks, apps, environments).

Partial migrations are valid: You don't have to pick one tool for everything. New projects can use the new tool. Existing infrastructure can stay where it is. The overhead of two tools is often less than the disruption of a full migration.

The version that no one talks about: OpenTofu

In 2023, HashiCorp changed Terraform's license from MPL to BSL — a non-open-source license that restricts commercial use by competing products. The community responded by forking Terraform at the last open-source version (1.5.x) to create OpenTofu, now maintained by the Linux Foundation.

OpenTofu is a drop-in replacement for Terraform with the same HCL syntax, provider compatibility, and state format. It diverges from Terraform where HashiCorp has added BSL-licensed features. For most users, OpenTofu and Terraform are functionally identical today. OpenTofu is the right choice if the license change concerns you or if you're vendor-averse; Terraform is the right choice if you want the closest alignment with the HashiCorp ecosystem and Terraform Cloud.

Whichever IaC tool you use, visualizing what it actually creates helps during both development and review. InfraSketch supports Terraform HCL, CDK synthesized JSON, and Pulumi TypeScript/Python — paste your code to see the architecture diagram without deploying anything.

Generate IaC Architecture Diagrams Inside Claude Code and Cursor with InfraSketch MCP

Raghvendra Pandey — Thu, 14 May 2026 00:51:46 +0000

TL;DR: Install the InfraSketch MCP server with one command (claude mcp add infrasketch -- npx infrasketch-mcp), then ask Claude Code to diagram any Terraform, Kubernetes, or Pulumi code without leaving your editor. The diagram opens in your browser, your code never leaves your machine.

The context-switching problem: diagrams while you code

AI editors like Claude Code, Cursor, and Windsurf have gotten surprisingly good at infrastructure work. You can paste a 2,000-line Terraform module into context, ask it to refactor a subnet configuration, and get a working diff in seconds. But ask it to show you an architecture diagram of that same code? Until recently, the answer was "open a browser, paste it somewhere else, render it there."

That break is annoying in a specific way. It's not just the thirty seconds — it's that you lose the thread. The AI has been building context about your codebase, your constraints, what you're trying to do. When you jump to another tool and come back, you're starting that context fresh.

The InfraSketch MCP server fixes this by adding diagram generation as a tool your AI assistant can call natively. Two tools: generate_diagram and detect_iac_format. Ask Claude "can you show me a diagram of this?" and it calls the tool, gets a URL, and includes it in its response. You never left the editor.

What is MCP and why it matters for infrastructure teams

Model Context Protocol (MCP) is an open standard developed by Anthropic that lets AI assistants communicate with external tools and services through a standardized interface. Instead of an AI assistant being limited to the text in its context window, MCP lets it call out to tools — read files, run searches, query APIs, generate artifacts — and incorporate the results back into the conversation.

For infrastructure teams the implication is straightforward: your AI assistant can now do things that previously required leaving the editor:

Look up the current state of a Terraform resource in a cloud account
Check a security scanner for misconfigurations
Generate an architecture diagram of whatever code it's currently looking at
Figure out what IaC format a file is before processing it

MCP servers are lightweight processes that run locally alongside your editor. They communicate with the AI assistant over a stdio or SSE transport — there's no cloud intermediary. The InfraSketch MCP server is published as infrasketch-mcp on npm and requires only Node.js to run.

Installing infrasketch-mcp (npx, global, Claude Code one-liner)

The fastest way to install is the Claude Code one-liner, which registers the MCP server in your Claude Code configuration in a single command:

claude mcp add infrasketch -- npx infrasketch-mcp

This tells Claude Code to start the InfraSketch MCP server on demand using npx, which downloads and runs the package without a global install. The server starts fresh each session and exits when Claude Code closes — no persistent daemon.

If you prefer a global install to avoid npx startup overhead:

npm install -g infrasketch-mcp

Then reference it directly in your MCP configuration as infrasketch-mcp instead of npx infrasketch-mcp.

Setup: Claude Code (claude mcp add command + settings.json method)

After running the claude mcp add command above, the server is registered automatically. To verify it's registered correctly, run:

claude mcp list

You should see infrasketch in the list. If you prefer to configure it manually, or if you're deploying a shared configuration to a team, edit your Claude Code MCP settings file at ~/.claude/settings.json and add the server to the mcpServers object:

{
"mcpServers": {
"infrasketch": {
"command": "npx",
"args": ["infrasketch-mcp"]
}
}
}

For project-level configuration (useful when you want to check the MCP config into your repository so the whole team gets it automatically), create or edit .claude/settings.json in your project root:

{
"mcpServers": {
"infrasketch": {
"command": "npx",
"args": ["infrasketch-mcp"],
"description": "Generate architecture diagrams from IaC code"
}
}
}

After editing the settings file, restart Claude Code for the changes to take effect. The MCP server will be available in all sessions in that project.

Setup: Cursor and Windsurf

Cursor uses the same MCP configuration format. Create or edit ~/.cursor/mcp.json (for global configuration) or .cursor/mcp.json in your project root:

{
"mcpServers": {
"infrasketch": {
"command": "npx",
"args": ["infrasketch-mcp"]
}
}
}

In Cursor, go to Settings → MCP to verify the server appears and shows a green connected status. You may need to click "Reload MCP Servers" after saving the config file.

Windsurf follows the same pattern. Edit ~/.windsurf/mcp.json with the same configuration block. Once the server is connected, both Cursor's Composer and Windsurf's Cascade will have access to the InfraSketch tools and can call them in response to natural language prompts about your infrastructure code.

Node.js required: The MCP server requires Node.js 18 or later. Run node --version to check. If you're using a version manager like nvm or fnm, make sure the active version is 18+ before starting your editor.

The generate_diagram tool in action (with example prompt + example output)

Once the MCP server is connected, you can ask Claude to generate diagrams in natural language. You don't need to know the tool name or its parameters — just describe what you want:

Generate a diagram for this terraform code:

resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
tags = { Name = "main-vpc" }
}

resource "aws_subnet" "public" {
vpc_id            = aws_vpc.main.id
cidr_block        = "10.0.1.0/24"
availability_zone = "us-east-1a"
}

resource "aws_internet_gateway" "igw" {
vpc_id = aws_vpc.main.id
}

resource "aws_route_table" "public" {
vpc_id = aws_vpc.main.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.igw.id
}
}

resource "aws_route_table_association" "public" {
subnet_id      = aws_subnet.public.id
route_table_id = aws_route_table.public.id
}

Claude will call the generate_diagram MCP tool with the code and format, and return something like:

Here's your architecture diagram:

**[View Interactive Diagram](https://infrasketch.cloud/#eJyVVdtu2zgQffdXEH...)**

The diagram shows:
- **VPC** (10.0.0.0/16) as the top-level container
- **Public Subnet** (10.0.1.0/24) in us-east-1a inside the VPC
- **Internet Gateway** attached to the VPC
- **Route Table** with a default route (0.0.0.0/0) pointing to the IGW
- **Route Table Association** linking the public subnet to the route table

You can click the link to open an interactive, zoomable diagram in your browser.

The diagram URL is a permanent shareable link. The iframe embed lets you paste the diagram directly into documentation, Notion, Confluence, or any HTML page. The AI assistant also provides a human-readable summary of what the diagram shows, which is useful for accessibility and for quickly confirming the rendering is correct without opening a browser.

The detect_iac_format tool

The second tool exposed by the MCP server is detect_iac_format. This is useful when you have an ambiguous file and want the AI to determine its format before processing it. You can invoke it explicitly:

What IaC format is this file?

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.21

The tool returns the detected format (kubernetes in this case) along with a confidence score. Claude Code will then use this information to call generate_diagram with the correct type parameter, producing an accurate diagram for Kubernetes resources rather than trying to parse it as Terraform or CloudFormation.

In practice, the format detection runs automatically when you ask for a diagram — you rarely need to invoke it explicitly. It's most useful in agentic workflows where code is being passed programmatically and the format isn't known in advance.

Real workflow: reviewing a Terraform PR with Claude + InfraSketch

Concrete example. A teammate opens a PR adding an ECS cluster with autoscaling to an existing VPC. You're reviewing it in Claude Code:

Ask Claude to show you what changed: "What's different in the terraform/ directory compared to main?" It reads the diff and summarizes.
"Generate a diagram of the full terraform/environments/prod/ directory so I can see how the ECS cluster fits in." Claude calls generate_diagram, gets a URL back.
Click the URL. You get an interactive, zoomable diagram — ECS cluster, task definitions, ALB, target groups, VPC subnets, security groups, all in one view.
Back in Claude: "The ECS tasks are in the public subnet. Intentional?" Claude already has all the code in context and can answer immediately.
Your review comment is specific — "the ECS tasks should be in the private subnet per our networking standards, the security group on line 43 is too permissive" — not just "looks good" or a vague worry about networking.

Reading HCL line by line is fine for small changes. For anything spanning multiple resources and subnets, the diagram gets you oriented in about ten seconds instead of ten minutes.

For teams that want this diagram to appear automatically on every PR without any manual step, the GitHub Action (pandey-raghvendra/infrasketch@v4) posts the diagram as a PR comment whenever Terraform files change.

How the URL is generated (no upload, privacy-first, works offline)

The MCP server uses the same URL-encoding approach as the CLI and browser tool. When generate_diagram is called:

The code is serialized into a JSON envelope with the detected format
The JSON is compressed with gzip and base64-encoded
The encoded string is appended as a URL fragment to https://infrasketch.cloud/
The resulting URL is returned to the AI assistant, which includes it in its response

The MCP server process runs entirely on your machine. It never makes outbound HTTP requests to InfraSketch servers. The URL it generates encodes your code in the fragment, which — by the HTTP specification — is never transmitted to a server. When you open the URL in a browser, the InfraSketch web app decodes and renders the diagram entirely client-side.

This means the tool works in air-gapped environments (as long as your browser can load the web app), needs no API keys, stores nothing server-side, and leaves no audit trail. If your team has strict data residency requirements, that matters. Cloud-hosted diagramming APIs typically process your code on their servers — this one doesn't, by design.

Combining MCP with CLI and embed for full workflow

The MCP server, CLI, and embed component are designed to complement each other across different parts of the development lifecycle:

MCP server — Use during active coding and code review inside your AI editor. Get diagrams inline without leaving your editor or conversation.
CLI (npx infrasketch) — Use for quick local checks, scripting, and CI pipelines. See the CLI guide for full details.
GitHub Action — Use for automated PR comments. Zero friction for reviewers — the diagram appears without anyone running a command.
Embed web component — Use for documentation, runbooks, and internal wikis. Keep diagrams live and synchronized with the source code.

In practice, a team that's using all four ends up with: MCP for daily coding and reviews, CLI for pre-commit sanity checks and one-off scripting, the GitHub Action for automated PR comments, and embed for the internal docs wiki. Every diagram comes from the same source of truth. Nothing ever gets uploaded.

Supported formats and limitations

The MCP server supports the same eight IaC formats as the web tool and CLI:

Format	Auto-detected	Multi-file support	Notes
terraform	Yes (.tf extension)	Yes (concatenated)	Best coverage of all formats
kubernetes	Yes (apiVersion field)	Yes	Helm chart values not rendered
pulumi	Partial (Pulumi.yaml)	Partial	Heuristic extraction from source
cloudformation	Yes (Resources key)	Yes	Nested stacks shown as references
cdk	Partial	Partial	Works best with synthesized output
bicep	Yes (.bicep extension)	Yes	Modules shown as references
terragrunt	Yes (terragrunt.hcl)	Yes	Module dependency graph included
docker-compose	Yes (services key)	Single file	Networks and volumes visualized

The main limitation to be aware of is code size. The URL fragment approach has a practical ceiling around 2 MB of raw IaC content before browser URL limits become a concern. For most individual modules and services this is not an issue, but for very large monorepos you should point the tool at a specific subdirectory rather than the entire repository root.

FAQ

Does the MCP server require any API key or account?

No. Completely free, no authentication. It runs locally and talks to your AI editor over stdio. No InfraSketch account, no key, no usage limits. Just Node.js.

Does my Terraform code get sent to InfraSketch servers?

No. The MCP server encodes your code into a URL fragment. Fragments are never transmitted in HTTP requests — that's part of the HTTP spec, not something special we're doing. The InfraSketch web app decodes and renders everything in the browser. The InfraSketch server only ever serves the static HTML/JS/CSS files, same as any website.

Can I use the MCP server with VS Code or other editors?

Anything that supports Model Context Protocol over stdio will work. Claude Code, Cursor, and Windsurf are the most common. VS Code has MCP support in some extensions. For editors that don't support MCP at all, the CLI and browser tool still cover most of the same use cases.

What happens if the diagram looks wrong?

The MCP server passes your code to the renderer exactly as-is — no modification, no interpretation. So if the diagram is wrong, it's the renderer's parsing that's off, not how the AI called the tool. Open the interactive diagram in your browser, check what's there, and if you spot a bug file it on GitHub with the example code.

Can I use the MCP server in a team setting with a shared configuration?

Yes, and this is actually the easiest setup. Add the MCP config to .claude/settings.json (Claude Code) or .cursor/mcp.json (Cursor) at the project root and check it into your repo. Anyone who clones gets it automatically. Since it uses npx, nobody needs to install anything manually.

Diagram your infrastructure without leaving your editor Add the InfraSketch MCP server to Claude Code in one command, or try the browser tool right now — no account needed. Try InfraSketch Free →

Visualize Terraform, Kubernetes, and Pulumi from the Terminal with npx infrasketch

Raghvendra Pandey — Thu, 14 May 2026 00:51:43 +0000

TL;DR: Run npx infrasketch . in any IaC repository to instantly open an interactive architecture diagram in your browser. No login, no install, no file upload. Works with Terraform, Kubernetes, Pulumi, CloudFormation, CDK, Bicep, Terragrunt, and Docker Compose.

The copy-paste problem with IaC diagrams

You finish a Terraform module, open Lucidchart or draw.io, manually drag in an EC2 box, an RDS cylinder, an ALB arrow — 45 minutes recreating what the code already describes. Two weeks later the code changes, the diagram doesn't, and you've got a lie pinned to your Confluence page. I've seen this on every team I've worked with.

The browser-based InfraSketch tool at infrasketch.cloud cuts most of that: paste your HCL, YAML, or JSON and get an interactive diagram instantly, no account required. But there was still friction. You had to leave your terminal, copy the file contents, navigate to a website. For anyone living in the command line, that adds up over a day.

The CLI removes that last step. Point it at a file, a directory, or a raw GitHub URL, and it generates the diagram and opens your browser — usually in under two seconds. In CI where there's no browser to open, --no-open prints the shareable URL to stdout so you can pipe it wherever you need it.

What npx infrasketch does (and what it doesn't)

The CLI is a thin Node.js wrapper published to npm as infrasketch. When you run it, it reads your IaC file or directory, serializes the content into a base64-encoded JSON fragment, appends that fragment to the InfraSketch web app URL, and opens the result in your default browser.

What it doesn't do matters more than you'd expect:

Your code never leaves your machine — no upload, no server processing.
No account, no API key, nothing to sign up for.
It doesn't touch your IaC files. Read-only.
No background process. It runs, opens the browser, and exits.

The diagram is rendered entirely in the browser using the URL fragment. Browsers don't send the hash to the server — they only transmit the path and query string. So when you open https://infrasketch.cloud/#eJy..., the server gets a plain GET / with no idea what's in the fragment. That's not a hack; it's just how HTTP works, and we're leaning into it deliberately. It also means the tool works in air-gapped environments as long as the browser can load the InfraSketch page.

Zero-install usage: your first diagram in 10 seconds

Because the CLI is on npm, you can run it immediately with npx without installing anything globally. Navigate to any directory that contains IaC files and run:

npx infrasketch .

That's it. The CLI will scan the current directory, detect the format automatically, and open your browser. If you have a mixed repository with, say, a terraform/ subdirectory and a k8s/ subdirectory, run it from the root — it will pick the dominant format or let you point it at a subdirectory.

If you prefer a permanent global install to skip the npx overhead on every invocation, add it to your PATH once:

npm install -g infrasketch

After that, infrasketch . works from any directory, just like terraform or kubectl.

Single file mode

When you want to diagram a specific file rather than an entire directory, pass the file path directly:

npx infrasketch main.tf

Single file mode is useful when you're working on a large module and want to check just the resources you're currently editing. The CLI reads only that file, so it's faster for large repositories where a full directory scan would pull in many files you don't care about right now.

The format is auto-detected from the file extension and content. .tf files are parsed as Terraform HCL, .yaml and .yml files trigger heuristic detection to distinguish between Kubernetes manifests, Docker Compose files, and CloudFormation templates. For ambiguous cases you can override with the --type flag:

npx infrasketch template.yaml --type cloudformation

Supported values for --type are: terraform, kubernetes, pulumi, cloudformation, cdk, bicep, terragrunt, docker-compose.

Directory scan mode — multi-file Terraform projects

Real Terraform projects rarely live in a single file. A production module typically spans main.tf, variables.tf, outputs.tf, locals.tf, and any number of resource-specific files. Directory scan mode reads all of them and stitches them into one diagram:

npx infrasketch ./terraform/environments/prod/

The CLI recursively finds all .tf files under the given path, concatenates their contents, and feeds the combined HCL to the diagram renderer. This gives you a complete picture of the environment rather than a partial view of whichever file you happened to open first.

For Terragrunt projects where each module lives in its own subdirectory with a terragrunt.hcl, point the CLI at the root of the deployment tree:

npx infrasketch ./live/prod/ --type terragrunt

It will discover all terragrunt.hcl files and combine the dependency graph across modules, giving you visibility into how modules reference each other — something that's notoriously hard to visualize manually.

Tip: For very large Terraform codebases (hundreds of files), consider running the CLI against a specific module subdirectory rather than the repository root. The URL fragment has a practical size limit around 2 MB of raw content, which covers most real-world modules comfortably.

Kubernetes and Pulumi projects

Kubernetes works the same way. Point it at a directory of YAML files and it picks up Deployments, Services, Ingresses, ConfigMaps, StatefulSets, DaemonSets — and renders how they connect:

npx infrasketch ./k8s/ --type kubernetes

The --type flag is optional here if your YAML files have the standard apiVersion and kind fields — the auto-detector recognizes Kubernetes manifests reliably. But if you have a directory that mixes Kubernetes YAML with other YAML (for example, a Helm chart with both templates and values files), being explicit avoids ambiguity.

For more detail on the Kubernetes diagram format and what relationships are visualized, see the Kubernetes diagram generator guide.

Pulumi projects work similarly. Navigate to a Pulumi project directory (the one containing Pulumi.yaml) and run:

npx infrasketch . --type pulumi

The CLI reads the Pulumi program source — TypeScript, Python, Go, or YAML — and extracts resource declarations. Because Pulumi programs are general-purpose code rather than declarative config, the extraction is necessarily heuristic: it looks for new aws.ec2.Instance()-style patterns in TypeScript/Python and equivalent patterns in other languages. Coverage is good for standard resource declarations but may miss resources created inside complex loops or conditional logic.

Remote GitHub raw URL mode

You don't have to clone a repository to diagram it. Pass any raw GitHub URL (or any other URL that returns plain text) and the CLI will fetch it, parse it, and open the diagram:

npx infrasketch https://raw.githubusercontent.com/hashicorp/terraform-provider-aws/main/examples/two-tier/main.tf

This mode is particularly useful for:

Reviewing open-source Terraform modules before adding them as dependencies
Quickly understanding the architecture of a reference implementation you found
Sharing a diagram of a public repository without cloning it locally
Generating diagrams in documentation pipelines where the source lives in a separate repo

The URL is fetched by the CLI process on your machine (not by the browser or any server), parsed locally, and then encoded into the diagram URL. Your network request goes to GitHub's raw content CDN, not to InfraSketch servers.

CI/CD pipeline integration (--no-open flag)

In a headless CI environment there's no browser to open, so the default behavior would hang or error. The --no-open flag suppresses browser launch and instead prints the diagram URL to stdout:

npx infrasketch main.tf --no-open

This emits a single line like:

https://infrasketch.cloud/#eJyVVdtu2zgQ...

You can capture this URL and use it anywhere: a PR comment, a Slack notification, a build artifact, a release note. Here's a complete GitHub Actions workflow that posts the diagram URL as a PR comment whenever Terraform files change:

name: Terraform Diagram

on:
pull_request:
paths:
- '**.tf'
- '**.tfvars'

jobs:
diagram:
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'

- name: Generate diagram URL
id: diagram
run: |
URL=$(npx infrasketch . --no-open)
echo "url=$URL" >> "$GITHUB_OUTPUT"

- name: Post diagram comment
uses: actions/github-script@v7
with:
script: |
const url = '${{ steps.diagram.outputs.url }}';
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `## Architecture Diagram\n\n[View interactive diagram](${url})\n\n> Generated by InfraSketch CLI`
});

If you'd rather use the purpose-built GitHub Action instead of the raw CLI in CI, see the GitHub Action guide — it handles the comment posting and diff detection automatically with less boilerplate.

How it works under the hood (URL encoding, no upload, privacy)

If you're going to run a tool against production Terraform code, you should know what it actually does. Here's the full flow:

Reads file contents from disk (or fetches from URL). No preprocessing — the raw content goes through as-is, including variable references and module calls.
Wraps the content in a JSON envelope: {"type":"terraform","code":"..."}. Multi-file mode concatenates everything into the code field with separator comments between files.
Gzip-compresses the JSON and base64-encodes the result into a compact ASCII string.
Appends the encoded string to the InfraSketch URL as a hash fragment: https://infrasketch.cloud/#.
Opens that URL in your default browser (open on macOS, xdg-open on Linux, start on Windows). The web app decodes the fragment entirely client-side.

The key detail is step 5. The URL fragment — everything after the # — is never transmitted in HTTP requests. When your browser loads https://infrasketch.cloud/#eJy..., the actual HTTP request to the server is just GET /. No server ever sees your infrastructure code. That's a browser standard, not something we're doing specially — we're just building on top of it.

This also means the diagram URL is a permanent, shareable link with no expiry. The entire diagram state is encoded in the URL itself — there's no server-side session or storage. You can bookmark it, share it in Slack, or save it in documentation and it will work forever, as long as the InfraSketch web app is running.

Combining CLI with GitHub Action for full automation

The CLI is for you while you're coding. The GitHub Action is for your reviewers when they're not.

Practically, the workflow goes: you write a new Terraform module for a VPC peering connection, run npx infrasketch . locally to make sure the diagram looks right (subnets, route tables, peering connections all there), then open a PR. The GitHub Action (pandey-raghvendra/infrasketch@v4) fires automatically and posts the diagram as a comment. Reviewers click the link, see the architecture interactively, no tooling required on their end.

The CLI is fast and zero-ceremony — just a command. The Action is automated and requires no one to remember to run anything. They cover different moments in the same workflow, not the same moment twice.

Comparison table: CLI vs browser vs GitHub Action vs MCP

Method	Best for	Install needed	Works in CI	Opens browser
Browser	Quick one-off diagrams, paste-and-view	None	No	Already in browser
CLI (npx infrasketch)	Local dev, scripting, remote URLs	Node.js (npx auto-installs)	Yes (--no-open)	Yes (default)
GitHub Action	Automated PR comments on IaC changes	None (uses Action)	Yes (built for it)	No
MCP Server	AI-assisted coding in Claude Code / Cursor	Node.js (npx auto-installs)	No	Yes (via AI prompt)

FAQ

Does npx infrasketch work without internet access?

First run via npx needs to download the package, so yes, internet required there. After that, npm install -g infrasketch once and it runs offline. The generated URL needs a browser to load the InfraSketch web app — so some internet to load the page initially, though your IaC code never gets uploaded to any server regardless.

What happens with .tfvars files that contain secrets?

The code goes into the URL fragment, which is never sent to a server. But the fragment does show up in browser history and in any URL you share. So treat the URL like you'd treat the source code — don't paste it in a public Slack channel if the file has real credentials in it. Stick to main.tf or module directories for diagramming; leave the .tfvars with the actual secrets out of it.

Can I use the CLI with Terraform workspaces?

Yes, workspaces only affect state — the HCL source files are the same regardless of which workspace you're on. npx infrasketch . works normally. If you have workspace-specific variable files like prod.tfvars, you can point the CLI at them directly.

How large can the IaC file be before the URL gets too long?

HCL compresses well — gzip typically gets 5-10x reduction before base64, so a 500 KB Terraform project ends up under 100 KB in the URL. In practice, URL length limits never bite you with normal module-sized codebases. If you're running against an entire monorepo root with hundreds of files, run it against a subdirectory instead.

Can I get the diagram URL without opening the browser, even locally?

--no-open works anywhere, not just CI. npx infrasketch main.tf --no-open on your laptop prints the URL to stdout without touching the browser. Pipe it to pbcopy, drop it in a script, whatever you need.

Diagram your infrastructure in 10 seconds Run npx infrasketch . in your IaC repo right now — or try the browser tool directly with no install required. Try InfraSketch Free →

How to Embed a Live IaC Architecture Diagram on Any Website with <infra-sketch>

Raghvendra Pandey — Thu, 14 May 2026 00:28:11 +0000

TL;DR: Add one script tag to your page, then use `` or inline your IaC code between the tags. The diagram renders interactively, updates when you update the source, and never sends your code to a server.

Why living diagrams beat static images in docs

There's a specific kind of documentation rot that hits infrastructure teams hard: the architecture diagram that was accurate when someone drew it, is now pinned to a Confluence page, and describes an environment that no longer exists. Someone added an RDS read replica. Someone else swapped EC2 for ECS Fargate. The diagram stayed put.

Static diagram images — PNGs and SVGs exported from draw.io or Lucidchart — have a structural problem. Keeping them current requires someone to remember to update them, have the original tool open, export a new image, and re-upload it. That chain breaks constantly in practice. Not because people are careless; they're just busy.

A living diagram solves this by rendering from the actual IaC file every time the page loads. When the Terraform changes, the diagram changes. No export, no re-upload, no manual step. It's accurate because it's always generated from the current code, not from a snapshot.

The InfraSketch embed component makes this work on any website, docs platform, or wiki that accepts custom HTML: one script tag, then drop the `` element wherever you want the diagram.

Two ways to embed: iframe vs web component

Two ways to embed, and which one you use depends on your platform:

The iframe approach works everywhere — Confluence, Notion, GitHub Pages, internal wikis, anything that allows an HTML iframe. You generate a diagram URL first (browser tool, CLI, or MCP server), then paste the iframe tag. It's a bit more manual but totally reliable across any platform, including those that restrict JavaScript.

The web component is cleaner for HTML-based sites. One script tag in the head, then use `` anywhere on your page — point it at a source URL or write IaC code inline between the tags. The component fetches, detects the format, and renders. For docs sites built with MkDocs, Docusaurus, Hugo, or Jekyll, this is the approach I'd recommend.

Getting the iframe embed code (from the app)

If you have an existing diagram open in the InfraSketch web app, the fastest way to get embed code is the share button. The app generates both an iframe snippet and a direct link:

`plaintext

Paste this into any HTML page or any platform with an HTML/embed block. The iframe is fully self-contained — the diagram state is in the URL, so there are no external dependencies beyond the InfraSketch web app itself.

For platforms that strip the src attribute from iframes for security reasons (some wiki platforms and CMS editors do this), see the CORS considerations section below.

The infra-sketch web component: one-line setup

The web component is loaded with a single script tag that you add once to your page's or just before :

`plaintext

After loading, the `` custom element is registered and available anywhere on the page. It renders as a responsive iframe-like widget with pan and zoom support, the same interactive experience as the full web app.

The script is small, loads from a CDN, and is cached aggressively. Including it on a documentation page adds negligible overhead — the component only fetches and renders when an `` element is actually present in the DOM, so unused script inclusions don't trigger any work.

Embedding by URL: the src= attribute

The simplest way to embed a diagram is to point the component at a URL that returns IaC code. The component fetches the URL, detects the format, and renders the diagram:

`plaintext

When the page loads, the component fetches the src URL, reads the content, auto-detects the IaC format, and renders the interactive diagram. If the file changes in GitHub, the diagram on your documentation page changes the next time someone loads it — without any manual intervention.

You can also pass an explicit type attribute to skip format detection and ensure correct rendering for ambiguous files:

`plaintext

The height attribute sets the diagram height in pixels. If omitted, a default of 480px is used. Width is always 100% of the container element, making the component responsive to its parent layout.

Embedding inline code

For documentation where you want to show a specific, curated snippet rather than an entire file, write the IaC code directly between the `` tags:


apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: api-server
template:
spec:
containers:
- name: api
image: your-org/api:latest
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-service
spec:
selector:
app: api-server
ports:
- port: 80
targetPort: 8080
type: ClusterIP

When no src attribute is provided, the component uses the element's text content as the IaC code. The type attribute is recommended for inline code since there's no file extension to auto-detect from. The content is treated as plain text — HTML entities inside the element are decoded before parsing, so you can use `
Tip: For documentation sites that process Markdown, wrap inline IaC code in a raw HTML block to prevent the Markdown parser from mangling the whitespace or escaping characters inside the element.

All supported attributes (src, type, height, width)

Attribute	Required	Default	Description
src	No	—	URL of the IaC file to fetch and render. Omit to use inline content.
type	No	auto-detect	IaC format: terraform, kubernetes, pulumi, cloudformation, cdk, bicep, terragrunt, docker-compose
height	No	480	Height of the diagram widget in pixels
width	No	100%	Width of the diagram widget. Accepts px or % values.

The component also exposes a custom event, infra-sketch-load, that fires when the diagram has finished rendering. You can listen to it for analytics, lazy-load triggers, or to show/hide a loading skeleton:

javascript document.querySelector('infra-sketch').addEventListener('infra-sketch-load', (e) => { console.log('Diagram rendered:', e.detail.resourceCount, 'resources'); });

Real examples: GitHub Pages docs, Backstage, Confluence, Notion

GitHub Pages / Jekyll / Hugo / MkDocs

For static site generators, add the embed script to your theme's base template and use the component in any page. For MkDocs with the Material theme, add the script to overrides/main.html. For Jekyll, add it to _layouts/default.html. Then use the component in any Markdown file within a raw HTML block:

`markdown

Network Architecture

The following diagram shows the production VPC layout:

Backstage TechDocs

Backstage TechDocs is built on MkDocs and renders Markdown for service catalogs. The same pattern applies — add the script to the TechDocs theme override and embed diagrams inline in your service's docs/ directory. Because TechDocs serves content from the same origin as Backstage, there are no CORS restrictions when fetching from GitHub raw URLs.

Confluence

Confluence's HTML macro lets you paste raw HTML including script tags. Create an HTML macro block on your page, paste in the embed script tag and the element. Alternatively, use the iframe approach — generate the iframe URL from the InfraSketch app and paste it using Confluence's iframe macro. The iframe approach is more reliable across Confluence versions since some restrict JavaScript in HTML macros by default.

Notion

Notion doesn't support arbitrary HTML or script tags in pages, but it does have an Embed block that accepts iframe URLs. Generate the diagram URL from the InfraSketch app (or via the CLI with --no-open), then use Notion's /embed command and paste the URL. Notion will render it as an embedded iframe. The result is an interactive diagram inside your Notion page that anyone with page access can pan and zoom.

Loading from GitHub raw URLs — the recommended pattern

Pointing the src attribute at a GitHub raw URL is the most maintainable pattern for living diagrams. The URL format is:

plaintext https://raw.githubusercontent.com/{owner}/{repo}/{branch}/{path}

For example:

`plaintext

When main.tf is updated and merged, the next page load will show the updated diagram. No documentation update required. No diagram export needed. The diagram is always synchronized with the code in your default branch.

For branch-specific documentation — for example, showing the architecture as it will look after a specific feature branch is merged — use the branch name in the URL:

`plaintext

This is particularly useful when combined with the GitHub Action, which posts a diagram to the PR. You can also link to branch-specific documentation that shows what the architecture will look like after the PR is merged, and the diagram will automatically become the "post-merge" diagram once the branch is merged to main.

For private repositories, GitHub raw URLs require authentication. In that case, you can either use the inline code approach (copy the relevant code into your documentation page directly) or host your own IaC files on an authenticated endpoint and point src at that.

Error handling and CORS considerations

When using the src attribute, the web component fetches the URL from the browser using the Fetch API. This means the target URL must be accessible from the user's browser and must allow cross-origin requests via CORS headers.

GitHub raw URLs (raw.githubusercontent.com) are served with permissive CORS headers — they allow requests from any origin, so they work out of the box. Most public file hosting services and CDNs also support CORS for public content.

If the src URL is on a private server or an origin that doesn't send CORS headers, the fetch will fail with a network error. In that case:

Use inline content instead of src
Configure your server to send Access-Control-Allow-Origin: * for the IaC files
Use a CORS proxy (only for non-sensitive content)
Generate the diagram URL server-side and embed it as a static iframe URL instead of using the web component's dynamic fetch

When a fetch fails, the component shows a visible error state — it doesn't silently disappear. Non-200 response, empty body, network error — you get "Could not load diagram source" in the widget and the actual error in the browser console, so you know what went wrong.

For inline content, there's no network request and no CORS consideration — the content is right there in the DOM, and the component reads it synchronously.

Privacy: what the embed does and doesn't send

Same privacy model as the rest of InfraSketch: your IaC code never touches InfraSketch servers. Here's exactly what happens when you use src:

The browser fetches the src URL — request goes to GitHub or wherever you're hosting the file, not to InfraSketch
File content lands in the browser
The component encodes it into a URL fragment (gzip + base64) in-memory
Creates an iframe pointing to https://infrasketch.cloud/#
Browser loads the InfraSketch web app — the HTTP request goes to our CDN, but fragments are never included in HTTP requests (that's the HTTP spec)
The web app decodes the fragment client-side and renders

What InfraSketch servers actually see: a request for the embed script, a request for the web app assets. That's it. No IaC code, no diagram content, no infrastructure metadata. Standard CDN logs with IP, user agent, timestamp — same as any website.

With inline content, step 1 doesn't happen at all. The only network contact is loading the web app assets.

A complete minimal embed page

Here's a full, self-contained HTML page that embeds a Terraform diagram using the web component. You can use this as a starting point for a GitHub Pages documentation page or any static HTML site:

`css

Network Architecture - Internal Docs

body { font-family: system-ui, sans-serif; max-width: 900px; margin: 40px auto; padding: 0 20px; }
h1 { font-size: 24px; margin-bottom: 8px; }
p { color: #555; margin-bottom: 24px; }

Production Network Architecture
This diagram is generated live from the Terraform source in the infra repository.

Diagram always reflects the current main branch. View source

This page requires no build step, no server-side processing, and no InfraSketch account. It works as a standalone HTML file served from any static host.

FAQ

Does the embed component work with private GitHub repositories?

The src fetch runs in the user's browser, so private GitHub raw URLs would need a token — which you obviously can't bake into a public HTML page. For private repos, use inline content instead: copy the relevant Terraform or Kubernetes code between the tags. Or generate the URL server-side with npx infrasketch main.tf --no-open and embed it as a static iframe. The state is in the fragment, so it doesn't expire.

Will the diagram break if infrasketch.cloud goes down?

If the web app is unavailable, iframes and components pointing to it will show an error. If you need high reliability for critical docs, pre-generate the URL with the CLI and embed it as a static iframe — then if you ever want to self-host the web app, the same URL works unchanged.

Can I control the diagram layout or colors through the embed?

Not through attributes currently. Users can pan, zoom, and move nodes interactively inside the embedded diagram. If you want a specific layout, open the diagram in the full web app, arrange it, and use the "share" URL — it encodes the adjusted node positions in the fragment. Custom theming via attributes is on the roadmap.

Does the component support lazy loading for pages with many diagrams?

Yes — it uses IntersectionObserver internally to defer fetching and rendering until the element is near the viewport. Put 10 diagrams on a page and only the visible ones actually load. The rest wait until you scroll to them. This makes diagram-heavy runbooks and wikis load fast enough to be practical.

Can I use the embed component with React, Vue, or other frameworks?

Web components work natively in all modern browsers, so any framework is fine. React treats it like any other HTML element. Vue might show an "unknown element" warning — add infra-sketch to compilerOptions.isCustomElement in your Vite/Vue config to silence it. No npm package needed — just the script tag.

Turn your IaC docs into living architecture diagrams Add one script tag to your docs site and your diagrams stay synchronized with your code automatically. Try it in the browser first — no account needed. Try InfraSketch Free →

Cloud Cost Estimation with Infracost: How It Works and What It Covers

Raghvendra Pandey — Tue, 12 May 2026 01:21:28 +0000

Cloud infrastructure bills are notoriously hard to predict. A Terraform change that looks like a small configuration tweak — upgrading an RDS instance class, enabling Multi-AZ, adding a NAT gateway — can add hundreds of dollars per month. Without tooling, engineers discover the cost impact after the change is deployed and the first bill arrives.

Infracost bridges the gap between infrastructure code and cost. It reads your Terraform (or Terragrunt) and queries cloud provider pricing APIs to generate a cost breakdown before anything is deployed. This guide covers how Infracost works, what it can and can't estimate, how to run it, and how to wire it into a CI/CD pipeline so cost changes appear on every pull request.

How Infracost works

Infracost has three main phases:

1. Parse the Terraform project. Infracost runs terraform init and terraform plan internally, generating a plan JSON file. It reads resource types, instance types, storage sizes, replication settings, and other cost-relevant attributes from the plan output.

2. Look up pricing. Infracost maintains a pricing database synced from AWS, GCP, and Azure price lists. It queries this database to find the on-demand price for each resource configuration. For usage-based resources (data transfer, Lambda invocations, DynamoDB read/write units), it applies usage estimates from a infracost-usage.yml file or uses zero as a baseline.

3. Generate the output. The result is a breakdown of monthly costs per resource, organized by module and resource type, plus a diff view when comparing two Terraform states.

$ infracost breakdown --path .

Name                                      Monthly Qty  Unit          Monthly Cost

aws_instance.web
Instance usage (Linux/UNIX, on-demand)          730  hours              $29.20
root_block_device
Storage (general purpose SSD, gp3)             50  GB                  $4.00

aws_db_instance.main
Database instance (db.t3.medium, Multi-AZ)      730  hours             $118.26
Storage (general purpose SSD, gp2)              100  GB                 $23.00

aws_nat_gateway.main
NAT gateway                                      730  hours              $32.85
Data processed                                     0  GB                  $0.00

OVERALL TOTAL                                                             $207.31

What Infracost can estimate

Infracost covers most compute, storage, and database resources where pricing is instance-type or size based:

EC2 instances — all instance families, Linux/Windows pricing, on-demand rates
RDS instances — all DB engines and instance classes, Multi-AZ, storage, IOPS
EKS — cluster cost, node group EC2 instances, Fargate task pricing
S3 — standard storage pricing (per GB), request costs with usage estimates
CloudFront — data transfer with usage estimates
ElastiCache — Redis/Memcached node pricing
OpenSearch — instance pricing, storage
NAT Gateway — hourly cost, data processing with usage estimates
Load balancers — ALB/NLB hourly cost, LCU usage
Lambda — requires usage estimates for invocation count and duration
DynamoDB — on-demand vs provisioned capacity, requires usage estimates for on-demand

What Infracost cannot estimate

Some costs are unknowable from Terraform configuration alone because they depend entirely on runtime behavior:

Data transfer between regions or to the internet (volume varies with traffic)
CloudWatch log storage and ingestion (volume depends on application log verbosity)
API Gateway invocations (depends on request volume)
Spot instance interruption rates
Reserved instance or Savings Plans discounts (Infracost uses on-demand rates)

Infracost marks these as $0.00 with a note that usage estimates are needed, which is accurate — zero is the floor, not the expected cost.

Usage estimates

For usage-based resources, Infracost reads a infracost-usage.yml file where you specify expected usage:

# infracost-usage.yml
version: 0.1

resource_usage:
aws_lambda_function.api:
monthly_requests: 5000000       # 5M invocations/month
request_duration_ms: 200        # average 200ms per invocation
memory_mb: 512

aws_cloudfront_distribution.cdn:
monthly_data_transfer_to_internet_gb: 500

aws_nat_gateway.main:
monthly_data_processed_gb: 100

aws_dynamodb_table.sessions:
monthly_read_request_units: 1000000
monthly_write_request_units: 200000

infracost breakdown --path . --usage-file infracost-usage.yml

Usage estimates are approximate by definition, but even rough estimates make cost modeling far more useful than showing $0.00 for all usage-based resources. Teams often store a shared infracost-usage.yml alongside production usage metrics, updated quarterly.

Running Infracost

Install

# macOS
brew install infracost

# Linux
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh

# Register for a free API key (rate limiting, no data stored)
infracost auth login

Basic cost breakdown

# Show costs for current Terraform directory
infracost breakdown --path .

# Output JSON for programmatic use
infracost breakdown --path . --format json --out-file infracost.json

# Show a diff between two Terraform states
infracost diff --path . --compare-to infracost-base.json

Cost diff between branches

The diff workflow is the most useful for pull request review: compare the base branch cost against the PR branch cost and surface only the delta.

# On the base branch, generate baseline
git checkout main
infracost breakdown --path . --format json --out-file infracost-base.json

# On the PR branch, generate diff
git checkout feature/upgrade-rds
infracost diff --path . --compare-to infracost-base.json

Project: my-infrastructure

+ aws_db_instance.main
+ Database instance class changed: db.t3.medium to db.r6g.large
+$168.44 (+142%)

~ aws_instance.web (no change)
~ aws_nat_gateway.main (no change)

Monthly cost change: +$168.44 (from $207.31 to $375.75)

CI/CD integration

Infracost's most valuable feature is the PR comment: on every pull request that changes Terraform, Infracost posts a cost diff directly in the PR. Engineers see the impact before merging.

# .github/workflows/infracost.yml
name: Infracost Cost Estimate
on: [pull_request]

jobs:
infracost:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write

steps:
- uses: actions/checkout@v4

- name: Setup Infracost
uses: infracost/actions/setup@v3
with:
api-key: ${{ secrets.INFRACOST_API_KEY }}

- name: Generate cost estimate for base branch
run: |
git fetch origin ${{ github.base_ref }}
git checkout origin/${{ github.base_ref }}
infracost breakdown --path . \
--format json \
--out-file /tmp/infracost-base.json

- name: Generate cost diff for PR branch
run: |
git checkout ${{ github.sha }}
infracost diff --path . \
--format json \
--compare-to /tmp/infracost-base.json \
--out-file /tmp/infracost-diff.json

- name: Post PR comment
run: |
infracost comment github \
--path /tmp/infracost-diff.json \
--repo $GITHUB_REPOSITORY \
--github-token ${{ secrets.GITHUB_TOKEN }} \
--pull-request ${{ github.event.pull_request.number }} \
--behavior update

The --behavior update flag updates the existing Infracost comment rather than creating a new one on each commit — keeps the PR clean.

Failing CI on large cost increases

You can configure Infracost to fail CI when a PR increases costs beyond a threshold. This works well as a guardrail for staging environments or cost-sensitive teams:

infracost comment github \
--path /tmp/infracost-diff.json \
--github-token ${{ secrets.GITHUB_TOKEN }} \
--pull-request ${{ github.event.pull_request.number }} \
--behavior update \
--policy-path infracost-policy.rego

# infracost-policy.rego — fail CI if monthly cost increase > $100
package infracost

deny[msg] {
maxDiff := 100.0
input.diffTotalMonthlyCost > maxDiff
msg := sprintf(
"Monthly cost increase ($%.2f) exceeds threshold ($%.2f)",
[input.diffTotalMonthlyCost, maxDiff]
)
}

Cost optimization patterns

Infracost makes cost tradeoffs explicit, which is most useful when evaluating architectural decisions. Common patterns where it helps:

Right-sizing instances

Oversized instances are one of the most common sources of waste. Infracost makes the cost difference between instance types visible before deployment:

db.r6g.xlarge (~$480/month) vs db.r6g.large (~$240/month) — if workload fits, this is pure savings
Multi-AZ doubles RDS cost — necessary for production, wasteful for staging
gp3 storage is consistently cheaper than gp2 for equivalent IOPS

NAT Gateway costs

NAT Gateway charges both per-hour ($0.045/hour = ~$33/month) and per-GB processed. For high-throughput workloads, data processing costs dominate. Common mitigation: VPC endpoints for S3 and DynamoDB bypass the NAT Gateway entirely, eliminating data processing charges for traffic to those services.

resource "aws_vpc_endpoint" "s3" {
vpc_id          = aws_vpc.main.id
service_name    = "com.amazonaws.${var.region}.s3"
route_table_ids = [aws_route_table.private.id]
}

Environment-specific sizing

Using production instance sizes in staging and dev environments is a common source of unnecessary spend. Infracost makes the cost difference concrete:

variable "instance_class" {
default = "db.t3.micro"  # $13/month in dev
}

# Production tfvars: instance_class = "db.r6g.large"  # $240/month

Storage class selection

S3 Intelligent-Tiering automatically moves objects to cheaper storage tiers based on access frequency. For large buckets with unpredictable access patterns, the monitoring fee ($0.0025/1000 objects) is typically far less than the savings from automatic tiering. Infracost can model this with usage estimates for access frequency.

Multi-project and monorepo setups

For repos with multiple Terraform projects, Infracost can aggregate costs across all of them:

# infracost.yml — project configuration
version: 0.1

projects:
- path: ./environments/production
name: Production
- path: ./environments/staging
name: Staging
- path: ./modules/shared-networking
name: Shared Networking

infracost breakdown --config-file infracost.yml --format table

The output shows each project's costs separately and a combined total — useful for understanding the cost split across environments or teams.

Reading Infracost output alongside your architecture

Cost numbers are more actionable when connected to the architecture they describe. When working with Terraform that has both a Checkov security scan and an Infracost estimate, InfraSketch lets you visualize the infrastructure from the same Terraform source: paste your configuration to generate an architecture diagram, then overlay the Infracost JSON output to see per-resource cost annotations directly on the diagram. This makes it easier to communicate cost breakdowns to stakeholders who aren't reading raw JSON.