DEV Community

Cover image for Building an Amazon EKS Cluster with raw Terraform Resources
AWS Community Builders profile image Chinmay Tonape
Chinmay Tonape for AWS Community Builders

Posted on

Building an Amazon EKS Cluster with raw Terraform Resources

#kubernetes #aws #eks #terraform

Most guides for provisioning Amazon EKS use the AWS community Terraform module for convenience. While this is great for speed, it often abstracts away what’s actually happening behind the scenes.

In this blog, we will build an EKS cluster from scratch using raw Terraform resources (without using the community module). By doing this, you’ll gain a clear understanding of the AWS networking, IAM, and compute resources that make up a Kubernetes cluster on AWS.

Finally, we’ll deploy a sample microservices application (Voting App) on the cluster and access it using port-forwarding.

Architecture

Here’s how the setup works at a high level:

  • VPC is created with 2 Availability Zones for high availability.
  • Each AZ contains both a public and a private subnet.
  • EKS worker nodes (EC2 instances) are launched in private subnets for better security.
  • A NAT Gateway is provisioned in a public subnet to allow worker nodes in private subnets to pull images and updates from the internet (e.g., from ECR, Docker Hub).
  • EKS control plane (managed by AWS) communicates with the worker nodes securely within the VPC. This setup ensures that your nodes are not directly exposed to the internet while still having outbound internet access via the NAT gateway.

Step 1: Create a VPC with Public and Private Subnets

EKS requires a properly configured VPC with both public and private subnets. Worker nodes generally live in private subnets, while the NAT gateway in public subnets provides outbound internet connectivity.

We also add Internet Gateway (IGW), NAT Gateway, and route tables to handle connectivity between public/private subnets.

################################################################################
# VPC
################################################################################
resource "aws_vpc" "custom_vpc" {
  cidr_block = var.networking.cidr_block

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-VPC"
  })
}

################################################################################
# PUBLIC SUBNETS
################################################################################
resource "aws_subnet" "public_subnets" {
  count                   = var.networking.public_subnets == null || var.networking.public_subnets == "" ? 0 : length(var.networking.public_subnets)
  vpc_id                  = aws_vpc.custom_vpc.id
  cidr_block              = var.networking.public_subnets[count.index]
  availability_zone       = var.networking.azs[count.index]
  map_public_ip_on_launch = true

  tags = merge(var.common_tags, {
    Name                                        = "${var.naming_prefix}-public-subnet-${count.index}"
    "kubernetes.io/role/elb"                    = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  })
}

################################################################################
# PRIVATE SUBNETS
################################################################################
resource "aws_subnet" "private_subnets" {
  count                   = var.networking.private_subnets == null || var.networking.private_subnets == "" ? 0 : length(var.networking.private_subnets)
  vpc_id                  = aws_vpc.custom_vpc.id
  cidr_block              = var.networking.private_subnets[count.index]
  availability_zone       = var.networking.azs[count.index]
  map_public_ip_on_launch = false

  tags = merge(var.common_tags, {
    Name                                        = "${var.naming_prefix}-private-subnet-${count.index}"
    "kubernetes.io/role/internal-elb"           = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  })
}

################################################################################
# INTERNET GATEWAY
################################################################################
resource "aws_internet_gateway" "i_gateway" {
  vpc_id = aws_vpc.custom_vpc.id

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-i-gateway"
  })
}

################################################################################
# EIPs
################################################################################
resource "aws_eip" "elastic_ip" {
  count      = var.networking.private_subnets == null || var.networking.nat_gateways == false ? 0 : length(var.networking.private_subnets)
  depends_on = [aws_internet_gateway.i_gateway]

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-eip-${count.index}"
  })
}

################################################################################
# NAT GATEWAYS
################################################################################
resource "aws_nat_gateway" "nats" {
  count             = var.networking.private_subnets == null || var.networking.nat_gateways == false ? 0 : length(var.networking.private_subnets)
  subnet_id         = aws_subnet.public_subnets[count.index].id
  connectivity_type = "public"
  allocation_id     = aws_eip.elastic_ip[count.index].id
  depends_on        = [aws_internet_gateway.i_gateway]

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-nat-gw-${count.index}"
  })
}

################################################################################
# PUBLIC ROUTE TABLE
################################################################################
resource "aws_route_table" "public_table" {
  vpc_id = aws_vpc.custom_vpc.id
}

resource "aws_route" "public_routes" {
  route_table_id         = aws_route_table.public_table.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.i_gateway.id
}

resource "aws_route_table_association" "assoc_public_routes" {
  count          = length(var.networking.public_subnets)
  subnet_id      = aws_subnet.public_subnets[count.index].id
  route_table_id = aws_route_table.public_table.id
}

################################################################################
# PRIVATE ROUTE TABLES
################################################################################
resource "aws_route_table" "private_tables" {
  count  = length(var.networking.azs)
  vpc_id = aws_vpc.custom_vpc.id
}

resource "aws_route" "private_routes" {
  count                  = length(var.networking.private_subnets)
  route_table_id         = aws_route_table.private_tables[count.index].id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = aws_nat_gateway.nats[count.index].id
}

resource "aws_route_table_association" "assoc_private_routes" {
  count          = length(var.networking.private_subnets)
  subnet_id      = aws_subnet.private_subnets[count.index].id
  route_table_id = aws_route_table.private_tables[count.index].id
}

################################################################################
# SECURITY GROUPS
################################################################################
resource "aws_security_group" "sec_groups" {
  for_each    = { for sec in var.security_groups : sec.name => sec }
  name        = "${var.naming_prefix}-${each.value.name}"
  description = each.value.description
  vpc_id      = aws_vpc.custom_vpc.id

  dynamic "ingress" {
    for_each = try(each.value.ingress, [])
    content {
      description      = ingress.value.description
      from_port        = ingress.value.from_port
      to_port          = ingress.value.to_port
      protocol         = ingress.value.protocol
      cidr_blocks      = ingress.value.cidr_blocks
      ipv6_cidr_blocks = ingress.value.ipv6_cidr_blocks
    }
  }

  dynamic "egress" {
    for_each = try(each.value.egress, [])
    content {
      description      = egress.value.description
      from_port        = egress.value.from_port
      to_port          = egress.value.to_port
      protocol         = egress.value.protocol
      cidr_blocks      = egress.value.cidr_blocks
      ipv6_cidr_blocks = egress.value.ipv6_cidr_blocks
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Step 2: Create IAM Roles and Policies

EKS needs roles for both the control plane and worker nodes.

  • Cluster Role – Grants permissions for EKS to manage AWS resources.
  • Node Role – Allows EC2 instances (worker nodes) to join the cluster. 
################################################################################
# EKS CLUSTER ROLE
################################################################################
resource "aws_iam_role" "EKSClusterRole" {
  name = "${var.naming_prefix}-EKSClusterRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "eks.amazonaws.com"
        }
      },
    ]
  })
}

// This policy provides Kubernetes the permissions it requires to manage resources on your behalf. 
// Kubernetes requires Ec2:CreateTags permissions to place identifying information on EC2 resources
// including but not limited to Instances, Security Groups, and Elastic Network Interfaces
resource "aws_iam_role_policy_attachment" "AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.EKSClusterRole.name
}

################################################################################
# NODE GROUP ROLE
################################################################################
resource "aws_iam_role" "NodeGroupRole" {
  name = "${var.naming_prefix}-EKSNodeGroupRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

//This policy allows Amazon EKS worker nodes to connect to Amazon EKS Clusters.
resource "aws_iam_role_policy_attachment" "AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.NodeGroupRole.name
}

// Provides read-only access to Amazon EC2 Container Registry repositories.
resource "aws_iam_role_policy_attachment" "AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.NodeGroupRole.name
}

// This policy provides the Amazon VPC CNI Plugin (amazon-vpc-cni-k8s) the permissions
// it requires to modify the IP address configuration on your EKS worker nodes. 
// This permission set allows the CNI to list, describe, and modify ENIs on your behalf
resource "aws_iam_role_policy_attachment" "AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.NodeGroupRole.name
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Provision the EKS Cluster

Following creates the EKS control plane.

################################################################################
# EKS Cluster
################################################################################
resource "aws_eks_cluster" "eks-cluster" {
  name     = var.cluster_config.name
  role_arn = aws_iam_role.EKSClusterRole.arn
  version  = var.cluster_config.version

  vpc_config {
    subnet_ids         = flatten([var.public_subnets_id, var.private_subnets_id])
    security_group_ids = flatten(var.security_groups_id)
  }

  depends_on = [
    aws_iam_role_policy_attachment.AmazonEKSClusterPolicy
  ]

}
Enter fullscreen mode Exit fullscreen mode

At this point, we create a managed set of EC2 worker nodes with basic required addons

################################################################################
# NODE GROUP
################################################################################
resource "aws_eks_node_group" "node-ec2" {
  for_each        = { for node_group in var.node_groups : node_group.name => node_group }
  cluster_name    = aws_eks_cluster.eks-cluster.name
  node_group_name = "${var.naming_prefix}-${each.value.name}"
  node_role_arn   = aws_iam_role.NodeGroupRole.arn
  subnet_ids      = flatten(var.private_subnets_id)

  scaling_config {
    desired_size = try(each.value.scaling_config.desired_size, 2)
    max_size     = try(each.value.scaling_config.max_size, 3)
    min_size     = try(each.value.scaling_config.min_size, 1)
  }

  update_config {
    max_unavailable = try(each.value.update_config.max_unavailable, 1)
  }

  ami_type       = each.value.ami_type
  instance_types = each.value.instance_types
  capacity_type  = each.value.capacity_type
  disk_size      = each.value.disk_size
  version        = var.cluster_config.version

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-nodes-${each.value.name}"
  })

  depends_on = [
    aws_iam_role_policy_attachment.AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.AmazonEC2ContainerRegistryReadOnly,
    aws_iam_role_policy_attachment.AmazonEKS_CNI_Policy
  ]
}

resource "aws_iam_openid_connect_provider" "eks_oidc_provider" {
  url             = aws_eks_cluster.eks-cluster.identity[0].oidc[0].issuer
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = ["9e99a48a9960b14926bb7f3b02e22da2b0ab7280"]
}


################################################################################
# EKS Addons
################################################################################
data "aws_eks_addon_version" "eks_addons" {
  for_each = var.addons

  addon_name         = each.value.name
  kubernetes_version = aws_eks_cluster.eks-cluster.version
  most_recent        = coalesce(each.value.most_recent, true)
}

resource "aws_eks_addon" "addons" {
  for_each = var.addons

  cluster_name                = aws_eks_cluster.eks-cluster.name
  addon_name                  = each.value.name
  addon_version               = coalesce(each.value.version, data.aws_eks_addon_version.eks_addons[each.key].version)
  resolve_conflicts_on_create = "OVERWRITE"

  tags = merge(var.common_tags, {
    Name = "${var.naming_prefix}-addon-${each.value.name}"
  })

  depends_on = [aws_eks_node_group.node-ec2]
}
Enter fullscreen mode Exit fullscreen mode

Step 4: Configure kubectl and Access Cluster 

################################################################################
# Update kubeconfig with created cluster
################################################################################
resource "null_resource" "update_kubeconfig" {
  provisioner "local-exec" {
    command = "aws eks --region ${var.aws_region} update-kubeconfig --name ${var.cluster_config.name}"
  }
  depends_on = [module.eks]
}
Enter fullscreen mode Exit fullscreen mode

Step 5: Deploy a Sample Microservices Application

We will deploy famous exmaple voting app from the https://github.com/dockersamples/example-voting-app. We will just deploy the manifests and not look into building the docker images at this moment.

  • A front-end web app in Python which lets you vote between two options
  • A Redis which collects new votes
  • A .NET worker which consumes votes and stores them in…
  • A Postgres database backed by a Docker volume
  • A Node.js web app which shows the results of the voting in real time

Note: The voting application only accepts one vote per client browser. It does not register additional votes if a vote has already been submitted from a client.

Apply the manifests and verify

cd /example-voting-app/k8s-specifications
kubectl apply -f .
deployment.apps/db created
service/db created
deployment.apps/redis created
service/redis created
deployment.apps/result created
service/result created
deployment.apps/vote created
service/vote created
deployment.apps/worker created


kubectl port-forward service/vote 8080:8080
kubectl port-forward service/result 8081:8081
Enter fullscreen mode Exit fullscreen mode

Access app using:

Vote:   http://localhost:8080
Result: http://localhost:8081
Enter fullscreen mode Exit fullscreen mode

Cleanup

Delete the k8s resources created

cd /example-voting-app/k8s-specifications
kubectl delete -f .
Enter fullscreen mode Exit fullscreen mode

And then terraform destroy the EKS infrastructure if you are not using it to save costs.

Conclusion

By building an EKS cluster from scratch with Terraform—without relying on the AWS community module—we gain a clear understanding of the essential AWS resources required to run Kubernetes in production.

We saw how networking forms the foundation, how the cluster and node groups tie everything together, and finally how workloads are deployed and accessed. This exercise provides not only a strong foundation in AWS and Kubernetes but also the confidence to customize infrastructure for real-world production use cases.

References

Top comments (0)