From Zero to a Working EKS Pipeline: Terraform, Ansible, and GitLab CI/CD (and Everything That Broke Along the Way)

I recently built an end-to-end deployment pipeline on AWS EKS using Terraform for infrastructure, Ansible for configuration, and GitLab CI/CD to tie it all together. On paper, that sentence sounds clean. In practice, it took several rounds of "why is this failing" before it actually worked.

This post is not a "here's how EKS works" tutorial. There are plenty of those. This is the version with the failures left in, the quota limits, the IAM permission walls, the pods that wouldn't schedule, and the resources that refused to die. If you're building something similar, I'm hoping this saves you a few hours of confused Googling.

Repo: gitlab.com/nenyeonyema/terraform-eks-ansible-cicd

What I Was Building

The goal was a full IaC-driven pipeline:

• Terraform to provision the EKS cluster and supporting AWS infrastructure (VPC, node groups, IAM roles)
• Ansible to handle configuration tasks on top of the provisioned infrastructure
• GitLab CI/CD to automate the whole thing — plan, apply, configure, deploy — on every push

Simple enough in theory. Four separate blockers said otherwise.

Blocker #1: Free Tier ASG Restrictions

The first wall I hit was with the Auto Scaling Group for my EKS node group. AWS Free Tier limits how much compute you can provision, and my initial node group sizing quietly ran into those limits — the kind of failure that doesn't always throw an obvious, single-line error.

Fix: I resized the node group to stay within Free Tier boundaries and got explicit about instance types and desired/min/max capacity in Terraform, instead of leaving Auto Scaling to make assumptions I couldn't afford.

Lesson: If you're building on Free Tier, hardcode your capacity expectations early. Don't let the defaults surprise you later.

Blocker #2: EKS Private Endpoint Access

By default, EKS clusters can be configured with private-only API server endpoint access. That's great for security, less great when your CI/CD runner or local machine suddenly can't reach the cluster to run kubectl commands.

Fix: I had to explicitly configure endpoint access (public, private, or both, depending on where the access was coming from) in the Terraform EKS module configuration, rather than relying on defaults.

Lesson: Decide your access model on purpose, not by accident. "It won't connect" is often a networking/endpoint configuration issue before it's anything else.

Blocker #3: The IAM Access Entry API

This one cost the most time. Managing who (which IAM roles/users) can access the EKS cluster used to be handled through the aws-auth ConfigMap. AWS has since introduced the Access Entry API as the newer approach to cluster access management, and the two don't always play nicely if you're mixing old tutorials with new AWS provider versions.

Fix: I moved to using the Access Entry API directly through Terraform's EKS resources instead of manually patching aws-auth, which meant re-reading provider documentation carefully since a lot of existing tutorials online still reference the older ConfigMap-based method.

Lesson: When Terraform provider behavior doesn't match a tutorial, check the provider version and changelog before assuming you made a mistake. AWS and the community are actively moving away from aws-auth.

Blocker #4: t3.micro Pod Limits

Even after the cluster came up, deployments started failing to schedule. The problem wasn't compute, it was pod density. Instance types like t3.micro have a maximum number of pods they can run based on their networking capacity (ENI/IP limits), not just CPU/RAM.

Fix: I scaled from a single-node setup to multiple nodes, spreading pods across more machines instead of trying to force everything onto one small instance.

Lesson: "Not enough compute" and "not enough pod slots" look identical from the outside. If pods are stuck in Pending and your node has CPU/memory to spare, check max-pods-per-node before you check anything else.

Blocker #5: Orphaned ELBs Blocking terraform destroy

The most frustrating one. After deploying a Kubernetes LoadBalancer service, Kubernetes provisions an AWS ELB behind the scenes — but Terraform doesn't know about it, because it wasn't the one that created it. Result: running terraform destroy would hang or fail, because AWS wouldn't let Terraform tear down the VPC/subnets while an orphaned ELB was still attached to them.

Fix: Before running terraform destroy, I had to manually delete any Kubernetes-created LoadBalancer services first (kubectl delete svc <service-name>), which triggers AWS to clean up the associated ELB. Only then would terraform destroy complete cleanly.

Lesson: Anything Kubernetes provisions on your behalf (LoadBalancers, EBS volumes via PVCs) lives outside Terraform's state. Always tear down Kubernetes-managed cloud resources before tearing down the infrastructure underneath them.

What I'd Do Differently Next Time

• Pin provider versions early. Several of these issues came from AWS's EKS access model evolving faster than the tutorials I was learning from.
• Build a teardown checklist, not just a build checklist. terraform destroy needs its own runbook when Kubernetes is in the mix.
• Size for pod density, not just compute. I'll check max-pods-per-instance-type before picking an instance size going forward.

Why This Matters

None of these blockers are exotic. They're the ordinary friction of running Terraform, Ansible, and Kubernetes together in a real AWS account — which is exactly why I think they're worth writing down. The polished tutorials rarely mention them, but if you're doing this hands-on, you will hit at least one of these five.

If you're working through something similar, or you've hit a different version of one of these walls, I'd love to hear about it — drop a comment or reach out.

Repo: gitlab.com/nenyeonyema/terraform-eks-ansible-cicd