3 Tier Application On EKS

I built a 3-tier application running on Amazon EKS, deployed by a GitHub Actions pipeline that runs on every merge to main. The first version of that pipeline had two secrets sitting in the repo settings: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

Github- https://github.com/itzmayank01/3-tier-user-platform-devops

They worked. They were also a bad idea, and getting rid of them taught me more about EKS authentication than anything else in the project.

This post is the setup I ended up with. It includes the part that broke for me and that most tutorials skip: getting past IAM is only half the job, because EKS has its own authorization layer on top.

Why long-lived keys are a problem

An IAM access key does not expire. If it leaks, it stays valid until someone notices and revokes it. In a CI pipeline that risk is real:

• The key lives in repository settings, and anyone with admin access to the repo can see who added it
• A malicious or careless workflow change can print it, exfiltrate it, or use it for something unrelated
• Rotating it means updating every repo that uses it, so in practice nobody rotates it
• A fork or a compromised third-party action widens the blast radius

OIDC replaces that with a token that lives for the length of a single job.

How OIDC actually works here

Four steps, and it helps to hold the whole picture in your head before you start:

1. GitHub Actions mints a short-lived JSON Web Token describing the job: which repo, which branch, which workflow
2. Your workflow sends that token to AWS STS
3. AWS checks the token signature against a registered identity provider, then checks the claims inside it against your IAM role's trust policy
4. If both pass, STS hands back temporary credentials that expire when the job ends

No secret is stored anywhere. The trust is in the claims, not in a shared string.

Step 1: Register GitHub as an identity provider in IAM

You only do this once per AWS account.

aws iam create-open-id-connect-provider \
  --url https://token.actions.githubusercontent.com \
  --client-id-list sts.amazonaws.com

If you have used GitHub OIDC before, you may remember passing a certificate thumbprint. AWS no longer requires you to manage that for this provider - it verifies and rotates the certificate itself. If your tooling still demands a value, older guides use 6938fd4d98bab03faadb97b34396831e3780aea1.

Check that it landed:

aws iam list-open-id-connect-providers

Step 2: Create the IAM role and its trust policy

This is where most of the security lives. The trust policy decides which GitHub jobs are allowed to assume this role.

trust-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::111122223333:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:my-org/my-repo:ref:refs/heads/main"
        }
      }
    }
  ]
}

aws iam create-role \
  --role-name GitHubActionsEKSDeployRole \
  --assume-role-policy-document file://trust-policy.json

Read the sub condition twice. It is the only thing standing between "my main branch can deploy" and "anyone who opens a pull request against my repo can deploy". Some patterns and what they mean:

Pattern	Who can assume the role
repo:my-org/my-repo:ref:refs/heads/main	Only jobs running on the main branch
repo:my-org/my-repo:environment:production	Only jobs targeting the production environment
repo:my-org/my-repo:pull_request	Any pull request, including from forks
repo:my-org/*	Every repo in the org
repo:*	Anyone on GitHub. Never do this.

I use the environment form for production, because GitHub environments let me add a required reviewer on top. IAM enforces which jobs can assume the role; GitHub enforces who can trigger those jobs.

Then attach permissions. The deploy role needs very little - enough to look up the cluster endpoint:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "eks:DescribeCluster",
      "Resource": "arn:aws:eks:ap-south-1:111122223333:cluster/my-cluster"
    }
  ]
}

If the same job also pushes images, add the ECR actions it needs (ecr:GetAuthorizationToken, ecr:BatchCheckLayerAvailability, ecr:PutImage, ecr:InitiateLayerUpload, ecr:UploadLayerPart, ecr:CompleteLayerUpload). I prefer two separate roles - one for build and push, one for deploy - so a compromised build step cannot touch the cluster.

Step 3: The workflow

name: Deploy to EKS

on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::111122223333:role/GitHubActionsEKSDeployRole
          role-session-name: gha-deploy-${{ github.run_id }}
          aws-region: ap-south-1

      - name: Update kubeconfig
        run: aws eks update-kubeconfig --name my-cluster --region ap-south-1

      - name: Deploy
        run: |
          kubectl set image deployment/frontend \
            frontend=111122223333.dkr.ecr.ap-south-1.amazonaws.com/frontend:${{ github.sha }}
          kubectl rollout status deployment/frontend --timeout=120s

Two things to notice.

permissions: id-token: write is not optional. Without it, GitHub never mints the token and the credentials step fails with something like Could not assume role with OIDC. This is the single most common failure, and the error message does not point at the missing permission. Also note that declaring a permissions block resets all permissions to none, so you have to list contents: read explicitly if your job checks out code.

Tag images by commit SHA, not latest. ${{ github.sha }} makes every deploy traceable to a commit, and rolling back is just pointing at the previous SHA.

Step 4: The part tutorials skip

At this point my pipeline authenticated to AWS perfectly and then died on the kubectl step:

error: You must be logged in to the server (Unauthorized)

The credentials were fine. The problem is that EKS has two separate layers. IAM decides whether you can talk to the cluster's API endpoint. Kubernetes RBAC decides what you can do once you are there. A brand new IAM role has an AWS identity and no Kubernetes identity at all.

You have to explicitly map the role into the cluster.

The modern way: EKS access entries

Access entries are an EKS API, so you manage cluster access with the same tooling as everything else. First check what your cluster is using:

aws eks describe-cluster --name my-cluster --query 'cluster.accessConfig'

If authenticationMode is CONFIG_MAP, switch it:

aws eks update-cluster-config \
  --name my-cluster \
  --access-config authenticationMode=API_AND_CONFIG_MAP

Then create the entry and give it a scope:

aws eks create-access-entry \
  --cluster-name my-cluster \
  --principal-arn arn:aws:iam::111122223333:role/GitHubActionsEKSDeployRole \
  --type STANDARD

aws eks associate-access-policy \
  --cluster-name my-cluster \
  --principal-arn arn:aws:iam::111122223333:role/GitHubActionsEKSDeployRole \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSEditPolicy \
  --access-scope type=namespace,namespaces=production

AmazonEKSEditPolicy scoped to one namespace is usually right for a deploy role. It can update workloads and nothing else. Resist reaching for AmazonEKSClusterAdminPolicy - it is the AdministratorAccess of EKS.

One direction-of-travel note: you can move CONFIG_MAP to API_AND_CONFIG_MAP to API, but you cannot go back. Migrate deliberately.

The older way: the aws-auth ConfigMap

If you are on an older cluster still using CONFIG_MAP mode, you edit aws-auth in kube-system:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: arn:aws:iam::111122223333:role/GitHubActionsEKSDeployRole
      username: github-actions
      groups:
        - deployers

Then bind deployers to a Role or ClusterRole with normal Kubernetes RBAC. Two warnings if you go this route: a bad edit here can lock everyone out of the cluster, including you, and if your IAM role has a path (/ci/GitHubActionsEKSDeployRole), you must strip the path out of the ARN before EKS will match it.

Gotchas worth knowing before you start

• Could not assume role with OIDC - the missing id-token: write permission, nine times out of ten
• Not authorized to perform sts:AssumeRoleWithWebIdentity - your sub claim does not match. Print github.ref and github.workflow_ref in a debug step and compare them character by character with the trust policy
• You must be logged in to the server (Unauthorized) - IAM is fine, the cluster mapping is missing. This is Step 4
• kubectl hangs, then times out - your cluster API endpoint is private and a hosted runner cannot reach it. You need a self-hosted runner inside the VPC, or a VPC-connected alternative
• The role works from your laptop but not from CI - you are probably testing with a user that has broader permissions than the role. Assume the role locally with aws sts assume-role and retest

What you actually get

After the migration my repo has zero AWS secrets. The credentials a job holds are valid for that job only, scoped to one namespace, on one branch. If someone gets a malicious workflow merged, the worst case is bounded by an IAM policy and a Kubernetes RBAC binding instead of by whatever those static keys happened to be allowed to do.

That is a much better position to defend, and it took an afternoon.

Next in this series: cutting that same pipeline from five minutes to two and a half with self-hosted runners and parallel stages.

---