DEV Community: MD Nur Ahmed

Different Ways to Manage AWS IAM to Connect to an EKS Cluster

MD Nur Ahmed — Fri, 12 Dec 2025 11:45:13 +0000

Managing access to Amazon EKS clusters can be confusing. Should you use IAM roles? What about service accounts? How do the new EKS Access Entries differ from the old aws-auth ConfigMap? If you've ever been overwhelmed by these questions, you're not alone.

Think of EKS access management as the security checkpoint at an airport. IAM is your passport (authentication), and EKS access policies are your boarding pass (authorization). You need both to get on the plane, and different passengers need different levels of access.

In this comprehensive guide, I'll walk you through everything you need to know about IAM for EKS, complete with four hands-on scenarios and working Terraform code you can deploy today.

What You'll Learn

✅ How IAM authentication and authorization works with EKS
✅ The OLD WAY (aws-auth ConfigMap) vs NEW WAY (Access Entries)
✅ Setting up developer access with read-only permissions
✅ Configuring CI/CD pipelines (GitHub Actions, GitLab, Jenkins)
✅ Implementing IRSA (IAM Roles for Service Accounts)
✅ Production-ready best practices

GitHub Repository: eks-iam-hands-on

All code examples in this post are fully tested and ready to deploy! 🚀

Understanding IAM with EKS
The Old Way vs The New Way
Scenario 1: Basic EKS Cluster Setup
Scenario 2: Developer Read-Only Access
Scenario 3: CI/CD Pipeline Access
Scenario 4: IRSA for Pods
Best Practices
Common Pitfalls
Conclusion

Understanding IAM with EKS

Before diving into the code, let's understand how IAM works with EKS.

Two Layers of Access Control

EKS uses two separate layers of access control:

1. AWS IAM (Authentication)

"Who are you?"
Verifies your AWS identity
Happens at the AWS API level

2. Kubernetes RBAC (Authorization)

"What can you do?"
Determines your permissions inside the cluster
Happens at the Kubernetes API level

┌──────────────────────────────────────────────────┐
│  User/Role/Service Account                       │
│  (Your AWS Identity)                             │
└────────────────┬─────────────────────────────────┘
                 │
                 ▼
┌──────────────────────────────────────────────────┐
│  AWS IAM Layer                                   │
│  ✓ Authenticated: You are who you say you are   │
└────────────────┬─────────────────────────────────┘
                 │
                 ▼
┌──────────────────────────────────────────────────┐
│  EKS Access Entry / aws-auth ConfigMap           │
│  ✓ Mapped: Your IAM identity → K8s user/group   │
└────────────────┬─────────────────────────────────┘
                 │
                 ▼
┌──────────────────────────────────────────────────┐
│  Kubernetes RBAC                                 │
│  ✓ Authorized: You can perform specific actions │
└──────────────────────────────────────────────────┘

OIDC: The Bridge Between AWS and Kubernetes

For pods to access AWS services, EKS uses an OIDC (OpenID Connect) provider:

Pod → Kubernetes Service Account → OIDC Token → IAM Role → AWS Service

This eliminates the need for long-lived AWS credentials in your pods!

The Old Way vs The New Way

AWS recently introduced EKS Access Entries as a modern alternative to the aws-auth ConfigMap. Let's compare:

❌ OLD WAY: aws-auth ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: arn:aws:iam::123456789012:role/DeveloperRole
      username: developer
      groups:
        - system:masters  # Too permissive!
  mapUsers: |
    - userarn: arn:aws:iam::123456789012:user/alice
      username: alice
      groups:
        - developers

Problems:

😱 Manual YAML editing (error-prone)
😱 Requires cluster access to modify
😱 No fine-grained permission control
😱 All-or-nothing access (system:masters or nothing)
😱 Poor audit trail

✅ NEW WAY: EKS Access Entries

# Create access entry
resource "aws_eks_access_entry" "developer" {
  cluster_name  = aws_eks_cluster.main.name
  principal_arn = aws_iam_role.developer.arn
  type          = "STANDARD"
}

# Associate managed access policy
resource "aws_eks_access_policy_association" "developer" {
  cluster_name  = aws_eks_cluster.main.name
  principal_arn = aws_iam_role.developer.arn

  # AWS-managed policy for read-only access
  policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy"

  access_scope {
    type       = "namespace"  # Limit to specific namespaces!
    namespaces = ["dev", "staging"]
  }
}

Benefits:

✅ Infrastructure as Code friendly
✅ AWS-managed policies
✅ API-driven (no ConfigMap editing)
✅ Fine-grained permissions
✅ Namespace-scoped access
✅ Better audit via CloudTrail
✅ No cluster access needed

AWS Managed Access Policies

AWS provides three managed policies:

Policy	Kubernetes Equivalent	Use Case
`AmazonEKSClusterAdminPolicy`	`cluster-admin`	Full cluster access
`AmazonEKSAdminPolicy`	`admin`	Admin without cluster-scoped resources
`AmazonEKSEditPolicy`	`edit`	Create/update resources
`AmazonEKSViewPolicy`	`view`	Read-only access

Scenario 1: Basic EKS Cluster Setup

Let's start by creating an EKS cluster using the new Access Entry API.

Architecture

┌─────────────────────────────────────────────────────┐
│                  AWS Account                         │
│                                                      │
│  ┌────────────────┐         ┌────────────────────┐ │
│  │  Admin Role    │────────▶│   EKS Cluster      │ │
│  │  (Your IAM)    │  Access │   + OIDC Provider  │ │
│  └────────────────┘  Entry  │   + Node Group     │ │
│                              └────────────────────┘ │
└─────────────────────────────────────────────────────┘

Step 1: Create the EKS Cluster

Create main.tf:

# VPC for EKS
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "demo-eks-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = true

  # Required tags for EKS
  public_subnet_tags = {
    "kubernetes.io/role/elb" = "1"
  }
  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = "1"
  }
}

# IAM Role for EKS Cluster
resource "aws_iam_role" "eks_cluster_role" {
  name = "demo-eks-cluster-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "eks.amazonaws.com"
      }
    }]
  })
}

# Attach required policies
resource "aws_iam_role_policy_attachment" "eks_cluster_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster_role.name
}

# EKS Cluster with Access Entry API
resource "aws_eks_cluster" "main" {
  name     = "demo-eks-cluster"
  role_arn = aws_iam_role.eks_cluster_role.arn
  version  = "1.28"

  vpc_config {
    subnet_ids = concat(
      module.vpc.private_subnets,
      module.vpc.public_subnets
    )
  }

  # 🎯 KEY: Enable the new Access Entry API
  access_config {
    authentication_mode = "API_AND_CONFIG_MAP"  # Supports both
    bootstrap_cluster_creator_admin_permissions = true
  }

  depends_on = [
    aws_iam_role_policy_attachment.eks_cluster_policy
  ]
}

# Node Group
resource "aws_eks_node_group" "main" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "demo-node-group"
  node_role_arn   = aws_iam_role.eks_node_role.arn
  subnet_ids      = module.vpc.private_subnets

  scaling_config {
    desired_size = 2
    max_size     = 4
    min_size     = 1
  }

  instance_types = ["t3.medium"]
}

Step 2: Create Admin Access

# IAM Role for Admin Access
resource "aws_iam_role" "cluster_admin" {
  name = "demo-eks-admin-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        AWS = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:root"
      }
    }]
  })
}

# 🎯 NEW: Create Access Entry
resource "aws_eks_access_entry" "admin" {
  cluster_name  = aws_eks_cluster.main.name
  principal_arn = aws_iam_role.cluster_admin.arn
  type          = "STANDARD"
}

# 🎯 NEW: Associate Admin Policy
resource "aws_eks_access_policy_association" "admin" {
  cluster_name  = aws_eks_cluster.main.name
  principal_arn = aws_iam_role.cluster_admin.arn

  # Grants full cluster admin access
  policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy"

  access_scope {
    type = "cluster"  # Full cluster access
  }
}

Step 3: Deploy and Test

# Initialize and apply
terraform init
terraform apply

# Configure kubectl
aws eks update-kubeconfig --region us-east-1 --name demo-eks-cluster

# Test access
kubectl get nodes
kubectl get pods -A

Why This Works:

The cluster creator gets automatic admin access
Additional admins are granted via Access Entry
No manual ConfigMap editing required!

Full code: scenario-1-basic-eks/

Scenario 2: Developer Read-Only Access

Now let's grant a developer read-only access to specific namespaces.

Use Case

Junior developers need to view pods and logs
No permission to create/delete resources
Limited to dev and staging namespaces only

Architecture

┌──────────────────────────────────────────────────┐
│  Developer                                       │
│  (IAM User/Role)                                 │
└────────────┬─────────────────────────────────────┘
             │ sts:AssumeRole
             ▼
┌──────────────────────────────────────────────────┐
│  Developer IAM Role                              │
│  + EKS Describe permissions                      │
└────────────┬─────────────────────────────────────┘
             │ Access Entry
             ▼
┌──────────────────────────────────────────────────┐
│  EKS Cluster                                     │
│  Policy: AmazonEKSViewPolicy                     │
│  Scope: namespaces [dev, staging]                │
└──────────────────────────────────────────────────┘

Implementation

# IAM Role for Developers
resource "aws_iam_role" "developer" {
  name = "demo-eks-developer-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        AWS = var.developer_principal_arns
      }
      # 🔒 Security: Require External ID
      Condition = {
        StringEquals = {
          "sts:ExternalId" = var.external_id
        }
      }
    }]
  })
}

# IAM Policy for EKS API Access
resource "aws_iam_policy" "eks_describe" {
  name = "demo-eks-developer-describe-policy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "eks:DescribeCluster",
          "eks:ListClusters"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "developer_eks_describe" {
  role       = aws_iam_role.developer.name
  policy_arn = aws_iam_policy.eks_describe.arn
}

# 🎯 Access Entry for Developer
resource "aws_eks_access_entry" "developer" {
  cluster_name  = var.cluster_name
  principal_arn = aws_iam_role.developer.arn
  type          = "STANDARD"
}

# 🎯 Grant Read-Only Access (Namespace-Scoped)
resource "aws_eks_access_policy_association" "developer" {
  cluster_name  = var.cluster_name
  principal_arn = aws_iam_role.developer.arn

  # View-only policy
  policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy"

  access_scope {
    type       = "namespace"
    namespaces = ["dev", "staging"]  # Limited scope!
  }
}

Testing Read-Only Access

# Assume developer role
aws sts assume-role \
  --role-arn arn:aws:iam::123456789012:role/demo-eks-developer-role \
  --role-session-name dev-session \
  --external-id my-external-id

# Export credentials
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_SESSION_TOKEN="..."

# Update kubeconfig
aws eks update-kubeconfig --region us-east-1 --name demo-eks-cluster

# These should WORK ✅
kubectl get pods -n dev
kubectl get pods -n staging
kubectl describe node <node-name>
kubectl logs <pod-name> -n dev

# These should FAIL ❌
kubectl create deployment test --image=nginx -n dev
# Error: deployments.apps is forbidden

kubectl delete pod <pod-name> -n dev
# Error: pods is forbidden

kubectl get pods -n production
# Error: pods is forbidden (wrong namespace)

What Makes This Secure?

External ID: Prevents confused deputy problem
Namespace-scoped: Can't access production
Read-only: Can't modify anything
No long-lived credentials: Must assume role each time

Full code: scenario-2-developer-access/

Scenario 3: CI/CD Pipeline Access

Modern CI/CD systems need to deploy to EKS. Let's configure GitHub Actions, GitLab CI, and Jenkins.

Why OIDC for CI/CD?

❌ Old way: Store long-lived AWS access keys as secrets
✅ New way: Use OIDC tokens (no credentials stored!)

GitHub Actions with OIDC

Architecture

┌────────────────────────────────────────────────┐
│  GitHub Actions Workflow                       │
│  (Repository: org/repo)                        │
└──────────────┬─────────────────────────────────┘
               │ OIDC Token
               ▼
┌────────────────────────────────────────────────┐
│  GitHub OIDC Provider (AWS IAM)                │
│  token.actions.githubusercontent.com           │
└──────────────┬─────────────────────────────────┘
               │ Trust Relationship
               ▼
┌────────────────────────────────────────────────┐
│  GitHub Actions IAM Role                       │
│  + EKS Admin Access Entry                      │
└──────────────┬─────────────────────────────────┘
               │
               ▼
┌────────────────────────────────────────────────┐
│  EKS Cluster - Deploy Application              │
└────────────────────────────────────────────────┘

Step 1: Create OIDC Provider

# GitHub OIDC Provider
resource "aws_iam_openid_connect_provider" "github_actions" {
  url = "https://token.actions.githubusercontent.com"

  client_id_list = ["sts.amazonaws.com"]

  # GitHub's SSL certificate thumbprints
  thumbprint_list = [
    "6938fd4d98bab03faadb97b34396831e3780aea1",
    "1c58a3a8518e8759bf075b76b750d4f2df264fcd"
  ]
}

Step 2: Create IAM Role

# IAM Role for GitHub Actions
resource "aws_iam_role" "github_actions" {
  name = "demo-eks-github-actions-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = {
        Federated = aws_iam_openid_connect_provider.github_actions.arn
      }
      Action = "sts:AssumeRoleWithWebIdentity"
      Condition = {
        StringEquals = {
          "token.actions.githubusercontent.com:aud" = "sts.amazonaws.com"
        }
        # 🔒 Restrict to specific repositories
        StringLike = {
          "token.actions.githubusercontent.com:sub" = [
            "repo:your-org/your-repo:*"
          ]
        }
      }
    }]
  })
}

# Grant EKS access
resource "aws_eks_access_entry" "github_actions" {
  cluster_name  = var.cluster_name
  principal_arn = aws_iam_role.github_actions.arn
  type          = "STANDARD"
}

resource "aws_eks_access_policy_association" "github_actions" {
  cluster_name  = var.cluster_name
  principal_arn = aws_iam_role.github_actions.arn
  policy_arn    = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy"

  access_scope {
    type = "cluster"
  }
}

Step 3: GitHub Workflow

.github/workflows/deploy.yml:

name: Deploy to EKS

on:
  push:
    branches: [main]

# 🎯 KEY: Request OIDC token
permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      # 🎯 Configure AWS credentials via OIDC (no keys!)
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: us-east-1

      # Now you have AWS access!
      - name: Update kubeconfig
        run: |
          aws eks update-kubeconfig \
            --region us-east-1 \
            --name demo-eks-cluster

      - name: Deploy
        run: |
          kubectl apply -f k8s/
          kubectl rollout status deployment/my-app

GitLab CI with OIDC

.gitlab-ci.yml:

deploy:
  stage: deploy
  image: amazon/aws-cli:latest
  id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.com
  before_script:
    - >
      export $(printf "AWS_ACCESS_KEY_ID=%s AWS_SECRET_ACCESS_KEY=%s AWS_SESSION_TOKEN=%s"
      $(aws sts assume-role-with-web-identity
      --role-arn ${ROLE_ARN}
      --role-session-name "gitlab-${CI_PROJECT_ID}-${CI_PIPELINE_ID}"
      --web-identity-token ${GITLAB_OIDC_TOKEN}
      --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]'
      --output text))
    - aws eks update-kubeconfig --region us-east-1 --name demo-eks-cluster
  script:
    - kubectl apply -f k8s/
  only:
    - main

Jenkins (Traditional Approach)

For Jenkins, use role assumption with External ID:

pipeline {
    agent any

    environment {
        ROLE_ARN = 'arn:aws:iam::123456789012:role/jenkins-eks-role'
        EXTERNAL_ID = 'jenkins-external-id'
    }

    stages {
        stage('Deploy to EKS') {
            steps {
                script {
                    // Assume role
                    def creds = sh(
                        script: """
                            aws sts assume-role \
                                --role-arn ${ROLE_ARN} \
                                --role-session-name jenkins-${BUILD_NUMBER} \
                                --external-id ${EXTERNAL_ID} \
                                --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' \
                                --output text
                        """,
                        returnStdout: true
                    ).trim().split()

                    env.AWS_ACCESS_KEY_ID = creds[0]
                    env.AWS_SECRET_ACCESS_KEY = creds[1]
                    env.AWS_SESSION_TOKEN = creds[2]
                }

                sh '''
                    aws eks update-kubeconfig --name demo-eks-cluster
                    kubectl apply -f k8s/
                '''
            }
        }
    }
}

Full code: scenario-3-cicd-access/

Scenario 4: IRSA for Pods

The crown jewel of EKS IAM: IAM Roles for Service Accounts (IRSA).

The Problem

How do pods access AWS services (S3, DynamoDB, etc.) securely?

❌ Bad: Put credentials in environment variables
❌ Bad: Use EC2 instance profile (all pods share permissions)
✅ Good: IRSA (each pod gets its own IAM role)

How IRSA Works

┌───────────────────────────────────────────────────┐
│  Pod                                              │
│  ServiceAccount: s3-access-sa                     │
│  ↓                                                │
│  Environment Variables (injected by EKS):         │
│  - AWS_ROLE_ARN=arn:aws:iam::123456789012:role/s3│
│  - AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets...│
└────────────────┬──────────────────────────────────┘
                 │
                 ▼
┌──────────────────────────────────────────────────┐
│  EKS OIDC Provider                               │
│  oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E  │
└────────────────┬──────────────────────────────────┘
                 │ Validates token
                 ▼
┌──────────────────────────────────────────────────┐
│  IAM Role (s3-access-role)                       │
│  Trust Policy: Only this ServiceAccount          │
└────────────────┬──────────────────────────────────┘
                 │
                 ▼
┌──────────────────────────────────────────────────┐
│  S3 Bucket                                       │
│  ✅ Pod can access!                              │
└──────────────────────────────────────────────────┘

Step 1: Create OIDC Provider

# Get OIDC provider details from EKS
data "tls_certificate" "cluster" {
  url = data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer
}

# Create OIDC provider
resource "aws_iam_openid_connect_provider" "cluster" {
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.cluster.certificates[0].sha1_fingerprint]
  url             = data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer
}

Step 2: Create S3 Bucket and IAM Role

# S3 bucket for app
resource "aws_s3_bucket" "app_data" {
  bucket = "demo-eks-app-data"
}

# IAM Role for pods
resource "aws_iam_role" "s3_access" {
  name = "demo-eks-s3-access-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = {
        Federated = aws_iam_openid_connect_provider.cluster.arn
      }
      Action = "sts:AssumeRoleWithWebIdentity"
      Condition = {
        StringEquals = {
          # 🔒 Only allow specific ServiceAccount
          "${local.oidc_provider_url}:sub" = "system:serviceaccount:app:s3-access-sa"
          "${local.oidc_provider_url}:aud" = "sts.amazonaws.com"
        }
      }
    }]
  })
}

# S3 access policy
resource "aws_iam_policy" "s3_access" {
  name = "demo-eks-s3-access-policy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "s3:GetObject",
        "s3:PutObject",
        "s3:ListBucket"
      ]
      Resource = [
        aws_s3_bucket.app_data.arn,
        "${aws_s3_bucket.app_data.arn}/*"
      ]
    }]
  })
}

resource "aws_iam_role_policy_attachment" "s3_access" {
  role       = aws_iam_role.s3_access.name
  policy_arn = aws_iam_policy.s3_access.arn
}

Step 3: Create Kubernetes ServiceAccount

resource "kubernetes_service_account" "s3_access" {
  metadata {
    name      = "s3-access-sa"
    namespace = "app"

    # 🎯 This annotation links SA to IAM role
    annotations = {
      "eks.amazonaws.com/role-arn" = aws_iam_role.s3_access.arn
    }
  }
}

Step 4: Deploy Pod Using ServiceAccount

apiVersion: v1
kind: Pod
metadata:
  name: s3-test-pod
  namespace: app
spec:
  serviceAccountName: s3-access-sa  # 🎯 Use the ServiceAccount
  containers:
  - name: app
    image: amazon/aws-cli:latest
    command: ["sleep", "3600"]

Step 5: Test IRSA

# Deploy the pod
kubectl apply -f pod.yaml

# Check environment variables (injected by EKS)
kubectl exec -it s3-test-pod -n app -- env | grep AWS

# Output:
# AWS_ROLE_ARN=arn:aws:iam::123456789012:role/demo-eks-s3-access-role
# AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token

# Test S3 access
kubectl exec -it s3-test-pod -n app -- \
  aws s3 ls s3://demo-eks-app-data/

# Upload a file
kubectl exec -it s3-test-pod -n app -- \
  aws s3 cp /etc/hostname s3://demo-eks-app-data/test.txt

# ✅ It works!

# Try to access a different bucket (should fail)
kubectl exec -it s3-test-pod -n app -- \
  aws s3 ls s3://some-other-bucket/
# ❌ Error: Access Denied (as expected!)

Real Application Example

Here's a Flask app that uses IRSA:

import boto3
from flask import Flask, jsonify

app = Flask(__name__)

# boto3 automatically uses IRSA credentials!
s3_client = boto3.client('s3')
BUCKET = 'demo-eks-app-data'

@app.route('/files')
def list_files():
    """List files in S3 bucket"""
    response = s3_client.list_objects_v2(Bucket=BUCKET)
    files = [obj['Key'] for obj in response.get('Contents', [])]
    return jsonify({'files': files})

@app.route('/upload/<filename>')
def upload_file(filename):
    """Upload a file to S3"""
    s3_client.put_object(
        Bucket=BUCKET,
        Key=filename,
        Body=b'Hello from IRSA!'
    )
    return jsonify({'message': f'Uploaded {filename}'})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Key points:

No AWS credentials in code!
boto3 automatically uses IRSA
Each pod can have different permissions

Full code: scenario-4-irsa/

Best Practices

1. Use the New Access Entry API

✅ Do:

resource "aws_eks_access_entry" "developer" {
  cluster_name  = aws_eks_cluster.main.name
  principal_arn = aws_iam_role.developer.arn
}

❌ Don't:

# Manual aws-auth ConfigMap editing

2. Principle of Least Privilege

✅ Do:

# Specific resources only
Resource = [
  aws_s3_bucket.app.arn,
  "${aws_s3_bucket.app.arn}/*"
]

❌ Don't:

# Wildcard access
Resource = "*"

3. Use Namespace-Scoped Access

✅ Do:

access_scope {
  type       = "namespace"
  namespaces = ["dev", "staging"]
}

❌ Don't:

# Give everyone cluster-wide access
access_scope {
  type = "cluster"
}

4. Enable CloudTrail Logging

resource "aws_cloudtrail" "main" {
  name           = "eks-audit-trail"
  s3_bucket_name = aws_s3_bucket.cloudtrail.id

  event_selector {
    read_write_type           = "All"
    include_management_events = true
  }
}

5. Use OIDC for CI/CD

✅ Do: GitHub Actions/GitLab with OIDC
❌ Don't: Store long-lived access keys

6. Implement Session Limits

resource "aws_iam_role" "developer" {
  # Limit session duration
  max_session_duration = 3600  # 1 hour
}

7. Require MFA for Production

Condition = {
  Bool = {
    "aws:MultiFactorAuthPresent" = "true"
  }
}

8. Use External IDs

Condition = {
  StringEquals = {
    "sts:ExternalId" = "unique-external-id"
  }
}

Common Pitfalls

1. Forgetting to Bootstrap Cluster Creator

❌ Problem:

# Cluster creator has no access!
access_config {
  bootstrap_cluster_creator_admin_permissions = false
}

✅ Solution:

access_config {
  bootstrap_cluster_creator_admin_permissions = true
}

2. Wrong OIDC Provider Thumbprint

❌ Problem:

Error: InvalidIdentityToken

✅ Solution:

# Get correct thumbprint
openssl s_client -servername oidc.eks.us-east-1.amazonaws.com \
  -showcerts -connect oidc.eks.us-east-1.amazonaws.com:443 2>&- \
  | openssl x509 -fingerprint -sha1 -noout -in /dev/stdin

3. ServiceAccount Annotation Typo

❌ Problem:

annotations:
  eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/wrong-role

✅ Solution:

# Verify with Terraform output
terraform output s3_role_arn

4. Overly Permissive Policies

❌ Problem:

Action = "*"
Resource = "*"

✅ Solution:

Action = [
  "s3:GetObject",
  "s3:PutObject"
]
Resource = "${aws_s3_bucket.app.arn}/*"

5. Not Testing Denied Actions

✅ Always test that restricted actions fail:

# Should succeed
kubectl get pods -n dev

# Should fail
kubectl delete pod test -n dev

Troubleshooting Guide

Issue: "You must be logged in to the server (Unauthorized)"

Check:

# Verify AWS credentials
aws sts get-caller-identity

# Update kubeconfig
aws eks update-kubeconfig --region us-east-1 --name demo-eks-cluster

# Check access entry exists
aws eks describe-access-entry \
  --cluster-name demo-eks-cluster \
  --principal-arn <your-role-arn>

Issue: IRSA Not Working

Check:

# Verify OIDC provider exists
aws iam list-open-id-connect-providers

# Check ServiceAccount annotation
kubectl get sa s3-access-sa -n app -o yaml

# Verify pod has environment variables
kubectl exec -it <pod> -n app -- env | grep AWS

# Check token file exists
kubectl exec -it <pod> -n app -- cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token

Issue: Access Denied

Check:

# Get caller identity from pod
kubectl exec -it <pod> -- aws sts get-caller-identity

# Check IAM role trust policy
aws iam get-role --role-name <role-name>

# Check attached policies
aws iam list-attached-role-policies --role-name <role-name>

Conclusion

We've covered a lot! Here's what you learned:

✅ Understanding: How IAM works with EKS (authentication vs authorization)
✅ New API: EKS Access Entries vs aws-auth ConfigMap
✅ Scenarios: 4 practical, production-ready examples
✅ IRSA: The secure way for pods to access AWS services
✅ CI/CD: OIDC-based authentication for GitHub, GitLab, Jenkins
✅ Best Practices: Security, least privilege, and troubleshooting

Next Steps

Clone the repo: eks-iam-hands-on
Start with Scenario 1: Get a cluster running
Experiment: Try different access policies
Deploy the sample app: See IRSA in action
Adapt for production: Use these patterns in real projects

Additional Resources

What challenges have you faced with EKS IAM?

Share your experiences in the comments! If this guide helped you, give it a ❤️ and share with your team.

Happy Kuberneting! 🚀

Found this helpful? Follow me for more AWS and Kubernetes content!

Connect:

GitHub: @mdnurahmed
LinkedIn: nur-ahmed

Mastering AWS IAM: A Complete Guide with Hands-On Scenarios

MD Nur Ahmed — Fri, 12 Dec 2025 10:56:57 +0000

Mastering AWS IAM: A Complete Guide with Hands-On Scenarios

AWS Identity and Access Management (IAM) is the backbone of security in AWS. Whether you're deploying your first Lambda function or architecting a multi-account enterprise solution, understanding IAM is crucial. In this comprehensive guide, I'll walk you through different types of IAM identities, policies, and real-world scenarios with practical Terraform examples.

What is AWS IAM?

AWS IAM is a web service that helps you securely control access to AWS resources. It enables you to manage authentication (who can sign in) and authorization (what actions they can perform) across your AWS infrastructure.

Think of IAM as the security guard of your AWS account. It decides:

Who can access your resources (authentication)
What they can do with those resources (authorization)
When and where they can access them (conditions)

Core IAM Components

1. IAM Users

Individual identities with long-term credentials. Best suited for:

Human users who need console or CLI access
Applications that require permanent credentials (though roles are preferred)
Service accounts for third-party integrations

2. IAM Groups

Collections of users with similar permissions. Groups help you:

Manage permissions at scale
Apply consistent policies across teams
Simplify user onboarding and offboarding

3. IAM Roles

Temporary identities that can be assumed by users, applications, or AWS services. Roles are perfect for:

AWS services (Lambda, EC2, ECS)
Cross-account access
Federated users
Applications running on AWS

4. IAM Policies

JSON documents that define permissions. Types include:

Identity-based policies: Attached to users, groups, or roles
Resource-based policies: Attached to resources like S3 buckets
Permission boundaries: Set maximum permissions
Service Control Policies (SCPs): Applied at the AWS Organizations level

IAM Best Practices

Before diving into scenarios, here are critical best practices:

Follow the Principle of Least Privilege: Grant only the permissions required to perform a task
Use Roles Instead of Users for Applications: Roles provide temporary credentials that rotate automatically
Enable MFA for Privileged Users: Add an extra layer of security
Regularly Rotate Credentials: Change passwords and access keys periodically
Use Policy Conditions: Add constraints like IP address ranges or time windows
Monitor with CloudTrail: Keep audit logs of all IAM actions
Avoid Using Root Account: Create IAM users for daily operations

Real-World IAM Scenarios

Let me walk you through practical scenarios that you'll encounter in production environments. All the code examples are available in my AWS IAM Hands-On Repository.

Scenario 1: Lambda Function with S3 Access

Use Case: You have a Lambda function that needs to read from an S3 bucket and write logs to CloudWatch.

Solution: Create an IAM role with the necessary permissions and attach it to the Lambda function.

# IAM Role for Lambda
resource "aws_iam_role" "lambda_role" {
  name = "lambda-s3-access-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

# Policy for S3 access
resource "aws_iam_policy" "lambda_s3_policy" {
  name = "lambda-s3-access-policy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:PutObject"
        ]
        Resource = "arn:aws:s3:::my-bucket/*"
      },
      {
        Effect = "Allow"
        Action = "s3:ListBucket"
        Resource = "arn:aws:s3:::my-bucket"
      }
    ]
  })
}

# Attach policy to role
resource "aws_iam_role_policy_attachment" "lambda_s3_attach" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = aws_iam_policy.lambda_s3_policy.arn
}

# Attach CloudWatch Logs policy
resource "aws_iam_role_policy_attachment" "lambda_logs" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

Why This Works:

The Lambda service can assume this role
The role has specific S3 permissions (read and write to a specific bucket)
CloudWatch Logs permissions allow Lambda to write logs
No hard-coded credentials needed

Scenario 2: Cross-Account Access

Use Case: You have a CI/CD pipeline in Account A that needs to deploy resources to Account B.

Solution: Create a role in Account B that Account A can assume.

# In Account B - Create role that Account A can assume
resource "aws_iam_role" "cross_account_role" {
  name = "cross-account-deploy-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          AWS = "arn:aws:iam::ACCOUNT_A_ID:root"
        }
        Action = "sts:AssumeRole"
        Condition = {
          StringEquals = {
            "sts:ExternalId" = "unique-external-id-123"
          }
        }
      }
    ]
  })
}

# Deployment permissions
resource "aws_iam_policy" "deploy_policy" {
  name = "deployment-policy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ec2:*",
          "s3:*",
          "lambda:*",
          "cloudformation:*"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "deploy_attach" {
  role       = aws_iam_role.cross_account_role.name
  policy_arn = aws_iam_policy.deploy_policy.arn
}

Security Highlights:

External ID prevents the "confused deputy" problem
Account A must explicitly assume the role
Permissions are isolated to Account B
Can be easily audited via CloudTrail

Scenario 3: EC2 Instance Profile

Use Case: Your EC2 instances need to access AWS services without storing credentials in the application.

Solution: Use an instance profile with an IAM role.

# IAM Role for EC2
resource "aws_iam_role" "ec2_role" {
  name = "ec2-app-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      }
    ]
  })
}

# Application permissions
resource "aws_iam_policy" "app_policy" {
  name = "ec2-app-policy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "dynamodb:GetItem",
          "dynamodb:PutItem",
          "dynamodb:Query"
        ]
        Resource = "arn:aws:dynamodb:us-east-1:*:table/my-table"
      },
      {
        Effect = "Allow"
        Action = [
          "sqs:SendMessage",
          "sqs:ReceiveMessage"
        ]
        Resource = "arn:aws:sqs:us-east-1:*:my-queue"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "app_attach" {
  role       = aws_iam_role.ec2_role.name
  policy_arn = aws_iam_policy.app_policy.arn
}

# Instance Profile
resource "aws_iam_instance_profile" "ec2_profile" {
  name = "ec2-app-profile"
  role = aws_iam_role.ec2_role.name
}

# Attach to EC2 instance
resource "aws_instance" "app_server" {
  ami                  = "ami-0c55b159cbfafe1f0"
  instance_type        = "t2.micro"
  iam_instance_profile = aws_iam_instance_profile.ec2_profile.name

  # Your other configurations...
}

Benefits:

No credential management in application code
Automatic credential rotation by AWS
Easy to audit and rotate permissions
Credentials never leave the AWS environment

Scenario 4: Least Privilege with Condition Keys

Use Case: Grant developers access to only their team's resources using tags.

Solution: Use condition keys in IAM policies.

resource "aws_iam_policy" "team_policy" {
  name = "team-based-access-policy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ec2:StartInstances",
          "ec2:StopInstances",
          "ec2:TerminateInstances"
        ]
        Resource = "*"
        Condition = {
          StringEquals = {
            "ec2:ResourceTag/Team" = "$${aws:username}"
          }
        }
      },
      {
        Effect = "Allow"
        Action = [
          "ec2:CreateTags"
        ]
        Resource = "*"
        Condition = {
          StringEquals = {
            "ec2:CreateAction" = "RunInstances"
            "aws:RequestTag/Team" = "$${aws:username}"
          }
        }
      },
      {
        Effect = "Allow"
        Action = [
          "ec2:Describe*"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_group" "developers" {
  name = "developers"
}

resource "aws_iam_group_policy_attachment" "dev_attach" {
  group      = aws_iam_group.developers.name
  policy_arn = aws_iam_policy.team_policy.arn
}

How It Works:

Developers can only manage instances tagged with their username
Forces tagging at creation time
Read-only access to all resources for visibility
Prevents accidental modification of other teams' resources

Advanced IAM Patterns

Permission Boundaries

Permission boundaries set the maximum permissions an entity can have, even if more permissive policies are attached.

resource "aws_iam_policy" "permission_boundary" {
  name = "developer-boundary"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:*",
          "dynamodb:*",
          "lambda:*"
        ]
        Resource = "*"
      },
      {
        Effect = "Deny"
        Action = [
          "iam:*",
          "organizations:*"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_user" "developer" {
  name                 = "junior-developer"
  permissions_boundary = aws_iam_policy.permission_boundary.arn
}

Session Policies

Provide additional restrictions when assuming a role:

# Assume role with session policy
resource "aws_iam_role" "assumed_role" {
  name = "limited-session-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          AWS = "arn:aws:iam::123456789012:user/admin"
        }
      }
    ]
  })
}

Then when assuming with AWS CLI:

aws sts assume-role \
  --role-arn arn:aws:iam::123456789012:role/limited-session-role \
  --role-session-name my-session \
  --policy '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::specific-bucket/*"
      }
    ]
  }'

IAM Policy Evaluation Logic

Understanding how AWS evaluates policies is crucial:

By default, all requests are denied (implicit deny)
Explicit allow in an identity-based or resource-based policy overrides the default
Permission boundary restricts the maximum permissions
Explicit deny overrides any allows

Explicit Deny → Deny
    ↓
Permission Boundary → Deny if not allowed
    ↓
Identity-based Policy → Allow
    ↓
Resource-based Policy → Allow
    ↓
Default → Deny

Testing IAM Policies

Always test your policies before deploying to production:

IAM Policy Simulator

Use the AWS IAM Policy Simulator to test policies:

aws iam simulate-principal-policy \
  --policy-source-arn arn:aws:iam::123456789012:role/my-role \
  --action-names s3:GetObject \
  --resource-arns arn:aws:s3:::my-bucket/file.txt

Access Analyzer

Enable IAM Access Analyzer to:

Identify resources shared with external entities
Validate policies against AWS best practices
Get recommendations for unused access

Common IAM Pitfalls and Solutions

1. Overly Permissive Policies

Problem: Using "Resource": "*" and "Action": "*"

Solution: Be specific with resources and actions

{
  "Effect": "Allow",
  "Action": "s3:GetObject",
  "Resource": "arn:aws:s3:::my-specific-bucket/*"
}

2. Not Using Roles for Applications

Problem: Storing access keys in code

Solution: Always use IAM roles with instance profiles or task roles

3. Forgetting to Test

Problem: Deploying untested policies

Solution: Use IAM Policy Simulator and Access Analyzer

4. Ignoring CloudTrail Logs

Problem: No visibility into who did what

Solution: Enable CloudTrail and review logs regularly

Hands-On Practice

Want to try these scenarios yourself? Check out my AWS IAM Hands-On Repository which contains:

Scenario 1: Lambda with S3 and DynamoDB access
Scenario 2: Cross-account role assumption
Scenario 3: EC2 instance profiles with multiple service integrations
Scenario 4: Tag-based access control with conditions
Lambda Code: Sample functions demonstrating IAM in action

Each scenario includes:

Terraform infrastructure code
Step-by-step deployment instructions
Testing procedures
Cleanup scripts

# Clone the repository
git clone https://github.com/mdnurahmed/aws-iam-hands-ons.git

# Navigate to a scenario
cd aws-iam-hands-ons/scenario-1

# Initialize Terraform
terraform init

# Plan the deployment
terraform plan

# Apply the configuration
terraform apply

IAM Security Checklist

Use this checklist to ensure your IAM setup is secure:

[ ] Root account MFA enabled
[ ] No access keys for root account
[ ] IAM users have MFA enabled
[ ] Access keys rotated regularly (90 days max)
[ ] Unused users and roles removed
[ ] CloudTrail enabled and logs analyzed
[ ] IAM Access Analyzer enabled
[ ] Password policy enforced (complexity, rotation)
[ ] Least privilege principle applied
[ ] Service roles used instead of user credentials
[ ] Cross-account access uses external IDs
[ ] Permission boundaries set for delegated admin
[ ] Regular IAM credential reports reviewed

Monitoring and Compliance

CloudTrail Integration

Monitor IAM activities:

resource "aws_cloudtrail" "iam_trail" {
  name                          = "iam-audit-trail"
  s3_bucket_name               = aws_s3_bucket.trail_bucket.id
  include_global_service_events = true
  is_multi_region_trail        = true

  event_selector {
    read_write_type           = "All"
    include_management_events = true
  }
}

AWS Config Rules

Set up compliance checks:

resource "aws_config_config_rule" "iam_password_policy" {
  name = "iam-password-policy-check"

  source {
    owner             = "AWS"
    source_identifier = "IAM_PASSWORD_POLICY"
  }

  depends_on = [aws_config_configuration_recorder.main]
}

Conclusion

IAM is fundamental to AWS security, and mastering it takes practice. The key principles to remember:

Always use least privilege - grant only what's needed
Prefer roles over users - for applications and services
Test before deploying - use simulators and analyzers
Monitor continuously - enable CloudTrail and Config
Automate with IaC - use Terraform or CloudFormation

By following these patterns and practicing with the scenarios in the AWS IAM Hands-On Repository, you'll build a strong foundation in AWS security.

What IAM challenges have you faced? Share your experiences in the comments below!

Additional Resources

If you found this guide helpful, give it a ❤️ and follow me for more AWS and DevOps content!

Are you really treating redis as cache storage ?

MD Nur Ahmed — Sat, 08 Mar 2025 12:43:30 +0000

TL,DR:

If your application goes down because of Redis outages while using Redis as cache storage, you need to check your Redis client configurations in your code. You're not actually using Redis as a cache storage properly in this case.

Redis cache anti-pattern

Redis is mostly used as a cache storage to speed up server response time. However, I've often seen applications crash when Redis servers fail (due to Redis memory being full or connection limits being reached). This is an anti-pattern and shouldn't happen. Your application should still function during Redis outages (although with increased latency). This problem typically occurs because the Redis client library configuration treats Redis as a primary data source and keeps retrying with exponential backoff when Redis crashes, instead of gracefully falling back to the primary database.

Demo

I created a simple demo for this using nodejs and ioredis library here.

In the main branch , we have a simple redis configuration like this :

const redis = new Redis({
  host: process.env.REDIS_HOST || "redis",
  port: process.env.REDIS_PORT || 6379,
  password: process.env.REDIS_PASSWORD || "",
});

Let's run it using docker:

docker compose up 

//Create a new item
curl -X POST http://localhost:8080/items \
  -H "Content-Type: application/json" \
  -d '{"name":"test item","value":"test value"}'

// Get the item
curl http://localhost:8080/items/1

Now , lets bring down redis using "docker stop ". And we see application crashes because ioredis connection error is unhandled.

Now let's checkout another branch where we handled redis connection errors :

// Handle errors without crashing
redis.on("error", (err) => {
  console.error("[ioredis] Connection error:", err.message);
  // Don't crash the app, just log the error
});

Now, if we stop the Redis Docker container, the app doesn't crash anymore. We've made some progress. However, our requests are taking forever to get a response.

Why is that? Because our configuration still isn't resilient to Redis outages. It keeps trying to connect to Redis indefinitely (ioredis default max retry attempt is 20, which is still too many—we don't want to retry so many times).

Let's make it even better :

// Redis connection
const redis = new Redis({
  host: process.env.REDIS_HOST || "redis",
  port: process.env.REDIS_PORT || 6379,
  password: process.env.REDIS_PASSWORD || "",
  connectTimeout: 100, // 100 millisecond timeout
  maxRetriesPerRequest: 1, // Try once and then move on
});

Now, our application is resilient to Redis outages. It remains responsive even when Redis goes down.

Conclusion

Of course, you should implement high availability setups and use max-memory policies so that Redis doesn't go down in the first place. And if it does go down, there should be alerts in place. But a Redis cache outage shouldn't turn into a P0 incident for your application.

Simple Scalable Search Autocomplete Systems

MD Nur Ahmed — Fri, 10 Sep 2021 05:51:06 +0000

Here I'll discuss 4 ways to build a simple scalable search autocomplete service that I know of and some pros and cons of each of them.

Requirements

One API for inserting texts/words that were searched by users. For example :

POST /insert
{
  "search string" : "Harry Potter and the Prisoner of Azkaban"
}

One API for getting autocomplete suggestions as the user keeps typing. For example :

GET /search?SearchString=potter

One API to clear all searches made by the user. For Example -

DELETE /delete

The system should be able to return top N frequent relevant queries/searches.
The system should be horizontally scalable using shards and read replicas.
The calculations should be In-memory to make the system fast and real-time.

Way 1: Using SQL

SELECT * FROM autocomplete 
WHERE search_items LIKE '%{SEARCH_STRING}% 
ORDER BY frequency DESC 
LIMIT 5';

PostgreSQL (and some other SQL databases) have LIKE operator which can be used to search patterns in text data. Percent sign (%) matches any sequence of zero or more characters. So %{SEARCH_STRING}% will return all rows in autocomplete table where {SEARCH_STRING} exists in any position of the search-items field. We would also want to get rid of less frequent search items before they pile up. There is no direct way to do this. We could have an "expire" field which we would update every time insert API is called, that is when a user searches something. Another process could periodically delete the expired items. This can do infix matching (i.e. matching terms at any position within the input). This is the simplest way to do this but this actually doesn't happen in memory.

Using Redis Sorted Set

Redis is an in-memory database. So we could try to exploit Redis's data structures to make a search autocomplete system. Salvatore Sanfilippo Antirez, the creator of Redis, wrote about 2 ways about how it could be done in his old blog post from 2010. You can read the original blog post but here I'm gonna explain both of the ways. Both of the ways use sorted set data structure of Redis which is implemented using the skip list data structure under the hood.

Way 2: Using 1 Sorted Set

If any 2 members in a Redis's sorted set have an equal score they are sorted lexicographically. For each prefix in the search string, we'll store them in a sorted set with the same score. So, they will be lexicographically sorted in the set. To mark an end word we'll add '*' to it in the end. For example, if we searched "bangladesh" and "bangkok", our sorted set will look like this -

 0 b
 1 ba
 2 ban
 3 bang
 4 bangk
 5 bangko
 6 bangkok
 7 bangkok*
 8 bangl
 9 bangla
10 banglad
11 banglade
12 banglades
13 bangladesh
14 bangladesh*

Each insertion in the sorted set happens at O(logN) complexity where N is the size of the sorted set. So total complexity of insert each time the insert API is called is O(L*logN) where L is the average length of a search string. We can use the ZRANK command to find any member's index/position in the sorted set. We then can use ZRANGE command to fetch M members from that position in O(M) complexity. If any fetched member has '*' in the end and it contains the search string as a prefix, then we can return it as a suggestion.
For example, if the user types 'bang' we can find the index of 'bang' in a sorted set, it's 3. Now if we fetch 12 items from position 3 we would fetch -

[
bang,bangk,bangko,bangkok,bangkok*,
bangl,bangla,banglad,banglade,
banglades,bangladesh,bangladesh*
]

Among these members bangkok* and bangladesh* has "bang" in the prefix and "*" in the end. So they can go into the suggestion. Our API would return [bangkok,bangladesh].
But this method cannot return top N frequent relevant searches. Also, there is no way to get rid of less frequent searched items. On top of that, this cannot do infix matching (i.e. matching terms at any position within the input).

Here is my full code implementing this method

Way 3: Using Multiple Sorted Sets

For each prefix of the searched string, we'll have 1 sorted set. The searched string will be a member of each of those sorted sets. The score of each member will be the frequency. The complexity of insert here is L*log(N) where L average length of a search string, N is the size of the sorted set. For example, if we searched "bangladesh" 2 times and "bangkok" 3 times, our Redis would look like this


|Sorted Set| Members                   | 
| ---------|---------------------------|
|b         | bangladesh:2 , bangkok:3  |           
|ba        | bangladesh:2 , bangkok:3  |
|ban       | bangladesh:2 , bangkok:3  |
|bang      | bangladesh:2 , bangkok:3  |
|bangk     | bangladesh:2 , bangkok:3  |
|bangko    | bangladesh:2 , bangkok:3  |
|bangkok   | bangladesh:2 , bangkok:3  |
|bangl     | bangladesh:2 , bangkok:3  |
|bangla    | bangladesh:2 , bangkok:3  |
|banglad   | bangladesh:2 , bangkok:3  |
|banglade  | bangladesh:2 , bangkok:3  |
|banglades | bangladesh:2 , bangkok:3  |
|bangladesh| bangladesh:2 , bangkok:3  |

So if the user types 'bang' we can fetch top N frequent items easily from the sorted set 'bang' using the ZREVRANGE command as our suggestions are sorted according to frequency. We can also use EXPIRE command to delete sorted sets containing less frequent search items.
Now we need to cap the length of the sorted set. We cannot let it reach infinite length. For the top 5 suggestions, keeping 300 elements is quite enough. Whenever a new member would come in the sorted set we'll pop the element with the lowest score and insert the new element with a score equal to the popped element's score plus 1. So this new element would have a chance to rise to the top. Now, this is a stream-based algorithm. So the accuracy of this algorithm depends on the distribution of input. If all the strings have very close frequency this might not be very accurate. But in the case of searching problems, usually, a small subset occupies a big percentage of all the searches. So in a real-life scenario, this will be very feasible. But this also can not do infix matching.

Here is my full code implementing this method

Way 4: Using Elasticsearch

There are multiple ways to do it using Elasticsearch. But almost all of them does most of the work at query time and are not guaranteed to be in memory. Except if we map our field as search_as_you_type data type. Then this does most of the work at index time by updating in-memory Finite State Transducers which are a variant of the trie data structure. Also, this can do infix matching. This is the most efficient way to solve this problem.
our mapping would look like this -

{
   "mappings":{
      "properties":{
         "search-text":{
            "type":"search_as_you_type"
         },
         "frequency":{
            "type":"long"
         },
         "expire":{
            "type":"long"
         }
      }
   }
}

"frequency" is the frequency of the search-text, so we can fetch top N relevant searches. "expire" field can be used to delete expired less frequent items by another process.

We don't want one search to appear multiple times in the database. One way to solve this is to use the hash of the search string as the document ID.

Our insert operation will be an upsert operation where we would increase the "frequency" of the document and update the "expire" field if it already exists or insert the document if it doesn't exist. We can take help of the "script" field for this -

POST autocomplete/_update/{DOCUMENT_ID}
{
   "script":{
      "source":"ctx._source.frequency++;ctx._source.expire={EXPIRE_TIME}",
      "lang":"painless"
   },
   "upsert":{
      "search-text":"{SEARCH_STRING}",
      "frequency":1,
      "expire":"{EXPIRE_TIME}"
   }
}

In search API we want to fetch the top N relevant element sorted by frequency -

GET /autocomplete/_search
{
   "size":5,
   "sort":[
      {
         "frequency":"desc"
      }
   ],
   "query":{
      "multi_match":{
         "query":{SEARCH_STRING},
         "type":"bool_prefix",
         "fields":[
            "search-text",
            "search-text._2gram",
            "search-text._3gram"
         ]
      }
   }
}

Here is my full code implementing this method.

Conclusion :

Thanks a lot for taking your time to read it. Let me know what you think of it in the comment. Let me know if there are any other solutions you know of or how I could improve my solutions. Any feedback is welcome.

DEV Community: MD Nur Ahmed

Different Ways to Manage AWS IAM to Connect to an EKS Cluster

What You'll Learn

Table of Contents

Understanding IAM with EKS

Two Layers of Access Control

1. AWS IAM (Authentication)

2. Kubernetes RBAC (Authorization)

OIDC: The Bridge Between AWS and Kubernetes

The Old Way vs The New Way

❌ OLD WAY: aws-auth ConfigMap

✅ NEW WAY: EKS Access Entries

AWS Managed Access Policies

Scenario 1: Basic EKS Cluster Setup

Architecture

Step 1: Create the EKS Cluster

Step 2: Create Admin Access

Step 3: Deploy and Test

Scenario 2: Developer Read-Only Access

Use Case

Architecture

Implementation

Testing Read-Only Access

What Makes This Secure?

Scenario 3: CI/CD Pipeline Access

Why OIDC for CI/CD?

GitHub Actions with OIDC

Architecture

Step 1: Create OIDC Provider

Step 2: Create IAM Role

Step 3: GitHub Workflow

GitLab CI with OIDC

Jenkins (Traditional Approach)

Scenario 4: IRSA for Pods

The Problem

How IRSA Works

Step 1: Create OIDC Provider

Step 2: Create S3 Bucket and IAM Role

Step 3: Create Kubernetes ServiceAccount

Step 4: Deploy Pod Using ServiceAccount

Step 5: Test IRSA

Real Application Example

Best Practices

1. Use the New Access Entry API

2. Principle of Least Privilege

3. Use Namespace-Scoped Access

4. Enable CloudTrail Logging

5. Use OIDC for CI/CD

6. Implement Session Limits

7. Require MFA for Production

8. Use External IDs

Common Pitfalls

1. Forgetting to Bootstrap Cluster Creator

2. Wrong OIDC Provider Thumbprint

3. ServiceAccount Annotation Typo

4. Overly Permissive Policies

5. Not Testing Denied Actions

Troubleshooting Guide

Issue: "You must be logged in to the server (Unauthorized)"

Issue: IRSA Not Working

Issue: Access Denied

Conclusion

Next Steps

Additional Resources

What challenges have you faced with EKS IAM?

Mastering AWS IAM: A Complete Guide with Hands-On Scenarios

Mastering AWS IAM: A Complete Guide with Hands-On Scenarios

What is AWS IAM?

Core IAM Components

1. IAM Users

2. IAM Groups

3. IAM Roles

4. IAM Policies

IAM Best Practices

Real-World IAM Scenarios

Scenario 1: Lambda Function with S3 Access

Scenario 2: Cross-Account Access

Scenario 3: EC2 Instance Profile

Scenario 4: Least Privilege with Condition Keys

Advanced IAM Patterns