Migrating from ECS to EKS - Key Considerations

Thinking about moving your containers from Amazon ECS to Amazon EKS? You're not alone. Many teams are making this shift to leverage Kubernetes flexibility and industry standard tooling.

While both ECS and EKS run containers on AWS, the migration isn't straightforward. ECS task definitions need to become Kubernetes manifests. Your IAM roles need restructuring. Your deployment pipelines need updates. And your team needs to learn Kubernetes.

This guide covers everything you need to consider: from converting your applications and configuring networking, to setting up monitoring and training your team. We'll focus on practical considerations that will help you plan and execute a smooth migration.

Whether you're moving one service or your entire platform, understanding these key areas will save you time and headaches.

Let's get started with the following considerations..

1. Architecture & Workload Analysis

Assess Current ECS Setup:

• Task Definitions → Need conversion to Kubernetes Deployments/StatefulSets
• Service types (Fargate vs EC2) → Node groups or Fargate on EKS
• Task placement strategies → Pod affinity/anti-affinity rules
• Service discovery (Cloud Map, ALB) → Kubernetes Services/Ingress
• Auto-scaling policies → HPA (Horizontal Pod Autoscaler)

Application Compatibility:

• Review applications for ECS-specific dependencies
• Check for AWS SDK calls that use ECS metadata endpoints
• Identify hardcoded ECS configurations in application code
• Assess container health check configurations

2. Container & Image Management

Container Images:

• ECS and EKS both use Docker/OCI images - no changes needed
• Ensure images are in Amazon ECR (or accessible registry)
• Review image tagging strategy
• Verify image security scanning is enabled

Startup Commands & Environment:

• ECS command and entryPoint → Kubernetes command and args
• Environment variables transfer directly
• Secret handling needs migration (see below)

3. Configuration Translation

ECS Task Definition → Kubernetes Manifests

ECS Task Definition:

{
  "family": "my-app",
  "cpu": "512",
  "memory": "1024",
  "containerDefinitions": [{
    "name": "app",
    "image": "my-app:latest",
    "portMappings": [{"containerPort": 8080}]
  }]
}

Kubernetes Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: app
        image: my-app:latest
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "500m"
            memory: "1Gi"
          limits:
            cpu: "500m"
            memory: "1Gi"

Resource Mapping:

• ECS CPU units (1024 = 1 vCPU) → Kubernetes millicores (1000m = 1 CPU)
• ECS memory (MB) → Kubernetes memory (Mi/Gi)
• Set both requests and limits in K8s

4. Networking Considerations

Load Balancing:

• ECS ALB integration → AWS Load Balancer Controller for EKS
• ECS Service Discovery (Cloud Map) → Kubernetes Services (DNS-based)
• Target Groups → Managed by ALB Ingress Controller

Service Mesh (Optional):

• Consider AWS App Mesh (works with both) or Istio/Linkerd for EKS
• Service-to-service communication patterns
• Traffic management and observability

Network Modes:

• ECS awsvpc mode → Similar to Kubernetes pod networking
• Choose appropriate CNI plugin (AWS VPC CNI, Calico, Cilium)
• Plan IP address space (pods consume VPC IPs with AWS VPC CNI)

Security Groups:

• ECS task-level security groups → SecurityGroupPolicy CRD in EKS
• Or use Network Policies for pod-to-pod restrictions
• Maintain database and service security group rules

5. IAM & Authentication

Task Roles → Service Accounts:

• ECS Task Roles → IAM Roles for Service Accounts (IRSA)
• Create OIDC provider for EKS cluster
• Map IAM roles to Kubernetes service accounts
• No more EC2 instance roles for pod permissions

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-sa
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/my-app-role

Access Control:

• ECS permissions → RBAC (Role-Based Access Control) in Kubernetes
• AWS IAM → aws-auth ConfigMap for cluster access
• Integrate with AWS SSO or IAM Identity Center

6. Secrets & Configuration Management

Secrets:
ECS Secrets (from Secrets Manager/SSM) → Multiple options:

• AWS Secrets Manager/Parameter Store with External Secrets Operator
• Kubernetes Secrets (less secure, stored in etcd)
• Sealed Secrets or SOPS for GitOps

Configuration:

• ECS environment variables → ConfigMaps and Secrets
• ECS Systems Manager parameters → External Secrets Operator
• Consider Helm for templating and configuration management

7. Storage & Persistence

Volumes:

• ECS EFS volumes → EFS CSI Driver for EKS
• ECS bind mounts → Kubernetes HostPath (not recommended) or emptyDir
• ECS Docker volumes → Persistent Volume Claims (PVCs) with EBS CSI Driver

StatefulSets:

• Applications requiring persistent storage need StatefulSets not Deployments
• Define StorageClass and PersistentVolumeClaims

volumeClaimTemplates:
- metadata:
    name: data
  spec:
    accessModes: ["ReadWriteOnce"]
    storageClassName: gp3
    resources:
      requests:
        storage: 20Gi

8. Logging & Monitoring

Logging:

ECS CloudWatch Logs → Multiple options:

• FluentBit/Fluentd DaemonSet → CloudWatch/S3/Elasticsearch
• CloudWatch Container Insights for EKS
• Maintain same log groups or migrate to new structure

Monitoring:

• ECS metrics → Prometheus + Grafana ecosystem
• CloudWatch Container Insights for both
• Application Performance Monitoring (APM) tools (Datadog, New Relic, etc.)
• Migrate existing dashboards and alerts

Tracing:

• Ensure distributed tracing (X-Ray, Jaeger) continues to work
• Update SDK configurations if needed

9. CI/CD Pipeline Changes

Deployment Pipeline:

• Update pipelines to use kubectl, Helm, or GitOps (ArgoCD/FluxCD)
• Replace ECS task definition updates with kubectl apply or Helm upgrades
• Image building remains the same (push to ECR)

Deployment Strategies:

• ECS rolling updates → Kubernetes rolling updates (more control)
• Configure RollingUpdate strategy with maxSurge and maxUnavailable
• Blue/green deployments using Argo Rollouts or Flagger

Testing:

• Update integration tests for Kubernetes environment
• Test in staging EKS cluster first
• Validate health checks and readiness probes

10. Cost Considerations

Cost Changes:

• EKS Control Plane: $0.10/hour per cluster (~$73/month)
• Worker nodes: Similar to ECS EC2 instances
• Fargate on EKS: Available but pricing differs from ECS Fargate
• Potential for better resource utilization with Kubernetes bin-packing

Optimization:

• Use Spot instances for cost savings
• Implement Cluster Autoscaler or Karpenter
• Right-size pods with appropriate resource requests/limits
• Consider Savings Plans or Reserved Instances

11. Migration Strategy

Phased Approach (Recommended):

Phase 1: Preparation:

• Set up EKS cluster in same VPC
• Install necessary controllers (ALB, EBS CSI, etc.)
• Convert task definitions to Kubernetes manifests
• Set up monitoring and logging

Phase 2: Pilot Migration:

• Migrate non-critical, stateless services first
• Run in parallel with ECS (blue/green at service level)
• Validate functionality and performance
• Gather team feedback

Phase 3: Gradual Migration:

• Service-by-service migration
• Update DNS/load balancer routing gradually
• Monitor each migration closely
• Keep ECS as fallback initially

Phase 4: Complete Migration:

• Migrate remaining services
• Decommission ECS cluster
• Update documentation and runbooks

Big Bang vs Strangler Pattern:

• Big Bang: All at once (high risk, faster)
• Strangler Pattern: Gradual service-by-service (lower risk, recommended)

12. Operational Changes

Team Training:

• Train teams on Kubernetes concepts (pods, services, namespaces, etc.)
• Learn kubectl commands and workflows
• Understand YAML manifests and Helm charts
• Practice troubleshooting techniques

New Tools & Skills:

• kubectl (CLI tool)
• Helm (package manager)
• Kustomize (configuration management)
• GitOps tools (ArgoCD, FluxCD)
• Kubernetes dashboard or Lens (GUI tools)

Runbooks & Documentation:

• Update incident response procedures
• Document new deployment processes
• Create troubleshooting guides for Kubernetes
• Update disaster recovery plans

13. Testing & Validation

Pre-Migration Testing:

• Load testing in EKS environment
• Failover testing (pod crashes, node failures)
• Database connection testing
• Performance benchmarking vs ECS

Post-Migration Validation:

• Monitor error rates and latency
• Verify all integrations working
• Check cost metrics
• Validate backup/restore procedures
• Test rollback procedures

14. Common Pitfalls to Avoid

• ⚠️ Resource limits: Not setting proper requests/limits causes instability
• ⚠️ Health checks: Incorrect probes cause pod restarts
• ⚠️ Secrets exposure: Storing secrets in ConfigMaps instead of Secrets
• ⚠️ Network policies: Forgetting to configure proper pod communication
• ⚠️ Storage: Not planning for persistent storage needs
• ⚠️ IAM roles: Not properly configuring IRSA
• ⚠️ Monitoring gaps: Losing observability during migration
• ⚠️ DNS caching: Application DNS caching issues with service discovery

15. Migration Checklist

• EKS cluster setup with multi-AZ configuration
• Install AWS Load Balancer Controller
• Install EBS and EFS CSI drivers
• Configure IAM OIDC provider
• Set up IRSA for applications
• Convert all task definitions to K8s manifests
• Migrate secrets to Secrets Manager with External Secrets Operator
• Configure monitoring and logging
• Set up CI/CD pipelines for K8s
• Update security groups and network policies
• Test in staging environment
• Create rollback plan
• Train team on Kubernetes operations
• Update documentation
• Plan maintenance window for cutover

Summary
The migration from ECS to EKS is significant but manageable with proper planning. Key success factors:

• Gradual migration (service by service)
• Thorough testing at each stage
• Team training on Kubernetes
• Robust monitoring throughout the process
• Clear rollback plans

Conclusion

Migrating from ECS to EKS is a significant undertaking, but with proper planning and a phased approach, it's absolutely achievable. The key is to start small, test thoroughly, and migrate service by service rather than attempting everything at once.

Remember that this migration isn't just about changing technology. It's about adopting a new way of thinking about container orchestration. Invest time in training your team, building robust monitoring, and documenting your processes. The flexibility and ecosystem that Kubernetes provides will be worth the effort.

Take your time, plan carefully, and don't hesitate to run both platforms in parallel during the transition. Your future self will thank you for the careful approach.

Happy migrating! 🤝😊