Originally published on graycloudarch.com.
Four weeks before the production cutover, we ran a platform review. Five microservices, 21 tasks running on EKS Fargate, Load Balancer Controller wired up, IRSA roles configured, health checks green. We walked through the production runbook for the first time as a group.
By the end of the meeting, we'd decided not to ship it.
Not because anything was broken. Because we kept arriving at the same four problems, and at every review they'd become harder to dismiss.
This isn't a post about how to migrate from EKS to ECS — that's covered in the companion post on running both orchestrators simultaneously. This is about why we made the decision, and what the evidence looked like.
The four problems that kept coming up
Operational complexity nobody wanted to own
EKS Fargate adds roughly fifteen concepts on top of "run a container." AWS Load Balancer Controller, TargetGroupBinding CRDs, Fargate profiles, IRSA pod annotations — each with its own version compatibility matrix, each capable of generating its own 2am incident.
The pattern we kept seeing: non-infrastructure engineers were touching Kubernetes concepts they shouldn't have needed to touch. Not because they were doing the wrong thing — because the operational surface area extended to them automatically. During incident reviews, every problem started with "is this a Kubernetes problem or an application problem?" That question has a cost. It requires someone who knows both layers well enough to triage in real time. At this team size, that person was always the same person.
The CDN edge problem EKS can't solve natively
One of the five services was structurally different. It ran on Proxmox VMs at a bare metal CDN provider — anycast-routed, where the CDN node IP is the service IP. The traffic routing requirement was physical: the service had to run on those specific nodes.
EKS's native answer for on-premises nodes is Virtual Kubelet, which adds complexity in a direction nobody wanted to manage. ECS Anywhere registers external compute via SSM activation. Same ECS API, same task definitions, same IAM model. The Proxmox VMs become an EXTERNAL capacity provider; the service runs on them identically to how it runs on Fargate.
That wasn't a tiebreaker. It was a primary reason. We weren't choosing ECS because EKS was technically inferior — we were choosing ECS because it could run all five services from one control plane without requiring a second orchestrator or a non-trivial workaround.
Standardization without the overhead
If the CDN service couldn't run on EKS natively, we had two options: accept two compute orchestrators permanently, or find one that handled both.
Two orchestrators means two IAM models (IRSA for EKS, task roles for ECS), two deployment pipelines, two mental models for anyone doing incident response. ECS flattened this. Task roles replace IRSA — same security model, simpler configuration. ALB integration is native — no Load Balancer Controller CRDs required. ECS Anywhere extends the same task definition and IAM model to external compute. One tooling surface for every service, regardless of where that service runs.
Cost at this scale
The EKS cluster fee runs ~$72/month before a single task. For five microservices at moderate traffic, the Fargate resource overhead for Kubernetes versus ECS Fargate adds up across instances. Not a dealbreaker on its own. But combined with the other three, it was additional weight on a scale that was already tilted.
What we removed
When the decision was made, the before-and-after was concrete:
| Component | EKS | ECS |
|---|---|---|
aws-load-balancer-controller |
Helm chart, version-pinned | Removed |
lbc-cross-account-role |
IAM role for LBC IRSA | Removed |
TargetGroupBinding CRDs |
Required per service | Removed — native ALB integration |
| IRSA pod annotations | Per-service, per-pod | ECS task roles |
| EKS cluster + Fargate profiles | common/modules/eks |
common/modules/ecs-cluster |
The LBC removal was the most meaningful. The AWS Load Balancer Controller is the right tool for what it does — but deploying and maintaining a CRD-based ingress controller to create an ALB is a layer of infrastructure that simply doesn't exist on ECS. Target group registration is native behavior. Removing it eliminated an entire failure domain.
The nonprod EKS cluster stayed running as the rollback target during the soak period, then was decommissioned once every service had passed 72 hours on ECS without incident.
What ECS doesn't give you
This is a post about why we moved, not a claim that ECS is always right.
No service mesh. If you need mTLS between services, you're adding Envoy manually or accepting unencrypted internal traffic.
ECS Anywhere runs in bridge networking mode — no awsvpc, no task-level ENIs, no security groups at the container level. On external nodes, port conflicts are your problem to manage.
No KEDA, no HPA with custom metrics, no cluster-level bin-packing. If you need those, ECS will constrain you.
For five stateless HTTP services with straightforward target-tracking autoscaling, none of those gaps mattered. The question isn't which orchestrator has more capability — it's which orchestrator can run these specific workloads at this team's operational capacity without the infrastructure becoming the bottleneck.
The thing that's easy to get wrong
The framing we almost fell into: EKS is the production-grade option and ECS is the stepping stone. That reverses the actual question.
EKS is the right answer when the team has — or is genuinely building toward — the operational capacity to run it: someone who knows the cluster internals, runbooks that account for Kubernetes-layer failures, and a service count and scaling complexity that justifies the overhead.
For most teams, that capacity isn't aspirational. It either exists now or it doesn't. The mistake is treating it as a destination you'll grow into while also running Kubernetes in production.
We'll migrate to EKS when — and if — the operational picture actually supports it. Until then, ECS is handling the load, the CDN edge service is running on-premises via ECS Anywhere, and the team can respond to incidents without a 15-minute "is this Kubernetes" tax.
The ECS → EKS migration later is largely mechanical: task definitions become Helm charts, task roles become IRSA annotations, ALB target group registration becomes ingress configuration. The container images don't change. If you build ECS as though you'll eventually migrate, the path is clear. What you don't want is to deploy EKS before the team is ready to operate it.
Evaluating EKS vs. ECS for a migration or a greenfield build? The on-premises and edge-compute dimension changes the calculus in ways most framework comparisons skip. Get in touch.
Top comments (0)