Blue/green deployments on ECS Fargate with CloudFormation.

#aws #python #docker #devops

I built this project to practice container deployments on AWS while preparing for the AWS Certified Developer – Associate exam. It is a small FastAPI notes API running on ECS Fargate behind an Application Load Balancer, with blue/green deployments defined in CloudFormation. This post walks through the design, what went wrong, and how I kept the cost close to zero.

Repo: https://github.com/PabsCueto/ecs-fargate-bluegreen

What it does

A FastAPI app with /health, /version and a small in-memory CRUD for notes.
A pipeline that builds the Docker image, tests it, and pushes it to ECR.
A CloudFormation template with an ALB, two target groups, and an ECS service that uses the built-in blue/green strategy.
Scripts to preview, deploy, and tear everything down.

The notes live in memory on purpose: I wanted the focus to be the deployment, not the database.

The pipeline: GitHub Actions with OIDC

The workflow assumes an IAM role through GitHub's OIDC provider, so there are no long-lived AWS keys in the repo. The trust policy is limited to one repository and branch, and the role can only push to one ECR repository.

Before pushing the image, the workflow starts the container inside the runner and calls /health and /version. I don't have a local Docker setup that can run current Docker Desktop, so this smoke test also acts as my "does it actually start?" check.

The infrastructure: ECS blue/green in CloudFormation

The ALB has two target groups and a listener rule that carries production traffic. The ECS service declares the blue/green strategy and tells ECS which rule and alternate target group to use:

EcsService:
  Type: AWS::ECS::Service
  Properties:
    DeploymentController:
      Type: ECS
    DeploymentConfiguration:
      Strategy: BLUE_GREEN
      BakeTimeInMinutes: 1
      DeploymentCircuitBreaker:
        Enable: true
        Rollback: true
    LoadBalancers:
      - ContainerName: api
        ContainerPort: 8000
        TargetGroupArn: !Ref TargetGroupBlue
        AdvancedConfiguration:
          AlternateTargetGroupArn: !Ref TargetGroupGreen
          ProductionListenerRule: !Ref ListenerProdRule
          RoleArn: !GetAtt EcsInfrastructureRole.Arn

Changing anything in the task definition (I change an environment variable) creates a new revision and starts a deployment. ECS starts the green tasks, waits for them to pass the health checks, moves the listener rule to the green target group, keeps both versions for the bake time, and then stops the old one. If the new version never becomes healthy, the circuit breaker rolls back.

Keeping the cost near zero

An ALB and Fargate tasks are billed per hour, so I designed around short-lived deployments:

deploy.sh --preview creates only a CloudFormation change set, so nothing is provisioned.
deploy.sh shows a cost warning and requires typing a confirmation before creating anything.
teardown.sh deletes the stack and runs check-costs.sh, which lists any load balancer, running task, NAT gateway, or Elastic IP left behind.
Tasks run in public subnets with a security group that only accepts traffic from the ALB, which avoids a NAT gateway.
Smallest Fargate size, one task, one-day log retention, and a 5-second deregistration delay.

Result

I deployed 1.0.0 (blue), then updated the version to 2.0.0 (green):

GET /version -> {"version":"1.0.0","color":"blue","hostname":"ip-172-31-67-143..."}
GET /version -> {"version":"2.0.0","color":"green","hostname":"ip-172-31-84-17..."}

The hostname changed, which shows traffic moved to a different set of tasks. ECS reported both deployments as successful (about 1 minute 25 seconds for the first one and 2 minutes 25 seconds for the blue/green update, including the bake time). Then I destroyed the stack.

What went wrong

1. OIDC: "Not authorized to perform sts:AssumeRoleWithWebIdentity". My trust policy matched the classic subject format (repo:owner/repo:ref:refs/heads/main), but the token for this repository included numeric IDs: repo:owner@<owner-id>/repo@<repo-id>:ref:refs/heads/main. I only found out by adding a temporary workflow step that decoded and printed the token claims.

2. Smoke test failing with curl exit code 56. The published port accepted connections before uvicorn was ready, and curl doesn't retry that error by default. Adding --retry-all-errors fixed it, and the workflow now prints the container logs when the smoke test fails.

3. CodeDeploy was not available. My first version used the CloudFormation CodeDeploy blue/green hook. The stack was created, but the update failed with SubscriptionRequiredException, and aws deploy list-applications returned the same error. I suspect a restriction of my account's plan, although I couldn't confirm that. I switched to ECS's native blue/green strategy, which needs no extra service and meets the same goal.

Limits

There is no live demo: I tear the stack down to avoid costs. The README has the evidence.
HTTP only, no domain or HTTPS.
I didn't measure downtime with a continuous request loop during the switch, and I didn't use a test listener to validate the green version before the shift. Those are natural next steps.

If you want to look at the code, it's here: https://github.com/PabsCueto/ecs-fargate-bluegreen. Feedback is welcome.

DEV Community