ECS Deployment circuit breaker with Serverless Fargate

Nilay — Fri, 19 Jan 2024 08:56:06 +0000

🚦 ECS Deployment Circuit Breaker with Serverless Fargate

Stop Bad Deployments Before They Break Your App

Deployments are exciting—until they fail in production. 😅 That’s where Amazon ECS Deployment Circuit Breaker comes in.
Think of it as a safety switch: it catches failed deployments early, rolls back automatically, and keeps your app running smoothly.

💡 Why You Should Care

Here’s what the circuit breaker does for you:

🔄 Auto Rollback – Bad deploy? ECS flips back to the last good version.
⏱ Less Downtime – Faulty tasks get stopped before they impact users.
🛡 Safety Net – ECS continuously checks health and cancels bad rollouts.
😴 Sleep Easy – No more 3 AM firefights over broken deployments.

⚙️ How It Works

ECS tracks deployment failures in two stages:

1️⃣ Task Startup

✅ Success: At least one task reaches RUNNING → move to health checks.
❌ Failure: Multiple tasks fail to start → deployment may be marked as FAILED.

2️⃣ Health Checks

✅ Success: At least one task passes ELB, Cloud Map, or container health checks.
❌ Failure: Health check failures exceed threshold → deployment fails → rollback triggered.

Failure Threshold Example:

Desired Task Count	Threshold (min 3, max 200)
1	3
25	13
400	200
800	200 (capped)

🚀 Setup with Serverless + Fargate

Enable circuit breaker in your serverless.yml:

cloudFormationResource:
  service:
    DeploymentConfiguration:
      DeploymentCircuitBreaker:
        Enable: true
        Rollback: true

Done ✅ — your service now has a safety net.

🧪 Test It Out

Deploy a simple Express app:

const express = require('express');
const app = express();
app.get('/', (_, res) => res.send('Health Check!'));
app.listen(3000, () => console.log(`Running on :3000`));

Build & push Docker image.
Simulate a bad deploy by forcing an error in Dockerfile:

CMD ["EXIT", "2"]

👉 ECS rolls back automatically, saving your users.

🎯 The Takeaway

The ECS Deployment Circuit Breaker is a simple switch with a huge payoff:

Fewer outages
Automatic rollbacks
Stress-free deployments

If you’re using Fargate + Serverless, flipping it on is a no-brainer.

DEV Community: Nilay