How to Run a Resilient AI Agent on ECS Fargate Spot — Service Connect, Retry Logic, and Blue/Green Deploys

Container‑based AI agents get tripped up by spot interruptions, but you can turn that weakness into a strength. In this guide we stitch together Service Connect, smart retry handling, and CDK‑driven blue/green pipelines so your agent stays alive and up‑to‑date. You’ll walk away with a production‑ready pattern you can drop into any Node.js/TypeScript project.

In plain English: Spot instances are cheap, but they can disappear at any moment. By catching the disappearance inside the container and letting ECS know the task failed, the platform will automatically start a fresh copy.

---

What Is an AI Agent and Why Run It on ECS?

AI agent – a piece of code that continuously reads input, makes a decision using a model (for example OpenAI’s Chat Completion API), and then performs an action. Think of it as a digital receptionist that never sleeps.

Why choose Amazon ECS over Lambda or Cloud Run?

Feature	Serverless functions (Lambda, Cloud Run)	Amazon ECS on Fargate
Execution time	Limited to a few minutes (max 15 min for Lambda)	Unlimited, runs as long as you need
Memory & CPU granularity	Fixed tiers, hard to fine‑tune	Choose exactly the vCPU and memory you want
Networking	Simple, but hard to keep persistent sockets	Full VPC networking, Service Connect for service‑to‑service calls
Debugging	Logs only, no interactive shell	You can attach a debugger to a running container

An autonomous agent often needs a persistent TCP connection (e.g., a WebSocket to a chat platform) and the freedom to adjust its resources while it learns. Containers give you that control.

Below is a tiny TypeScript “agent” that prints a heartbeat every 30 seconds. It runs forever – the kind of loop you would replace with real AI logic later.

// src/heartbeatAgent.ts
// This file shows the shape of a long‑running process.
// No AWS code yet – just a Node.js loop.

import { setInterval } from "timers";

// Print a timestamp every 30 seconds so we can see the container stays alive.
setInterval(() => {
  const now = new Date().toISOString();
  console.log(`[heartbeat] ${now} – agent is alive`);
}, 30_000);

// Keep the process from exiting.
// In a real agent you would replace this with your AI‑driven logic.
process.stdin.resume();

Key takeaway: ECS gives you a stable home for processes that need to run indefinitely, something serverless platforms struggle with.

---

Setting Up the ECS Fargate Spot Cluster with Service Connect

Spot Fargate in a nutshell

Spot Fargate lets you run containers on spare capacity that AWS sells at a discount (often 70 % cheaper). The trade‑off is that AWS can reclaim the underlying compute at any time with a two‑minute warning. The warning shows up inside the container as a sudden network error, not as a special ECS status code.

Service Connect

Service Connect is AWS’s built‑in service mesh for ECS. It gives each task a DNS name and a stable endpoint, and it can route traffic between services without exposing public ports. The downside is a small latency penalty (usually a few milliseconds) that becomes noticeable only under heavy load.

CDK code to create the cluster, capacity provider, and service

The following CDK (Cloud Development Kit) snippet does three things:

1. Creates an ECS cluster with a default VPC.
2. Adds a Fargate Spot capacity provider so tasks can be scheduled on cheap spare capacity.
3. Defines a Service Connect namespace and attaches the AI‑agent service to it.

// infra/ecs-stack.ts
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import {
  Cluster,
  FargateTaskDefinition,
  ContainerImage,
  AwsLogDriver,
  CapacityProviderStrategy,
  Service,
  ServiceConnectOptions,
} from "aws-cdk-lib/aws-ecs";
import { SubnetType } from "aws-cdk-lib/aws-ec2";

export class AiAgentEcsStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // 1️⃣  Create a VPC (the default VPC is fine for demos)
    const vpc = cdk.Stack.of(this).vpc ?? new cdk.aws_ec2.Vpc(this, "Vpc", {
      maxAzs: 2,
      subnetConfiguration: [{ name: "public", subnetType: SubnetType.PUBLIC }],
    });

    // 2️⃣  Build the ECS cluster
    const cluster = new Cluster(this, "AiAgentCluster", { vpc });

    // 3️⃣  Enable Service Connect namespace (optional but helpful)
    const scNamespace = cluster.addDefaultCloudMapNamespace({
      name: "ai-agent.local",
    });

    // 4️⃣  Define the task – we use Spot capacity later
    const taskDef = new FargateTaskDefinition(this, "AiAgentTask", {
      // Spot tasks need the "spot" capacity provider strategy later
      cpu: 256, // 0.25 vCPU
      memoryLimitMiB: 512,
      // Execution role gives the container permission to pull images and write logs
      executionRole: new cdk.aws_iam.Role(this, "ExecRole", {
        assumedBy: new cdk.aws_iam.ServicePrincipal("ecs-tasks.amazonaws.com"),
        managedPolicies: [
          cdk.aws_iam.ManagedPolicy.fromAwsManagedPolicyName(
            "service-role/AmazonECSTaskExecutionRolePolicy"
          ),
        ],
      }),
    });

    // 5️⃣  Add the container that runs our agent code
    taskDef.addContainer("AiAgentContainer", {
      image: ContainerImage.fromAsset("../app"), // Dockerfile lives in ../app
      logging: new AwsLogDriver({ streamPrefix: "AiAgent" }),
      environment: {
        // Pass any required env vars, e.g. OpenAI key
        OPENAI_API_KEY: process.env.OPENAI_API_KEY ?? "",
      },
    });

    // 6️⃣  Create the service that will run on Spot
    const service = new Service(this, "AiAgentService", {
      cluster,
      taskDefinition: taskDef,
      desiredCount: 1,
      // Spot capacity provider strategy
      capacityProviderStrategies: [
        {
          capacityProvider: "FARGATE_SPOT",
          weight: 1,
          base: 0,
        },
      ],
      // Hook Service Connect so other services can call us by name
      serviceConnectConfiguration: {
        enabled: true,
        namespace: scNamespace,
        services: [
          {
            portMappingName: "http", // default mapping
            discoveryName: "agent", // becomes agent.ai-agent.local
            dnsName: "agent",
            port: 8080,
          },
        ],
      },
    });

    // 7️⃣  Output the DNS name for debugging
    new cdk.CfnOutput(this, "AgentDns", {
      value: `${service.serviceName}.${scNamespace.namespaceName}`,
    });
  }
}

Tip: The most common permission snag is mixing up the task role (what the container can do at runtime) and the execution role (what ECS needs to start the container). Make sure the execution role has AmazonECSTaskExecutionRolePolicy; otherwise the task will silently fail to pull the image.

---

Implementing Retry Logic for Spot Interruptions

The hidden error

When AWS stops a Spot task, the container’s network stack drops connections, and the next HTTP request typically throws ECONNRESET or ETIMEDOUT. There is no special “SpotInterrupted” exception from the SDK, so we have to infer it ourselves.

Why retry inside the process?

If we simply let the error bubble up and exit, ECS will see the container exit with a non‑zero code and launch a new task – that works, but we lose any in‑flight request data. By catching the error, logging, and then exiting cleanly, we give the platform a chance to restart while preserving observability.

Analogy

Think of a bus driver who encounters a sudden road closure. The driver can either keep trying to drive through (wasting fuel) or pull over, inform the dispatcher, and wait for a replacement bus. Our retry loop is the driver politely stopping and notifying the dispatcher (ECS).

Code: a resilient OpenAI call with exponential back‑off

// src/openaiClient.ts
import fetch from "node-fetch";

/**
 * Calls OpenAI's chat completion endpoint.
 * Retries on network errors that are typical for Spot interruptions.
 *
 * @param prompt The user message we want a response for.
 * @returns The assistant's reply text.
 */
export async function fetchChatCompletion(prompt: string): Promise<string> {
  const maxAttempts = 5;
  const baseDelayMs = 1_000; // start with 1 second

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      // -----------------------------------------------------------------
      // 1️⃣  Perform the HTTP request.
      // -----------------------------------------------------------------
      const response = await fetch("https://api.openai.com/v1/chat/completions", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
        },
        body: JSON.stringify({
          model: "gpt-4o-mini",
          messages: [{ role: "user", content: prompt }],
        }),
      });

      // -----------------------------------------------------------------
      // 2️⃣  If the API returned an error status, surface it.
      // -----------------------------------------------------------------
      if (!response.ok) {
        const errBody = await response.text();
        throw new Error(`OpenAI error ${response.status}: ${errBody}`);
      }

      // -----------------------------------------------------------------
      // 3️⃣  Parse the successful response.
      // -----------------------------------------------------------------
      const data = (await response.json()) as any;
      return data.choices[0].message.content;
    } catch (err: any) {
      // ---------------------------------------------------------------
      // 4️⃣  Detect Spot‑related network failures.
      // ---------------------------------------------------------------
      const isSpotInterrupted =
        err.code === "ECONNRESET" ||
        err.code === "ETIMEDOUT" ||
        err.message?.includes("socket hang up");

      if (!isSpotInterrupted) {
        // Not a Spot problem – rethrow so the outer process can see it.
        throw err;
      }

      // ---------------------------------------------------------------
      // 5️⃣  If we have exhausted retries, give up and let ECS restart.
      // ---------------------------------------------------------------
      if (attempt === maxAttempts) {
        console.error(
          `[retry] Spot interruption persisted after ${maxAttempts} attempts – exiting`
        );
        // Signal failure to ECS by exiting with a non‑zero code.
        process.exit(1);
      }

      // ---------------------------------------------------------------
      // 6️⃣  Wait with exponential back‑off before the next try.
      // ---------------------------------------------------------------
      const delay = baseDelayMs * Math.pow(2, attempt - 1);
      console.warn(
        `[retry] Spot interruption detected (attempt ${attempt}/${maxAttempts}). Waiting ${delay} ms before retry...`
      );
      await new Promise((res) => setTimeout(res, delay));
    }
  }

  // This line is unreachable but TypeScript expects a return.
  throw new Error("Unreachable");
}

In plain English: The function tries the OpenAI call up to five times. If it sees a network error that looks like a Spot stop, it waits a little longer each time. After the last try it quits the container, which tells ECS “this task died” and triggers a fresh start.

---

Blue/Green Deploys with CodeDeploy and CDK

What is blue/green deployment?

Instead of replacing the running containers in place (a “rolling” update), blue/green creates a second set of tasks (the “green” version) alongside the current (“blue”) version. Once the green tasks pass health checks, traffic is shifted to them. If anything goes wrong, you can instantly roll back to blue.

Why pair it with Service Connect?

Service Connect can act as a traffic router that moves requests from the old DNS name to the new one. This lets you switch without changing client code.

CDK constructs for CodeDeploy

The code below builds a CodeDeploy Application and DeploymentGroup that knows how to shift traffic between the two ECS services. It also creates the green service (identical to the blue one) but initially with desiredCount: 0.

// infra/deploy-stack.ts
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import {
  Application as CodedeployApp,
  ServerDeploymentGroup,
  ServerDeploymentConfig,
} from "aws-cdk-lib/aws-codedeploy";
import {
  Cluster,
  FargateService,
  FargateTaskDefinition,
  ContainerImage,
  AwsLogDriver,
  CapacityProviderStrategy,
} from "aws-cdk-lib/aws-ecs";

export class AiAgentDeployStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Reuse the same cluster we defined earlier
    const cluster = Cluster.fromClusterAttributes(this, "ExistingCluster", {
      clusterName: "AiAgentCluster",
      securityGroups: [], // add as needed
      vpc: cdk.Stack.of(this).vpc!,
    });

    // ------------------------------
    // 1️⃣  Create a CodeDeploy application (logical name)
    // ------------------------------
    const cdApp = new CodedeployApp(this, "AiAgentCodeDeployApp", {
      applicationName: "AiAgentApp",
    });

    // ------------------------------
    // 2️⃣  Define the task definition (shared by both blue & green)
    // ------------------------------
    const taskDef = new FargateTaskDefinition(this, "SharedTaskDef", {
      cpu: 256,
      memoryLimitMiB: 512,
    });

    taskDef.addContainer("AiAgentContainer", {
      image: ContainerImage.fromAsset("../app"),
      logging: new AwsLogDriver({ streamPrefix: "AiAgentDeploy" }),
      environment: {
        OPENAI_API_KEY: process.env.OPENAI_API_KEY ?? "",
      },
    });

    // ------------------------------
    // 3️⃣  Blue service – the one that is already serving traffic
    // ------------------------------
    const blueService = new FargateService(this, "BlueService", {
      cluster,
      taskDefinition: taskDef,
      desiredCount: 1,
      capacityProviderStrategies: [
        {
          capacityProvider: "FARGATE_SPOT",
          weight: 1,
        },
      ],
    });

    // ------------------------------
    // 4️⃣  Green service – initially idle
    // ------------------------------
    const greenService = new FargateService(this, "GreenService", {
      cluster,
      taskDefinition: taskDef,
      desiredCount: 0, // start stopped
      capacityProviderStrategies: [
        {
          capacityProvider: "FARGATE_SPOT",
          weight: 1,
        },
      ],
    });

    // ------------------------------
    // 5️⃣  Deployment group that knows how to switch traffic
    // ------------------------------
    new ServerDeploymentGroup(this, "AiAgentDeployGroup", {
      application: cdApp,
      deploymentGroupName: "AiAgentDG",
      service: blueService,
      // Tell CodeDeploy that the green service is the target for the new version
      targetService: greenService,
      deploymentConfig: ServerDeploymentConfig.CANARY_10PERCENT_5MINUTES,
      // Give CodeDeploy permission to call ECS APIs
      autoRollback: {
        failedDeployment: true,
        stoppedDeployment: true,
        deploymentInAlarm: true,
      },
    });

    // ------------------------------
    // 6️⃣  Grant the CodeDeploy role permissions (gotcha!)
    // ------------------------------
    const codedeployRole = cdApp.role!;
    codedeployRole.addManagedPolicy(
      cdk.aws_iam.ManagedPolicy.fromAwsManagedPolicyName(
        "AWSCodeDeployRoleForECS"
      )
    );
  }
}

Tip: A frequent mistake is forgetting to attach the AWSCodeDeployRoleForECS policy. Without it, the deployment will stall silently because CodeDeploy cannot tell ECS to start the green tasks.

When you push a new container image to ECR and run cdk deploy, the pipeline will:

1. Update the task definition (new image digest).
2. Trigger CodeDeploy to start the green service with the fresh task.
3. Shift 10 % of traffic for five minutes (canary), then 100 % if no alarms fire.
4. If something breaks, CodeDeploy rolls back automatically.

All of this happens without touching the client code because Service Connect resolves the DNS name agent.ai-agent.local to whichever service is currently “green”.

---

Wiring the OpenAI Call and Adding Observability

Putting the pieces together

Our container’s entry point will import the retry‑aware client, call it in a loop, and push some simple metrics to CloudWatch. Observability lets us see whether Spot interruptions are happening often enough to tweak the discount level.

// src/main.ts
import { fetchChatCompletion } from "./openaiClient";
import * as AWS from "aws-sdk";

// CloudWatch client for custom metrics
const cw = new AWS.CloudWatch({ region: process.env.AWS_REGION });

/**
 * Sends a single metric datapoint.
 * @param name Metric name
 * @param value Numeric value
 */
async function putMetric(name: string, value: number) {
  await cw
    .putMetricData({
      Namespace: "AiAgent",
      MetricData: [
        {
          MetricName: name,
          Timestamp: new Date(),
          Value: value,
          Unit: "Count",
        },
      ],
    })
    .promise();
}

/**
 * Main loop – fetch a response from OpenAI and log it.
 * In a real agent you would replace the static prompt with dynamic input.
 */
async function runAgent() {
  while (true) {
    try {
      const reply = await fetchChatCompletion(
        "Explain the difference between blue/green and rolling deploys."
      );
      console.log(`[assistant] ${reply}`);

      // Record a successful call
      await putMetric("SuccessfulCalls", 1);
    } catch (err) {
      console.error("[error] Unexpected failure:", err);
      // Record a failure metric – helpful for alerting
      await putMetric("FailedCalls", 1);
    }

    // Wait a bit before the next request to avoid hitting rate limits.
    await new Promise((res) => setTimeout(res, 15_000));
  }
}

// Start the loop
runAgent().catch((e) => {
  console.error("[fatal] Agent crashed:", e);
  process.exit(1);
});

In plain English: Each successful OpenAI request increments a SuccessfulCalls counter. If the retry logic eventually exits because Spot kept interrupting, the container will die, ECS will restart it, and the metric will help you spot a pattern.

Observability gotchas

• Task role vs execution role: The code above needs the task role to call cloudwatch:PutMetricData. If you only grant this permission to the execution role, the call will be denied with a silent AccessDenied that appears only in CloudWatch logs.
• Service Connect latency: Adding the extra hop adds ~2–3 ms per request. If you are latency‑sensitive, measure it in a staging environment before enabling it for all traffic.

---

The Takeaway

Key points to remember

• Spot Fargate cuts compute cost dramatically, but you must treat its interruptions as ordinary network failures inside the container.
• Service Connect gives each task a stable DNS name and simplifies traffic shifting, at the cost of a tiny latency increase.
• A retry loop that recognises ECONNRESET/ETIMEDOUT, backs off exponentially, and exits after a few attempts lets ECS automatically replace a stopped Spot task.
• Blue/green deployments with CodeDeploy and CDK provide a safe, zero‑downtime upgrade path; remember to attach the AWSCodeDeployRoleForECS policy.
• Separate execution role (pull images, write logs) from task role (call CloudWatch, other AWS services) to avoid silent permission failures.
• Adding a few custom CloudWatch metrics gives you visibility into how often Spot interruptions happen and whether your discount level is appropriate.

By following the pattern above you can ship a long‑running AI agent that stays cheap, stays online, and stays observable—all with familiar TypeScript and AWS tooling. Happy building!

---

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-08-19 · Primary focus: ECS

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.