DEV Community: Tendong Brain Nkengafac

How to Build an AI Agent on AWS Using AWS AgentCore

Tendong Brain Nkengafac — Mon, 05 Oct 2026 18:34:25 +0000

Introduction

In under 100 lines of Python, you can build an AI agent that picks its own tools, reasons with Amazon Nova Pro on Amazon Bedrock, and runs as a serverless HTTP endpoint on Amazon Bedrock AgentCore Runtime. This article walks through that build step by step, from an empty folder to a deployed agent you can call from AWS Lambda.

I built this while following the Build Your First AI Agents with AWS AgentCore workshop from EduCloud Academy. I have expanded the session into a complete, reproducible guide, checked the code against the current SDK releases, and added the pitfalls I hit along the way.

Why agents need a serverless home. An agent's resource use is unpredictable: one request may answer instantly, the next may chain five tool calls and several model invocations. Paying for idle servers makes little sense for that pattern. AgentCore Runtime runs each session in an isolated microVM, scales to zero, and bills only for active consumption.

What AgentCore adds beyond hosting. AgentCore is a set of services for running agents in production: Runtime (hosting), Gateway (turning APIs and MCP servers into agent tools), Identity (safe access to external services on a user's behalf), Memory, Observability and more. This guide uses Runtime; the others plug into the same agent later.

What you will build

A Strands agent with two community tools (calculator, current_time) and one custom tool (letter_counter).
Debug logging that shows every model call and tool choice.
An explicit Amazon Nova Pro model on Bedrock.
An AgentCore-compatible HTTP service, tested locally with curl.
A cloud deployment on AgentCore Runtime, invoked from the CLI and from Lambda.

Tested with: Python 3.10+, strands-agents 1.57, strands-agents-tools 0.8, bedrock-agentcore 1.24 and bedrock-agentcore-starter-toolkit 0.3.13 (October 2026). These SDKs move fast, so check the versions if a command differs.

Architecture

Every request takes the same path: a caller sends JSON to the AgentCore endpoint, the Strands agent loops between Nova Pro and its tools, and a plain JSON answer comes back.

The upper half is the request path; the lower half is the one-time deployment path that packages agent.py into a container AgentCore can run.

Component	Role in this build
Strands Agents SDK	Open-source Python framework that runs the agent loop and tool calls
Amazon Bedrock (Nova Pro)	Managed foundation model that does the reasoning, via the Converse API
`bedrock-agentcore` SDK	Wraps the agent in an HTTP service with `/invocations` and `/ping`
AgentCore Runtime	Serverless host: one isolated microVM per session, billed on active use
AgentCore starter toolkit	CLI that configures, builds (CodeBuild), stores (ECR) and deploys the agent
AWS Lambda / Amazon ECS	Example callers that invoke the deployed agent through the AWS SDK
Amazon CloudWatch	Receives the runtime's logs and traces

Prerequisites and project setup

You need an AWS account, credentials configured locally, Python 3.10 or newer, and access to Amazon Nova Pro in your Region.

Requirement	How to check or set it up
AWS CLI v2 with credentials	`aws sts get-caller-identity` returns your account ID
Python 3.10+	`python3 --version`
Bedrock model access	Bedrock console → Model access → enable Amazon Nova Pro (us-east-1 used here)
IAM permissions (local dev)	`bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` on the Nova Pro inference profile
IAM permissions (deploy)	Rights to create IAM roles, ECR repositories, CodeBuild projects and AgentCore runtimes (the toolkit creates these for you)

Create the project and a virtual environment:

mkdir first-agent && cd first-agent
python3 -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

Create requirements.txt:

strands-agents
strands-agents-tools
bedrock-agentcore

Install the dependencies, plus the starter toolkit, which provides the agentcore CLI used for deployment:

pip install -r requirements.txt
pip install bedrock-agentcore-starter-toolkit

The toolkit is a development tool, so it stays out of requirements.txt; the runtime container only needs the three packages above. Install everything inside the active virtual environment. In the workshop, the most common stumble was a package installed globally and missing at runtime.

The final project is deliberately small:

first-agent/
├── agent.py           # agent, tools, model and AgentCore entrypoint
├── requirements.txt   # runtime dependencies
└── invoke_lambda.py   # optional: calling the deployed agent from Lambda

Step 1: A Strands agent with tools

A Strands agent is a model, a list of tools and a prompt; the framework runs the loop that lets the model decide which tool to call. Start with two community tools and one custom tool.

The custom tool counts letters in a word. It is a classic example because language models are unreliable at counting characters, while a five-line function is exact. The @tool decorator turns the function's signature and docstring into the tool specification the model reads, so a clear docstring matters.

# agent.py (Step 1)
from strands import Agent, tool
from strands_tools import calculator, current_time


@tool
def letter_counter(word: str, letter: str) -> int:
    """Count how many times a single letter appears in a word (case-insensitive).

    Args:
        word: The word or text to search.
        letter: The single letter to count.
    """
    if not isinstance(letter, str) or len(letter) != 1:
        raise ValueError("letter must be exactly one character")
    return word.lower().count(letter.lower())


agent = Agent(tools=[calculator, current_time, letter_counter])

if __name__ == "__main__":
    agent("What is the current time in UTC?")
    agent("What is 3111696 divided by 74088?")
    agent("How many times does the letter 'r' appear in 'strawberry'?")

Run it:

python agent.py

If you see ModuleNotFoundError: No module named 'strands_tools', the tools package went to another environment. Activate .venv and rerun pip install -r requirements.txt.

Expected output (the agent streams its answer to the terminal; wording varies between runs):

Tool #1: current_time
The current time in UTC is 2026-10-04T14:32:07+00:00.

Tool #2: calculator
3111696 divided by 74088 equals 42.

Tool #3: letter_counter
The letter 'r' appears 3 times in 'strawberry'.

Each prompt was routed to a different tool. That routing is the core agent loop: the model reads the tool specifications, chooses one, Strands executes it, and the result goes back to the model to write the final answer.

A note on the model. No model is configured in this snippet, but the agent is still calling one. When you omit model, Strands defaults to a Claude model on Amazon Bedrock (global.anthropic.claude-sonnet-4-6 in version 1.57). That default needs Bedrock access for that model, which is a good reason to set the model explicitly, as Step 2 does.

Step 2: Debug logging and an explicit Bedrock model

Two changes make the agent production-ready in spirit: logs that show its decisions, and a model you chose on purpose.

Logging. Setting the strands logger to DEBUG prints model configuration, tool registration and each step of the event loop. That visibility pays off most in multi-agent systems, where the trace shows exactly which agent or tool misbehaved.

Model. BedrockModel wraps the Bedrock Converse API. Here it targets Amazon Nova Pro through the US cross-region inference profile us.amazon.nova-pro-v1:0, which routes requests across US Regions for better availability. A low temperature keeps tool-heavy answers consistent.

# agent.py (Step 2): additions shown at the top of the file
import logging
import os

from strands import Agent, tool
from strands.models import BedrockModel
from strands_tools import calculator, current_time

# Strands debug logs show every model call and tool selection
logging.getLogger("strands").setLevel(logging.DEBUG)
logging.basicConfig(
    format="%(levelname)s | %(name)s | %(message)s",
    handlers=[logging.StreamHandler()],
)

AWS_REGION = os.getenv("AWS_REGION", "us-east-1")
MODEL_ID = os.getenv("MODEL_ID", "us.amazon.nova-pro-v1:0")  # cross-region inference profile

# ... letter_counter tool from Step 1 ...

model = BedrockModel(model_id=MODEL_ID, region_name=AWS_REGION, temperature=0.3)

agent = Agent(
    model=model,
    tools=[calculator, current_time, letter_counter],
    system_prompt="You are a concise assistant. Use a tool whenever one fits the request.",
)

Reading the Region and model ID from environment variables means you can switch to another model, such as Claude or Nova Lite, without touching code.

Expected output. The first lines below are what Strands 1.57 prints at start-up; the event-loop lines are abridged and vary by version.

DEBUG | strands.models.bedrock | config=<{'model_id': 'us.amazon.nova-pro-v1:0', 'include_tool_result_status': 'auto', 'temperature': 0.3}> | initializing
DEBUG | strands.models.bedrock | region=<us-east-1> | bedrock client created
DEBUG | strands.tools.registry | tool_name=<calculator>, tool_type=<function>, is_dynamic=<False> | registering tool
DEBUG | strands.tools.registry | tool_name=<current_time>, tool_type=<function>, is_dynamic=<False> | registering tool
DEBUG | strands.tools.registry | tool_name=<letter_counter>, tool_type=<function>, is_dynamic=<False> | registering tool
DEBUG | strands.event_loop.event_loop | ... | streaming messages
Tool #1: letter_counter
DEBUG | strands.event_loop.event_loop | ... | tool execution completed
The letter 'r' appears 3 times in 'strawberry'.

With the trace on, you can see the full path of a request: model configured, tools registered, model asked, tool chosen and executed, answer produced. When something breaks later, this is the first place to look.

Step 3: Wrap the agent for AgentCore Runtime and test locally

AgentCore Runtime expects an HTTP service with two routes: POST /invocations for requests and GET /ping for health checks, on port 8080. The bedrock-agentcore SDK provides both through four lines of code: import the app, initialise it, decorate an entrypoint, and run it.

Here is the complete agent.py:

"""First AI agent: Strands Agents + Amazon Bedrock (Nova Pro) on AgentCore Runtime."""
import logging
import os

from bedrock_agentcore.runtime import BedrockAgentCoreApp
from strands import Agent, tool
from strands.models import BedrockModel
from strands_tools import calculator, current_time

# Strands debug logs show every model call and tool selection
logging.getLogger("strands").setLevel(logging.DEBUG)
logging.basicConfig(format="%(levelname)s | %(name)s | %(message)s", handlers=[logging.StreamHandler()])
logger = logging.getLogger("first-agent")

AWS_REGION = os.getenv("AWS_REGION", "us-east-1")
MODEL_ID = os.getenv("MODEL_ID", "us.amazon.nova-pro-v1:0")  # cross-region inference profile


@tool
def letter_counter(word: str, letter: str) -> int:
    """Count how many times a single letter appears in a word (case-insensitive).

    Args:
        word: The word or text to search.
        letter: The single letter to count.
    """
    if not isinstance(letter, str) or len(letter) != 1:
        raise ValueError("letter must be exactly one character")
    return word.lower().count(letter.lower())


model = BedrockModel(model_id=MODEL_ID, region_name=AWS_REGION, temperature=0.3)

agent = Agent(
    model=model,
    tools=[calculator, current_time, letter_counter],
    system_prompt="You are a concise assistant. Use a tool whenever one fits the request.",
)

app = BedrockAgentCoreApp()


@app.entrypoint
def invoke(payload: dict) -> dict:
    """AgentCore entrypoint: receives the JSON body of each /invocations request."""
    prompt = payload.get("prompt", "Hello! What can you do?")
    logger.info("Received prompt: %s", prompt)
    result = agent(prompt)
    # AgentResult is not a clean API response; return plain text instead
    return {"result": str(result)}


if __name__ == "__main__":
    app.run()  # serves POST /invocations and GET /ping on port 8080

Test locally with two terminals. In the first, start the service:

python agent.py

In the second, check health, then send a prompt:

curl http://localhost:8080/ping

curl -X POST http://localhost:8080/invocations \
  -H "Content-Type: application/json" \
  -d '{"prompt": "How many times does the letter s appear in Mississippi?"}'

Expected output:

{"status":"Healthy","time_of_last_update":1791145026}

{"result": "The letter 's' appears 4 times in 'Mississippi'."}

The pitfall: returning the raw result. The workshop demo hit a runtime error at exactly this point, because the handler returned the agent's response object as-is. agent(prompt) returns an AgentResult, which holds the message plus metrics, traces and event-loop state. Depending on the SDK version and what that state contains, returning it directly either fails to serialise or leaks a large internal object to your callers. When I tested with bedrock-agentcore 1.24, the raw return succeeded but the response carried every metric and trace field. Returning {"result": str(result)} gives callers a small, stable contract; str() on an AgentResult yields the final text.

Step 4: Deploy to AgentCore Runtime and invoke it

The starter toolkit turns agent.py into a running cloud endpoint with two commands: configure and deploy. By default it builds an ARM64 container in AWS CodeBuild, so you do not need Docker on your machine.

Configure. Point the toolkit at your entrypoint. It detects requirements.txt and offers to create the IAM execution role and ECR repository for you; accepting the defaults is fine for a first deployment.

agentcore configure --entrypoint agent.py --name first_agent

This writes a .bedrock_agentcore.yaml file holding the agent's settings. Keep it under version control, but never commit credentials.

Deploy. In toolkit 0.3, this command replaced the older agentcore launch, which many tutorials still show.

agentcore deploy

When it finishes, the toolkit prints the agent runtime ARN. Check the status at any time:

agentcore status

Expected output (abridged; the account ID and ARN suffix are placeholders):

Deployment completed successfully
Agent ARN: arn:aws:bedrock-agentcore:us-east-1:123456789012:runtime/first_agent-AbCdEf1234

Invoke from the CLI:

agentcore invoke '{"prompt": "What is 15% of 2,480?"}'

Response:
{"result": "15% of 2,480 is 372."}

Invoke from AWS Lambda. Any AWS service that can call an API can use the agent, including Lambda and Amazon ECS. The bedrock-agentcore client's invoke_agent_runtime operation needs the runtime ARN, a JSON payload and a session ID of at least 33 characters. Requests that share a session ID reach the same isolated session, which keeps conversation context.

# invoke_lambda.py: Lambda handler that calls the deployed agent
import json
import logging
import os
import uuid

import boto3

logger = logging.getLogger()
logger.setLevel(logging.INFO)

client = boto3.client("bedrock-agentcore", region_name=os.environ.get("AWS_REGION", "us-east-1"))
AGENT_ARN = os.environ["AGENT_RUNTIME_ARN"]  # set in the Lambda configuration


def lambda_handler(event, context):
    prompt = event.get("prompt", "Hello!")
    session_id = event.get("session_id") or f"session-{uuid.uuid4()}"  # must be 33+ characters
    logger.info("Invoking agent, session=%s", session_id)

    response = client.invoke_agent_runtime(
        agentRuntimeArn=AGENT_ARN,
        runtimeSessionId=session_id,
        payload=json.dumps({"prompt": prompt}).encode(),
        contentType="application/json",
        accept="application/json",
    )
    body = json.loads(response["response"].read())  # response is a streaming body
    return {"statusCode": 200, "session_id": session_id, "answer": body.get("result")}

The Lambda execution role needs bedrock-agentcore:InvokeAgentRuntime on the runtime ARN. If your Lambda runtime's bundled boto3 predates AgentCore, package a current boto3 with the function.

Expected output for the test event {"prompt": "How many vowels are in 'Cameroon'?"}:

{
  "statusCode": 200,
  "session_id": "session-3f6c1a9e-2b7d-4c1e-9a55-0d8e7f4b2c11",
  "answer": "There are 4 vowels in 'Cameroon': a, e, o, o."
}

Troubleshooting, cost and cleanup

Most first-run failures come from environments, permissions or model access, not from the agent code.

Symptom	Likely cause	Fix
`ModuleNotFoundError: strands_tools`	Package installed outside the virtual environment	Activate `.venv`, reinstall from `requirements.txt`
`AccessDeniedException` calling the model	Model access not enabled, or IAM policy missing	Enable Nova Pro in Bedrock Model access; grant `bedrock:InvokeModel*`
`ValidationException` on the model ID	Inference profile not available in your Region	Use a profile valid for your Region, or set `MODEL_ID`
Raw metrics or a serialisation error in the response	Handler returns the `AgentResult` object	Return `{"result": str(result)}`
`agentcore launch` not found	Toolkit 0.3 renamed the command	Use `agentcore deploy`
Invocation works locally but fails in the cloud	Execution role lacks Bedrock permissions	Add `bedrock:InvokeModel*` to the runtime's execution role
`ValidationException` on `runtimeSessionId`	Session ID shorter than 33 characters	Prefix a UUID, as in the Lambda example

What it costs. You pay for three things: AgentCore Runtime (billed on active CPU and memory consumption per session, nothing while idle), Bedrock model tokens for Nova Pro, and small amounts of ECR storage, CodeBuild minutes and CloudWatch logs. For a tutorial-sized test, model tokens are usually the largest line. Check the Amazon Bedrock AgentCore pricing and Amazon Bedrock pricing pages for your Region, and set an AWS Budgets alert before you experiment.

Clean up when you are done, so nothing keeps billing:

agentcore destroy

This removes the runtime and the resources the toolkit created, such as the ECR repository and CodeBuild project. Confirm in the console that no AgentCore runtime remains, and delete the CloudWatch log groups if you no longer need them.

Conclusion and next steps

You now have an agent that chooses between three tools, reasons with Amazon Nova Pro, and runs on AgentCore Runtime behind an HTTP endpoint that Lambda can call. The code is under 100 lines; the platform handles isolation, scaling and session management.

Technical takeaways

Tools beat prompting for exact work. Counting, arithmetic and time lookups belong in functions; the model's job is to choose and explain.
Turn on debug logging from day one. The trace shows each decision, and it becomes essential once you add more agents.
Design the response contract. Return a small, explicit JSON shape from your entrypoint, never the framework's internal objects.
Test locally before deploying. Two terminals and curl catch most bugs in seconds instead of after a cloud build.

Lessons from the workshop beyond the code. The second half of the session was about how to grow as a cloud and AI engineer, and three points stayed with me. Real skill matters more than collecting certificates. Structured learning paths still matter, because AI tools and endless content make it easy to drift without depth. And public building, through weekend projects and blog posts like this one, creates opportunities that waiting never will.

Where to go next

Add AgentCore Memory so the agent remembers users across sessions.
Expose an existing API or MCP server as tools through AgentCore Gateway.
Use AgentCore Identity to let the agent act on a user's behalf in services like GitHub or Slack.
Add the Code Interpreter and Browser built-in tools for more autonomous tasks.
Wire up AgentCore Observability dashboards in CloudWatch.

References

How to Deploy a Machine Learning Project on AWS Using ECR, ECS Fargate, and EFS.

Tendong Brain Nkengafac — Sat, 09 May 2026 15:39:37 +0000

A step-by-step walkthrough from Docker image to a live, serverless ML application running in the cloud

Introduction

Deploying a machine learning project is often where things get humbling. You've trained a model, built a pipeline, maybe even wired up a slick dashboard, and then you stare at your terminal wondering how to get any of it onto a real server that other people can access. I've been there.

In this article, I'll walk you through exactly how I deployed a real-time machine learning application on AWS, from pushing a Docker image to ECR, to running two live containers on ECS Fargate, to watching my dashboard update in real time. Every command here actually worked. I'll explain what each step does, why it matters, and what you should expect to see as output.

Whether you're deploying your first ML project or looking for a reference you can actually trust, this guide is for you.

The Project: Real-Time Bike Rental Forecasting

Before we get into the AWS infrastructure, let me briefly describe what we're deploying.

I built a Real-Time Bike Rental Forecasting Application a live machine learning system that predicts hourly bike rental demand and visualizes the results in real time. The dashboard refreshes every second, showing a live comparison of predicted vs. actual bike rental counts over a rolling time window you control.

The application is built around three core services running simultaneously:

Model Training Pipeline : trains a CatBoost forecasting model on historical bike rental data
Inference Service : runs every second, simulating 1 hour of dataset time per tick, and produces predictions
Real-Time Dashboard : a Dash web app that fetches and plots the latest predicted vs. actual counts

The interface gives operators a simple control panel (you can adjust the display window from as little as 1 hour to 24 hours of history) and a live chart that updates continuously. When the model is tracking well, you see the green predicted line closely shadowing the pink actual line. It's a satisfying thing to watch.

Tech stack:

Kedro: ML Pipeline Orchestration
Forecasting ModelCatBoost: Forecasting Model
Dash (Plotly): Dashboard
Docker: Containerization
Amazon ECR: Image Registry
Amazon ECS with Fargate: Container Orchestration
Amazon EFS: Shared Storage

The UI and inference service run as separate Docker containers. They share data through a mounted volume, the inference container writes predictions, and the dashboard reads them. On AWS, that shared volume is handled by Amazon EFS.

Now let's deploy it.

Prerequisites

Before starting, make sure you have the following:

AWS CLI installed and configured (aws configure with your credentials)
Docker installed and running
PowerShell (these commands use PowerShell syntax. Adapt to bash if on Linux/Mac)
An AWS account with sufficient permissions (IAM, ECR, ECS, EC2, EFS)

Step 1 — Define Your Session Variables

Start every deployment session by setting these variables. They're referenced throughout every subsequent command, so getting them right upfront saves you a lot of pain.

$REGION = "us-east-1"
$ACCOUNT_ID = aws sts get-caller-identity --query Account --output text
$SERVICE_NAME = "bike-rental-forecasting"

What this does: $REGION sets the AWS region for all operations. $ACCOUNT_ID dynamically fetches your 12-digit AWS account number using the STS (Security Token Service) API, no hardcoding needed. $SERVICE_NAME is the name we'll use consistently for ECR repositories, images, and services.

Expected output for $ACCOUNT_ID:

048908710060

A 12-digit number. If this fails, your AWS CLI isn't configured. Run aws configure first.

Step 2 — Push Your Docker Image to Amazon ECR

Amazon ECR (Elastic Container Registry) is AWS's managed Docker image registry. Think of it like Docker Hub, but private and tightly integrated with the rest of AWS. ECS will pull your image from here when launching containers.

2.1 — Authenticate Docker with ECR

aws ecr get-login-password --region $REGION |
docker login --username AWS --password-stdin "$ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com"

This pipes a temporary ECR authentication token directly into the Docker login command. The token is valid for 12 hours.

Expected output:

Login Succeeded

If you see an authentication error, verify your IAM user has the AmazonEC2ContainerRegistryFullAccess policy attached.

2.2 — Create the ECR Repository (First Time Only)

If you haven't already created a repository for this project:

aws ecr create-repository --repository-name $SERVICE_NAME --region $REGION

Expected output: A JSON block describing your new repository, including the repositoryUri which looks like:

048908710060.dkr.ecr.us-east-1.amazonaws.com/bike-rental-forecasting

2.3 — Build Your Docker Image

docker build -f Dockerfile.aws -t $SERVICE_NAME .

We use a dedicated Dockerfile.aws here. It's good practice to have a separate Dockerfile for cloud deployments that strips out local dev dependencies and optimises for image size.

Expected output: A series of build steps ending in:

Successfully built <image-id>
Successfully tagged bike-rental-forecasting:latest

2.4 — Tag and Push the Image

docker tag "${SERVICE_NAME}:latest" `
  "$ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/${SERVICE_NAME}:latest"

docker push "$ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/${SERVICE_NAME}:latest"

The tag command gives your local image the full ECR address. The push uploads it layer by layer.

Expected output:

The push refers to repository [048908710060.dkr.ecr.us-east-1.amazonaws.com/bike-rental-forecasting]
latest: digest: sha256:abc123... size: 1234

Tip: If the push fails mid-way with a TLS timeout, don't panic — just re-run the push command. Docker is smart enough to detect already-uploaded layers and resume from where it left off. Only the remaining layers will be transferred.

Step 3 — Create the IAM Task Execution Role

ECS containers don't have your credentials. They need an IAM role to do things like pull images from ECR and write logs to CloudWatch. This role is called the task execution role, and it must exist before you can register any task definition.

3.1 — Write the Trust Policy and Create the Role

$trustPolicy = @"
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "Service": "ecs-tasks.amazonaws.com" },
    "Action": "sts:AssumeRole"
  }]
}
"@

[System.IO.File]::WriteAllText(
  "$PWD\ecs-trust-policy.json", $trustPolicy,
  [System.Text.UTF8Encoding]::new($false))

aws iam create-role --role-name ecsTaskExecutionRole `
  --assume-role-policy-document file://ecs-trust-policy.json

The trust policy tells AWS: "ECS tasks are allowed to assume this role." We write it to a file first (without a BOM, hence the UTF8Encoding trick) because the AWS CLI reads it from disk.

Expected output: A JSON block describing the newly created role, including its ARN:

{
  "Role": {
    "RoleName": "ecsTaskExecutionRole",
    "Arn": "arn:aws:iam::048908710060:role/ecsTaskExecutionRole",
    ...
  }
}

3.2 — Attach the Required Policies

aws iam attach-role-policy --role-name ecsTaskExecutionRole `
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

aws iam attach-role-policy --role-name ecsTaskExecutionRole `
  --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly

AmazonECSTaskExecutionRolePolicy covers CloudWatch Logs and Secrets Manager access. AmazonEC2ContainerRegistryReadOnly lets the task pull images from ECR. Both are AWS-managed policies, so they're always up to date.

Expected output: No output means success. AWS CLI returns nothing for successful attach-role-policy calls.

Note: If the role already exists from a previous deployment, you'll get an error on create-role. That's fine — skip creation and jump straight to the attach commands.

Step 4 — Create the ECS Cluster

The cluster is the logical grouping that holds all your services and tasks. With Fargate, there are no EC2 instances to manage, the cluster is purely an organizational boundary.

aws ecs create-cluster --cluster-name bike-rental-cluster --region $REGION

Expected output:

{
  "cluster": {
    "clusterName": "bike-rental-cluster",
    "status": "ACTIVE",
    "registeredContainerInstancesCount": 0,
    "runningTasksCount": 0,
    ...
  }
}

registeredContainerInstancesCount: 0 is expected with Fargate, there are no managed EC2 instances. Fargate provisions compute on-demand when tasks launch.

Step 5 — Configure Networking (VPC, Subnet, Security Group)

ECS Fargate tasks run inside your VPC. We need to identify the right subnet for them to launch in, and create a security group that controls what traffic can reach them.

5.1 — Get the Default VPC and Subnet

$VPC_ID = aws ec2 describe-vpcs `
  --filters "Name=isDefault,Values=true" `
  --query "Vpcs[0].VpcId" --output text --region $REGION

$SUBNET_ID = aws ec2 describe-subnets `
  --filters "Name=vpc-id,Values=$VPC_ID" `
  --query "Subnets[0].SubnetId" --output text --region $REGION

Expected output:

vpc-0ebc1234dar56786a
subnet-0328456779xbvhet0

We use the default VPC here for simplicity. In a production setup, you'd use a custom VPC with private subnets and a load balancer in front.

5.2 — Create a Security Group and Open Required Ports

$SG_ID = aws ec2 create-security-group `
  --group-name bike-rental-sg `
  --description "Bike Rental App Security Group" `
  --vpc-id $VPC_ID --query "GroupId" --output text --region $REGION

Expected output:

sg-1abc026def476580a

Now open the two ports we need:

# Port 8050 — Dash dashboard (HTTP traffic from the internet)
aws ec2 authorize-security-group-ingress `
  --group-id $SG_ID --protocol tcp --port 8050 `
  --cidr 0.0.0.0/0 --region $REGION

# Port 2049 — NFS/EFS (shared storage between containers)
aws ec2 authorize-security-group-ingress `
  --group-id $SG_ID --protocol tcp --port 2049 `
  --cidr 0.0.0.0/0 --region $REGION

Port 8050 is where Dash serves the dashboard. Port 2049 is the NFS protocol port that EFS uses. Both rules accept traffic from anywhere (0.0.0.0/0). You can restrict this by IP range for a more secure setup.

Expected output for each: A JSON block confirming the rule was added, showing the protocol, port range, and IP range.

Step 6 — Register Task Definitions

A task definition is ECS's equivalent of a docker-compose.yml. It describes what image to run, how much CPU and memory to allocate, what command to execute, and what ports to expose. We need two: one for the dashboard and one for the inference service.

6.1 — UI / Dashboard Task Definition

$taskDef = @"
{
  "family": "bike-rental-task",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512",
  "executionRoleArn": "arn:aws:iam::${ACCOUNT_ID}:role/ecsTaskExecutionRole",
  "containerDefinitions": [{
    "name": "bike-rental-app",
    "image": "${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/${SERVICE_NAME}:latest",
    "command": ["python", "entrypoints/app_ui.py"],
    "portMappings": [{"containerPort": 8050, "protocol": "tcp"}],
    "environment": [{"name": "PORT", "value": "8050"}]
  }]
}
"@

[System.IO.File]::WriteAllText(
  "$PWD\task-def.json", $taskDef,
  [System.Text.UTF8Encoding]::new($false))

aws ecs register-task-definition `
  --cli-input-json file://task-def.json --region $REGION

A few things worth noting:

"cpu": "256" means 0.25 vCPU. "memory": "512" means 512 MB. These are the smallest Fargate allocations, enough for a lightweight dashboard.
networkMode: awsvpc gives each task its own elastic network interface (ENI) with its own IP required for Fargate.
The command overrides the Dockerfile's default CMD, so we can reuse one image for both services.

Expected output: A large JSON block containing taskDefinitionArn:

arn:aws:ecs:us-east-1:038304770010:task-definition/bike-rental-task:1

The :1 at the end is the revision number. Every time you register a new version of the same task definition, this increments.

6.2 — Inference Task Definition

$inferenceTaskDef = @"
{
  "family": "bike-rental-inference-task",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512",
  "executionRoleArn": "arn:aws:iam::${ACCOUNT_ID}:role/ecsTaskExecutionRole",
  "containerDefinitions": [{
    "name": "bike-rental-inference",
    "image": "${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/${SERVICE_NAME}:latest",
    "command": ["python", "entrypoints/inference.py"]
  }]
}
"@

[System.IO.File]::WriteAllText(
  "$PWD\inference-task-def.json", $inferenceTaskDef,
  [System.Text.UTF8Encoding]::new($false))

aws ecs register-task-definition `
  --cli-input-json file://inference-task-def.json --region $REGION

Notice this task has no portMappings.The inference container doesn't serve HTTP traffic. It just runs the pipeline and writes results to the shared volume. The only difference from the UI task is the command.

Expected output:

arn:aws:ecs:us-east-1:038304770010:task-definition/bike-rental-inference-task:1

Step 7 — Set Up EFS Shared Storage

Both containers need to share data, the inference service writes predictions, and the dashboard reads them. In Docker locally, this is a named volume. On AWS, we use Amazon EFS (Elastic File System) — a managed, serverless NFS file system that multiple containers can mount simultaneously.

$EFS_ID = aws efs create-file-system `
  --performance-mode generalPurpose `
  --query "FileSystemId" --output text --region $REGION

Expected output:

fs-0abc1234def56789a

Then create a mount target. This is the network endpoint that makes EFS accessible from within your subnet:

aws efs create-mount-target `
  --file-system-id $EFS_ID `
  --subnet-id $SUBNET_ID `
  --security-groups $SG_ID --region $REGION

Expected output: A JSON block describing the mount target, including its IP address within the subnet. Wait a minute or two for the state to transition from creating to available before proceeding.

Why EFS? Unlike S3, EFS behaves like a traditional file system containers can open, write, and read files using normal file I/O. No special SDK needed. For ML pipelines that exchange files between services, this is the simplest approach on AWS.

To use EFS in your task definitions, add a volumes block and mountPoints to each container definition. Here's the pattern to add to your task JSON:

"volumes": [{
  "name": "shared-data",
  "efsVolumeConfiguration": {
    "fileSystemId": "<YOUR_EFS_ID>",
    "rootDirectory": "/"
  }
}],
"containerDefinitions": [{
  ...
  "mountPoints": [{
    "sourceVolume": "shared-data",
    "containerPath": "/app/data",
    "readOnly": false
  }]
}]

Step 8 — Create and Deploy ECS Services

An ECS service is what keeps your task running. If a task crashes, the service restarts it automatically. It also handles scaling, load balancer integration, and rolling deployments.

8.1 — Create the UI Service

aws ecs create-service `
  --cluster bike-rental-cluster `
  --service-name bike-rental-service `
  --task-definition bike-rental-task `
  --desired-count 1 `
  --launch-type FARGATE `
  --network-configuration `
  "awsvpcConfiguration={subnets=[$SUBNET_ID],securityGroups=[$SG_ID],assignPublicIp=ENABLED}" `
  --region $REGION

--desired-count 1 tells ECS to keep exactly one task running at all times. assignPublicIp=ENABLED gives the task a public IP so we can reach the dashboard from a browser.

Expected output: A large JSON block with "status": "ACTIVE" and "runningCount": 0. The count starts at 0 and increments as the task pulls the image and starts — usually takes 30–90 seconds.

8.2 — Create the Inference Service

aws ecs create-service `
  --cluster bike-rental-cluster `
  --service-name bike-rental-inference `
  --task-definition bike-rental-inference-task `
  --desired-count 1 `
  --launch-type FARGATE `
  --network-configuration `
  "awsvpcConfiguration={subnets=[$SUBNET_ID],securityGroups=[$SG_ID],assignPublicIp=ENABLED}" `
  --region $REGION

Expected output: Same structure as above with "serviceName": "bike-rental-inference".

8.3 — Force a Re-Deploy After Updates

Whenever you push a new image or update the task definition, trigger a forced re-deployment:

aws ecs update-service `
  --cluster bike-rental-cluster `
  --service bike-rental-service `
  --task-definition bike-rental-task `
  --force-new-deployment --region $REGION

aws ecs update-service `
  --cluster bike-rental-cluster `
  --service bike-rental-inference `
  --task-definition bike-rental-inference-task `
  --force-new-deployment --region $REGION

ECS will launch new tasks with the updated image, wait for them to become healthy, then stop the old ones, zero-downtime rolling deployment by default.

Step 9 — Get the Public IP of Your Running Task

Once the services are running, you need the public IP to access the dashboard.

$TASKS = aws ecs list-tasks --cluster bike-rental-cluster `
  --query "taskArns" --output text --region $REGION

foreach ($TASK in $TASKS.Split()) {
  $ENI = aws ecs describe-tasks --cluster bike-rental-cluster `
    --tasks $TASK `
    --query "tasks[0].attachments[0].details[?name=='networkInterfaceId'].value" `
    --output text --region $REGION

  $IP = aws ec2 describe-network-interfaces `
    --network-interface-ids $ENI `
    --query "NetworkInterfaces[0].Association.PublicIp" `
    --output text --region $REGION

  $CONTAINER = aws ecs describe-tasks --cluster bike-rental-cluster `
    --tasks $TASK `
    --query "tasks[0].containers[0].name" --output text --region $REGION

  echo "Container: $CONTAINER -> http://${IP}:8050"
}

This script loops over all running tasks, finds their network interface, resolves the public IP, and prints a clickable URL.

Expected output:

Container: bike-rental-app -> http://54.123.45.68:8050
Container: bike-rental-inference -> http://54.123.45.69:8050

Open the UI container's URL in your browser and you should see the live dashboard — predictions updating every second, the chart scrolling in real time.

Step 10 — Monitoring and Debugging

Once deployed, here are the commands you'll reach for most often.

Check overall service health:

aws ecs describe-services `
  --cluster bike-rental-cluster `
  --services bike-rental-service bike-rental-inference `
  --region $REGION `
  --query "services[*].{Name:serviceName,Running:runningCount,Pending:pendingCount,Status:status}"

Expected output:

[
  { "Name": "bike-rental-service", "Running": 1, "Pending": 0, "Status": "ACTIVE" },
  { "Name": "bike-rental-inference", "Running": 1, "Pending": 0, "Status": "ACTIVE" }
]

Running: 1 for both means everything is healthy.

Check recent service events (where errors appear):

aws ecs describe-services `
  --cluster bike-rental-cluster `
  --services bike-rental-service `
  --region $REGION --query "services[0].events"

This is the first place to look when a deployment fails. ECS logs events like image pull failures, task launch errors, and health check failures here in chronological order.

List all running tasks:

aws ecs list-tasks --cluster bike-rental-cluster --region $REGION

Verify security group rules are correct:

aws ec2 describe-security-groups `
  --group-ids $SG_ID --region $REGION `
  --query "SecurityGroups[0].IpPermissions"

If your dashboard is unreachable, a missing or misconfigured security group rule is the most common culprit. Check that port 8050 is open with 0.0.0.0/0 CIDR.

Architecture Summary

Here's a bird's-eye view of what we built:

Key Takeaways

Deploying an ML application on AWS doesn't require Kubernetes or a platform team. With ECR, ECS Fargate, and EFS, you get:

Managed container orchestration : no EC2 instances to patch or maintain
Serverless scaling : Fargate provisions compute on-demand
Shared persistent storage : EFS lets multiple containers share a file system like they're on the same machine
Fast iteration : push a new image, run update-service --force-new-deployment, done

The full deployment from a working Docker image to a live public URL took me less than an hour following this sequence. The trickiest parts were the IAM role setup (easy to get wrong, hard to debug) and remembering that ECS events — not CloudWatch Logs — is where deployment errors show up first.

I hope this guide saves you some of the trial and error I went through. If you have questions or run into something unexpected, drop a comment below.

All commands were tested and verified during an actual deployment. Account IDs in examples are illustrative.

Find Attached the Link to the deployed application: https://drive.google.com/file/d/1tQp4gdg3AauQ8eNbeLZFdivRdCnvMRB9/view?usp=sharing

Learning AI Evaluation on AWS Without the Complexity

Tendong Brain Nkengafac — Sat, 09 May 2026 13:51:02 +0000

Tidding Ramsey

May 9

How I Evaluated an AI Model on AWS Without Writing a Single Line of Training Code

#ai #aws #machinelearning #cloudcomputing

6 min read