szymon-szym for AWS Community Builders

Posted on Nov 23

Deploy Rust Agent to AWS AgentCore Runtime with GitHub actions

#aws #ai #rust #cdk

AgentCore Runtime works out of the box with Python frameworks. It also allows for deploying agents created with other languages. Using Rust to build an agent sounds like a nice excuse to explore this territory

Goal

For this blog post, I would like to:

deploy containerised Rust application to AgentCore Runtime
define all infrastructure as code (AWS CDK in my case)
test agent locally
build and deploy in CI/CD pipeline (GitHub Actions)

The agent itself will be simple, as I want to focus on the deployment process.

Architecture

The code is available IN THE REPOSITORY (on the 01-initial branch)

Agent

The agent in the AgentCore Runtime is nothing but a web application that exposes two endpoints: /ping and /invocations

For my project, I use the axum framework. Let's define request and response types:

// main.rs
//...
#[derive(Serialize)]
pub struct StatusResponse {
    #[serde(rename = "status")]
    status_msg: String,
}

#[derive(Debug, Serialize, Deserialize)]
pub struct InvocationsRequest {
    prompt: String,
}

#[derive(Serialize)]
pub struct InvocationsResponse {
    message: String,
}

//...

/ping endpoint is expected to return {"status": "healthy"}
/invocations will accept an object with prompt and return message.

/Invocations

The handler looks like this:

#[debug_handler]
async fn invocations(
    State(state): State<AppState>,
    Json(payload): Json<InvocationsRequest>,
) -> Json<InvocationsResponse> {
    let prompt = payload.prompt;

    let response = state.agent.prompt(prompt).await.unwrap();

    let response = InvocationsResponse { message: response };
    Json(response)
}

I use axum extractors for getting a state with a prepared rig agent, and for working with json

Main function

I initialise the AWS SDK client to use it with rig, and register handlers

// main.rs
// ...
#[tokio::main]
async fn main() {
    tracing_subscriber::fmt().json().init();

    let aws_config = aws_config::from_env().region("us-east-1").load().await;

    let bedrock_runtime_client = aws_sdk_bedrockruntime::Client::new(&aws_config);

    let client = Client::from(bedrock_runtime_client);

    let agent = client
        .agent(AMAZON_NOVA_PRO)
        .preamble("You are helpful assistant. Respond only with the answer. Be concise.")
        .build();

    let state = AppState { agent: agent };

    let app = Router::new()
        .route("/ping", get(ping))
        .route("/invocations", post(invocations))
        .with_state(state);

    let listener = TcpListener::bind("0.0.0.0:8080").await.unwrap();
    axum::serve(listener, app).await.unwrap();
}
// ...

Now I can test my agent locally.
I can run cargo run and CURL the prompt:

Deployment

With the agent working locally is now a time to start working on the deployment process.

From here on, I iterated a lot to figure out how to make it work. To keep things clear, I won't describe all my struggles in chronological order. Some things were obvious, while others turned out to be rather tricky.

Dockerfile

AgentCore Runtime supports only ARM images. I will build the artefact using GitHub Actions ARM runner, so I don't need any emulation.

FROM rust:1.91 AS builder

RUN rustup target add aarch64-unknown-linux-gnu

WORKDIR /app

COPY . .

RUN cargo build --release --target aarch64-unknown-linux-gnu

FROM ubuntu:24.04 AS runtime

RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates libssl3 && update-ca-certificates && rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY --from=builder /app/target/aarch64-unknown-linux-gnu/release/agent .

ENV AWS_REGION=us-east-1 \
    AWS_DEFAULT_REGION=us-east-1 \
    RUNNING_ENV=agentcore

RUN useradd -m appuser
USER appuser

CMD ["./agent"]

For my application to work, I need some system dependencies. That's why I am installing libssl3. Initially, I tried to use a smaller image for the runner, but I ended up with a default ubuntu to have a new version of GLIBC available. For now, I finished with a ~40MB image in the ECR, which is significantly smaller than the default Python-based one anyway.

Infratructure

The tricky part of preparing the infrastructure for AgentCore Runtime is that specific elements need to be ready in a specific order. What I mean is that I am not able to host the agent until the actual image is uploaded to ECR. In other words, initial deployment needs to be thought through.

There are a few ways to tackle this problem. You might, for example, use custom resources in CloudFormation. I decided to keep things simple and split the infrastructure into two stacks. Once the ECR repository is created, I build a Docker image and then deploy AgentcoreRuntime.

The hardest part is to have the first deployment right. Later on, it is much easier, as most of the components are already deployed.

Deployment pipeline

Let's go through the GitHub Actions workflow step by step

Set up AWS credentials and install CDK

# .github/workflows/deploy.yaml
name: Deploy Agent

on:
  push:
    branches: ["main"]

env:
  AWS_REGION: ${{ vars.AWS_REGION }}
  AWS_ROLE: ${{ secrets.AWS_ROLE }}

permissions:
  id-token: write
  contents: read

jobs:
  deploy-agent:
    runs-on: ubuntu-24.04-arm

    defaults:
      run:
        working-directory: agent

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ env.AWS_ROLE }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Install CDK
        run: |
          npm install -g aws-cdk
# ...

I have a role created in IAM to grant GitHub access to the AWS. The process of creating it is described here: LINK
I store the role ARN in the GitHub secrets.

Deploy ECR repository

The repository definition:

import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import * as ecr from "aws-cdk-lib/aws-ecr";

export class AcInfraStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const acRepository = new ecr.Repository(this, "RustAgentRepository", {
      repositoryName: "agentcore-rust-agent-repo",
    });

    new cdk.CfnOutput(this, "ECRRepositoryURI", {
      value: acRepository.registryUri,
    });

    new cdk.CfnOutput(this, "ECRRepositoryName", {
      value: acRepository.repositoryName,
    });
  }
}

I could create exports for repository name and URI, but I am not a huge fan of exports, so I just return them as outputs and extract in the GitHub action.

# ...
- name: Deploy ECR
        working-directory: aws/ac-infra
        run: |
          npm i
          cdk deploy -O cdk-ecr-outputs.json --require-approval never

      - name: Parse ECR CDK outputs
        id: cdk_ecr_outputs
        working-directory: aws/ac-infra
        run: |
          REPO_NAME=$(jq -r '.RustAgentAgentcoreECRStack.ECRRepositoryName' cdk-ecr-outputs.json)
          REPO_URI=$(jq -r '.RustAgentAgentcoreECRStack.ECRRepositoryURI' cdk-ecr-outputs.json)

          echo "repo_name=$REPO_NAME" >> $GITHUB_OUTPUT
          echo "repo_uri=$REPO_URI" >> $GITHUB_OUTPUT

Build the image

Once the Dockerfile is prepared, there is not much magic in the build step. The best part is that there are ARM runners available on GitHub for free for public repositories. Pretty nice!

I use a dynamic repository name and URI stored after the previous steps

# ...
- name: Docker login to ECR
        run: |
          aws ecr get-login-password --region $AWS_REGION \
            | docker login \
              --username AWS \
              --password-stdin ${{ steps.cdk_ecr_outputs.outputs.repo_uri }}

      - name: Build ARM64 Docker image
        run: |
          docker build \
            -t ${{ steps.cdk_ecr_outputs.outputs.repo_name }}:latest \
            .

      - name: Tag image
        run: |
          docker tag ${{ steps.cdk_ecr_outputs.outputs.repo_name }}:latest \
            ${{ steps.cdk_ecr_outputs.outputs.repo_uri }}/${{ steps.cdk_ecr_outputs.outputs.repo_name }}:latest

      - name: Push to ECR
        run: |
          docker push \
          ${{ steps.cdk_ecr_outputs.outputs.repo_uri }}/${{ steps.cdk_ecr_outputs.outputs.repo_name }}:latest

Deploy AgentCore Runtime

With the ECR repository in place and the image uploaded, I can now deploy AgentCore Runtime. For the initial version, I stick to the simple public endpoint with IAM authorisation.

// ...
const AGENT_NAME = "rust_agent";

    const repositoryName = this.node.tryGetContext("REPO_NAME");
    const repositoryURI = this.node.tryGetContext("REPO_URI");

    const runtimeRole = new iam.Role(this, "AgentCoreRustAgent", {
      assumedBy: new iam.ServicePrincipal("bedrock-agentcore.amazonaws.com", {
        conditions: {
          StringEquals: {
            "aws:SourceAccount": this.account,
          },
          ArnLike: {
            "aws:SourceArn": `arn:aws:bedrock-agentcore:${this.region}:${this.account}:*`,
          },
        },
      }),
    });

// -- a lot of "runtimeRole.addToPolicy" to fulfil requirements for execution role --

    const agentRuntime = new agentcore.CfnRuntime(this, "RustAgent", {
      agentRuntimeArtifact: {
        containerConfiguration: {
          containerUri: `${repositoryURI}/${repositoryName}:latest`,
        },
      },
      agentRuntimeName: AGENT_NAME,
      networkConfiguration: {
        networkMode: "PUBLIC",
      },
      roleArn: runtimeRole.roleArn,
    });

    agentRuntime.node.addDependency(runtimeRole);

    new cdk.CfnOutput(this, "RustAgentId", {
      value: agentRuntime.attrAgentRuntimeId,
    });

    new cdk.CfnOutput(this, "AgentRuntimeRoleArn", {
      value: runtimeRole.roleArn,
    });
  }

Again, I use the simple output to pass variables down the GitHub pipeline.

# ...
- name: Deploy AgentCore Runtime
        working-directory: aws/ac-runtime
        run: |
          npm i
          cdk deploy -O cdk-runtime-outputs.json --require-approval never \
          --context REPO_URI=${{ steps.cdk_ecr_outputs.outputs.repo_uri }} \
          --context REPO_NAME=${{ steps.cdk_ecr_outputs.outputs.repo_name }}
# ...

Bump agent version

AgentCore Runtime will pull the current image from the repository from time to time. I am not sure how often, though. Probably the safest way is to bump the Agent version to make sure that the current image is in use, and to track changes across versions.

I run AWS CLI to update the Runtime using variables returned from the deployment step

# ...
- name: Parse Runtime CDK outputs
        id: cdk_runtime_outputs
        working-directory: aws/ac-runtime
        run: |
          AGENT_RUNTIME_ROLE=$(jq -r '.AcRuntimeStack.AgentRuntimeRoleArn' cdk-runtime-outputs.json)
          AGENT_RUNTIME_ID=$(jq -r '.AcRuntimeStack.RustAgentId' cdk-runtime-outputs.json)

          echo "agent_runtime_role=$AGENT_RUNTIME_ROLE" >> $GITHUB_OUTPUT
          echo "agent_runtime_id=$AGENT_RUNTIME_ID" >> $GITHUB_OUTPUT

      - name: Update Agent version
        run: |
          aws bedrock-agentcore-control update-agent-runtime \
          --agent-runtime-id ${{ steps.cdk_runtime_outputs.outputs.agent_runtime_id }} \
          --agent-runtime-artifact "{
                  \"containerConfiguration\": {
                    \"containerUri\": \"${{ steps.cdk_ecr_outputs.outputs.repo_uri }}/${{ steps.cdk_ecr_outputs.outputs.repo_name }}:latest\"
                  }
                }" \
          --role-arn ${{ steps.cdk_runtime_outputs.outputs.agent_runtime_role }} \
          --network-configuration '{ "networkMode": "PUBLIC" }' \
          --region ${{ env.AWS_REGION }}

Finding out

When I merge this code into the main, it will deploy the whole environment and bump the agent version. This is pretty cool. However, the agent itself will... fail in the runtime due to missing permissions.

MicroVM Metadata Service (MMDS)

AgentCore Runtime uses MMDS to get credentials for the given actor. It is similar to the Instance Metadata Service on EC2. The bottom line is that developers don't need to worry about how it works under the hood, as a AWS SDK covers the authentication part.
That's why it works in Python out of the box.

However, the Rust AWS SDK fails to authenticate on the AgentCore Runtime, which is a bummer.

I didn't really find the answer in the documentation, but this is what I understood. Rust SDK is new and it supports only IMDS v2 authorising schema. It is understandable, as on EC2, we shouldn't use v1 anymore. Other libraries, like botocore, still have a fallback to IMDS v1 if v2 is not available.

From what I saw, the MicroVM Metadata Service uses the same schema as IMDS v1. That's why it fails on Rust, but works on Python.

To fix this issue, I just needed to create my own CredentialsProvider that would follow the schema of IMDS v2. The code is in the repo, but it took me a while to figure this out.

Resources provisioning

I struggled a bit with creating the AgentCore Runtime for the first time using CDK. CloudFormation under the hood runs validation when creating a new runtime. It looks like we need to explicitly say that CloudFormation should wait for the IAM role to be created before triggering provisioning new Runtime. Otherwise, creation will fail. In CDK, this is done by adding agentRuntime.node.addDependency(runtimeRole);