DEV Community: Matt Lewis

Strands Agents + AgentCore Runtime - a perfect match

Matt Lewis — Wed, 20 May 2026 21:17:54 +0000

This is the third in a series of posts documenting the architecture, implementation, and lessons learned from building the AWS Briefing Agent - a personalised AWS assistant deployed on Amazon Bedrock AgentCore Runtime.

Part 1: Building a Full-Stack AI Agent on Bedrock AgentCore
Part 2: Data Ingestion: RSS Feeds, Knowledge Base, S3 Vectors, and Metadata Filtering
Part 3: Strands Agents + AgentCore Runtime - a perfect match
Part 4: Adding Memory to the Agent
Part 5: Experimenting with API Gateway
Part 6: Observability and Evaluations
Part 7: Third Party Integrations - Identity, Gateway and Slack Notifications

The initial implementation of the AWS Briefing Agent called the AWS News Feed RSS feed on every invocation. After setting up an Amazon Bedrock Knowledge Base, the next step was to refactor the code to take advantage of an agentic framework. The decision was made to adopt Strands Agents SDK as an open source SDK that helps you build and run AI agents in just a few lines of code. In our case, switching to the Knowledge Base and adopting Strands Agents SDK helped us to reduce the number of lines of code in our implementation logic by 75%.

Using Strands Agents SDK

The core of the Strands Agents code is straightforward and shown in the code snippet below:

from strands import Agent
from strands.models import BedrockModel
from strands.agent.conversation_manager import SlidingWindowConversationManager
from strands_tools import retrieve
from agent.tools.slack_formatter.tool import format_slack_message

model = BedrockModel(
    guardrail_id=GUARDRAIL_ID,
    guardrail_version=GUARDRAIL_VERSION,
    guardrail_trace="enabled",
)

agent = Agent(
    system_prompt=_load_system_prompt(),
    model=model,
    tools=[retrieve, format_slack_message] + gateway_tools,
    session_manager=session_manager,
    conversation_manager=SlidingWindowConversationManager(
        window_size=20,
        should_truncate_results=True,
        per_turn=True,
    ),
    callback_handler=None,
)

result = agent(message)

We start by importing a number of classes and functions from two packages (strands-agents and strands-agents-tools) and one local module. Agent is the core class for the agent itself, BedrockModel is the model provider, SlidingWindowConversationManager controls how conversation history is trimmed, and retrieve is a pre-built tool that is used to query a Bedrock Knowledge Base. The format_slack_message is a local custom tool within this project - a Python function decorated with the @tool annotation.

We instantiate the BedrockModel() without specifying a model_id. At this point, Strands uses its default model, which is current Claude Sonnet on Bedrock. We include details of a Bedrock Guardrail when we instantiate the model, purely to demonstrate the use of guardrails which we cover this later in the blog post.

Finally, we create the agent by wiring together its core components.

Deploy to Amazon Bedrock AgentCore Runtime

The AgentCore Runtime Python SDK provides a lightweight wrapper that helps to deploy your agent function as HTTP services

# Import the runtime
from bedrock_agentcore.runtime import BedrockAgentCoreApp

# Initialise the app
app = BedrockAgentCoreApp()

# Decorate the function
@app.entrypoint
def invoke(payload: Dict[str, Any], context: Any = None) -> Dict[str, Any]:
    """Entry point for AgentCore Runtime."""
    message = payload.get("prompt", payload.get("message", ""))
    ...
    return response

BedrockAgentCoreApp wraps your function in an HTTP server that listens om port 8080 with two endpoints:

/invocations - a POST endpoint for agent interactions. This gets invoked when customers call the InvokeAgentRuntime action with the payload in JSON format
/ping - a GET endpoint for health checks to verify your agent is operational and ready to handle requests

The @app.entrypoint decorator registers your invoke function as the handler for incoming requests. When AgentCore Runtime receives a request, it deserialises the JSON body into payload, provides a context object (with session_id, request_headers, etc.), calls your function, and serialises the returned dict back as the HTTP response.

Using the Container Build

When using the @aws/agentcore CLI and running agentcore deploy, the CLI needs to turn the Python source code into a runnable container image on AgentCore Runtime. This is controlled by the build field in the agentcore.json file. The default setting is CodeZip, in which the CLI zips up the Python source code, uploads it, and AgentCore resolves dependencies using uv --no-build. This is fast but has a hard constraint, as every dependency must have a pre-built wheel. In our code, we have a package that only ships source distributions, which required us to switch to the Container build setting. This also makes our build more production-ready.

When you run agentcore deploy with the Container build type, the CLI synthesis a CloudFormation stack that includes a CodeBuild project, an ECR repository, the AgentCore Runtime resource, and IAM roles. The CLI packages the codeLocation directory (agent/) and uploads it to S3 as the CodeBuild source artefact. CodeBuild pulls the provided Dockerfile and builds the container image. You can see all the steps in the CodeBuild project below:

After the image builds successfully, CodeBuild tags it and pushes it to the ECR repository as shown below:

The stack updates the Runtime resource to point at the new ECR image URI. AgentCore pulls the image from ECR the next time it starts a container for an invocation.

Built-In Conversation Managers

In the Strands Agents SDK, the user messages and agent responses are all added to the context. As the conversation grows within a session, this starting having a material impact on response times. We modified the default SlidingWindowConversationManager manager:

reducing the windowSize from the default of 40 to 20. This sets the maximum number of messages to keep
setting the per_turn parameter to false. This runs the sliding window before every model call within the same invocation, rather than waiting until after the agent loop completes.

This reduced the average response time from around 80 seconds down to 15 seconds.

Adding Bedrock Guardrails

Amazon Bedrock Guardrails are designed to help you safely build and deploy responsible generative AI applications with confidence. We decided to include a guardrail in the architecture, to understand where it fits in and what it can provide.

The guardrail itself was defined in CDK with content filters (sexual, violence, hate, insults, misconduct and prompt attack), a topic policy (deny off-topic sports questions), and a managed profanity word list:

# ----------------------------------------------------------------
# Bedrock Guardrail — content safety for the agent
# ----------------------------------------------------------------
guardrail = bedrock.CfnGuardrail(
    self,
    "BriefingAgentGuardrail",
    name="briefing-agent-guardrail",
    description="Content safety guardrail for the AWS Briefing Agent",
    blocked_input_messaging="I'm sorry, I can't process that request. Please rephrase your question about AWS announcements.",
    blocked_outputs_messaging="I'm sorry, I can't provide that response. Let me try a different approach.",
    content_policy_config=bedrock.CfnGuardrail.ContentPolicyConfigProperty(
        filters_config=[
            bedrock.CfnGuardrail.ContentFilterConfigProperty(
                type="SEXUAL",
                input_strength="HIGH",
                output_strength="HIGH",
            ),
            bedrock.CfnGuardrail.ContentFilterConfigProperty(
                type="VIOLENCE",
                input_strength="HIGH",
                output_strength="HIGH",
            ),
            # HATE, INSULTS, MISCONDUCT, PROMPT_ATTACK
        ],
    ),
    topic_policy_config=bedrock.CfnGuardrail.TopicPolicyConfigProperty(
        topics_config=[
            bedrock.CfnGuardrail.TopicConfigProperty(
                name="Sports",
                definition="Questions about sports scores, match results, player transfers, league standings, fixtures, or any sporting events.",
                type="DENY",
            ),
        ],
    ),
    word_policy_config=bedrock.CfnGuardrail.WordPolicyConfigProperty(
        managed_word_lists_config=[
            bedrock.CfnGuardrail.ManagedWordsConfigProperty(
                type="PROFANITY",
            ),
        ],
    ),
)

When the agent is invoked, the request first reaches the AgentCore Runtime and runs the handler code first. The guardrail itself is only applied when the handler makes the Bedrock inference call. Bedrock evaluates the input before running the model inference, and then inspects the output before returning it. We did encounter some interesting behaviour when implementing the guardrail.

IAM Permission Gap

The first invocation after adding the guardrail failed with:

AccessDeniedException: User is not authorized to perform: bedrock:ApplyGuardrail
on resource: arn:aws:bedrock:eu-west-1.xxx

The AgentCore execution role (auto-created by the @aws/agentcore-cdk construct) includes bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream, but not bedrock:ApplyGuardrail. The construct doesn’t know about guardrails — they’re a Bedrock feature, not an AgentCore feature. We ended up having to use the aws iam put-role-policy CLI command to add the missing permission

Topic policies can false-positive on legitimate queries

The initial topic policy denied "questions not related to AWS services, cloud computing, or technology". The intention was that it would be easy to demonstrate, and would ensure that the user input was relevant. However, when the user asked questions such as "what are the top announcements today", the classifier ended up deciding this was a blocked topic. In the end, to demonstrate how topic policies work, we changed it to explicitly deny sporting questions.

Guardrail versions can be deleted by CDK updates

When we updated the topic policy, we changed the version description for the guardrail. The CDK stack updated the guardrail version resource, so that CloudFormation deleted version 1 and created version 2. Unfortunately, the version number is also defined in the agentcore.json file. This meant that the AgentCore Runtime container still had version 1 baked into its environment, which meant calls now failed with the following exception:

ValidationException: The guardrail identifier or version provided in the request does not exist.

In the end it was a case of having to update the version number in agentcore.json, redeploy the agent, and start a new session.

Data Ingestion: RSS Feeds, Knowledge Base, S3 Vectors, and Metadata Filtering

Matt Lewis — Wed, 20 May 2026 21:16:06 +0000

This is the second in a series of posts documenting the architecture, implementation, and lessons learned from building the AWS Briefing Agent - a personalised AWS assistant deployed on Amazon Bedrock AgentCore Runtime.

Part 1: Building a Full-Stack AI Agent on Bedrock AgentCore
Part 2: Data Ingestion: RSS Feeds, Knowledge Base, S3 Vectors, and Metadata Filtering
Part 3: Strands Agents + AgentCore Runtime - a perfect match
Part 4: Adding Memory to the Agent
Part 5: Experimenting with API Gateway
Part 6: Observability and Evaluations
Part 7: Third Party Integrations - Identity, Gateway and Slack Notifications

When I started building the AWS Briefing Agent, the first version queried the AWS What's New RSS feed on every invocation. This worked in terms of showing the agent could return tailored information back to the client. However, it was costly and wasteful, with the same data fetched repeatedly, which added latency to every invocation. The RSS feed also only covers recent information, and it was likely we would want to start searching for releases that had been launched in the past 6 months or more. The next step therefore, was to separate the retrieval by the agent from the ingestion.

Amazon Bedrock Knowledge Base

One of the key design goals was to allow the agent to match a natural language query "what's new in Bedrock this week?" against a large corpus of documents to return the most semantically similar results. This is where Amazon Bedrock Knowledge Base comes into its own. It allows the agent to use RAG (Retrieval-Augmented Generation). By querying the Knowledge Base, we can retrieve relevant documents at query time, and then inject them into the prompt as context. The LLM then generates a response from this retrieved information which we know to be factual.

The python CDK code that creates the Knowledge Base is shown below:

knowledge_base = bedrock.CfnKnowledgeBase(
    self,
    "AnnouncementKnowledgeBase",
    name="aws-briefing-agent-announcements",
    ...
    knowledge_base_configuration=bedrock.CfnKnowledgeBase.KnowledgeBaseConfigurationProperty(
        type="VECTOR",
        vector_knowledge_base_configuration=bedrock.CfnKnowledgeBase.VectorKnowledgeBaseConfigurationProperty(
            embedding_model_arn=f"arn:aws:bedrock:{self.region}::foundation-model/amazon.titan-embed-text-v2:0",
        ),
    ),
    storage_configuration=bedrock.CfnKnowledgeBase.StorageConfigurationProperty(
        type="S3_VECTORS",
        s3_vectors_configuration=bedrock.CfnKnowledgeBase.S3VectorsConfigurationProperty(
            index_name="announcements",
            vector_bucket_arn=f"arn:aws:s3vectors:{self.region}:{self.account}:bucket/briefing-agent-vectors",
        ),
    ),
)

This declares the embeddings model to be used as amazon.titan-embed-text-v2:0 and the vector store as being of type S3_VECTORS. There is no code required to handle aspects such as embeddings. Instead, Bedrock manages all of this for us.

Amazon S3 Vectors

Amazon Bedrock Knowledge Bases support several vector stores. A vector store is the retrieval engine that makes RAG work. It stores documents as numerical embeddings (vectors) that are generated by an embeddings model. At query time, the user's question is embedded, and the vector store finds documents whose embeddings are closest in meaning.

The prototype uses Amazon S3 Vectors as the underlying vector store. S3 Vectors provides cost-effective, elastic, and durable vector storage at up to 90% lower costs for uploading, storing, and querying vectors than alternatives such as OpenSearch Serverless. There is no infrastructure to manage, and it still provides a sub-second query latency which is acceptable for this use case.

Scheduling the Ingestion

The ingestion pipeline is run every 6 hours using Amazon EventBridge Scheduler. This service provides capabilities such as built-in retry policies, time zone support, and dead-letter queues. The schedule triggers an AWS Lambda function that carries out the required processing. This includes:

Lists existing document hashes in S3
Fetches the AWS What’s New RSS feed (~100 announcements)
Fetches 13 AWS blog RSS feeds (aws, machine-learning, compute, security, database, containers, devops, networking, storage, infrastructure-and-automation, developer, big-data, iot)
Fetches the AWS Security Bulletins RSS feed
For each new blog post, fetches the canonical URL and extracts the full article body using a stdlib HTML parser
Parses publication dates into YYYYMMDD integers
Writes .txt and .metadata.json files per new item to S3
Triggers a Bedrock KB ingestion job

Deduplication and Incremental Writes

When the ingestion pipeline runs, most of the content in the various RSS feeds is not new. It was important to find a way to prevent re-fetching and re-writing hundreds of announcements every 6 hours.

To support this, we created an MD5 hash of the blog posts URL, truncated to 12 hex characters. This hash is used as the S3 filename. The sample code snippet is shown below:

def write_to_s3(items, existing_keys=None):
    existing = existing_keys or set()
    for item in items:
        url_hash = hashlib.md5(item["link"].encode()).hexdigest()[:12]
        if url_hash in existing:
            continue # Already in S3, skip
        # ... write doc + metadata files

At startup, get_existing_keys() lists all the .txt files in S3 and extracts the hash from each filename into a set. When processing the blog posts, the Lambda functions computes the URL hash and checks to see if it is already in the set. If it already exists, then it has been ingested in a previous run, and there is no need to re-fetch the page. If the hash does not exist, then the function fetches the page, extracts the content, and writes to S3. The hash gives a stable, deterministic filename derived from the URL. The same URL always produces the same hash.

Chunking Strategy

The chunking strategy is set on the Data Source resource in the CDK stack as shown below:

data_source = bedrock.CfnDataSource(
    self,
    "AnnouncementDataSource",
    name="aws-announcements-s3",
    knowledge_base_id=knowledge_base.attr_knowledge_base_id,
    data_source_configuration=bedrock.CfnDataSource.DataSourceConfigurationProperty(
        type="S3",
        s3_configuration=bedrock.CfnDataSource.S3DataSourceConfigurationProperty(
            bucket_arn=data_bucket.bucket_arn,
        ),
    ),
    vector_ingestion_configuration=bedrock.CfnDataSource.VectorIngestionConfigurationProperty(
        chunking_configuration=bedrock.CfnDataSource.ChunkingConfigurationProperty(
            chunking_strategy="SEMANTIC",
            semantic_chunking_configuration=bedrock.CfnDataSource.SemanticChunkingConfigurationProperty(
                breakpoint_percentile_threshold=92,
                buffer_size=1,
                max_tokens=600,
            ),
        ),
    ),
)

We utilise a SEMANTIC chunking strategy. This uses the embedding model itself to decide where to split. The following three parameters control this behaviour:

breakpoint_percentile_threshold=92 - controls the percentile threshold that will result in a split. A higher threshold requires sentences to be more distinguishable to split the document into different chunks.
max_tokens=600 - the maximum number of tokens that should be included in a single chunk, while honoring sentence boundaries.
buffer_size=1 - for a given sentence, the buffer size defines the number of surrounding sentences to be added for embeddings creation. A larger buffer size might capture more context but can also introduce noise, while a smaller buffer size might miss important context but ensures more precise chunking.

Filtering by Date

One of the goals in writing the agent was that a user could ask to constrain information by how recent it is e.g. "what is new in the past 7 days?".

To help achieve this, at ingestion time for each document, we create an associated metadata.json sidecar file that attaches structured, filterable attributes to a document so the agent can narrow search results without relying only on semantic similarity. An example companion file is shown below:

{
  "metadataAttributes": {
    "published_date": 20260415,
    "service": "amazon-bedrock",
    "category": "artificial-intelligence",
    "source_type": "announcement"
  }
}

During the Knowledge Base sync, Bedrock reads this sidecar and attaches those attributes to every vector chunk generated from that document. At query time, the agent can combine semantic search with metadata filters:

"What's new in Bedrock this week?" → vector similarity for "Bedrock" + greaterThanOrEquals filter on published_date
"Show me security bulletins" → vector similarity + equals filter on source_type: "security-bulletin"
"Lambda announcements from the last month" → vector similarity + filters on both service and published_date

Without the metadata file, the agent would get the most semantically similar results regardless of date or service — so a question about "this week" might return announcements from 3 months ago that happen to be textually similar. The metadata filters let the agent constrain results to the correct time window or service before ranking by relevance.

The naming convention (.metadata.json) is a Bedrock KB convention — it automatically associates the sidecar with its parent document during ingestion. No code links them; the filename pattern is enough.

Bedrock Knowledge Base metadata supports four types: STRING, NUMBER, BOOLEAN and STRING_LIST. There is no native data type. The comparison operators (greaterThan, greaterThanOrEquals, lessThan, lessThanOrEquals) only work with NUMBER. Our original implementation stored published_date as a string ("2026-05-14"). When the agent tried to filter, we got back the following exception:

ValidationException: The filter value type provided isn't supported
for the given operation: GREATER_THAN_OR_EQUALS

The fix was to store dates as YYYYMMDD numbers (so using "20260514" instead of "2026-05-14"). We also inject today's date into the system prompt at runtime so the LLM can easily calculate relative dates.

Note that Amazon S3 Vectors has a strict 2 KB limit on filterable metadata per vector. We found the Bedrock Knowledge Base internal metadata keys (AMAZON_BEDROCK_TEXT and AMAZON_BEDROCK_METADATA) were set as filterable by default, which caused frequent ValidationException errors. The fix was mark both of these keys as non-filterable when creating the vector index:

vector_index = s3vectors.CfnIndex(
    self, "AnnouncementVectorIndex",
    index_name="announcements",
    vector_bucket_name=vector_bucket.vector_bucket_name,
    dimension=1024,  # Titan Embed Text v2
    distance_metric="cosine",
    data_type="float32",
    metadata_configuration=s3vectors.CfnIndex.MetadataConfigurationProperty(
        non_filterable_metadata_keys=[
            "AMAZON_BEDROCK_TEXT",
            "AMAZON_BEDROCK_METADATA",
        ],
    ),
)

This meant the only filterable metadata is contained in the .metadata.json fields, which are the only fields we filter on.

The next post covers how we used an agentic framework (Strands Agents SDK) in combination with AgentCore to really start bringing the briefing agent to life.

Building a Full-Stack AI Agent on Amazon Bedrock AgentCore

Matt Lewis — Wed, 20 May 2026 21:14:23 +0000

This is the first in a series of posts documenting the architecture, implementation, and lessons learned from building the AWS Briefing Agent - a personalised AWS assistant deployed on Amazon Bedrock AgentCore Runtime.

Part 1: Building a Full-Stack AI Agent on Bedrock AgentCore
Part 2: Data Ingestion: RSS Feeds, Knowledge Base, S3 Vectors, and Metadata Filtering
Part 3: Strands Agents + AgentCore Runtime - a perfect match
Part 4: Adding Memory to the Agent
Part 5: Experimenting with API Gateway
Part 6: Observability and Evaluations
Part 7: Third Party Integrations - Identity, Gateway and Slack Notifications

Why build an agent?

The last few years have seen a rapid shift from Generative to Agentic AI. Most of us will remember our first experience with ChatGPT where we entered a prompt and got a response back. This was impressive at the time, but was reliant on a user typing a prompt and reacting to the response. We then saw the emergence of early AI agents that could break down tasks into smaller steps and execute them independently. Over the past year, this has evolved into fully autonomous multi-agent systems capable of completing complex tasks with minimal or even no human supervision.

This shift is accelerating quickly. Gartner predicts that by 2028, more than a third of all enterprise software apps will include Agentic AI, and at least 15% of day-to-day work decisions will be made autonomously by AI agents. For organisations, the question is no longer whether agents will become part of enterprise systems, but how to build them securely, reliably and operate them at scale. From an AWS perspective, Amazon Bedrock AgentCore provides a way to help enterprises achieve this goal.

I decided to build an agent utilising AgentCore and its supporting capabilities and which served a purpose ... helping me keep up to date with all the latest announcements from AWS. This agent brings together Memory, Observability, Gateway, Identity, Evaluations and Registry alongside AgentCore Runtime. It allows the agent to personalise briefings just for me from 13 different RSS feeds including What's New, Blog Posts and Security Bulletins. I can get a daily update, as well as automatically post any briefings I'm really interested in to a Slack channel. And I learnt a lot in the process. This blog series covers my experience in building out this agent.

Why AgentCore Runtime?

Amazon Bedrock AgentCore is an AWS service that has been designed specifically for the task of hosting agents. A common saying I keep on hearing is that Bedrock AgentCore is to agentic applications what AWS Lambda is to event driven applications.

At the heart is AgentCore Runtime, which provides the secure runtime for executing the agent code. AgentCore Runtime provides session-based isolation, where every session is assigned a dedicated Firecracker microVM with isolated CPU, memory and filesystem resources (the same lightweight virtualisation technology that underpins AWS Lambda and AWS Fargate). When the session finishes, the LLM's state information is copied to long-term memory and the entire microVM is destroyed. There is no shared state between sessions, which prevents any cross-session data leakage.

AgentCore Runtime is framework-agnostic and supports all popular frameworks such as Strands Agents, LangGraph and CrewAI. It also works with any LLM, such as models offered by Amazon Bedrock, Anthropic Claude, Google Gemini and OpenAI or even hosted on-premises. It supports long sessions up to 8 hours, which means it can handle complex multi-step tasks or time-consuming background processes. Unlike traditional compute services that charge for pre-allocated resources, AgentCore Runtime uses consumption-based pricing where you only pay for active CPU and memory usage. With this, I/O wait and idle time is free, and you're only charged for actual resource consumption calculated at per-second increments. The runtime automatically scales from zero to thousands of concurrent sessions on demand, with no capacity planning needed, and includes reliability features like checkpointing to recover gracefully from interruptions.

AWS Briefing Agent Architecture

A high-level architecture overview of the AWS Briefing Agent is shown below:

AWS Briefing Agent Client is a next.js static site hosted on AWS Amplify Hosting. It integrates directly with Amazon Cognito using the amazon-cognito-identity-js SDK, implementing a full sign-in, sign-up and email verification flow.
AWS Briefing Agent itself is a Python application built with the Strands Agents SDK and deployed to AgentCore Runtime as a Docker container. The @aws/agentcore CLI handles the full deployment lifecycle. When you run agentcore deploy, the CLI triggers AWS CodeBuild to build the Docker image (ARM64), pushes it to Amazon ECR, and deploys it to AgentCore Runtime.
AgentCore Memory provides persistent user knowledge across sessions using two built-in memory strategies. The SEMANTIC memory strategy extracts factual information and knowledge from conversations that have taken place e.g. that a user works with Lambda and EKS. The USER_PREFERENCE memory strategy identifies and extracts user preferences from conversations e.g. that the user prefers technical deep dives. The agent retrieves relevant memory records at the start of each invocation and injects them as context, enabling personalised briefings from the first message of a new session.
AgentCore Observability is used to instrument all Bedrock API calls, tool invocations and memory operations. This is carried out entirely by setting enableOtel: true in the runtime config and using the opentelemetry-instrument wrapper command. Spans show up in CloudWatch Transaction Search and the CloudWatch GenAI Observability dashboard is populated with the sessions and traces, and provides the ability to drill into individual invocations.
AgentCore Evaluations is configured to run online quality assessments against agent responses using built-in evaluators for Helpfulness, Goal Success Rate, and Correctness. These are shown in the front-end to give an indication on how well the agent is performing for each user.
Bedrock Knowledge Base is created and backed by Amazon S3 Vectors that stores all announcements, blog posts and security bulletins. An ingestion Lambda runs every 6 hours that writes each item as a .txt file alongside a metadata.json file to the S3 bucket, before triggering a Knowledge Base sync. The agent queries the KB via the Strands retrieve tool with metadata filters for date ranges and service names, enabling questions like "what's new in Bedrock this week?"
AgentCore Gateway exposes a managed MCP (Model Context Protocol) endpoint that the agent connects to at runtime for tool discovery. The Slack integration is defined as an OpenAPI spec pointing at the Slack chat.postMessage API, and is registered as a Gateway target. The agent discovers available tools dynamically via the MCP protocol. The Gateway handles authentication and credential injection for this integration with Slack, attaching the stored bot token as a Bearer header on outbound Slack API calls.
AgentCore Identity stores the Slack bot token as an API key credential in its token vault (encrypted at rest via Secrets Manager). When the agent calls the tool to send a briefing to Slack, AgentCore Identity retrieves the bot token and injects it into the outbound request automatically. The agent code never sees or handles the token directly.
AgentCore Registry is a governed catalog for agents, MCP servers, tools, skills, and custom resources. Teams can publish resources, control access through approval workflows, and enable both humans and AI agents to discover tools using semantic and keyword search. Once the Slack integration was working, the briefing agent and the Slack tool where registered in the AgentCore Registry. This makes the tool discoverable by other agents in the organisation.

AWS Briefing Agent in Action

We create a new user and login to the home screen for the AWS Briefing Agent front end. The first time we use the agent, we are asked to provide information about our interests and the type of briefing style we are interested in. These get added to memory, so that the agent can personalise its responses:

We can provide the details of the services we are most interested to the agent. At this point, the agent will pull back the top announcements that it has retrieved from the Knowledge Base, and display them in a briefing summary.

We have also integrated with Slack through Gateway. This means we can ask the Briefing Agent to post the details to our Slack channel:

This means that when we go to our Slack channel, we can see a new message with our briefing, alongside all the links we can click to take us to the original blog posts and announcement articles.

In the next post we cover design decisions made to ingest the data into a Bedrock Knowledge Base to support the agent

Moving from Node Groups to NodePools on Amazon EKS

Matt Lewis — Sat, 21 Feb 2026 17:38:16 +0000

Background

In November 2019, AWS introduced the concept of Amazon EKS Managed Node Groups. With this, Amazon EKS would provision and manage the underlying EC2 instances as worker nodes, as part of an EC2 Auto Scaling Group. You could create, update or terminate a node with a single operation. When updating or terminating a node, EKS would handle these operations gracefully by automatically draining nodes to ensure applications stayed available. Futher enhancements allowed for node configuration and customisation through EC2 Launch Templates and custom AMIs, alongside support for EC2 spot instances.

However, the modern trend in Kubernetes is moving away from static node groups to dynamic node provisioning with tools like Karpenter for more flexible and cost-effective infrastructure management. With Amazon EKS Auto Mode, the recommendation is no longer to create traditional node groups. Instead, you create a Karpenter NodePool that defines the compute requirements. Amazon EKS Auto Mode provides two built-in node pools - system and general-purpose - which you cannot modify, but you can enable or disable. The general-purpose node pool provides support for launching nodes for general purpose workloads. It supports only amd64 architecture and uses only on-demand EC2 capacity in the C, M or R instance families.

What happens if you want to take advantage of spot instances?
What happens if you want to take advantage of Graviton?

Let's show how you can create a node pool to do just that.

Creating a Karpenter NodePool

The complete configuration files for this post can be found in the k8s\node-pool section of the code repository here. We can create it using the following command.

$ kubectl apply -f arm-nodepool.yaml
nodepool.karpenter.sh/arm-mixed-capacity created

The start of the arm-nodepool.yaml configuration file is as follows:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: arm-mixed-capacity

This tells us we are using the NodePool API with Karpenter. This uses the nodepools.karpenter.sh CRD which is installed by default with Auto Mode. The spec element provides the contract with Karpenter. It has the following high-level structure, and we will go through each one in order.

spec:
  disruption:   # when and how nodes can be replaced
  template:     # what a node looks like
  limits:       # optional safety rails
  weight:       # optional priority

Disruption

The disruption section describes the ways in which Karpenter can disrupt and replace nodes. This is used when Karpenter wants to remove empty nodes, replace under-utilised nodes with better fitting ones, or shrink the cluster to save money.

  # Disruption settings for node lifecycle management
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 10m  # Wait 10 minutes before consolidating

    # Disruption budgets to control how many nodes can be disrupted
    budgets:
      # During business hours: more conservative
      - nodes: "2"
        schedule: "0 9 * * mon-fri"  # 9 AM Mon-Fri
        duration: 8h
      # Outside business hours: more aggressive
      - nodes: "10%"

The consolidationPolicy describes which types of nodes Karpenter should consider for consolidation. There are 2 options:

WhenEmptyOrUnderutilized - Karpenter will consider all nodes for consolidation and attempt to remove or replace nodes when it discovers that the node is empty or underutilised and could be changed to reduce cost
WhenEmpty - Karpenter will only consider nodes for consolidation that contain no workload pods

The consolidateAfter field is the amount of time Karpenter should wait to consolidate a node after a pod has been added or removed from the node. We set this to 10 minutes to make sure the behaviour is not too aggressive, and gives the scheduler time to stabilise.

Disruption budgets are used to control how many nodes can be disrupted. There are two rules defined in this section. The first rule states that between 09:00 and 17:00 on Monday to Friday, Karpenter may disrupt at most 2 nodes at a time. The second rule states that Karpenter may disrupt up to 10% of all nodes at any time. This will not apply between 09:00-17:00 on Monday to Friday as the first rule is more restrictive and so wins out.

Template

The template section defines the exact shape, rules and constraints of every node that Karpenter is allowed to create as part of this NodePool.

  # Node template specification
  template:
    spec:
      # Termination grace period (24 hours)
      terminationGracePeriod: 24h

      # Node requirements
      requirements:
        # ARM architecture
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64"]

        # Support spot and on-demand (prefer spot for cost)
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]

        # ARM instance types (Graviton) - diverse selection
        - key: node.kubernetes.io/instance-type
          operator: In
          values:
            # General purpose (M7g)
            - "m7g.medium"
            - "m7g.large"
            - "m7g.xlarge"
            - "m7g.2xlarge"
            # Burstable (T4g) - cost-effective for variable workloads
            - "t4g.medium"
            - "t4g.large"
            - "t4g.xlarge"
            # Compute optimized (C7g)
            - "c7g.large"
            - "c7g.xlarge"
            # Memory optimized (R7g)
            - "r7g.large"
            - "r7g.xlarge"

      # Node class reference (Auto Mode creates this automatically)
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default


      # Taints (optional - for dedicated ARM workloads)
      taints:
        - key: arch
          value: arm64
          effect: NoSchedule

The terminationGracePeriod field defines the amount of time that a node can be draining before Karpenter forcibly cleans it up.

The spec.requirements section provides more details about the nodes that can be created. There are a specified here as an example.

The kubernetes.io/arch key sets out the architecture for the node. Karpenter supports amd64 and arm64 nodes. This is how we support Graviton.

The karpenter.sh/capacity-type key is analogous to EC2 puchase options. The general-purpose NodePool only supports on-demand as a value, whereas he we specify both spot and on-demand. As multiple capacity types are specified, Karpenter will prioritise spot where available, but fallback to on-demand.

Note: AWS automatically applies Amazon EC2 Reserved Instance discounts to matching running on-demand EC2 usage, regardless of how these instances were launched. This means that you will get these discounts for instances launched by Karpenter

There are a number of instance type options

key: node.kubernetes.io/instance-type
key: karpenter.k8s.aws/instance-family
key: karpenter.k8s.aws/instance-category
key: karpenter.k8s.aws/instance-generation
key: karpenter.k8s.aws/instance-capability-flex

Note: Generally, instance types should be a list and not a single value. Leaving these requirements undefined is recommended, as it maximizes choices for efficiently placing pods.

Each NodePool must reference a NodeClass. A Node Class defines infrastructure-level settings that apply to groups of nodes in your EKS cluster, including network configuration, storage settings, and resource tagging. When you need to customize how EKS Auto Mode provisions and configures EC2 instances beyond the default settings, creating a Node Class gives you precise control over critical infrastructure parameters. For example, you can specify private subnet placement for enhanced security, configure instance ephemeral storage for performance-sensitive workloads, or apply custom tagging for cost allocation. In this case, we just reference the default Auto Mode NodeClass.

There is also an example shown on how to apply a taint to a NodePool. When a taint is applied to a NodePool, Karpenter will only place pods on the nodes that explicitly tolerate the taint. In the example, Karpenter will only place a workload on the node that explicitly states that it supports the ARM architecture.

# Toleration for the taint (if you added one)
tolerations:
- key: arch
    operator: Equal
    value: arm64
    effect: NoSchedule

Limits

The limits section is used to constrain the total size of the NodePool. The limits that are set prevent Karpenter from creating new instances, once they have been exceeded. This is done to prevent runaway costs.

  # Limits for this node pool
  limits:
    cpu: "1000"
    memory: 1000Gi

Weight

The weight field controls prioritisation when Karpenter has multiple NodePools to choose from for scheduling a pod. When multiple NodePools can satisfy the requirements for a pod, Karpenter will give priority to the NodePool with the highest weight. If the weight attribute is not specified, it will default to 0.

  # Weight for prioritization (higher = preferred)
  weight: 10

Karpenter will look to choose the cheapest feasible instance. It prefers NodePools where it can pack the pod more efficiently with other pending pods, and minimise wasted CPU / memory on the node.

Note: Based on the way that Karpenter performs pod batching and bin packing, it is not guaranteed that Karpenter will always choose the highest priority NodePool given specific requirements. For example, if a pod can’t be scheduled with the highest priority NodePool, it will force creation of a node using a lower priority NodePool, allowing other pods from that batch to also schedule on that node. The behavior may also occur if existing capacity is available, as the kube-scheduler will schedule the pods instead of allowing Karpenter to provision a new node.

Targetting the NodePool with a Deployment

In order to test the NodePool and show it working, we created a Deployment, which is a simple Nginx container. It can be deployed using the following command from the code repository.

$ kubectl apply -f arm-deployment.yaml
deployment.apps/arm-app created

We define a Deployment and give it the name of arm-app, which is also assigned a label of the same name.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: arm-app
  labels:
    app: arm-app

The next part of the manifest file tells Kubernetes to run 3 copies of the application, and to make sure they are labelled as app=arm-app

spec:
  replicas: 3
  selector:
    matchLabels:
      app: arm-app
  template:
    metadata:
      labels:
        app: arm-app

The manifest file then defines a nodeSelector which is a rule that states that these pods can only run on nodes with an architecture type of arm64. This matches the architecture of our NodePool. Kubernetes will only schedule the Pod onto nodes that match the labels specified.

# Node selector for ARM architecture
nodeSelector:
  kubernetes.io/arch: arm64

The next part of the manifest file moves onto affinity. Node affinity functions like the nodeSelector field but is more expressive and allows you to specify soft rules. In this case, we use preferredDuringSchedulingIgnoredDuringExecution with a weight of 100 to state that we want the Pod to run on a Spot instance, but if this cannot be scheduled, then it is fine to drop back to on-demand. This means that the Pod will not remain in a pending state if a Spot instance was not available, and so it is considered a soft rule.

# Prefer spot instances for cost savings
affinity:
nodeAffinity:
  preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      preference:
        matchExpressions:
          - key: karpenter.sh/capacity-type
            operator: In
            values: ["spot"]

Finally, we use the containers section to say that we want to run a copy of nginx in each Pod, which half a CPU and 512 MB of memory reserved, but this can grow to a whole CPU and 1 GB or memory.

containers:
  - name: nginx
    image: nginx:latest  # Multi-arch image supports ARM64
    ports:
      - containerPort: 80
        name: http
    resources:
      requests:
        cpu: 500m
        memory: 512Mi
      limits:
        cpu: 1000m
        memory: 1Gi

We open up a number of additional terminal windows as we apply the Deployment, to give us more information on what exactly is happening in the background.

The first command lists all the nodes in the EKS cluster including a column showing their architecture and a column showing their capacity type. We can see that a node is in the Ready state which uses ARM and is a spot instance.

$ kubectl get nodes -L kubernetes.io/arch,karpenter.sh/capacity-type
NAME                  STATUS   ROLES    AGE     VERSION               ARCH    CAPACITY-TYPE
i-0d336c28e588123ae   Ready    <none>   2m22s   v1.34.3-eks-3c60543   arm64   spot

The second command lists the NodeClaim resources. A NodeClaim is a custom resource created by Karpenter. Here we can see the generated NodeClaim name is taken from the name of the NodePool with a random suffix. We can also see it is using spot capacity, and a supported instance family type.

kubectl get nodeclaims
NAME                       TYPE         CAPACITY   ZONE         NODE                  READY   AGE
arm-mixed-capacity-zw6vh   m7g.xlarge   spot       eu-west-2a   i-0d336c28e588123ae   True    3m

The next command describes all pods that have the label app=arm-app. This is the label that gets applied as part of the deployment. It filters the output to show the pod lifecycle events. Again, we can see from this that the pod is running on an ARM-based Graviton spot instance. The event timeline shows the lifecycle involved here. The pod is bound to a compatible node, it then downloads the latest nginx image from the container registry, the container is then created, and finally started.

kubectl describe pod -l app=arm-app | grep -A 20 Events

Name:             arm-app-6674bd9849-ld6fm
Namespace:        default
Priority:         0
Service Account:  default
Node:             i-0d336c28e588123ae/10.1.3.225
Start Time:       Tue, 27 Jan 2026 11:42:06 +0000
Labels:           app=arm-app
                  pod-template-hash=6674bd9849
Annotations:      <none>
Status:           Running
IP:               10.1.3.97
--
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  2m14s  default-scheduler  Successfully assigned default/arm-app-6674bd9849-ld6fm to i-0d336c28e588123ae
  Normal  Pulling    2m12s  kubelet            spec.containers{nginx}: Pulling image "nginx:latest"
  Normal  Pulled     2m9s   kubelet            spec.containers{nginx}: Successfully pulled image "nginx:latest" in 3.874s (3.874s including waiting). Image size: 61200811 bytes.
  Normal  Created    2m8s   kubelet            spec.containers{nginx}: Created container: nginx
  Normal  Started    2m8s   kubelet            spec.containers{nginx}: Started container nginx

We ran a similar command to list all of the pods running, and to show that the 3 replicas as specified in the deployment are running.

kubectl get pods -l app=arm-app -w
NAME                       READY   STATUS             RESTARTS   AGE
arm-app-6674bd9849-ld6fm   0/1     Pending             0          0s
arm-app-6674bd9849-76wzg   0/1     Pending             0          0s
arm-app-6674bd9849-2vnwx   0/1     Pending             0          0s
arm-app-6674bd9849-ld6fm   0/1     ContainerCreating   0          0s
arm-app-6674bd9849-76wzg   0/1     ContainerCreating   0          0s
arm-app-6674bd9849-2vnwx   0/1     ContainerCreating   0          0s
arm-app-6674bd9849-2vnwx   0/1     Running             0          7s
arm-app-6674bd9849-ld6fm   0/1     Running             0          7s
arm-app-6674bd9849-76wzg   0/1     Running             0          7s
arm-app-6674bd9849-ld6fm   1/1     Running             0          13s
arm-app-6674bd9849-76wzg   1/1     Running             0          13s
arm-app-6674bd9849-2vnwx   1/1     Running             0          13s

Get started with the Argo CD EKS Capability

Matt Lewis — Fri, 23 Jan 2026 14:49:00 +0000

Argo CD Overview

EKS Capabilities was announced at re:Invent 2025. These are Kubernetes-native platform features managed by AWS, that provide higher-level functionality. This post looks at the Argo CD capability.

Argo CD is a GitOps based continuous deployment tool. Your git repository becomes the source of truth, and Argo CD ensures that your cluster state matches what you have defined in git. AWS have been consistently guiding their customers towards GitOps for a number of years. AWS describe GitOps as being like a reference implementation of best practice with these 4 characteristics:

Desired state expressed declaratively
Desired state is immutable and versioned
Desired state is automatically applied from source
Desired state is continuously reconciled

Argo CD is used by most of AWS customers practicing GitOps in 2025, and has really emerged as its own de facto standard. More than 45% of Kubernetes end-users reported production or planned production use of Argo CD in the 2024 CNCF survey.

Setting up Argo CD Capability via Console

The quickest way to get up and running with Argo CD is via the console. In the EKS console there is a capabilities tab that shows which managed capabilities are deployed in the cluster and which are available.

From here, we can click the Create capabilities button, and tick the checkbox against Argo CD in the Deployment section, before clicking next.

This brings up the page where we configure the selected capabilities. The first thing to do is to either select an existing role for the capability role, or select the button to create a new role.

Finally, the Argo CD managed capability integrates with AWS Identity Centre for authentication, and uses RBAC roles for authorization. You select the existing instance of IAM Identity Centre which should be pre-populated when you click the drop down. The next step is to assign RBAC roles. This involves specifying an AWS user or group from AWS Identity Centre and assigning them to an Argo CD RBAC role of "Admin", "Editor" or "Viewer".

At this point, you can review and create the managed capability for Argo CD. To understand this process in more detail, we can step through how to setup the capability using IaC.

Setting up Argo CD Capability via IaC and CLI

The code samples required for this section are contained in the CloudFormation section of this GitHub repository.

Create an IAM Capability Role

The first step is to create an IAM Capability Role. EKS Capabilities use this role to act on your behalf, running controllers in EKS. EKS Capabilities introduced a new service principle called capabilities.eks.amazonaws.com. When you create the capability role, you need to ensure the trust policy trusts this new service principle. An example of the required trust policy is shown below which is available in a file named "argocd-trust-policy.json":

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "capabilities.eks.amazonaws.com"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:TagSession"
            ]
        }
    ]
}

We can create an IAM role with this trust policy using the following command:

aws iam create-role \
  --role-name ArgoCDCapabilityRole \
  --assume-role-policy-document file://argocd-trust-policy.json

Next we need to attach permissions to this Capability IAM Role based on the capability needs and what integrations are required. For example, for Argo CD you may need to give permissions to ecr or codecommit or codeconnection, depending on where your source is coming from. When choosing "Create Argo CD role" through the console, an IAM role with the AWSSecretsManagerClientReadOnlyAccess managed policy pre-selected is created. This managed policy provides read access to all secrets stored in Secrets Manager in your AWS account and is intended for getting started quickly. You have the flexibility to modify these permissions by unselecting this policy or adding different policies as needed.

We can this managed policy to the created role by using the following command:

aws iam attach-role-policy \
  --role-name ArgoCDCapabilityRole \
  --policy-arn arn:aws:iam::aws:policy/AWSSecretsManagerClientReadOnlyAccess

We can achieve the same in CloudFormation but in a single template. The AWS::IAM::Role config required is shown below, although we will use this as part of the CloudFormation template that creates the EKS Capability, so we can control it in a single stack.

  # IAM Capability Role for ArgoCD
  ArgoCDCapabilityRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: "RoleName"
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: capabilities.eks.amazonaws.com
            Action:
              - sts:AssumeRole
              - sts:TagSession
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AWSSecretsManagerClientReadOnlyAccess

Create EKS Capability

Finally, we can create the EKS Capability for Argo CD itself. The Argo CD capability is integrated with AWS Identity Centre (IDC). This ensures that single sign-on is enabled for the fully managed and hosted Argo UI instance and for Argo CLI. For this, you need to ensure that your identity centre configuration is passed into the capability when creating it.

The Argo CD IDC instance identifies the name of the IAM Identity Center instance that is used by your organization to get permissions for Argo CD to access your EKS cluster. Once the capability has been created, the Argo CD IDC instance cannot be edited.

We can retrieve the IDC Instance ARN directly from the console, or by running the following command:

aws sso-admin list-instances --region eu-west-2 --query 'Instances[0].InstanceArn' --output text

We need to identify the users or groups that should be assigned to be a particular Argo CD role. The users and groups you identify from your IDC instance defines the permissions that the Argo CD capability has to access your EKS cluster. In the command below I retrieve the User ID for the user called mattlewis:

aws identitystore list-users \
  --region eu-west-2 \
  --identity-store-id $(aws sso-admin list-instances --region eu-west-2 --query 'Instances[0].IdentityStoreId' --output text) \
  --query 'Users[?UserName==`mattlewis`].UserId' --output text

If I wanted to assign a Group rather than a User, then I need to retrieve a Group ID from Identity Centre. The following command retrieves the Group ID for a group called Admin:

aws identitystore list-groups \
  --region eu-west-2 \
  --identity-store-id $(aws sso-admin list-instances --region eu-west-2 --query 'Instances[0].IdentityStoreId' --output text) \
  --query 'Groups[?DisplayName==`Admin`].GroupId' \
  --output text

There is a JSON file called aws-identity-centre-configuration.json which is made available for convenience. The configuration below assigns the User ID to the Argo CD role of ADMIN, and the Group ID to the Argo CD role of VIEWER.

{
    "argoCd": {
      "awsIdc": {
        "idcInstanceArn": "REPLACE_WITH_IDC_INSTANCE_ARN",
        "idcRegion": "REPLACE_WITH_REGION"
      },
      "rbacRoleMappings": [{
        "role": "ADMIN",
        "identities": [{
          "id": "REPLACE_WITH_USER_ID",
          "type": "SSO_USER"
        }]
      },{
        "role": "VIEWER",
        "identities": [{
          "id": "REPLACE_WITH_GROUP_ID",
          "type": "SSO_GROUP"
        }]
      }
    ]}
  }

We can then create the Argo CD capability using the following AWS CLI command:

aws eks create-capability \
  --capability-name argocd-capability \
  --cluster-name eks-test-cluster\
  --type ARGOCD \
  --role-arn arn:aws:iam::{ACCOUNT_ID}:role/ArgoCDCapabilityRole \
  --delete-propagation-policy RETAIN \
  --configuration file://aws-identity-centre-configuration.json

This is equivalent to the following that can be used in a CloudFormation template.

ArgoCDCapability:
  Type: AWS::EKS::Capability
  Properties:
    CapabilityName: !Ref CapabilityName
    ClusterName: !Ref ClusterName
    Type: ARGOCD
    RoleArn: !GetAtt ArgoCDCapabilityRole.Arn
    DeletePropagationPolicy: !Ref DeletePropagationPolicy
    Configuration:
      ArgoCd:
        AwsIdc:
          IdcInstanceArn: !Ref IdentityCenterInstanceArn
          IdcRegion: !Ref IdentityCenterRegion
        RbacRoleMappings:
          - Identities:
              - Id: !Ref AdminUserId
                Type: SSO_USER
            Role: ADMIN
          - Identities:
              - Id: !Ref ViewerGroupId
                Type: SSO_GROUP
            Role: VIEWER

You can deploy the full CloudFormation stack using the following command:

aws cloudformation create-stack \
  --stack-name argocd-capability-stack \
  --template-body file://argocd-capability.yaml \
  --parameters \
    ParameterKey=ClusterName,ParameterValue=eks-test-cluster \
    ParameterKey=IdentityCenterInstanceArn,ParameterValue=arn:aws:sso:::instance/ssoins-{REPLACE} \
    ParameterKey=IdentityCenterRegion,ParameterValue=eu-west-2 \
    ParameterKey=AdminUserId,ParameterValue={REPLACE_WITH_USER_ID} \
    ParameterKey=ViewerGroupId,ParameterValue={REPLACE_WITH_GROUP_ID} \
  --capabilities CAPABILITY_NAMED_IAM \
  --region eu-west-2

Some of the properties include:

Type - This defines the type of EKS Capability to create with valid values of
- ACK - Amazon Web Services Controllers for Kubernetes (ACK), which lets you manage resources directly from Kubernetes
- ARGOCD – Argo CD for GitOps-based continuous delivery
- KRO – Kube Resource Orchestrator (KRO) for composing and managing custom Kubernetes resources
DeletePropagationPolicy - This only supported value is RETAIN, which keeps all resources managed by the capability when the capability is deleted
Configuration - This property defines the configuration settings, with the structure depending on the capability type
Role - The role under the RbacRoleMappings property defines the Argo CD role to be assigned. The value values are:
- ADMIN - Full administrative access to Argo CD
- EDITOR - Edit access to Argo CD resources
- VIEWER - Read-only access to Argo CD resources

Once the capability has been created (which will take a while), the capabilities tab for the cluster in the EKS console will provide the Argo API endpoint and a link to go to the managed hosted Argo UI:

We can click on the link to open up the managed Argo UI. At this point, we will need to click the button to LOG IN VIA SSO.

In most cases, this will automatically log you directly into the console. At this point, we can see there are no applications currently available.

We can also check the Argo CD Role Based Access Control (RCAC) assignments in the console, and make sure that it matches what we set up in the previous JSON file or in the CloudFormation template.

Argo CD Capability

Now that the Argo CD capability has been created, we can take a quick look at the same of the changes that have been made to our cluster.

Firstly, we run a command to look at the EKS Access Entries for the cluster.

aws eks list-access-entries \
  --cluster-name eks-test-cluster

EKS Access Entries are the recommended way to grant users access to the Kubernetes API. Fundamentally, it associates a set of Kubernetes permissions with an IAM identity such as an IAM Role. Running the command above generated the following output.

{
    "accessEntries": [
        "arn:aws:iam::{ACCOUNT_ID}:role/ArgoCDCapabilityRole",
        "arn:aws:iam::{ACCOUNT_ID}:role/aws-reserved/sso.amazonaws.com/eu-west-2/AWSReservedSSO_Developer_b664db2de4791f77",
        "arn:aws:iam::{ACCOUNT_ID}:role/aws-service-role/eks.amazonaws.com/AWSServiceRoleForAmazonEKS",
        "arn:aws:iam::{ACCOUNT_ID}:role/eks-test-cluster-eks-auto-20260116165710199100000002"
    ]
}

We can see that four access entries currently exist in the cluster:

The SSO Developer role entry, which is the role I assumed to create the EKS cluster
An EKS Auto Mode generated role that is used to enable Auto Mode to make authenticated Kubernetes API calls
An EKS service-linked role that is used to manage the control plane and AWS-side resources of the cluster
The ArgoCDCapabilityRole role entry, which has been created when enabling the Argo CD EKS Capability which allows the capability to authenticate to the cluster

You can see the permissions that have been automatically granted to each entry in the console.

By enabling the EKS Argo CD Capability, the following Custom Resource Definitions (CRDs) have been added to the cluster.

kubectl get crds | grep argo  

applications.argoproj.io                        2026-01-16T17:10:56Z
applicationsets.argoproj.io                     2026-01-16T17:10:56Z
appprojects.argoproj.io                         2026-01-16T17:10:57Z

Deploy a sample application

Register your EKS cluster with Argo CD

In order to deploy a sample application, the first step is to register the EKS cluster where we want to deploy an application. We do this by by creating a Kubernetes secret, and passing the label of argocd.argoproj.io/secret-type: cluster. We give the cluster a name, and this is where the mapping between the actual cluster and the ARN happens. With EKS Capabilities you only need to provide the ARN and not the Kubernetes API Server URL as with a self-managed instance. In our case, we are registering a local cluster, as it is the same one as Argo CD is running. The following manifest file is found in the code repository under k8s/argocd.

apiVersion: v1
kind: Secret
metadata:
  name: local-cluster
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: cluster
stringData:
  name: local-cluster
  server: arn:aws:eks:eu-west-2:{ACCOUNT_ID}:cluster/eks-test-cluster
  project: default

We then apply this to the cluster

kubectl apply -f local-cluster.yaml

Register an Argo CD Application

Now we can register an Argo CD application. In this case, we will use the guestbook example from the Argo CD project itself.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/argoproj/argocd-example-apps
    targetRevision: HEAD
    path: guestbook
  destination:
    server: arn:aws:eks:eu-west-2:{ACCOUNT_ID}:cluster/eks-test-cluster
    namespace: default

This is a Kubernetes manifest file for a custom resource (an Argo CD Application). Our cluster knows how to handle this as it has installed the CRD when deploying the capability itself. This application tells Argo CD to deploy the guestbook application from the specific public Git repository into the default namespace of the EKS cluster. We can run this file using the following command.

kubectl apply -f guestbook-application.yaml

We should get back the information that the application has been created application.argoproj.io/guestbook created.

At this point we can log in to the Argo UI, and see something similar to the following:

If we go into settings, we can see there is a failed connection status with our cluster.

And if we go further and click into the application itself, we see that there are 3 errors in the application conditions.

The issue here is that Argo CD cannot build its cache of what exists in the cluster, because it does not have permission to list the cluster-scoped resource PersistentVolume. It also cannot list resource "ingressclassparams" in API group "eks.amazonaws.com" at the cluster scope to connect to the cluster.

Associate an access policy

This issue is caused because the two access policies associated with the Argo CD Capability role (AmazonEKSArgoCDPolicy and AmazonEKSArgoCDClusterPolicy) do not give the required permissions to access or mutate the cluster resources.

The best practice in this case is to determine the minimum permissions required, and add these to an access policy. However, the fastest solution is to add the AmazonEKSClusterAdminPolicy with cluster scope to the access entry, which we can do using the following command:

aws eks associate-access-policy \
  --cluster-name eks-test-cluster \
  --principal-arn arn:aws:iam::424727766526:role/ArgoCDCapabilityRole \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \
  --access-scope type=cluster

Once we have run this command, Argo CD will now be able to connect successfully to the cluster:

And the application can be synced and is healthy.

AWS Certified Machine Learning Engineer Core Concepts

Matt Lewis — Sat, 04 Oct 2025 13:32:38 +0000

Overview

Earlier this year I passed the AWS Machine Learning Engineer - Associate exam. I spent time making sure I understood the core concepts before taking the exam, and made a lot of notes. The intent of this post is to summarise the concepts essential to pass the exam. Based on my experience, knowing these concepts will get you at least half way there. Layer on knowledge of the AWS AI Services, and focus on SageMaker and all its capabilities, and the certification will be yours.

Data Preprocessing
SageMaker Built-In Algorithms
Model Development
Evaluating Model Accuracy
Improving Model Accuracy
Additional Topics to Study
Other Study Guides

Data Preprocessing

Data preprocessing ensures that data is in the right shape and of the right quality to be used for training.

Labelling data is important for models to learn effectively, and this is where services such as Mechanical Turk and SageMaker Ground Truth come in. Mechanical Turk is an online marketplace to access an on-demand global workforce. SageMaker Ground Truth provides built-in workflows to automate data labelling, and can use your own workforce, third-party vendors from the AWS marketplace, or Mechanical Turk.

Cleaning data can include removing outliers and duplicates, replacing inaccurate or irrelevant data, and correcting missing data.

Approaches for inputting missing data include:

Mean replacement – replacing the missing values with the mean value from the rest of the column. The mean value is the average, which means it can be distorted by outliers. Therefore, the median value (which is the middle value when data is sorted) may be a better choice
KNN (K-Nearest Neighbour) – Find the ‘K’ nearest’ most similar rows and average their values. This assumes numeric data.

Balancing data (for datasets with underrepresented categories) can be achieved using one of the following methods:

Random oversampling – this method randomly duplicates samples from the minority category. For example, if you were building a fraud detection model and you had 1000 examples of normal transactions and only 50 of fraudulent transactions, you would duplicate the fraudulent transactions until you had an equal proportion
Random Undersampling – this method randomly removes samples from the overrepresented category to achieve the equal proportion. This would typically be used when you have a large dataset, or you would want to reduce the size of your dataset to make training the model quicker
Synthetic Minority Oversampling Technique (SMOTE) – this approach generates new synthetic samples of the minority category by interpolating between existing samples using nearest neighbours.

Encoding is the concept around converting data (typically categorical data where the data represents a category or group) into a numerical format that can be well understood by a model. The main types of encoding are the following:

Label Encoding – assigns each category a unique number e.g. Red=0, Green=1, Blue=2. There is no order implied by this encoding
One-Hot Encoding – creates binary columns for each category. This means if there is a category called colour there would be additional columns created for each unique value such as a column for Red and one for Blue and for Green. The value for each column would be assigned 1 if it is true, else 0.
Ordinal Encoding – this is similar to label encoding but is used when there is a ranked ordering between values in a category. For a category called ‘size’ you could map Small to 0, Medium to 1 and Large to 2.

Outliers are data points in a data set that deviate significantly from the general patterns. One way of detecting outliers in training data is to measure how many standard deviations a data point is from the mean of a dataset. This is often called a z-score or standard score. Data points that lie more than one standard deviation from the mean can be considered unusual.

Outliers can be handled in different ways:

Delete the record - if the outlier is clearly an error and there is enough training data, you can just delete that record.
Feature scaling or normalisation – this aims to transform the numeric values so that are all values are on the same scale, often between 0 and 1. This rescaling makes the values more comparable
Standardisation – is similar to normalisation but instead of scaling values from 0 to 1, it rescales the features to have a mean of 0 and standard deviation of 1
Binning – takes a continuous numerical feature and splits into a set of intervals or bins. Each value is then assigned to a bin which can cover up imprecision or uncertainty e.g. someone aged 110 could end up in a bin which is “70+”.

After data has been cleaned up and encoded, you can fine tune or create new features in your dataset through feature engineering. There are other methods such as bag-of-words and N-gram that can be used to extract key information from text data.

In machine learning, dimensionality refers to the number of features in your dataset. If you have a dataset with 3 features (age, income and height), it exists in a 3-dimensional space. The curse of dimensionality refers to the problems that arise when you have too many features. When this happens, the data becomes sparse (e.g. spread out too thinly in the feature space), and it is hard to find meaningful patterns.

There are a number of unsupervised reduction techniques that can help to distil many features into a smaller more manageable number:

Principal component analysis (PCA) – this technique retains most of the variation in the original features but reduces the overall number of features. It works by transforming features into a new set of features called principal components
K-Means – this technique uses a clustering algorithm to group similar data points into K clusters. It does not create new features but assigns a cluster label to each data point.

SageMaker Built-In Algorithms

Before you can train your model, you need to select a machine learning algorithm to use. Amazon SageMaker provides a suite of built-in algorithms, pre-trained models, and pre-built solution templates to help data scientists and machine learning practitioners get started on training and deploying machine learning models quickly. It is worth understanding each of these algorithms at a high-level, understanding which ones are used for supervised versus unsupervised learning, and their main uses.

Linear Learner - A supervised learning algorithm suited for general classification (Logistic Regression) and regression (Linear Regression) tasks. It makes predictions based on labelled data. In simpler terms, the model is given examples where each example has some features (like size, price, or age) and an outcome (like a house price or a category). For classification tasks, the model sorts data into categories, such as whether a house is expensive or not. For regression tasks, it predicts a specific value, like the actual price of a house. It assumes linear regression and must pre-process missing values.

K-Means - An unsupervised algorithm designed for clustering or grouping data points based on their features (chosen attribute), without needing labelled data

BlazingText - A supervised algorithm used for Natural Language Processing tasks like text classification.

Seq2Seq - A supervised algorithm specifically designed for sequence-to-sequence tasks, such as predicting the next word in a sequence, making it ideal for tasks like language translation or text generation

DeepAR - A supervised algorithm used to forecast time-series predictions by using recurrent neural networks (RNN)

XGBoost - A supervised learning algorithm used for both classification and regression tasks — especially when you care about speed and performance. It is an optimized, scalable implementation of gradient boosting that builds an ensemble of decision trees in a sequential manner. Often outperforms other models in competitions (e.g., Kaggle)

Random Cut Forest - An unsupervised algorithm used to identify abnormal data points within a data set e.g. anomaly detection

Semantic Segmentation - A supervised algorithm that provides pixel-level classification but does not label objects with bounding boxes. It is typically used to classify individual pixels by tagging each pixel with a specified class

Principal Component Analysis (PCA) - An unsupervised algorithm used for dimensionality reduction

Image Classification - A supervised algorithm that is used to label entire images, not individual objects. It simply assigns a single label to an entire image, categorising it based on the predominant features. It cannot identify or count multiple objects within a single image.

Object Detection - A supervised algorithm used to identify and classify multiple objects within an image, assigning bounding boxes and confidence scores. It draws bounding boxes around detected objects and classifies them into different categories, making it very useful for tasks where you need to recognise what is in the image and determine the exact location of each object. This algorithm is well-suited for scenarios that require counting specific items, such as animals, in drone imagery, as it can distinguish between individual objects even in complex scenes.

Object2Vec - A supervised algorithm primarily used to learn vector embeddings of discrete objects. Its typically used in recommendation systems, document classification or semantic similarity tasks, not computer vision or image processing.

IP Insights - An unsupervised algorithm used to detect anomalies in IP address usage patterns. It captures associations between these IP addresses and various entities, such as user IDs or account numbers. For instance, you can use it to detect a user attempting to log into a web service from an anomalous IP address. Additionally, it helps identify accounts that create computing resources from unexpected IP addresses.

Latent Dirichlet Allocation (LDA) - An unsupervised learning technique designed to represent a collection of documents as a combination of various topics. LDA is primarily used to identify a specified number of topics within a set of text documents. The LDA algorithm is a powerful tool for text mining and natural language processing tasks. It allows companies to sift through vast amounts of textual data and discern patterns that might take time to be apparent. Since LDA is an unsupervised method, the topics are not specified up front, and the discovered topics may not necessarily match human categorisations. Instead, LDA learns the topics as a probability distribution over the words in the documents, and each document is characterised as a mixture of these topics.

Neural Topic Model (NTM) - An unsupervised algorithm used for organising documents into topics. It is just like LDA — but it's based on neural networks rather than probabilistic graphical models.

Factorization Machines - A supervised algorithm designed to handle sparse data, making it ideal for recommendation systems where user-item interactions are often sparse. It is primarily used for recommendation systems and ranking predictions.

Text Classification – TensorFlow algorithm - A supervised algorithm designed to classify text into predefined categories.

Model Development

Hyperparameters are external configuration variables used to control a training model, improve model performance and the model outcome. Hyperparameters are set before training. This can be done manually, although SageMaker offers an intelligent version of hyperparameter tuning methods based on Bayesian search theory designed to find the best model in the shortest time. Amazon SageMaker AI automatic model tuning (AMT) finds the best version of a model by running many training jobs on your dataset. Amazon SageMaker AI automatic model tuning (AMT) is also known as hyperparameter tuning and supports Hyperband, a new search strategy.

Common hyperparameters include:

Epoch – the number of times the entire training dataset is shown to the network during training. A smaller epoch value means faster training, but the model might not learn enough patterns and end up underfitting. A larger epoch value gives more opportunity to refine weights and give better convergence, but will take longer to train and may end up memorising training data and so overfitting
Learning rate – the rate at which an algorithm updates estimates. Too high a learning rate means you might overshoot the optimal solution. Too small a learning rate will take too long to find the optimal solution
Batch size – how many batch training samples are used within each batch of each epoch. Large batch sizes are faster per epoch as it will fill the GPU, but risk worse generalisation and can end up getting stuck in the wrong solution. Small batch sizes are slower per epoch but can provide better generalisation.

Note that hyperparameters are not related to inference parameters. Inference parameters are settings you can adjust during inference, that influence the response from the model. The most common are:

Temperature: Temperature is a value between 0 and 1, and it regulates the creativity of the model's responses. Use a lower temperature if you want more deterministic responses, and use a higher temperature if you want creative or different responses for the same prompt
Top K: The number of most-likely candidates that the model considers for the next token. Choose a lower value to decrease the size of the pool and limit the options to more likely outputs.
Top P: The percentage of most-likely candidates that the model considers for the next token. Choose a lower value to decrease the size of the pool and limit the options to more likely outputs.

Evaluating Model Accuracy

Metrics are used to measure the performance and accuracy of a machine learning model. These metrics can typically be broken down into classification metrics and regression metrics.

With classification, the goal of the model is to predict a label or class (category) for the given input. With binary classification, there are only two possible outputs (positive or negative). This is used to predict whether an image is a dog or not, or whether an email is spam or not. With multi-class classification, there are more than two possible outputs, such as predicting whether an animal is a dog, cat or cow.

With regression, the goal of the model is to predict a numerical value. This could be predicting a house or stock price, or a person's annual income given certain inputs.

Classification Metrics

The confusion matrix is a great way to help understand common classification metrics.

Recall - Recall is the percentage of positives correctly predicted. It is focused on how many of the actual positives did the model get right. You use it when you prefer to catch as many positives as possible, even if some are incorrect. It is a good metric when false negatives are costly e.g. fraud detection or cancer screening where it is better to flag more cases (even if some are wrong) than to miss a true positive

It is calculated as: Recall = TP / (TP + FN)

Precision - Precision is all around correct positives. All positive predictions include both true positives and false positives (those predicted as positive but are actually negative). This means it is a good metric when false positives are costly e.g. spam email when you don't want to mark legitimate emails as spam, or object detection in autonomous vehicles, where a false positive can induce sudden unnecessary braking.

It is calculated as: Precision = TP / (TP + FP)

A model with high precision and low recall catches few positives but is rarely wrong.

A model with high recall and low precision catches most positives but includes many false alarms.

F1 Score - The F1 Score is the harmonic mean of precision and recall. It is used when you need a balance betweeb both.

It is calculated as: F1 Score = 2 x ((Precision x Recall) / (Precision + Recall))

Accuracy - Accuracy measures overall correctness — how often the model was right, regardless of class. It considers all predictions (true positives, true negatives, false positives, and false negatives).

It is calculated as: Accuracy = TP + TN / TP + TN + FP + FN

AUC and ROC - The ROC curve is a graphical plot that helps visualise how well a binary classification model performs across different threshold values. It is a plot of true positive rate (recall) versus false positive rate and helps you see this trade off between True Positives and False Positives.

The Area under the Curve (AUC) is a single scalar value between 0 and 1 of how well the classification model can separate the positive and negative predictions. A value of 0.5 means the model performs no better than a random classifier. A value of 1.0 is a perfect classifier.

Regression Metrics

If you are using regression where you are predicting a number and not just a classification, then there are other metrics:

Mean Absolute Error (MEA) - MEA measures the average absolute difference between the predicted values and the actual values. It tells you on average, how much your models predictions are off from the true values. A lower score means a better model. It is simple to understand and is not affected by outliers. MAE is robust to outliers because it handles them linearly, not exponentially. This makes it a good choice when you don’t want a few bad predictions to dominate the error metric.

Mean Squared Error (MSE) - MSE averages the squared difference between actual and predicted values. Because it squares the values, outliers become amplified. This makes MSE more sensitive to outliers. You would choose MSE (Mean Squared Error) over MAE (Mean Absolute Error) when you want to penalize large errors more heavily and are more concerned with model performance on extreme values.

RMSE (Root Mean Square Error) - RMSE is a metric used to measure the differences between predicted values and actual values in a regression problem. It calculates the square root of the average squared differences between the predicted and actual values. A lower Root Mean Square Error value indicates better model performance. Since the errors are squared before averaging, larger errors have a bigger impact (this makes RMSE sensitive to outliers).

You would use RMSE (Root Mean Squared Error) over MSE (Mean Squared Error) when you want the error metric to be in the same units as the target variable, making it more interpretable. For example, if you're predicting house prices in dollars, RMSE is in dollars, while MSE is in squared dollars, which is less intuitive.

R Squared - R-Squared measures the square of the correlation coefficient between observed outcomes and predicted. It measures how well your regression model explains the variability of the target (dependent) variable. A score of 1 means the model explains all the variance perfectly. A score of 0 means the model explains none of the variance.

Improving Model Accuracy

Understanding model fit is important when understanding the root cause for poor model accuracy.

Two common terms that come up to describe model performance are:

Overfitting – a model that is overfitting has learned patterns in the training data that don’t generalise out to the real world. This means that it has high accuracy on the training data set, but lower accuracy on evaluation data sets
Underfitting – a model that is underfitting performs poorly on the training data and in the real world. This is because the model is unable to capture the relationship between the input examples and the target values.

Regularization techniques are intended to prevent overfitting. Common techniques include:

Dropout – this is a technique where random neurons are temporarily dropped out (e.g. ignore) during each training iteration. This means the network can’t rely too heavily on any specific neuron or connection
Early Stopping – this is a technique where you stop training the neural network before it overfits the training data. It works by monitoring validation loss and accuracy and stopping training when the model stops improving.
L1 and L2 Regularization

L1 and L2 Regularization are techniques used to prevent overfitting by penalising large model weights. In a machine learning model, a weight is a numeric parameter that connects an input feature to an output. A large weight value means the model is putting an extremely strong emphasis on that specific feature, which means the model becomes very sensitive to small changes in inputs, which can lead to overfitting. These techniques add a penalty term to the loss function to discourage large weights:

L1 Regularization (Lasso) – the penalty is the sum of the absolute value of the weights. It shrinks some weights entirely to zero to create sparse models. This is a form of feature selection (removing irrelevant features). You should use this when you suspect only a few features are important. It is computationally inefficient.
L2 Regularization (Ridge) – the penalty is the sum of the square of the weights. This shrinks the weights but does not make them zero. It helps keep the model simpler and reduces sensitivity. It is computationally efficient.

Additional Topics to Study

The two other main topic areas you need to understand are:

AWS AI Services - these are the managed AWS services that offer a simpler entry point than building your own model. You will need to a good understanding of each service and what it is used for, so you can distinguish between Amazon Lex and Amazon Polly, and between Amazon Translate and Amazon Transcribe.
Amazon SageMaker - Amazon SageMaker is a service that provides a whole host of features and capabilities you need to be aware of. You need to understand which feature you can use to detect bias; which to import, prepare and transform data; which to share curated features, and so on.

Other Study Guides

AWS Certification Page - the AWS certification home page for this exam includes the study guide and links to additional resources
AWS SkillBuilder - the AWS official learning plan which is available for free alongside an official set of practice exam questions. Additional material including longer review sections, extra questions and labs are available with a subscription.
Udemy - the certification course provided by Stephane Maarek and Frank Kane comes highly recommended
Pluralsight - this certification course by Pluralsight also includes labs. Pluralsight offer a 10 day free individual trial and monthly subscriptions which may work for some
Tutorials Dojo - this set of practice questions is a great way to get used to the style of exam in various modes

Next Gen Developer Experience with Amazon Q Developer

Matt Lewis — Thu, 08 May 2025 08:55:15 +0000

There have been massive advances in the capabilities and features supported by Amazon Q Developer over the last few months. A number of these really stood out for me:

At the beginning of March the new enhanced Amazon Q Developer CLI agent was released with the power of Claude 3.7 Sonnet step-by-step reasoning. This also gave the agent access to tools such as the AWS CLI. Read more in the blog post
At the end of April the Amazon Q Developer CLI was further enhanced with support for Model Context Protocol (MCP) to provide even more context. Read more in the blog post
At the beginning of May, Amazon Q Developer support was integrated into GitHub in preview. Read more in the blog post

With all of these improvements, I wanted to see if there was a way of bringing them together to meet a coherent use case. This use case was to:

Take a task assigned to me from a Jira Kanban board
Implement the requested functionality
Push the code up to GitHub as the source code repository
Run a check for security vulnerabilities and code quality issues
Raise a Pull Request
Move the task along on the Kanban board

The goal was to show how these tools can make life easier for a software engineer, and greatly increase their productivity. Let's see how I got on.

Step 1: Set up Jira

I am using a hosted version of Jira Cloud using the free tier provided by Atlassian. The first thing I did was to create a new Jira project that sets up a Kanban board using a software development supporting template.

Next I created a new Jira task. The task was to "create a classic snake game written in python using pygame", and I assigned it to myself. Although this is a contrived example, you could easily equate this to a new feature on an existing service.

Once created, we can see this task in to "To Do" section of the Kanban board.

I also created a new GitHub repository which is cloned to my workspace.

Step 2: Configure MCP Servers

The next step was to setup the Amazon Q CLI to access both Atlassian and GitHub. Amazon Q CLI acts as an MCP Client, and it can access MCP Servers that have been configured in a mcp.json file. This files needs to be located in ~/.aws/amazonq. You can find out more details in this blog post by Ricardo Sueiras.

I want to run these MCP Servers in a container, and without access to Docker Desktop I configure them to use Podman. My configuration is shown below.

{
  "mcpServers": {
    "github-mcp-server": {
      "command": "podman",
      "args": [
        "run",
        "--rm",
        "--interactive",
        "--env",
        "GITHUB_PERSONAL_ACCESS_TOKEN",
        "ghcr.io/github/github-mcp-server"
      ],
      "env": {"GITHUB_PERSONAL_ACCESS_TOKEN": "github_pat_xxx"}
    },
    "mcp-atlassian": {
      "command": "podman",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e", "JIRA_URL",
        "-e", "JIRA_USERNAME",
        "-e", "JIRA_API_TOKEN",
        "ghcr.io/sooperset/mcp-atlassian:latest"
      ],
      "env": {
        "JIRA_URL": "https://xxx.atlassian.net/",
        "JIRA_USERNAME": "xxx@email.com",
        "JIRA_API_TOKEN": "XXXXX"
      }
    }
  }
}

This required creating a Personal Access Token in GitHub, and an API Token in Atlassian.

Step 3: Run Amazon Q Developer CLI

After making the changes to the mcp.json configuration, I launched Amazon Q CLI from the terminal window in my Visual Studio Code IDE. You can see that the two MCP Servers have been loaded and are accessible.

I start by asking the Amazon Q CLI to “get the latest task from Jira that is assigned to me”. The Amazon Q CLI returns wanting to know more details, before it can use the configuration to retrieve information about the task.

I tell the Amazon Q CLI that I am using Jira Cloud and to search across all projects. I am then told that the Amazon Q CLI wants to run a tool provided by the mcp_atlassian MCP Server. I am prompted to either press t to always trust the tool for the session, y to allow the tool to be executed this time without trusting for the session, or n to not let the tool be executed. I will be answering y to all of these prompts.

After running the tool, the Amazon Q CLI has found the task and displays all of the details.

I ask Amazon Q CLI to help me implement the functionality, and it goes away and generates all the code required. At this point, the code is in memory, and I am asked if I want to save the code to a file in my current directory.

I tell the Amazon Q CLI to create a new branch and add the file to this new branch. After running the relevant git commands to create a new branch, the Amazon Q CLI switches to this branch, and then writes the code to a new file in this branch.

After successfully writing the code to a new branch, the Amazon Q CLI commits the changes with a message referencing the specific Jira task number that it still has in its current session context.

At this point, I could prompt to make more enhancements to the game. The Amazon Q CLI even gives suggestions of areas of the game it could improve. Instead, I just ask it to update the README file with instructions on how to play the game, and then make another commit.

The Amazon Q CLI now asks if I want to make any other changes. I'm happy with those that have been made so far, so ask it to make a pull request for these. Notice the first request to create a pull request fails. Amazon Q CLI apologises, and tries another approach, this time pushing the branch to GitHub and then creating the pull request which succeeds.

After creating the pull request, the Amazon Q CLI knows from its session context and its reasoning, that we should also update the original task in Jira. It interacts with tools in the Atlassian MCP Server to transition the task to the "In Progress" state and add a relevant comment.

Step 4: Jira Kanban Board

At this point in time, I go across to the Kanban board in Jira and can see that the task has been transitioned to "In Progress".

Clicking into the task, I can see the comment that has been added to the task. This gives details about the functionality implemented to meet the task description, alongside a working link to the open PR in GitHub.

Step 5: Code Scanning in GitHub

The final task I wanted to carry out was to run a check for security vulnerabilities and code quality issues. I have already configured my GitHub account with the Amazon Q Developer application. This means that as soon as the pull request was raised, the amazon-q-developer application automatically scanned the changes in the PR. Happily, there were no security or code quality issues found. If there were, the new application would have automatically generated code suggestions to fix the findings.

Conclusion

I can't remember the last time I tested out new features and services and genuinely felt so convinced we are now starting to see a change in how we will engineer applications in the software industry. The value of software engineering still remains, even more so when working on complex problems for which generative AI solutions do not have the corpus of knowledge to be trained on. However, this showed to me how these new capabilities can help in reducing the context switching, and the need to move between various tools, copying data between them. The next generation of developer experience is well and truly upon it, so I'd urge everyone to try it out.

Biography

As Chief AWS Architect at IBM in the UK, I am responsible for growing the AWS capability and community within one of the fastest growing AWS consulting partners globally. This often gives me the opportunity to try out the latest features in preview before they go into general availability. You'll often find me blogging about my experience, but please reach out if there are services you'd like to know more about.

Java code transformation using the Amazon Q Developer GitHub Integration

Matt Lewis — Tue, 06 May 2025 10:30:37 +0000

AWS have launched the Amazon Q Developer integration with GitHub. I was keen to try this out, and in this post, I walk through how to get started and use the integration to upgrade a Java project.

Getting Setup

The first step is to install the Amazon Q Developer application from the GitHub Marketplace found at this URL - https://github.com/apps/amazon-q-developer.

Then click on Install and select which repositories you want to allow the application to access.

You can also optionally register the application installation with your AWS account to increase your usage limits. This is a two-step process. A landing page in your AWS console allows you to authorise Amazon Q Developer to access your GitHub account.

This redirects you to GitHub to complete the authorisation process.

You are then returned the AWS Console to provide a registration name and complete the registration process.

Setup GitHub Actions

Before you can get started on running a transformation request, you need to enable GitHub Actions for the repository and ensure a runner is available online.

This involves creating a main.yml file within a .github/workflows/ folder structure. The workflow I used is shown below:

name: Q Code Transformation

on:
  push:
    branches:
      - 'Q-TRANSFORM-issue-*'

env:
   MAVEN_CLI_OPTS: >-
     -Djava.version=${{ contains(github.event.head_commit.message, 'Code transformation completed') &&  '17' || '11' }}

jobs:
  q-code-transformation:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-java@v3
        with:
          java-version: ${{ contains(github.event.head_commit.message, 'Code transformation completed') && '17' || '11' }}
          distribution: 'adopt'

      - name: Build and copy dependencies
        run: |
          mvn ${{ env.MAVEN_CLI_OPTS }} clean install -U
          mvn ${{ env.MAVEN_CLI_OPTS }} verify
          mvn ${{ env.MAVEN_CLI_OPTS }} dependency:copy-dependencies -DoutputDirectory=dependencies -Dmdep.useRepositoryLayout=true -Dmdep.copyPom=true -Dmdep.addParentPoms=true

      - name: Upload artifacts
        uses: actions/upload-artifact@v4
        with:
          name: q-code-transformation-dependencies
          path: dependencies

This creates a workflow named Q Code Transformation that is triggered when code is pushed to a branch that matches the pattern Q-TRANSFORM-issue-*. The job itself runs on Ubuntu, and starts by checking out the code in the repository into the GitHub Action runner.

It then installs and sets up a Java installation using the AdoptOpenJDK distribution on the runner. The choice of Java version is chosen dynamically. If the commit message contains "Code transformation completed", Java version 17 is installed, else Java 11.

The script then uses maven commands to clean and rebuild the project, builds the projects and runs all tests, and finally copies all project dependencies into a specific folder. These are then uploaded as a project artifact from the runner machine into GitHub.

Triggering the Code Transformation

Triggering the Java code transformation is as simple as raising a GitHub issue, and applying the Amazon Q transform agent label to the issue.

The amazon-q-developer application then takes over, adding comments to the issue to keep you up to date. It starts by running the GitHub Actions workflow required to transform the code. You can view the progress of the runner in the Actions tab.

Once successful, your code to be transformed is uploaded, and a transformation plan created. This code transformation plan sets out the changes that the agent is initially expecting to apply.

The agent then starts upgrading the code to Java 17. Once complete, the agent updates the issue with a comment and opens a pull request.

Viewing the transformed code

The pull request opened by the agent starts off with a code transformation summary detailing the changes made during the transformation process.

One area that the transform agent has significantly improved over the past few months is in summarising these changes. Not only are these nicely laid out in a list format, but for each significant change, there are details provided why it has been made and the benefits it offers.

Security vulnerabilities and code quality issues

One benefit that the GitHub integration brings is the scanning of all pull requests for security vulnerabilities and code quality issues. The application will raise a comment in the pull request, and then highlight any findings it discovers. In my case, the application generates a high severity warning.

For each vulnerability found, a detailed description of the fix needed alongside code that can be committed is provided.

If multiple issues are discovered, you can go into the Files Changed and select to batch up all of the suggestions into a single commit.

Observations and Conclusion

There are a few areas that are worth drawing out that initially caught me out.

Supported Target Code Versions
The transform agent in the IDE currently only supports Java 17. The transform agent in the IDE has recently enabled support for Java 21 as a target code version as well as Java 17.
Code Review
The code review run on the pull request is only carried out against the changes made in the diff, and not against unchanged content in these files, or the entire repo. To do this, you will need to go back to the IDE and run the /review agent. It would be great to trigger a full code review on an entire repo through the GitHub integration
Code Suggestions for Security Vulnerabilities
Ironically, as the code review is only carried out against new changes, the vulnerability detected by the agent is against code that the agent itself created. I'm not entirely sure how I'm supposed to feel about this. In fairness, its most likely a reflection of the wider codebase and the code suggestion following the existing styling. In addition, the code suggestion made to resolve the security vulnerability which I automatically accepted failed compilation. This was detected almost straight away as it triggered another run of the GitHub Action. It turned out simpler to fix this compilation error manually and add this as a commit to the pull request. I'm not sure why this was the case, but I feel confident this will be fixed soon.

For Java transformations this now provides an alternative approach to using the agent in the IDE. Using the transform agent in the IDE feels more synchronous, as you watch the progress taking place in the Transformation Hub terminal. I really enjoyed the asynchronous nature of the GitHub integration. I can simply create a new issue, use a label to assign to the transform agent, and then wait for the email notification that it is complete. This frees up time to carry on with other value add activities.

Running the transform agent in the IDE, you are also responsible for creating a separate branch, committing the changes to this branch, and raising the PR, all of which is taken care of for you with the GitHub integration. Even better, once merged into the main branch, the full conversation history is still maintained in the closed Pull Request for transparency.

The project I used can be found here - https://github.com/mlewis7127/bicycle-licence-GH-integration. If you have a requirement to transform old Java code projects, I would definitely recommend checking out this integration, and see how it fits into your developer workflow.

Biography

Amazon Q Developer transform for .NET

Matt Lewis — Mon, 20 Jan 2025 10:19:13 +0000

Introduction

A fantastic use case for AI Coding Assistants is upgrading applications to modern and supported versions of programming languages, libraries and frameworks. All too often, engineering effort is spent building new features, whilst existing applications are left untouched, until they become unsupported with all kinds of inherent vulnerabilities.

Amazon Q Developer introduced a code transformation agent for Java when launched in preview back in November 2023. This agent has undergone multiple iterations and become more accurate and powerful since then. On 3rd December 2024 at AWS re:Invent, the public preview of new transformation capabilities for .NET, mainframe, and VMware workloads was announced.

In this blog post, I take a look at the .NET transformation capability to get a better understanding of how it works. This feature is available in the Visual Studio IDE. However, Visual Studio for Mac has been retired, which gave me an opportunity to try out the Amazon Q Developer transformation web experience.

Why port .NET Framework?

.NET Framework is the original implementation of .NET by Microsoft. It supports running websites, services, desktop apps and more, but only on Windows.

.NET (sometimes called .NET Core) is a more modern, open-source version of .NET that can run on multiple operating systems including Windows, Linux and MacOS. The reasons to continue using .NET Framework are specific and limited according to Microsoft, and relate to use cases where your application is using third-party libraries or NuGet packages or .NET Framework technologies that are not available for .NET.

Where possible you should look to utilise .NET.

Getting started with web experience

To get started with the web experience, I had to subscribe to Amazon Q Developer from your management (root) account, as it is not currently possible to use a delegated administrator account. Note that you need to be in the us-east-1 region at this point.

This takes you to a Getting started with Amazon Q page. I was prompted to switch back to my home region (eu-west-2) which then automatically connected my AWS Organization instance of IAM Identity Center to Amazon Q. At this point, I clicked the button to "subscribe" and added a user from IAM Identity Center. It is also possible to add a Group instead of an individual user or users.

This successfully created an Amazon Q Developer Pro subscription for my chosen user. At this point, I clicked on the button which took me to the Amazon Q Developer console to complete the setup.

Opening up Amazon Q Developer in the AWS Console gave the option to click on a settings button.

In the settings, I enabled the transform settings, which is required to give access to the transformation web experience.

At this point, I navigated in a browser window to https://transform.developer.q.aws.com/ and signed in using IAM Identity Centre.

Once logged in, I was presented with the option of creating my first transformation job with Amazon Q.

Running transformation for .NET

Once I asked Q to create a transformation job, I was given the choice of the type of transformation to work on. There are three options available:

Modernize .NET applications to cross-platform .NET
Migrate VMware applications to Amazon EC2
Perform mainframe modernization (z/os to AWS)

I chose the option to modernise a .NET application. Amazon Q then populated a number of details about the .NET modernisation project. I could change these details, or in this case, confirm they are correct and let Amazon Q create the job itself.

At this point I had to connect Amazon Q to my source repository for the project I want to transform. I have forked the SharpZeroLogon GitHub repository to my own profile. This is an archived repository that was a rework of the NCC Group's tool specifically for .NET Framework 3.5.

The connection is made using AWS CodeConnections. Within an AWS account, you use CodeConnections to create a connection to a third party Git-based source provider. Currently, the only supported provider is GitHub. To create a connection, you need to go to AWS CodeArtifact, click on Settings and then Connections. I am using GitHub which installs the AWS Connector for GitHub as an application in GitHub. You can configure the connector with access to only specific repositories.

To setup the Amazon Q transformation job you first specify the account number for the AWS account where the connection is configured.

You then specify the AWS CodeConnection ARN.

You then go back into the AWS console to approve this connection request.

Once the connection request has been approved, you click on Send to Q, which will allow Amazon Q to access the repositories in the connected account.

Amazon Q analyses all of the repositories it has access too, to discover which ones run a .NET Framework application that is capable of being transformed. Amazon Q Developer transformation capabilities for .NET supports porting C# code projects of the following types: console application, class library, unit tests, web API, Windows Communication Foundation (WCF) service, and business logic layers of Model View Controller (MVC) and Single Page Application (SPA). Types of jobs that Amazon Q currently cannot transform include WebUI, SQLServer and ASP.NET.

In my example, the SharpZeroLogin repository has been detected as a supported project, and I am given the option to specify a target version (.NET 8.0). I can also specify the name of the new branch that will be created or keep the default.

Note that the web experience gives you the option of carrying out a .NET transform of multiple repositories. This is something not available within the IDE, which only allows one .NET solution at a time.

Amazon Q now automatically ports the selected .NET application to the target version following a transformation plan it has created. It commits all of the transformed code to a new branch in my GitHub repository, preserving the original source code.

I can click on the Dashboard tab to monitor the progress. In this case, I am told that the application has been transformed with no issues.

I can now go to my GitHub repository and look at the new branch that has been created. I can also view the diffs to see what changes have been made. In the file below, we can see that the target framework version has been updated from "3.5" to "net8.0".

Conclusion

The goal of this blog post is to show you how to simple it is to get up and running with the new Amazon Q Developer transformation web experience. If you have existing .NET Framework applications that you want to port to .NET to gain performance improvements and cross-platform support, it is definitely worth giving this feature a go.

Amazon GuardDuty Extended Threat Detection

Matt Lewis — Mon, 02 Dec 2024 15:19:53 +0000

Introduction

I was lucky to get the opportunity to try out the new "Extended Threat Protection" feature for Amazon GuardDuty whilst in beta. With the announcement of this new feature, I wanted to share more around my experience, and the value this new feature brings. Before jumping into this, let's start by providing some background to Amazon GuardDuty and the benefits it provides, to those who may not be familiar with the service.

Why Amazon GuardDuty?

Amazon GuardDuty is a threat detection service that continuously monitors, analyses and processes AWS logs and other data sources for malicious and abnormal activity. It uses its own internal feeds, alongside other intelligence feeds from CrowdStrike and Proofpoint to detect the latest threats and attack techniques. As someone who has worked for many years in heavily regulated industries processing sensitive data sets in areas of critical national infrastructure, I have been a huge advocate of Amazon GuardDuty.

In modern cloud environments, the quantity of logs and events that is captured is enormous. When it comes to threat detection, you require real-time and accurate visibility into this data. When your workloads reside on AWS, shipping this data externally to another cloud provider or back on-premises adds significant egress costs and latency. This is why I always look to use GuardDuty, so the data can be analysed at source, and threat detection can be consumed as a managed service.

GuardDuty uses a baseline of foundational data sources, and processes these logs using independent streams of data so it does not affect existing configurations. These foundational data sources are:

AWS CloudTrail - showing a history of AWS API calls and management events.
VPC Flow Logs - showing details of IP traffic going to and from network interfaces attached to your EC2 instances.
Route 53 Resolver DNS logs - showing a history of DNS queries.

On top of this baseline, you have the option to enable protection plans, which are specialised features within GuardDuty that provide enhanced threat detection for specific AWS services, such as:

S3 Protection - helps detect risks such as data exfiltration and destruction in your S3 buckets.
EKS Protection - monitors EKS audit logs to identify potential security issues such as unauthenticated actor attempts to collect secrets or AWS credentials, and suspicious container deployments with images not commonly used in the cluster.
Runtime Monitoring - observes and analyses operating-system level, networking, and file events to help detect potential threats for EC2 instances and container workloads in EKS and ECS including Fargate.
Malware Protection for EC2 - detect the potential presence of malware by scanning the EBS volumes attached to EC2 instances.
Malware Protection for S3 - detect the potential presence or malware by scanning newly uploaded objects to selected S3 buckets.
RDS Protection - profile and monitor access activity to Aurora databases in your AWS account without impacting database performance, to detect potential threats such as high severity brute force attacks, suspicious logins, and access by known threat actors.
Lambda Protection - identifies potential security threats when an AWS Lambda gets invoked in your AWS environment by monitoring Lambda network activity logs.

Integration with AWS Services

Amazon GuardDuty is tightly integrated with other AWS services to enable fast responses.

Amazon Detective

Amazon Detective ingests GuardDuty findings and allows you to quickly analyse and investigate these events.

AWS Security Hub

AWS Security Hub is a cloud security posture management (CSPM) service. It collects findings from the security services enabled across your AWS accounts, such as intrusion detection findings from GuardDuty, vulnerability scans from Inspector, and sensitive data identification findings from Macie. It runs continuous and automated account and resource-level configuration checks against the controls in the AWS Foundational Security Best Practices standard and other supported industry best practices and standards such as NIST and PCI DSS. The screenshot below shows GuardDuty findings in Security Hub.

Amazon EventBridge

GuardDuty creates an event whenever a new finding occurs. These are routed to the default event bus in Amazon EventBridge. You can configure an EventBridge rule with a pattern that listens for GuardDuty findings in order to automatically respond to these events.

Common use cases include sending automatic alerts for high severity findings, or automating remediation (e.g. disabling a compromised access key)

Extended Threat Detection with Attack Sequences

GuardDuty Extended Threat Detection is a new feature of Amazon GuardDuty that uniquely identifies attack sequences spanning multiple AWS data sources and resources within a 24-hour time window within an AWS account.

This addresses the risk where an attack could be comprised of a number of related suspicious activities over a period of time. Each of these suspicious activities may generate their own individual finding. However, these may be of a lower severity and act as a weak signal, and so not seen as presenting a real threat. However, when these weak signals are considered together, and the sequence of these activities align to a more suspicious activity, GuardDuty will generate an attack sequence finding.

In this case, we have triggered a finding of type AttackSequence:IAM/CompromisedCredentials. We can see looking at the summary of findings that this has been given a critical severity level.

Clicking into the finding and selection "View details" brings up the overview page. This provides a compact view of the attack sequence details, including signals, MITRE tactics, and potentially impacted resources. In the screenshot below, (1) shows the signals, (2) shows the MITRE tactics, and (3) shows the indicators.

Signals displays a timeline of events that are involved in the attack sequence. Each individual signal could be an API activity or finding that GuardDuty used to detect the attack sequence. Each signal, that is a GuardDuty finding, has it's own severity level and value assigned to it. In the GuardDuty console, you can select each signal to view the associated details.

One of my favourite aspects of the new feature is the mapping of the finding to both MITRE ATT&CK(™️) tactics and techniques. This "compromised credentials" attack sequence was comprised of the following 3 MITRE ATT&CK tactics. GuardDuty uses the MITRE ATT&CK framework to add context to the entire attack sequence. The colours GuardDuty uses to specify the threat purposes used by the threat actor, align with the colours that indicate the critical, high, medium, and low findings severity level.

The indicators section shows observed data that matches the pattern of a security issue, and is the reason why this collection of signals was identified as an attack sequence. For example, the "High risk API" indicator is flagged as the cloudtrail:DeleteTrail and iam:CreateUser API calls were made, which are actions commonly used by threat actors.

I setup a rule in EventBridge to capture an attack sequence finding. A small subset of the JSON event message is shown below. This message also provides details of the associated signals.

Overall, this is a fantastic new feature in GuardDuty and I am excited to see more attack sequence detections being added over time.

Accelerating builds with Amazon Q Developer Agent

Matt Lewis — Thu, 21 Nov 2024 12:07:45 +0000

Background

I have been using the Amazon Q Developer Agent for software development for some time, looking at ways to get the agent to consistently generate more accurate output. This has highlighted the importance of the prompt, as the primary way of instructing the agent about the type and style of content to be generated. I have found the more precise guidance the prompt contains, the better the output. However, crafting these prompts can be time consuming. This leads to another challenge of ensuring that prompts are used in a consistent fashion across all engineers in a team.

In this post we look at a simple way of introducing “prompt templates” as a better way of ensuring the accuracy of content generated. The goal is to generate a complete project using a number of AWS services written in Python, with the infrastructure managed and provisioned by Terraform. This is being carried out with limited knowledge of these languages.

Bootstrapping our project

We start off by bootstrapping the project with just 2 files:

requirements.md - a markdown file containing our requirements in terms of programming languages and other guidance. This acts as our prompt template
architecture.puml - a Plant UML diagram with the design of the application we want to build

From here, we can now pass in a simple prompt to the Amazon Q Developer Agent:

“Create a complete project implementation including source code, unit tests and a requirements.txt file that meets all of the project requirements as set out in the requirements.md file. Use Terraform following standard best practice to define and deploy all services as detailed in the Plant UML diagram format below:”

I have not found an accurate way of getting Amazon Q Developer to recognise the .puml file, so for the moment, I copy and paste the Plant UML diagram content into the prompt:

The Amazon Q Developer Agent for software development plans out all the steps it needs to take to meet the ask. In the very first step, the agent states:

I need to open requirements.md to review the project requirements. Based on Plant UML diagram, I can see it's an AWS serverless architecture with an API Gateway, Lambda functions and DynamoDB. I'll first create the necessary project structure

The complete summary of changes are shown in the screenshot below. It is interesting to see the agent is clever enough to recognise that some of the imports are not working correctly, and it continually iterates after each change, until it is confident that the requirements have been met.

I accept the proposed code changes, and now I have a full project structure that has been created for me, with a README.md file that gives me more details about the project.

Running the application

Next we want to get the application up and running. I recorded a video to show this working. There were no code changes needed. I just needed to execute a python script to package up the source code for each Lambda function into a Zip file. I also used the inline chat functionality of Amazon Q Developer to output the API endpoint URL so I could run some cURL commands.

Design versus Build Comparison

The only details of what AWS services that made up the application were shown in the Plant UML diagram. What I found most impressive was how accurately the software development agent created the infrastructure to match the design.

The design shows 4 individual AWS Lambda functions called CreateItem, ReadItem, UpdateItem and DeleteItem. These are the exact names of the Lambda functions created.
The design shows the HTTP verbs (POST, GET, DELETE) and the URL Path Notation. These have been created in API Gateway exactly as they are shown in the diagram.
The design shows a DynamoDB table called Item with a partition key of id. This has also been created exactly as shown in the diagram

In addition, we can see that the code generated conforms to the specification set out in the requirements file, being written in Python with headers and comments and meaningful variable names.

Conclusion

I am really excited to see the potential that exists today with the approach taken here. I love seeing how visual design artefacts can be taken and converted directly into the underlying infrastructure on AWS. This helps to provide a seamless transition between design and build. It also solves another challenge of adopting a prompt template that can be used consistently by engineers within a team and across teams in an organisation. This approach allows for the consistent generation of code artefacts that meet organisation guidelines, by ensuring that project repositories are bootstrapped when created with a standard requirements file.

Amazon Q Developer Agent for Code Transformation

Matt Lewis — Mon, 18 Nov 2024 09:56:30 +0000

Background

Here in the UK, the National Cyber Security Centre (NCSC) is the 'technical authority' for cyber incidents with a view that:

patching remains the single most important thing you can do to secure your technology, and is why applying patches is often described as 'doing the basics' blog post

All too often, all engineering effort is spent building new features, and existing applications are left untouched, until they are running out of date versions of libraries and frameworks with all kinds of inherent vulnerabilities. This is where the Amazon Q Developer Agent for Code Transformation comes into its own. In this article, we take the agent for a spin looking to upgrade a Java 11 application to Java 17.

Bicycle Licence Application

In March 2020, I presented an online AWS tech talk demonstrating features of Amazon QLDB in an application written in Java 11 and Spring Boot. In the past couple of months, I have replaced Amazon QLDB with Amazon DynamoDB, and made sure that it is simple to start up and run as a Java 11 application. This is now a great project to test out the code transformation agent. It is a more complex application with around 750 lines of code than a basic "Hello World" application, but not so large it is difficult to understand. You can clone this project yourself here.

Setting up the Java 11 application

All screenshots are taken using the Intellij IDEA. The first step is to clone the repository and open as a new project. Make sure the project structure is setup to use version 11 of the Java SDK:

The application requires a DynamoDB table with a Global Secondary Index to be setup in the eu-west-1 region, and there is a CloudFormation template provided you can use to set this up. I couldn't remember the exact command to use, but luckily Amazon Q on the command line came to the rescue.

Run a mvn clean and then a mvn compile, which will compile all of the code successfully. You can then launch the application using a new Spring Boot configuration:

Creating a new Bicycle Licence record

Once launched, the application will be running at http://localhost:8080/. Opening this in a browser window will render the landing page for the application:

From here, you can start by creating a new fictitious bicycle licence by specifying a name, telephone number and email address:

Once created, you can go into the view/update licence tab and add some points to the licence:

Finally, you can click in the history tab and view all the events that have taken place against that record:

Now we have this up and running as a Java 11 application, we want to look to migrate to Java 17.

Running the code transformation agent

The first step in running a code transformation is to type /transform into the Amazon Q chat window.

From here, Amazon Q will analyse the open workspace to identify if there is a module available running Java 8 or Java 11 that can be transformed. It will automatically recognise the bicycle-licence-ui module, and pre-select this for you. We can simply confirm we want to transform this module to JDK17.

There is also an option for the agent to build your module with or without running unit tests. The default is to run tests, and a number of unit tests are included as part of the project to show this feature working.

The first step is that Amazon Q builds your module locally, downloading all dependencies, and then running the unit tests. Assuming this is successful, Amazon Q will then scan the project files and get ready to start the job. To do this, the project artifacts are uploaded to a managed secure build environment on AWS.

Once the files have been uploaded, the transformation job has been accepted and is ready to start. The application is built again using Java 11. Following this, the code is analysed in order to generate a transformation plan.

This is the plan used by the agent to transform your code. The whole concept of an agent is that it can run autonomously to complete a complex task, and take actions based upon its findings as it progresses, without direct human intervention. The agent can make use of RAG and custom models, which are all abstracted away from the end user. It does mean that the final code updates may end up being slightly different than the initial plan, as the agent will continue to re-evaluate its progress after each step.

The most impressive feature for me is watching the agent apply the updated dependencies and code changes, and then building the module in a Java 17 environment. You can see from the screenshot below that each time updated dependencies were added, the application failed to compile. When this took place, the agent is able to access other underlying models to work out what further changes are required, until eventually the application can now be built successfully in Java 17.

After just over 12 minutes, the transformation job was complete, and it was time to review the code diff and see the proposed changes:

Clicking "view diff" opens up a new window highlighting the files that have been modified:

You can click on each one, to see these changes. In the example below, we can see that a new method has been added to the BicycleLicenceDynamoDBRepository.java class. This is because the CrudRepository interface provided by Spring Data that this class implements has had this new method added to it in the version upgrade.

You can also view the Code Transformation Summary provided by Amazon Q. This provides details around how many lines of code were analysed, how many files have been changed, how many planned dependencies have been added and so on. It also provides a build log summary. In this case, I can see that all source files were successfully compiled and 6 tests ran without any failures, errors or skips.

We are now in a position where we can test the application after it has been transformed to Java 17. After accepting the code changes, we run an mvn clean and mvn compile and reload all project dependencies. It also involves making sure the project structure is setup to use version 17 of the Java SDK, alongside the Run Configuration.

Running the Java 17 application

After making the previous changes, we run the application and can interact with the bicycle licence with no issues. However, one thing we notice is a warning message in the console that the AWS SDK for Java 1.x has entered maintenance mode and will reach end of support on December 31, 2025.

In reality, it's a little disappointing that the AWS SDK library was not updated as part of the code transformation. The reality is that there are significant changes involved, and the AWS SDK for Java 2.x is a major rewrite of the 1.x code base. Out of curiosity, I wanted to see how the software development agent would handle this.

AWS SDK v1 to v2 upgrade with software development agent

With the AWS SDK for Java v1 already in maintenance mode and not updated by the code transformation agent, I wanted to see how the software development agent would handle the upgrade. I typed in \dev in the chat window, and entered the simple prompt of "Rewrite this application to use the AWS SDK for Java 2.x". The agent analysed the application and then worked through a whole set of changes

I really liked the fact that the developer agent created a migration plan which it then used iteratively to update all dependencies. This was a markdown file as shown below:

Although the developer agent came up with a large number of correct changes, there were still some errors left behind. This really helped to highlight the differences between the two agents currently available. The two key points for me are:

The software development agent does not allow you to test the generated code before it is accepted. This means that any issues such as compilation errors will need to be fixed manually (supported by the chat interface), or thrown back to the agent consuming another code generation from your quota
The code transformation agent uploads your artifacts to a secure build environment, and will continually try to build and compile your code, fixing any errors as it goes along, until it is complete.

Conclusion

The Amazon Q Developer Agent for code transformation is a great example of the value that agents can bring, and why I believe they are the future for coding assistants. It handles the complex task of upgrading an application from an older version to a newer version autonomously, significantly reducing developer effort.

The two main drawbacks I encountered are:

The agent did not look to transform the outdated AWS SDK for Java 1.x libraries to the latest AWS SDK for Java 2.x libraries
The target version is currently at Java 17. This is now outdated itself. Amazon Corretto 17 was released in September 2021, whilst Amazon Corretto 21 was released in September 2023 and Amazon Corretto 23 was released in September 2024, and hopefully we will see these more recent versions supported shortly.

Nevertheless, I really hope you try this agent out for yourself, and I'd love to hear your feedback.