Speaker: Lahiru Ratnayaka @ AWS Community Day Hong Kong 2025
Summary by Amazon Nova
https://www.youtube.com/watch?v=qoPjyt-MyWk
Building Next-Generation Agent Applications on AWS
Use Case: Cafe Manager Agent
Cafe Manager Agent: Built using Crew AI framework.
Tools: Connected APIs with OpenAPI specification.
Scenario: Custom base (customer) communicates with the cafe manager agent to request items like cappuccino or long black coffee.
Distributed Agent: Another agent (e.g., vendor agent) communicates with the cafe manager agent to get information or place orders.
Frameworks: Cafe manager agent uses Crew AI, while the distributed agent uses Autogen.
Identity Provider: Custom base uses Okta as the identity provider.
Tools: APIs are not hosted on AWS Lambda but follow OpenAPI specification.
AWS Support for Agentic Applications
Agent Core Service: A comprehensive service in AWS that supports building and deploying agent applications.
Modules: Agent Core Service consists of six to seven interdependent modules that can be run separately.
Managed Services: All components are managed services offered by AWS.
Agent Core Runtime
Serverless Hosting: A serverless offering where you can deploy agents written in any framework (e.g., Crew AI, AWS strands, Microsoft Autogen, Google Agentic Development Kit).
Model Communication: Supports communication with any models, including those outside the AWS ecosystem (e.g., OpenAI models).
Protocols:
[ 1 ] MCP (Model Context Protocol): Used to communicate with tools and resources.
[ 2 ] A2A (Agent to Agent) Protocol: A new protocol for open communication between agents.
Execution Time: Supports real-time execution for quick responses and long-running tasks up to 8 hours.
Payload Size: Supports payloads up to 100 MB (text, blogs, etc.), with potential for increase in the future.
How Agent Core Runtime Works
Non-Deterministic Flow and Session Isolation
Non-Deterministic Flow: Agentic applications do not follow a deterministic flow; the LLM model decides the path.
Session Isolation: Agent Core Runtime provides session isolation out-of-the-box using a microVM kernel.
[ 1 ] MicroVM Kernel: When a user starts a session with a prompt, a new session is created with a microVM kernel.
[ 2 ] Isolation: Ensures no file system, memory, or CPU is shared between users, maintaining data privacy and security.
[ 3 ] Example: If Jonas asks to be notified when his coffee is done and Jane asks for a phone number, their sessions remain isolated.
Stateful Service
State Management: Agent Core Runtime is a stateful service that memorizes all states.
Active Sessions: Supports up to 500 active sessions at a time.
Session Lifecycle
Session Initiation: When a session starts, it is initiated with a unique session ID (managed by the service).
Resource Management:
[ 1 ] Inactive Time: After 5 minutes of inactivity, the service suspends the session's CPU and resources to save costs.
[ 2 ] Timeout: Requests time out after 15 minutes of inactivity or up to 60 minutes if streaming (e.g., chat).
[ 3 ] Long-Running Tasks: Supports tasks running up to 8 hours.
Agent Core Memory
Importance of Memory in Agent Applications
- Customer Experience: Memory is crucial for providing a better customer experience. For example, remembering a customer's favorite coffee order (e.g., cappuccino) from one day to the next.
Types of Memory
Short-Term Memory: Stores recent interactions and events.
Long-Term Memory: Stores patterns and structured data for long-term use.
Features of Agent Core Memory
Built-in Encryption: Ensures data security.
Flexible Namespaces: Allows for organized storage of memory records.
Time to Live (TTL): Users can set how long memories are retained.
Observability: Built-in observability for monitoring memory usage.
Short-Term Memory
[ 1 ] Event Creation: Each user interaction with the agent creates an event.
Example: User asks a question, agent responds—each action is an event.
[ 2 ] Event Metadata: Additional information about the event (e.g., location, time).
Example: Metadata could include the location (e.g., Hong Kong) where the interaction occurred.
[ 3 ] Actor ID and Session ID: Unique identifiers for storing data in the database.
[ 4 ] Checkpoints: Snapshots of previous chat sessions for quick retrieval.
[ 5 ] Chat Messages: Stores chat messages for session-wise interactions.
[ 6 ] Session State Management: Automatically manages session state and triggers for long-term memory.
Long-Term Memory
Structured Data Storage: Converts raw interactions into structured format for storage in a vector store.
Data Storage Methods:
[ 1 ] Use of Reference: Checks and uses existing information.
[ 2 ] Facts and Knowledge: Stores factual data and knowledge.
[ 3 ] Summarization: Summarizes data for efficient storage.
Custom Stores: Users can define their own storage solutions if needed, otherwise, default stores are used.
Strategies for Data Storage
Built-in Strategies: Automatically manages data storage without user intervention.
Customizable Strategies: Allows users to customize how data is stored.
Highly Customized Strategies: Users can define exactly what data goes to long-term memory and how it is stored.
Inbuilt Semantic Search: Helps extract memory data appropriately for queries.
Agent Core Gateway
Unified Access and Tool Integration
Purpose: Provides unified access to integrate tools with agents.
Intelligent Tool Discovery: Built-in functions to discover the best tools for specific purposes.
Pre-built Integrations: Easy integration with common tools like Salesforce, Slack, and SendGrid with one or two clicks.
Authentication and Security: Includes authentication and security features.
Connection Models
Smithy Models: For configurations supporting Smithy targets.
OpenAPI Target: For APIs supporting OpenAPI schema.
AWS Lambda Target: For tools implemented as AWS Lambda functions.
MCP Server: Agent Core Gateway acts as an MCP server, with agents communicating as clients through this gateway.
Agent Core Identity
Identity Management Service
Purpose: Ensures secure connections for all processes through Agent Core Runtime and Memory.
Inbound Authorization/Authentication: Verifies that the user connecting to the agent is valid and intends to communicate with that agent.
Outbound Authorization: Ensures the agent has the correct credentials and security measures to communicate with tools.
Existing Identity Providers
Integration with External Providers: Supports integration with existing identity providers like Microsoft Enterprise, or Okta.
Portability: Allows users to port their existing identity frameworks into Agent Core Identity, providing enterprise-level capabilities without requiring a complete migration to Amazon Cognito.
Agent Core Observability
Real-Time Visibility
AWS CloudWatch Integration: When the observability module is enabled, it automatically connects to Amazon CloudWatch.
General Dashboard: Provides a dashboard to monitor logs, traces, and metrics.
OpenTelemetry Standard: Supports integration with existing observability dashboards that use the OpenTelemetry standard.
Telemetry Data
Logs: Record what happened during agent interactions.
Traces: Show how events occurred, providing a detailed path of execution.
Metrics: Provide quantitative data such as the number of tokens processed or time taken for tasks.
Service-Level Telemetry
- Consolidation: Consolidates telemetry data from all modules (runtime, memory, gateway, identity) to provide comprehensive service-level information.
Observability Portal
Agent Runtime: Displays deployed agents, sessions, logs, and tracing.
Memory: Shows short-term and long-term memory storage.
Gateways: Allows creation and management of gateways for tool integration.
Identity: Supports integration with existing identity providers.
Observability: Enables monitoring of agents, memory logs, and traces in CloudWatch.
Agent Code Starter Kit
Simplified Deployment: Use the Agent Code Starter Kit to configure and launch agent applications easily.
Command Prompt: Install the starter kit and use commands like “configure” And “launch” to set up and deploy agents.
Entry Point Definition: Define the entry point for your application during configuration.
Demonstration
Portal View: Shows a dashboard with deployed agents, memory, gateways, and identity integrations.
CloudWatch Segment: Highlights the "Agent Core Observability" segment in CloudWatch for detailed monitoring.
Conclusion
- Ease of Use: The Agent Code Starter Kit simplifies the process of configuring and deploying agent applications, making it accessible for developers to build and monitor agentic applications on AWS.
Deploying and Configuring Agent Applications
Using Agent Code Starter Kit
Crew AI Application: Demonstrates deploying a Crew AI application (not a strand) using the Agent Code Starter Kit.
Bedrock Agent Core Runtime: Define the runtime and app, specifying the entry point.
Configuration:
Recommend File: The starter tool creates a recommended file; confirm with "yes".
Execution Wall: Automatically created.
Containerization: The application is placed in a container.
Authorization: Options for authorization and security configurations (e.g., Okta).
Memory: Choose between long-term and short-term memory.
Launch: Use agent co launch to deploy the application to the Agent Core Runtime.
Flexibility and Integration
Frameworks: Full flexibility to use existing frameworks and development practices.
Database: Not bound to a full database; can integrate with existing systems.
A2A Protocol
Introduction: A2A (Agent to Agent) is an open standard protocol, now managed by Linux, adopted by major companies.
Purpose: Allows different companies to create agents and communicate securely without needing detailed knowledge of each other's enterprise APIs.
Secure by Default: Complements MCP (Model Context Protocol) and is designed to be secure.
Use Case: Demonstrated by connecting to vendors using Okta, running a normal application alongside a distributed agent with A2A, and supporting memory and observability.
Reference Architecture
[1] Components:
Runtime: Host for applications.
Memory: Short-term and long-term memory support.
Observability: Monitoring and logging.
Gateway: Connects to other tools and services.
[ 2 ] Integration:
Shows how all components work together in a reference architecture for agentic applications on AWS.
Team:
Top comments (0)