AWS re:Invent 2025 - Build agentic workflows on AWS with third-party agents and tools (AIM3311)

🦄 Making great presentations more accessible.
This project enhances multilingual accessibility and discoverability while preserving the original content. Detailed transcriptions and keyframes capture the nuances and technical insights that convey the full value of each session.

Note: A comprehensive list of re:Invent 2025 transcribed articles is available in this Spreadsheet!

Overview

📖 AWS re:Invent 2025 - Build agentic workflows on AWS with third-party agents and tools (AIM3311)

In this video, AWS presenters Andrei Okhotnikov and Dr. Salman Taherian, along with Workday's Meera Rajaram, demonstrate building agentic workflows using AWS services and third-party tools. The session covers AWS's AgentCore stack including Runtime, Gateway, Identity, and observability features. Meera shares Workday's production experience implementing a Planning Agent with AgentCore Code Interpreter, achieving 50% token reduction and production deployment in just three days with minimal resources. Salman provides a hands-on demonstration building a Company Research Agent using LangChain framework, Anthropic Claude models, and Tavily search tool integrated through AgentCore Gateway. The presentation concludes with deploying a pre-built CRM AI sales agent from AWS Marketplace in minutes, showcasing how containerized agents run securely within customer VPCs while maintaining IAM-based access controls and full observability.

; This article is entirely auto-generated while preserving the original presentation content as much as possible. Please note that there may be typos or inaccuracies.

Main Part

Introduction: Building Agentic Workflows on AWS with Third-Party Agents and Tools

Good evening everyone and thank you for joining us, whether you're here in person or joining us virtually. We're excited to have you. My name is Andrei Okhotnikov, and I lead product management for AWS Marketplace deployment experience, which includes the fast-growing category of AI agents and tools, and I have two amazing co-presenters with me today. Dr. Salman Taherian is the global AI partner lead who's working closely with many of the partners who are building the next generation of AI agents and tools, and we have a special guest today, Meera Rajaram, head of AI core services at Workday Planning, who's bringing the practitioner experience of building and operationalizing AI agents. We're excited to take you through today's session, Build Agentic Workflows on AWS with Third-Party Agents and Tools.

Before we dive in, let's do a little warm-up. Please raise your hands if you have built an AI agent. Nice, let's keep them up. I see a few, keep them up, keep them up. Keep them up if you have more than five AI agents running in production in the organization. Nice, I see a few. More than ten? Okay, last hands, but still we got a few. Cool.

The goal of today's session is to show you how AWS and the partner ecosystem give you a strong foundation for building agentic solutions. By the end of the session, we want to make sure you get a strong understanding of all the foundation and actually get ready to put your hands on the keyboard and get started building the agentic workflows. So here's the plan for the next 60 minutes. We'll start with the basics, the overview of AWS Agentic stack, which includes the core services, some of the architectural principles, and the recommendations which we have collected from our customers who are building agentic applications right now.

After that, I'll give a quick overview of AWS Marketplace and how AI agents and tools from third-party vendors can help you build agentic applications, and specifically how you can use them in the new set of agentic services on AWS in Amazon Bedrock AgentCore Runtime and AgentCore Gateway. After that, I'll pass to Meera to walk us through Workday's example of building the planning agent, so you'll hear some inside stories of what's going well, what's not going well, of building and operationalizing AI. Next, Salman will walk us through a hands-on experience of building agentic workflow using third-party tools. So you'll see how to get started from zero to ready-to-go agent, and we will wrap up this presentation with the demo of a pre-built agent which you can get from Marketplace and which you can deploy in a few minutes. So let's get started.

Understanding AI Agents: Agency, Evolution, and AWS's Core Focus Areas

Fantastic, thank you so much, Andrei. Hi everyone. So agents, when we talk about agents, it's important to acknowledge their agency. They need to be able to act. And with AI, there are two additional attributes that are important. They need to be able to take on instructions and form a goal and an objective that they would be able to achieve. And then the second one is that they should be able to reason and plan in order to be able to take advantage of those tools that are available. So agency is super important, and with AI it's able to capture a high-level goal and it's able to reason and plan in order to invoke the right tools in order to serve the outcome to the user. And today, we'll see that, you know, we'll see it, we'll hear about these experiences that we've had with AgentCore and building agents, and Meera's going to be talking about it later on.

So it's very much an evolution. We've had bots in the past where for static workflows we've been able to bring automation. Over time, that's evolved into cognitive automation, enhanced workflows with machine learning capabilities and AI capabilities, and generative AI assistants came about where we were forming these prompts and queries, sending them over to a foundation model, getting a response back. But now with agents, as I highlighted, they can really do reasoning, planning, and they can form that collaboration where they can take on an objective, more than just a task, fulfill that objective, and return a response to us.

Over time, we're going to see fully autonomous agentic systems. There's a maturity curve shifting from low agency to higher agency by the agents, and the role of a human from actually doing the work in the past is shifting over to more of an oversight role.

Now in AWS, there are three areas that we're deeply focused on. First one is choice, bringing the latest market-leading capabilities to our customers and partners so they can freely innovate, innovate fast, and realize ROI and value quickly. It's important to move along with the latest capabilities that come within the market landscape. Second one is trust. AI has been probabilistic. It will likely remain probabilistic, so it's important to be able to control, govern, establish guardrails, and compliance for using these agents in production. The third one is around scale, because while the key components of these LLMs and some of the tools are fundamental, the end-to-end solution is comprised of multiple components. Having enterprise-grade capabilities that would allow you to quickly innovate, move those into production, and then scale that production as your demand and adoption grows is super important. That's what we bring from AWS.

AWS's Approach to Agentic AI: Choice, Trust, and Security

I'm going to briefly touch upon each one of these. Firstly, around choice. With choice, the ability to buy or build or partner, depending on the use case and depending on your strategy, is extremely important. At the top, you see some of our AWS native capabilities with Quick Suite and third-party capabilities that Andrei is going to be talking about from Marketplace. In the second layer, you have Amazon Bedrock AgentCore as well as frameworks like LangGraph Agents, where you can build any custom agentic solution of your choice. This is where, if your use case in particular does not fit one of those out-of-the-box models, the ability to be able to build your custom agent is important.

Lastly, bringing partners who have the expertise and experience within AWS, with the AWS Generative AI Innovation Center, but also our partners' professional services, is important. That's an area where we can partner in order to build those solutions and take advantage of the lessons learned and the best practices that have been accumulated over many runs and many build scenarios. In this context, we are providing the capability that would be able to work with any models and any frameworks, and that's the flexibility and choice that speaks to our value.

Now in terms of trust, security, governance, and audit, I want to highlight three points here. The first one is around transparency. It's extremely important to acknowledge what are the models that are available, how services are developed, and what are the intended use cases. In fact, we started this with our AI services where we had AI service cards, and we've now extended those into our agentic AI capabilities where we talk about how they've been trained, what type of data has been used, what are the intended applications, and how they fare against bias, transparency, and explainability. It's extremely important when you consider responsible AI, as well as providing IP indemnification for Amazon Nova models.

In the second part, it's around having control. This is a shared responsibility model. We're working closely with model providers around how to make the models themselves safe, but also we need to have the right guardrails around the models. This is where Bedrock Guardrails comes into play in order to be able to help you govern those for specific applications, make sure that you control the ingress as well as the egress of those LLM models. Lastly, when it comes to audit, data lineage, and observing what's happening across the agent runs, this is where I'll talk about AgentCore observability, but also our services such as CloudTrail and CloudWatch have been there in order to support that.

Amazon Bedrock AgentCore: Tools, Memory, Runtime, and Observability

Now, I talked about AgentCore. I want to double-click on it slightly more. There are three key areas with AgentCore. We talked about agents having agency. That's the first part. It's around having tools and memory to be able to realize what is the session information, what is the short-term as well as the long-term memory.

When an agent is run, it's able to learn from its experiences, take that information into account as memory, and enhance its output. It also has access to tools, and here I've highlighted two of our native capabilities. The code interpreter allows you to generate code and securely test and run that code within a sandbox environment, as well as a browser tool that would allow your agent to browse and utilize the browser much like a human would do.

In the second part, it's around deploying these securely at scale. Runtime and identity, which I'll cover in the next slide, are the key services in this area. Lastly, but not least, it's important to have a view on the whole end-to-end execution of these agents. That's where AgentCore observability comes into it.

Briefly, when it comes to deploying these securely, we have AgentCore Runtime, which would allow you to containerize your code, deploy it into a cloud environment within a customer VPC securely, and operate that. So that's extremely important to allow customers to leverage the IP of a partner, for instance, and run it within their own environment and have guarantees around what data can go outside and what data can stay within their own customer VPC environment. AgentCore Identity in particular addresses one of the key challenges, which is access control. Access control for the user who's using the agent, because based on your credentials you would have access to certain resources and information, and every one of us would have a different level of access. But also access control for the agent itself as it tries to access resources across systems.

When it comes to tools and memory, we have AgentCore Gateway that would provide access to a list of tools and capabilities that are available for the agent to be able to take advantage of. AgentCore Memory would allow you to, as we talked about, improve your run based on taking account of the shared context between different agents across sessions, be it short term or long term. And the last two I talked about, these browser and code interpreter capabilities, and later on we'll see a demo where we're actually using a third-party tool. So these two are the native tools provided by AWS.

Lastly, when it comes to AgentCore observability, we have integrations with standard models such as OpenTelemetry, and we're bringing that data into CloudWatch and CloudTrail so that users can inspect agent runs, perhaps diagnose any kind of root cause of issues, understand why certain patterns and why certain traces were executed, and improve that over time. But that's it, I'm going to hand over to Andrei to talk about Marketplace.

AWS Marketplace: Streamlining Access to AI Agents and Tools

Thank you, Salman. So when we talk about agents, they by default consist of multiple components including the LLMs, in many cases the tools, the knowledge bases, and so on. One of the key challenges that I hear from customers building or experimenting with Agentic AI is not how to build the agent, but it is how to get access to all the tools that they need. Think of the new vendor onboarding, security reviews, legal reviews, payment setup. All that can take from weeks to months, slowing down innovation and adoption, and AWS Marketplace removes that friction.

Marketplace is an enterprise-ready digital catalog of software, services, and data, and now also AI agents and tools for the developers who are building Agentic AI. The Marketplace brings three key benefits. The first one, the procurement time drops from months to days or minutes if Marketplace is already configured in your organization. Your developers can go to Marketplace, find the tool they need, and subscribe in a single click with full transparency about the pricing and contractual terms, so that's a big advantage for the getting started time.

The second benefit is the unified billing and the cost control transparency. All the spend through AWS Marketplace goes through your AWS bill, so your developers don't need to put their own credit card and do the expense reports for any tools that they're experimenting with. But in addition to that, Marketplace integrates with AWS cost tools such as Budgets and the Cost Explorer, so you could set up certain spend goals and have full transparency on who's spending and what they're spending on building agentic AI.

And finally, that's the actual developer experience. It's a simplicity of deployment. Marketplace products natively integrate with AWS services, so in a single click you can deploy an agent on AgentCore Runtime. I'll show you that in a later part of my presentation. So when a developer wants to start with building an AI agent or exploring the tool, evaluating the tool, they can get started in the marketplace with just a few clicks.

Making it a one-stop shop for AI agents and tools, we launched this category just a few months ago in July, and it's been growing very fast. Right now we have over 2,300 agents, tools, and agentic applications, including professional services from our partners. They fall into different categories, so let me break this down for you. On the right-hand side of the slide are the API-based agents and tools. Those are the remote tools that are ISV hosted. So for example, Tavily search tool, you'll see it later in Salman's section of the presentation.

From the developer standpoint, they act as external targets. So developers go to marketplace, they subscribe in a single click, and they can also in a single click import those tools into AgentCore Gateway as external targets, and then they can use or reuse those tools in multiple agents for different kinds of applications they're building. And what's great is the automation because AgentCore Gateway automatically pulls down all the metadata, the API schema, actions. Also we automate the API key rotation and give the full transparency into what's happening with those tools. So no manual wiring, no schema translation. Gateway handles all of that.

The second category on the left portion of the slide are container-based agents and tools. Those are containerized agents that are designed specifically to run on AgentCore Runtime. The key benefit is those run inside your VPC on AgentCore Runtime. So you're getting all the benefits of AgentCore service including the VPC isolation, IAM-based identity and resource policies, tool-level permissions, and of course observability. So you're getting all the benefits of the partner logic, the partner-defined agents who have pre-built, who've done all the prompt engineering, but it runs within your VPC.

And the third category on the bottom are agentic applications and also the professional services. We hear from many of our customers who are saying we need some help to get started. You can go on AWS Marketplace and find support from a broad network of the partners. So before we go deeper into building the agents and showing how that works, I want to ground all the theory that Salman and I have covered in a real customer implementation.

Workday's Planning Agent: From Multiple Iterations to the Right Solution

Workday has been one of the early adopters of AgentCore and other AWS agentic services, and please welcome on stage Meera, who will share their experience of building a planning agent. Thank you, Andrei. So I'm Meera Rajaram, and I head the AI core services for Workday Planning. I think we are perfectly positioned to help our customers manage their money, people, and agents. Now, this is a marketing slide. These are the list of agents I'm allowed to show you because these agents are actually going to be coming out by end of this year in the hands of customers. There's a whole other spreadsheet of agents that we're working on, and that are going to be following shortly.

So how are we getting all these agents out? Workday has an agent platform team that's essentially making some of these common stack consistency code to how we're building agents. So we leverage LangGraph, we're using LangSmith for our evals and traces, and we have something like a centralized gateway to the LLM models.

So this is what we end up leveraging across the board. And the Planning Agent is no different. We are also leveraging that platform.

But let's dive into the Planning Agent. The Planning Agent is one agent that represents four personas to give this complete planning intelligence. So the personas here are analyst, modeler, planner, and the admin persona. Based on the role, we are essentially offering the skills for those personas. So if we dive a little into the analyst role, one of the key things an analyst does is explore data. So let's dive into that.

The data exploration skill that we offer essentially allows our end users, these analysts, to leverage natural language and conduct ad hoc analysis on vast amounts of data. And trust me, when I say vast, it is vast. So how did this go about for us? How are we successful at it? Well, I'll tell you, it took us multiple tries.

Iteration one: let's get all this data, let's send it to the LLM, and have the LLM do the data analysis for us. We ran into a bunch of issues with that. First, we ran into token limits because it was a lot of data. Second, if we didn't run into token limits, it was still an expensive solution because we were sending a large number of tokens. And third, LLMs aren't really very accurate with math, so this was not a good solution for us.

Let's see what else we can do. The team came up with another solution, so this was iteration two. Token limits, we know how to solve it. We will chunk the data and send it to the LLM. That's exactly what we did. So it was a multi-step process. We would chunk the data, send it to the LLM, get the output, and send that output and the next chunk of data to the LLM. And of course, this was not a perfect solution again.

One, it introduced latency. Two, we did solve for the token limits, but the solution was still expensive. And three, we still hadn't solved for accuracy. Why? Because at the heart of it, LLMs aren't great at math. But LLMs are really good at writing code. And so we figured that we would rather get the LLM to write code, Python code that can do the data analysis that we need, and then the LLM can leverage that output and be able to explain it in natural language to our end users. So that's really what we went with.

Implementing the Sandboxed Code Interpreter: Workday's Production Success Story

Now, why do we need a Sandboxed Code Interpreter, and what exactly is this? Well, show of hands for how many people have raised kids. We've got a few folks. Well, if you've got a high-energy toddler at home, one way to deal with that toddler is to maybe create a playpen, put them with all their toys. They can create whatever mess they want to in that playpen, and the beauty is your fancy carpet and walls are not disturbed. So that's one way of doing it, and that's exactly what the Sandboxed Code Interpreter is.

Think of your code like your toddler. It can go wild, it can create a mess, and you want to put it in a sandbox. So that's exactly what a Sandboxed Code Interpreter allows you to do. It is to ensure that your code can run in a safe, secure environment. Now, we evaluated a lot of different options. One is, should Workday build our own Sandboxed Code Interpreter? That's not the core of what Workday does, so that's not going to be a good solution for us.

We then started looking at different frameworks and tools. Now, if you've been in this industry long enough, you've seen a lot of demos, and these demos are amazing because they tell you how to build agents in an hour, you can create POCs in an hour.

But taking that POC from POC stage to production is a whole other matter because you have to incorporate all the enterprise controls. You have to think about security. You have to think about customer data, which is so precious. And so we said, hey, you know what, we need a good solution that's going to help us address it. And Amazon Bedrock AgentCore essentially provided that solution. That was the answer to what we were looking for because, one, it manages security for you. So as far as your code is concerned, that code is running in a very secured virtual computer. The data that you have, your customer data, is uploaded into that environment. And as soon as you're done, all of that data is wiped clean, and you're now starting with a clean slate. So that was super critical.

Second thing that the AgentCore Code Interpreter does, which is kind of a differentiator compared to a lot of the other tools, is that it handles stateful multi-step analysis. So for that entire session, it's actually able to do a multi-step analysis process to ensure that you get the most accurate results. And then scalability. We're actually able to send a ton of data to the Code Interpreter. At least with what we saw last, we were able to upload anywhere between 100 MB file for inline processing. And of course, we are compressing that file and sending it over, but you could also leverage S3 for it, where you can have up to gigabytes of data.

So that's where we landed. Now, what did it take us to get this solution with AgentCore Code Interpreter in production? It took us one software developer, two engineering managers, and three days to get this from POC to something that can be production ready. And that is huge. How many folks can say that you can get something from POC to production in such a short span of time? That's exactly what we got with the AgentCore Code Interpreter. And by the way, we just went live with our solution two weeks back, so customers are actually using this in production, which is amazing.

So let's talk about some of the results with the Code Interpreter solution. What we ended up seeing is, I mean, yeah, we are adding some latency with Code Interpreter, somewhere in the ballpark of 400 to 500 milliseconds. If you look at the 90th percentile, somewhere between one to two seconds. But what we gained is the token size reduced significantly. So we're almost seeing, when we have these large payloads, we are seeing up to a 50% reduction in the tokens that we end up sending. And that's significantly reducing the cost for us. Two, the other benefit that we're getting is the accuracy. We ended up getting much better results with this solution than what we had done previously. So this was huge. And three, scalability. We now didn't have to worry about are we going to be hitting token limits or any of that. The solution is scalable. And at the end of the day, what is it our customers are looking to do? They want to analyze large reports, vast amounts of data, and this is exactly something that's solving for it.

With that, I do want to thank the close partnership we have had with the Planning Agent team, our own Workday Agent Platform team, and AWS folks. This was a very strategic partnership that helped us to innovate at a really rapid pace. With that, I will hand it off to Salman.

Building a Company Research Agent: Architecture and Cloud Setup

Thank you, Meera. OK, we're going to be building an agent now. Disclaimer number one, we're going to be seeing some code. Disclaimer number two, you can definitely use AI to help you with any form of coding. I actually did use AI to generate this code.

Third, I've got to do this in ten minutes, so I'm going to focus on the bare bones of the capability that I talked about. How do you bring that tool and agency within the agent? Where do you bring the LLM that does the reasoning? How do you bring in the query? And then I'll give some pointers around how you can start off but also strengthen that in order to move such a solution into production.

Now in this example, I'm going to be showing how to build an agent using a third-party tool from Tavily. Tavily is an internet search tool that provides this capability. I'll talk to you about how you can take that tool, integrate it within the code, and run it. First things first, what is the use case? That's important, right? And in this particular case, I am going to be demonstrating a Company Research Agent for Sales Teams. This happens to be a popular use case that I'm seeing in production. The reason being that many of the sales teams focus on searching information from the internet and creating these tailored briefing documents ahead of customer meetings. So this is one use case that I've seen emerge in production, and I thought I would create a simple bare-bones capability to show how these are built.

Next, the steps that we need to follow. Step number one, we need to architect the solution, so we're going to go over that. Step number two, we need to set up the tools in the cloud environment. I'll briefly touch upon that as well. And then step number three, we have to have the code and then run it, right? So step number one, solution components. This is an overly simplified version of solution components involved in an agent. You have the user that will be interacting with an Agent Executor, and that executor takes on the role of an orchestrator, right? It determines where's the LLM that I need to take advantage of in order to do that reasoning and the planning. What are the tools that have access to me and what does that mean?

So in this case, in this example, we have two tools. One to represent information that is coming from the web and the internet, and the second one is information that is residing within the enterprise environment. In this particular case, I just have a file sitting on an Amazon S3 storage just to show how the two information sources can come together, be combined, curated, and present a response to the user. Now, with each one of them, we need to have some decisions around the technical capabilities. So with the Agent Executor, there's a number of frameworks that are available. Meera talked about LangGraph. LangChain is one that is very close to that and comes from the LangChain provider. So that's a framework we're going to be using in order to orchestrate these capabilities.

From a Reasoning LLM perspective, we have access to more than two hundred models from Amazon Bedrock. In this particular case, we're going to see an example from Anthropic Claude. And then when it comes to the tooling, the AgentCore Gateway is going to be presenting us the option of connecting to Tavily's tool in order to search the internet, and the Local Data Search is simply a tool function within the Python environment. I may have skipped that part for the interest of time. So we're going to go over the example, and I'm going to cover how to now set up the tools and the cloud environment, and then we'll go into the code piece, right? And I'll walk you through what each section of the code is doing so you can follow.

So the first thing around the choice of the model. You can go to Amazon Bedrock, and as I said, you've got access to more than two hundred models. In this particular case, the use case seems simple enough, so a model like Claude 3.5 Haiku should be able to provide a decent result. Now within Bedrock, you have the Bedrock evaluation capability where you can actually test models side by side. And those are some of the best practices when you're moving and executing on a proof of concept to be able to identify what is the optimal model for your particular use case. But here, for the sake of simplicity, I'm going ahead with the Claude 3.5 Haiku model.

You can select the model and then you can generate keys, either short-term or long-term. So in this instance, we're generating short-term keys, and then we're storing these within the Secrets Manager as well as taking it and incorporating it into the code development that I'll show you later on. The second part is around the tooling. So as I talked about, we're using Tavily's tool to search the internet. You'll be able to access and get keys from Tavily.com.

And then again, same practice with those API keys, you can store them securely within your Amazon environment in the Secrets Manager. Within AgentCore Gateway, you can register this. As it so happens within the AgentCore Gateway, we have a number of native integrations. So one of the options that is the furthest to the right talks about some of the built-in integrations. You'll be selecting that, and Tavily is one of the options tools available there along with some other third-party tools. You'll be able to select that, select your key, and you'll be given an AgentCore Gateway URL and ARN that you will be using as part of your tool. So that's your cloud environment now set up.

Now, let's move on to the code. So in the simplest form, having that LangChain that I talked about and setting up the Agent Executor, it's not that difficult. If you could just follow the Python code here, you'd be basically pulling in some of the libraries, in this case LangChain. And then there are a couple of lines of code where this Agent Executor comes and orchestrates all the capabilities from the LLM, the tools, and the prompts. You create that tool calling capability, and then you form an Agent Executor. Once you have this object, you can actually call invoke against that, so then you can give a user's query in order to invoke this agent, and it would operationalize and generate a response.

Coding the Agent: LLM Integration, Tools, and Execution Results

Three parts are key: the LLM, the tools, and the prompt. These are the three parameters that are provided. So I'm going to be talking about those three parameters in the subsequent slide. So this code highlights all three. The first one is the LLM. In this case, again within LangChain, you have native integration with Amazon Bedrock. You can simply use Chat Bedrock in order to select the model and provide information around the region and its credentials that you have. So here you see the example of Anthropic Claude 3.5 Sonnet being mentioned there.

On the second one, you need to provide a prompt. So we talked about agents following an objective. Here I have an objective, and it reads such as you're a company research agent. Your goal is to answer questions about companies using available data sources. You have access to two tools. Number one, search local documents, search internal company documents for information. Number two, search the web, search the web for current company information. Fun fact, with AgentCore, you can even have a separate tool to say you can access, you know, inquire on the list of tools, so you can get what are the tools available, pass it on to this Reasoning LLM so that it can actually in the subsequent stages determine out of those tools which ones to use. But in this case, again, I've kept it simple, so I've provided those tools that are relevant to this agent directly within the prompt.

And the third one, you see that we have tools as an array that talks about these two: search local documents and search web. I'm going to skip the search local documents, but again, I want to show you the code for search web so you can see how easy it is to bring all of this capability into a Python code and execute it. First part, again, bringing some of those configurations that we have set up in a cloud environment, in the AWS environment, from your AWS region, your S3 buckets for the local search that I talked about, and then the AgentCore URL and tool. So those are the information that we already had from the AWS console. You're going to be able to take advantage of it.

And then the search web that was mentioned in the array is basically a function, and this function is decorated as a tool. So then the Python interpreter environment will be able to understand this is one of the tools available for that agent. And within that, we capture the name of the tool from the AgentCore service. We're going to be able to package this as a JSON RPC, sign the query, and then send it over to AWS for invocation. So in this case, what you're seeing, the payload is packaged up with some of the information around the JSON RPC information, also having arguments. And the argument says query is query. The query parameter is what the user provides, and I'll show you the example that we have in the recorded run.

And then post that, you're going to create this, encapsulate this as a request, an AWS request, and basically send this request as a POST operation.

So that's it. Whenever the agent decides to call the search web tool, it would go through this process. It would have the response, and the response is being sent to the LLM for further decision making as to whether it needs to further call tools or whether it needs to just curate the response and provide an output to the user. That's a decision that is left with the LLM to decide in this case.

Now I want to show you how this comes together when you run it. In this particular case, we need to have an example of the query. So the query here is, "How has AWS's approach to sustainability and green computing evolved, particularly focusing on their commitment to run on 100% renewable energy?" So that's a query we're going to be providing to this agent, and we're going to have the agent run. I'm going to show you the screenshots of the run in this case.

Firstly, once you run this, it would actually try to figure out, "Hey, do I have the information, or do I need to go and use the tools that are available?" The green is the JSON output of what the LLM has determined. It's determined that it needs to actually invoke the search web tool that it has available, and it's also formed a query that is different to the query that we have in order to determine what it's going to be searching the web for. So in this case, it says it wants to search the web with the information "AWS Amazon Web Services, sustainability, renewable energy, 100% commitment progress." So this is the determination of the LLM saying, "I've seen the query, and I'm going to be using this tool, and the tool is going to be used with this argument." It's formed its own argument.

This tool is going to be invoked through, as I said, Amazon Bedrock AgentCore Gateway and Tavily's tool, and it returns with information. Now you see a dense piece of text because it brings back search results from the Internet relating to that argument, relating to that query. In this particular case that you see, there is information around the Amazon Web Services sustainability, renewable energy commitments, URLs so that you can have citations and linkage to the original material, as well as text.

Now I'm not going to read over this text, but what I'll highlight is ultimately the LLM processes information combined with the S3 information that it has, and it brings it and provides this raw format in JSON, which can be processed further in order to have a much more human-readable text. But in this case, the output that is provided is based on the web search results. "I can provide a comprehensive overview of AWS's approach to sustainability and green computing. Renewable energy commitment: AWS has achieved its 100% renewable energy goal seven years ahead of scheduled target in 2021." I'm not going to read the whole text, but as you can see, three steps. Step one, we provided the query. Step number two, it determined which tools to call and which parameters to provide for each one of those tools. Step number three, it curated the responses and provided a response as an outcome.

So next steps from here. This is the core building block of building an agent, but this in itself should not be moved into production. Number one is around security and governance. With Amazon Bedrock AgentCore Runtime, and I didn't cover this within today's session, you'll be able to containerize this code and push it into AgentCore Runtime so it's operationalized. So that's step number one. Step number two is, as I've talked about these LLMs, it's important to place guardrails around the application. So having those guardrails implemented with Bedrock Guardrails is quite important.

Step number three is in terms of performance and reliability. In many of these applications, we see caching to be effective because there's a lot of repeat queries with the same kind of company names or information. So it very much depends on the application context and having reliability in terms of failing over properly in case a tool cannot be called or the answer does not exist. How do you recover from that? Lastly, cost control. You need to have mechanisms in place in order to realize how much the LLMs are calling, what kind of token-based pricing is being generated here per request, what about the tool calls that you have. All of those aspects would help you to strengthen this in order to move it into a production-grade environment.

Deploying Pre-Built Agents: The Fast Path to Production

With that said, I'm going to hand over to Andrei to talk about deploying a pre-built agent. There we go. Thanks. So you just saw how to build an agent from scratch, pick your LLM, pick your tools, do the prompt engineering, test that it's working, which is great when you want to have flexibility and when you have the granular control over the tool. But many teams don't want to start from scratch, and

sometimes the fastest path is to start with a production-ready pre-built agent that you can get from a third-party vendor. I'll be using a CRM AI sales agent that is built by one of the vendors available on AWS Marketplace that can help you automate the routine CRM tasks. This is a fully packaged solution with defined action schemas, with defined prompt engineering, which has the recommended LLM already baked in, and it is already packaged and optimized to run on AgentCore Runtime. That means you can deploy it in minutes, and I'm going to show you in just a few slides a demo of how that's going to be done. This is the power of pre-built solutions. You inherit all the vendor's work and their testing, making sure that all things and components work together, but it can still run securely in your environment within your VPC.

The getting started experience is extremely simple. It is just three steps. You first discover the agent or the solution based on your business needs. We have the new agent mode on AWS Marketplace where you can describe your business problem or attach your internal requirements document or your plan for what you're trying to build, and we'll give you the best optimal solution for that. The second is the subscription flow, which is a single click. This particular tool is free, but it still requires you to accept the vendor's terms and conditions. Finally, there is the deployment process, which is almost as simple as a single click. I'll talk about it a little bit more.

AWS Marketplace has standardized the framework for how to build and package those containers, and they come with vendor-authored instructions and all the configuration, requiring users to provide only minimal inputs. In this case, users deploying the agents need to provide the environment variables, such as the CRM access credentials. What's important about AWS Marketplace is that it acts as the supply chain guardrail. Think of it this way: all the agents that are containerized applications are stored in the Marketplace private ECR repository. That means that vendors cannot just swap the image whenever they want. All the images that they provide go through a rigorous scanning process, and they are behind entitlement-enforced access, so only users who are entitled to use this product can read them. The vendors, before pushing the new version, have to go through the full publishing flow and full security reviews.

Once it goes to the AgentCore Runtime side, that's where you get all the benefits of running these agents within your VPC. You get IAM role configurations where you start by either creating a new IAM role or selecting an existing IAM role if you have one already, where you define the minimum required set of permissions for this agent to operate properly. This usually requires access to Bedrock LLMs, access to the tools you're using, or other internal resources that are required for the agents. The secure runtime environment with AgentCore gives you full control. The agents would never leave the boundaries of your VPC. It's isolated from inbound requests from the internet unless you allow that, and you remain in full control over what the agent does and how it operates. AWS Marketplace in this case ensures the trusted delivery of the agent, while AgentCore Runtime enforces secure governance of the agents at the execution step.

Just to summarize, here is the flow if you want to get a third-party pre-built agent. Again, it's very simple. You subscribe and find it on AWS Marketplace, you provide the environment variables, in this case CRM credentials, you deploy it on AgentCore Runtime, and you invoke the agent. So let's see that in action.

Live Demo and Conclusion: From Marketplace to AgentCore Runtime in Minutes

I've got a short pre-recorded demo just to put all this together. We're starting on the AWS Marketplace website, and we go to the new search experience where you can describe your problem in natural language. In this case, I'm looking for an AI agent to automate my routine CRM actions.

As you can see, I'm typing very slowly. It will sync for a few seconds. You can also upload a document with your requirements if you need more precise recommendations here. You will see a few options with the recommended journey summary, and we just launched a new version of that experience where you can also create an AI build comparison of different tools so you can compare them across different dimensions.

Once you identify the tool that works for you, you go to the product detail page where you can read all the information about how the tool works and also see the deployment instructions, including the tool, or in this case agent schema and all the inputs that are required from you to make sure that this agent operates properly. Once you're ready, you move to the purchase step where you review pricing and terms and conditions. This tool is free, but you still need to hit the subscribe button to make sure you agree with the vendor's terms and conditions.

Once you're subscribed, you can proceed to launching the software. At this step there are two options. You can either proceed with CLI if this is your preferred method, and we will provide you with a few pre-built, pre-generated prompts for the CLI. The first step would be creating an IAM role. Let's see. Here it is. So the first step will be creating an IAM role so you can copy that and execute this command, attaching permission policies and finally actually deploying the agents with the container image provided by the partner.

Alternatively, you can get the same work done directly in the AgentCore console, which we'll be using for this demo. In the AgentCore console, again, your inputs are fairly simple. Provide a name for the agent. It's pre-populated already, but I prefer a different one. Let's call it a Salesforce agent. The ECR container URI is already pre-populated from the marketplace, the solution that you have just purchased. You select or create a new IAM role. I have one already selected, and I'm almost ready to hit the hosting engine, but not yet.

I need to provide the environment variables. In this case, I need to provide four keys with respective values, and that's my CRM username. This is my CRM password and the token and the domain. Again, this information never leaves the boundaries of your VPC. This information is never shared with the vendor, and you can also store those credentials in the Secrets Manager. Instead of providing environment variables here, you can reference them from the agent directly, which is much better for production and when you need to do regular rotation.

So now let's test this agent and see if that works. I'm asking it to provide me a summary for a specific account. Let's see. It will take a few more seconds to sync, and here we go. So here's an example of an agent that can help automate the routine work of your sales teams, and there are multiple ways how we can plug in this agent into your own individual workflows with all the data never leaving the boundaries of your VPC, with you having full control, but at the same time with no building, no coding, and no prompt engineering.

The one thing I also want to mention is this particular agent was built by partners using the LangGraph framework, which already has Bedrock LLMs baked in. So in the previous example, you saw how we need to pick an LLM, how we need to configure the LLM access. In this case, you don't need to do any of that. It kind of comes prebuilt. And with that, that's the end of our presentation. We'll be wrapping up in a second. Please join me on stage. We are running a little short on time, but as soon as we wrap up, we'll be around here for Q&A. Thank you, Andre.

Despite his slow typing, I'm still impressed, you know, getting an agent into production in five minutes is pretty impressive. So what we saw today was AWS capabilities, you know, providing pre-built agents, tools, models, infrastructure, and expertise. Generally, the best way to go about this is to identify a use case and start this flywheel cycle, right? What is the use case, what is the business pain point or opportunity, identify whether we should be building or buying or partnering, and then create this flywheel effect that you can improve over time.

There are some resources that are available, and this recording is going to be available as well. So we've got information around some of the sample code with AgentCore, but also some training material to improve your skills with our agentic AI offerings. And lastly, I want to close off with some of the relevant sessions that are also covering similar topics at re:Invent. Thank you so much for being with us. Thank you.

---

; This article is entirely auto-generated using Amazon Bedrock.