🦄 Making great presentations more accessible.
This project aims to enhances multilingual accessibility and discoverability while maintaining the integrity of original content. Detailed transcriptions and keyframes preserve the nuances and technical insights that make each session compelling.
Overview
📖 AWS re:Invent 2025 - Build agentic workflows on AWS with third-party agents and tools (AIM3311)
In this video, Andrea Kotnikov, Salman Taherian, and Meera Rajaram from Workday demonstrate building agentic workflows on AWS using third-party agents and tools. The session covers AWS's agentic stack including Amazon Bedrock AgentCore, AgentCore Runtime, AgentCore Gateway, and AWS Marketplace's 2,300+ AI agents and tools. Meera shares Workday's production experience building a planning agent with AgentCore's code interpreter, achieving 50% token reduction and production deployment in just three days with three team members. Salman provides a hands-on demonstration building a company research agent using LangChain, Claude 3.5 Haiku, and Tavily search tool, explaining the integration of LLMs, tools, and prompts. The presentation concludes with a live demo deploying a pre-built CRM AI sales agent from AWS Marketplace to AgentCore Runtime, showcasing the simplified procurement, unified billing, and VPC-isolated deployment capabilities that reduce setup time from months to minutes.
; This article is entirely auto-generated while preserving the original presentation content as much as possible. Please note that there may be typos or inaccuracies.
Main Part
Introduction: Building Agentic Workflows on AWS with Third-Party Agents and Tools
Good evening everyone, and thank you for joining us, whether you're here in person or joining us virtually. We're excited to have you. My name is Andrea Kotnikov, and I lead product management for AWS Marketplace deployment experience, which includes the fast-growing category of AI agents and tools. I have two amazing co-presenters with me today. Salman Taherian is the global AI partner lead who's working closely with many of the partners who are building the next generation of AI agents and tools. We have a special guest today, Meera Rajaram, head of AI core services at Workday planning, who's bringing the practitioner experience of building and operationalizing AI agents.
We're excited to take you through today's session: Build agentic workflows on AWS with third-party agents and tools. Before we dive in, let's do a little warm-up. Please raise your hands if you have built an AI agent. Nice. Let's keep them up. I see a few. Keep them up. Keep them up if you have more than five AI agents running in production in your organization. Nice, I see a few. More than ten? Okay, last hands, but still we got a few. Cool. The goal of today's session is to show you how AWS and the partner ecosystem give you a strong foundation for building agentic solutions. By the end of the session, we want to make sure you get a strong understanding of all the foundations and actually get ready to put your hands on the keyboard and get started building agentic workflows.
Here's the plan for the next sixty minutes. We'll start with the basics: the overview of the AWS agentic stack, which includes the core services, some of the architectural principles, and the recommendations we have collected from our customers who are building agentic applications right now. After that, I'll give a quick overview of AWS Marketplace and how AI agents and tools from third-party vendors can help you build agentic applications, and specifically how you can use them in the new set of agentic services on AWS in Agent Core Runtime and Agent Core Gateway.
After that, I'll pass to Meera to walk us through Workday's example of building the planning agent, so you'll hear some inside stories of what's going well and what's not going well with building and operationalizing AI. Next, Salman will walk us through a hands-on experience of building agentic workflows using third-party tools. You'll see how to get started from zero to a ready-to-go agent. We will wrap up this presentation with a demo of a pre-built agent which you can get from Marketplace and which you can deploy in a few minutes.
Understanding AI Agents: Agency, Reasoning, and AWS's Core Principles
So let's get started. Fantastic. Thank you so much, Andrea. Hi everyone. So when we talk about agents, it's important to acknowledge their agency. They need to be able to act. With AI, there are two additional attributes that are important. They need to be able to take on instructions and form a goal and an objective that they would be able to achieve. The second one is that they should be able to reason and plan in order to take advantage of those tools that are available. Agency is super important, and with AI, it's able to capture a high-level goal and it's able to reason and plan in order to invoke the right tools to serve the outcome to the user.
Today, we'll see that through our experiences with Agent Core and building agents, and Meera's going to be talking about it later on. It's very much an evolution. We've had bots in the past where, for static workflows, we've been able to bring automation. Over time, that's evolved into cognitive automation with enhanced workflows with ML capabilities and AI capabilities. Generative AI assistance came about where we were forming these prompts and queries, sending them over to a foundation model, and getting a response back. But now with agents, as I highlighted, they can really do reasoning and planning, and they can form that collaboration where they can take on an objective, not just a task, fulfill that objective, and return a response to us.
Over time, we're going to see fully autonomous agentic systems. There's a maturity curve shifting from low agency to higher agency by the agents, and the role of a human is shifting from actually doing the work in the past to more of an oversight role. In AWS, there are three areas that we're deeply focused on. First is choice, bringing the latest market-leading capabilities to our customers and partners so they can freely innovate, innovate fast, realize ROI and value quickly. It's important to move along with the latest capabilities that come within the market landscape. Second is trust. AI has been probabilistic and will likely remain probabilistic, so it's important to be able to control, govern, establish guardrails, and compliance for using these agents in production. The third is around scale because while the key components of these LLMs and some of the tools are fundamental, the end-to-end solution is comprised of multiple components. Having enterprise-grade capabilities that allow you to quickly innovate, move those into production, and then scale that production as your demand and adoption grows is super important. That's what we bring from AWS.
AWS AgentCore: Choice, Trust, and Scale for Agentic AI Development
I'm going to briefly touch upon each one of these. First, around choice. With choice, the ability to buy, build, or partner depending on the use case and your strategy is extremely important. At the top, you see some of our AWS native capabilities with QuickSuite and third-party capabilities that are available from Marketplace. In the second layer, you have Amazon Bedrock AgentCore as well as LangGraph as a framework where you can build any custom agentic solution of your choice. If your use case in particular does not fit one of those out-of-the-box models, the ability to build your custom agent is important. Lastly, bringing partners who have the expertise and experience within AWS with the AWS Generative AI Innovation Center, as well as our partners' professional services, is important. That's an area where we can partner to build those solutions and take advantage of the lessons learned and best practices that have been accumulated over many runs and many build scenarios. In this context, we are providing the capability to work with any models and any frameworks, and that's the flexibility and choice that speaks to our value.
Now in terms of trust, security, governance, and audit, I want to highlight three points. The first is around transparency. It's extremely important to acknowledge what models are available, how services are developed, and what the intended use cases are. We started this with our AI services where we had AI service cards, and we've now extended those into our agentic AI capabilities where we talk about how they've been trained, what type of data has been used, what the intended applications are, and how they fare against bias, transparency, and explainability. Transparency is extremely important when you consider responsible AI as well as providing IP identification for Amazon Nova models. The second part is around having control. This is a shared responsibility model. We're working closely with model providers around how to make the models themselves safe, but we also need to have the right guardrails around the models. This is where Bedrock Guardrails comes into play to help you govern those for specific applications and make sure that you control the ingress as well as the egress of those LLM models. Lastly, when it comes to audit, data lineage, and observing what's happening across the agent runs, this is where I'll talk about AgentCore observability, as well as our services such as CloudTrail and CloudWatch, which have been there to support that.
Now I talked about AgentCore. I want to double-click on it slightly more. There are three key areas with AgentCore. We talked about agents having agency. That's the first part. It's around having tools and memory to be able to realize what is the session information, what is the short-term as well as the long-term memory, so that each time an agent is run, it's able to learn from its experiences.
Taking that information into account as memory allows agents to enhance their output. Additionally, having access to tools is critical. I've highlighted two of our native capabilities: the code interpreter, where you can generate code and securely test and run that code within a sandbox environment, as well as a browser tool that allows your agent to browse and utilize the browser much like a human would do. The second part focuses on deploying these securely at scale. Runtime and identity, which I'll cover in the next slide, are the key services in this area. Lastly, it's important to have visibility into the whole end-to-end execution of these agents, which is where agent core observability comes into play.
When it comes to deploying these securely, we have agent core runtime, which allows you to containerize your code, deploy it into a cloud environment within a customer VPC securely, and operate that infrastructure. This is extremely important because it allows customers to leverage the intellectual property of a partner, for example, and run it within their own environment while having guarantees around what data can go outside and what data can stay within their own customer VPC environment.
Agent core identity addresses one of the key challenges, which is access control. Access control applies to the user who's using the agent because based on your credentials, you would have access to certain resources and information, and each of us would have a different level of access. It also applies to access control for the agent itself as it tries to access resources across systems. When it comes to tools and memory, we have agent core gateway that provides access to a list of tools and capabilities available for the agent to leverage. Agent core memory allows you to improve your runs by taking into account the shared context between different agents across sessions, whether short-term or long-term.
The last two capabilities I mentioned are the browser and code interpreter capabilities, and later on we'll see a demo where we're actually using a third-party tool. These two are the native tools provided by AWS. When it comes to agent core observability, we have integrations with standard models such as OpenTelemetry, and we're bringing that data into CloudWatch and CloudTrail so that users can inspect agent runs, diagnose any root cause of issues, understand why certain patterns and traces were executed, and improve that over time.
AWS Marketplace: Accelerating Agent Development with AI Agents and Tools
When we talk about agents, they by default consist of multiple components including the LLMs, in many cases the tools, and the knowledge bases. One of the key challenges I hear from customers building or experimenting with agentic AI is not how to build the agent, but how to get access to all the tools that they need. Think of new vendor onboarding, security reviews, legal reviews, and payment setup—all of that can take from weeks to months, slowing down innovation and adoption. AWS Marketplace removed that friction.
Marketplace is an enterprise-ready digital catalog of software, services, and data, and now also AI agents and tools for developers who are building agentic AI. The marketplace brings three key benefits. First, the procurement time drops from months to days or minutes if Marketplace is already configured in your organization. Your developers can go to Marketplace, find the tool they need, and subscribe in a single click with full transparency about the pricing and contractual terms, which is a big advantage for getting started time. Second, there is unified billing and cost control transparency. All the spend through AWS Marketplace goes through your AWS bill, so your developers don't need to put their own credit card and do expense reports for any tools that they're experimenting with. Additionally, Marketplace integrates with AWS cost tools such as budgets and cost explorer, so you could set up certain spend goals and have full transparency on who's spending and what they're spending.
Finally, there's the actual developer experience—the simplicity of deployment. Our marketplace products natively integrate with AWS services, so in a single click you can deploy an agent on AgentCore Runtime. I'll show you that in a later part of my presentation. When a developer wants to start building an AI agent or exploring and evaluating the tool, they can get started in the marketplace with just a few clicks.
We've made it a one-stop shop for AI agents and tools. We launched this category just a few months ago in July, and it's been growing very fast. Right now we have over 2,300 agents, tools, and agentic applications, including professional services from our partners. They fall into different categories, so let me break this down for you.
On the right side of the slide are the API-based agents and tools. Those are remote tools that are ISV hosted. For example, you'll see the Tavily search tool later in Salman's section of the presentation. From the developer standpoint, they act as external targets. Developers go to the marketplace, subscribe in a single click, and they can also in a single click import those tools into AgentCore Gateway as external targets. Then they can use or reuse those tools in multiple agents for different kinds of applications they're building.
What's great is the automation. AgentCore Gateway automatically pulls down all the metadata, the API schema, and actions. We also automate the API key rotation and give full transparency into what's happening with those tools. There's no manual wiring, no schema translation—the gateway handles all of that. The second category on the left portion of the slide are container-based agents. Those are containerized agents designed specifically to run on AgentCore Runtime.
The key benefit is that they run inside your VPC on AgentCore Runtime. You're getting all the benefits of AgentCore service, including VPC isolation, IAM-based identity and resource policies, tool-level permissions, and of course observability. You're getting all the benefits of the partner logic and partner-defined agents who have pre-built everything and done all the prompt engineering, but it runs within your VPC. The third category on the bottom are agentic applications and also the professional services. We hear from many of our customers who are saying they need help to get started. You can go on the AWS Marketplace and find support from a broad network of partners.
Workday's Planning Agent: From POC to Production in Three Days
Before we go deeper into building the agents and showing how that works, I want to ground all the theory that Salman and I have covered in a real customer implementation. Workday has been one of the early adopters of AgentCore and other AWS agentic services. Please welcome on stage Meera Rajaram, who will share their experience of building a planning agent.
Thank you. I'm Meera Rajaram, Head of AI Core Services for Workday Planning. Workday is perfectly positioned to help our customers manage their money, people, and agents. These are the list of agents I'm allowed to show you because these agents are actually going to be coming out by the end of this year in the hands of customers. There's a whole other spreadsheet of agents that we're working on that are going to be following shortly.
How are we getting all these agents out? Workday has an agent platform team that's essentially making some of these common stack consistency code for how we're building agents. We leverage LangGraph, we're using LangSmith for our evals and traces, and we have something like a centralized gateway to the LLM.
So this is what we end up leveraging across the board. The planning agent is no different. We are also leveraging that platform. But let's dive into the planning agent. The planning agent is one agent that represents four personas to give this complete planning intelligence. The personas here are analyst, modeler, planner, and admin. Based on the role, we are essentially offering the skills for those personas.
If we dive a little into the analyst role, one of the key things an analyst does is explore data. The data exploration skill that we offer essentially allows our end users, these analysts, to leverage natural language and conduct ad hoc analysis on vast amounts of data. When I say vast, it is vast. How did this go about for us? How are we successful at it? Well, I'll tell you, it took us multiple tries.
In iteration one, we thought, let's get all this data, let's send it to the LLM, and have the LLM do the data analysis for us. We ran into a bunch of issues with that. First, we ran into token limits because it was a lot of data. Second, if we didn't run into token limits, it was still an expensive solution because we were sending a large number of tokens. Third, LLMs aren't really very accurate with math, so this was not a good solution for us.
Let's see what else we can do, and the team came up with another solution. This was iteration two. We know how to solve token limits. We will chunk the data and send it to the LLM. That's exactly what we did. It was a multi-step process. We would chunk the data, send it to the LLM, get the output, and send that output and the next chunk of data to the LLM. Of course, this was not a perfect solution again.
One is it introduced latency. Two, we did solve for the token limits, but the solution is still expensive. Three, we're still not solved for accuracy. Why? Because at the heart of it, LLMs aren't great at math, but LLMs are really good at writing code. We figured that we would rather get the LLM to write code, Python code that can do the data analysis that we need, and then the LLM can leverage that output and be able to explain it in natural language to our end users. That's really what we went with.
Now, why do we need a sandbox code interpreter, and what exactly is this? Show of hands for how many people have raised kids. We've got a few folks. Well, if you've got a high energy toddler at home, one way to deal with that toddler is to maybe create a playpen, put them with all their toys. They can create whatever mess they want in that playpen, and the beauty is your fancy carpet and walls are not disturbed. That's one way of doing it, and that's exactly what the sandbox code interpreter is.
Think of your code like your toddler. It can go wild, it can create a mess, and you want to put it in a sandbox. That's exactly what a sandbox code interpreter allows you to do. It ensures that your code can run in a safe, secure environment. We evaluated a lot of different options. One is, should Workday build our own sandbox code interpreter? That's not the core of what Workday does, so that's not going to be a good solution for us.
We then started looking at different frameworks and tools. If you've been in this industry long enough, you've seen a lot of demos, and these demos are amazing because they tell you how to build agents in an hour, you can create POCs in an hour.
But taking that POC from the POC stage to production is a whole other matter because you have to incorporate all the enterprise controls. You have to think about security and customer data, which is so precious. We said we need a good solution that's going to help us address it, and Agent Core essentially provided that solution.
First, it manages security for you. Your code runs in a very secured virtual computer. The customer data you have is uploaded into that environment, and as soon as you're done, all of that data is wiped clean, and you're starting with a clean slate. That was super critical. Second, the code interpreter handles stateful multi-step analysis. For that entire session, it's able to do a multi-step analysis process to ensure that you get the most accurate results.
Third, there's scalability. We're able to send a ton of data to the code interpreter. We were able to upload anywhere between 100 MB files for inline processing. Of course, we compress that file and send it over, but you could also leverage S3 for it, where you can have up to gigabytes of data. That's where we landed.
Now, what did it take us to get this solution with Agent Core and the code interpreter into production? It took us one software developer, two engineering managers, and three days to get this from POC to something that can be production ready. That is huge. How many folks can say that you can get something from POC to production in such a short span of time? That's exactly what we got with the Agent Core code interpreter. By the way, we just went live with our solution two weeks back, so customers are actually using this in production. Which is amazing.
Let's talk about some of the results with the code interpreter solution. We are adding some latency with the code interpreter, somewhere in the ballpark of 400 to 500 milliseconds. If you look at the 90th percentile, somewhere between one to two seconds. But what we gained is the token size reduced significantly. When we have these large payloads, we are seeing up to a 50 percent reduction in the tokens that we end up sending, and that's significantly reducing the cost for us.
The other benefit that we're getting is accuracy. We ended up getting much better results with this solution than what we had done previously. That was huge. Third, there's scalability. We now didn't have to worry about hitting token limits or any of that. The solution is scalable. At the end of the day, what our customers are looking to do is analyze large reports and vast amounts of data, and this solution is exactly addressing that.
Hands-On: Building a Company Research Agent with Tavily and LangChain
I do want to thank the close partnership we have had with the Planning Agent team, our own Workday Agent Platform team, and AWS folks. This was a very strategic partnership that helped us innovate at a really rapid pace. With that, I will hand it off to Salman. Thank you, Mira. We're going to be building an agent now. Disclaimer number one, we're going to be seeing some code. Disclaimer number two, you can definitely use AI to help you with any form of coding. I actually did use AI to generate this code.
I need to do this in 10 minutes, so I'm going to focus on the bare bones of that capability that I talked about. How do you bring that tool and agency within the agent? Where do you bring the LLM that does the reasoning? How do you bring in the query? Then I'll give some pointers around how you can start off but also strengthen that in order to move such a solution into production.
In this example, I'm going to show how to build an agent using a third-party tool from Tavily. Tavily is an Internet search tool that provides a capability. I'll talk to you about how you can take that tool, integrate it within the code, and run it. First things first, what is the use case? That's important. In this particular case, I am going to demonstrate a company research agent for a sales team. This happens to be a popular use case that I'm seeing in production. Many sales teams focus on searching for information from the Internet and creating tailored briefing documents ahead of customer meetings. This is one use case that I've seen emerge in production, and I thought it would be good to create a simple bare bones capability to show how these are built.
Next, let me outline the steps that we need to follow. Step number one, we need to architect the solution, so we're going to go over that. Step number two, we need to set up the tools in the cloud environment. I'll briefly touch upon that as well. Step number three, we have to have the code and then run it. So for step number one, let me show you the solution components. This is an overly simplified version of the solution components involved in an agent. You have the user that will be interacting with an agent executor, and that executor takes on the role of an orchestrator. It determines where the LLM is that I need to take advantage of in order to do the reasoning and planning. What are the tools that I have access to and what does that mean?
In this case, in this example, we have two tools: one to represent information that is coming from the web and the Internet, and the second one is information that is residing within the enterprise environment. In this particular case, I just have a file sitting on Amazon S3 storage to show how the two information sources can come together, be combined, curated, and present a response to the user. Now, with each one of them we need to have some decisions around the technical capabilities. So with the agent executor, there are a number of frameworks that are available. Mira talked about LangGraph. LangChain is one that is very close to that and comes from the LangChain provider. That's a framework we're going to be using in order to orchestrate these capabilities from a reasoning LLM.
We have access to more than 200 models from Amazon Bedrock. In this particular case, we're going to see an example from Anthropic Claude. When it comes to the tooling, the Amazon Bedrock AgentCore Gateway is going to present us the option of connecting to Tavily's tool in order to search the Internet, and the local data search is simply a tool function within the Python environment. I think I've skipped that part for the interest of time. So we're going to go over the example, and I'm going to cover how to set up the tools in the cloud environment, and then we'll go into the code piece. I'll walk you through what each section of the code is doing so you can follow.
So the first thing around the choice of the model: you can go to Amazon Bedrock, and as I said, you've got access to more than 200 models. In this particular case, the use case seems simple enough, so a model like Claude 3.5 Haiku should be able to provide a decent result. Now within Bedrock, you have the Bedrock evaluation capability where you can actually test models side by side. Those are some of the best practices when you're moving and executing on a proof of concept to be able to identify what is the optimal model for your particular use case. But here, for the sake of simplicity, I'm going ahead with the Claude 3.5 Haiku model. You can select the model and then you can generate keys either short-term or long-term. In this instance, we're generating short-term keys and then we're storing these within the secrets manager as well as incorporating it into the code development that I'll show you later on.
The second part is around the tooling. As I talked about, we're using Tavily's tool to search the Internet. You'll be able to access and get keys from Tavily.com.
Then, following the same practice with those API keys, you can store them securely within your Amazon environment in the Secrets Manager. Within Agent Core Gateway, you can register this. As it happens, within Agent Core Gateway, we have a number of native integrations. One of the options on the furthest right talks about some of the built-in integrations you'll be selecting. Tevili is one of the options available there, along with some other third-party tools. You'll be able to select your key, and you'll be given an Agent Core Gateway URL and ARN that you will be using as part of your tool. So your cloud environment is now set up.
Now let's move on to the code. In its simplest form, having that LangChain that I talked about and setting up the agent executor is not that difficult. If you follow the Python code here, you'd basically be pulling in some of the libraries, in this case LangChain. Then there are a couple of lines of code where the agent executor comes and orchestrates all the capabilities from the LLM, the tools, and the prompts you create. That tool calling capability forms an agent executor. Once you have this object, you can actually call invoke against it, so you can give a user's query in order to invoke this agent. It would operationalize and generate a response.
Three parts are key: the LLM, the tools, and the prompt. These are the three parameters that are provided. I'm going to be talking about those three parameters in the subsequent slide . This code highlights all three. The first one is the LLM. In this case, within LangChain, you have native integration with Amazon Bedrock. You can simply use ChatBedrock in order to select the model and provide information around the region and credentials that you have. Here you see the example of Anthropic Claude 3.5 being mentioned.
On the second one, you need to provide a prompt. We talked about agents following an objective. Here I have an objective that reads: "You're a company research agent. Your goal is to answer questions about companies using available data sources. You have access to two tools. Number one: search local documents, search internal company documents for information. Number two: search the web, search the web for current company information." With Agent Core, you can even have a separate tool to inquire on the list of tools so you can get what tools are available and pass that on to the reasoning LLM so it can determine which ones to use in subsequent stages. But in this case, I've kept it simple and provided those tools that are relevant to this agent directly within the prompt.
The third one you see is that we have tools as an array that talks about these two: search local documents and search web. I'm going to skip the search local documents, but I want to show you the code for search web so you can see how easy it is to bring all of this capability into Python code and execute it. First, bringing some of those configurations that we have set up in the cloud environment in the AWS environment from your AWS region, your S3 buckets are for the local search that I talked about, and then the Agent Core URL and tool . Those are the information that we already had from the AWS console. You're going to be able to take advantage of it.
Then the search web that was mentioned in the array is basically a function , and this function is decorated as a tool. So the Python interpreter environment will be able to understand this is one of the tools available for that agent. Within that, we capture the name of the tool from the Agent Core service. We're going to be able to package this as a JSON RPC, sign the query, and then send it over to AWS for invocation .
In this case, what you're seeing is the payload is packaged up with some of the information around the JSON RPC information, also having arguments. The argument says query is query. The query parameter is what the user provides, and I'll show you the example that we have in the recorded run. Then after that, you're going to create this, encapsulate this as a request, an AWS request, and basically send this request as a post operation.
Once the agent calls the search web tool through this process, it receives the response, and that response is sent to the LLM for further decision-making to determine whether it needs to call additional tools or simply curate the response and provide output to the user. This decision is left with the LLM to make.
Now I want to show you how this comes together when you run it. In this particular case, we need an example query. The query here is: how has AWS's approach to sustainability and green computing evolved, particularly focusing on their commitment to run on 100% renewable energy? This is the query we're going to provide to the agent, and I'll show you screenshots of the run.
When you run this, the agent first tries to figure out whether it has the information or needs to use the available tools. The green text shows the JSON output of what the LLM has determined. It has determined that it needs to invoke the search web tool and has formed a query different from the original one. In this case, it says it wants to search the web for: AWS Amazon Web Services, sustainability, renewable energy, 100% commitment progress. The LLM has determined which tool to use and what argument to provide.
This tool is invoked through Amazon Bedrock AgentCore Gateway and returns information. You see a dense piece of text because it brings back search results from the Internet relating to that query. In this case, there is information around Amazon Web Services sustainability, renewable energy commitments, URLs for citations and linkage to the original material, as well as text. The LLM processes this information combined with the data it has and provides it in raw JSON format, which can be processed further to create more human-readable text. The output provided is based on the web search results: I can provide a comprehensive overview of AWS's approach to sustainability and green computing. AWS has achieved its 100% renewable energy goal seven years ahead of schedule in 2021.
To summarize, there are three steps: first, we provided the query. Second, we determined which tools to call and what parameters to provide for each tool. Third, we curated the responses and provided an outcome. These are the core building blocks of building an agent, but this should not be moved into production as-is.
The first consideration is security and governance. With Amazon Bedrock AgentCore Runtime, you can containerize this code and push it into AgentCore Runtime to operationalize it. The second consideration is placing guardrails around the application. As I've discussed, it's important to implement guardrails with Bedrock Guardrails. The third consideration is performance and reliability. In many applications, caching is effective because there are often repeat queries with the same company names or information. You also need reliability in terms of failing over properly if a tool cannot be called or the answer does not exist. How do you recover from that situation?
Deploying Pre-Built Agents: The Fast Path to Production-Ready Solutions
Lastly, there is cost control. You need mechanisms in place to understand how much the LLMs are being recalled, what token-based pricing is being generated per request, and what costs are associated with the tool calls. All of these aspects help you strengthen the application to move it into a production-grade environment. With that said, I'm going to hand over to Andre to talk about deploying a pre-built agent. You just saw how to build an agent from scratch: pick your LLM, pick your tools, do the prompt engineering, and test that it's working. This is great when you want flexibility and granular control over the tools, but many teams don't want to start from scratch.
And sometimes the fastest path is to start with a production-ready pre-built agent that you can get from a third-party vendor. I'll be using a CRM AI sales agent that is built by one of the vendors available on AWS Marketplace, which can help you automate routine CRM tasks. This is a fully packaged solution with defined action schemas, defined prompt engineering, and the recommended LLM already baked in. It is already packaged and optimized to run on AgentCore runtime, which means you can deploy it in minutes. I'll show you in just a few slides how that's going to be done.
This is the power of pre-built agents. You inherit all the vendor's work and their testing, ensuring that all components work together. It can still run securely in your environment within your VPC. The getting started experience is extremely simple. It is just three steps. First, you discover the agent or solution based on your business needs. We have a new agent mode on AWS Marketplace where you can describe your business problem or attach your internal requirements document or your plan for what you're trying to build, and we'll give you the best optimal solution for that.
The second step is the subscription flow, which is a single click. This particular tool is free, but it still requires you to accept the vendor's terms and conditions. Finally, there is the deployment process, which is almost as simple as a single click. Marketplace has standardized the framework for how to package and build those containers. They come with vendor-authored instructions and all the configuration, requiring users to provide only minimal inputs. In this case, users deploying the agents need to provide environment variables, such as CRM access credentials.
What's important about Marketplace is that it acts as a supply chain guardrail. All the agents, which are containerized applications, are stored in the Marketplace private ECR repository. This means that vendors cannot swap the image whenever they want. All images go through a rigorous scanning process and are behind entitlement-enforced access, so only users who are entitled to use this product can access them. Vendors must go through the full publishing flow and security reviews before pushing a new version.
Once it goes to the AgentCore runtime site, that's where you get all the benefits of running these agents within your VPC. You get IAM role configurations where you start by either creating a new IAM role or selecting an existing one if you already have one. You define the minimum required set of permissions for this agent to operate properly, which usually requires access to Bedrock LLMs, access to the tools you're using, or other internal resources that are required for the agents.
The secure runtime environment with AgentCore gives you full control. The agents would never leave the boundaries of your VPC. They are isolated from inbound requests from the Internet unless you allow that, and you retain full control over what the agent does and how it operates. Marketplace ensures the trusted delivery of the agent, while AgentCore runtime enforces secure governance of the agents at the execution step. To summarize, here is the flow if you want to get a third-party pre-built agent. It's very simple: you find the agent on Marketplace, you subscribe, you provide the environment variables—in this case, CRM credentials—you deploy it on an AgentCore runtime, and you invoke the agent.
Demo: Deploying a CRM AI Sales Agent from AWS Marketplace
Let's see that in action. I have a short pre-recorded demo to put all this together. We're starting on the Marketplace website and going to the new search experience. You can describe your problem in natural language.
In this case, I'm looking for an AI agent to automate my routine CRM actions. As you can see, I'm typing slowly. It will sync for a few seconds. You can also upload a document with your requirements if you need more precise recommendations. You will see a few options with the journey I recommended summary, and we just launched a new version of that experience where you can also create an AI build comparison of different tools so you can compare them across different dimensions.
Once you identify the tool that works for you, you go to the product detail page where you can read all the information about how the tool works and also see the deployment instructions, including the tool or in this case agent schema and all the inputs that are required from you to make sure that this agent operates properly. Once you're ready, you move to the purchase step where you review pricing and terms and conditions. This tool is free, but you still need to hit the subscribe button to make sure you agree with the vendor's terms and conditions.
Once you're subscribed, you can proceed to launching the software. At this step, there are two options. You can either proceed with CLI if this is your preferred method, and we will provide you a few pre-built, pre-generated prompts for the CLI. The first step would be creating an IAM role. So the first step will be creating an IAM role so you can copy that and execute this command attaching permission policies. Finally, you deploy the agents with the container image provided by the partner.
Alternatively, you can get the same work done directly in the AgentCore console, which we'll be using for this demo. In the AgentCore console, your inputs are fairly simple. Provide a name for the agent. It's pre-populated already, but I prefer a different one, so let's call it a Salesforce agent. The ECR container URI is already pre-populated from the marketplace solution that you have just purchased. You select or create a new IAM role. I have one already selected, and I'm almost ready to hit the hosting engine, but not yet. I need to provide the environment variables.
In this case, I need to provide four keys with respective values: my CRM username, my CRM password, the token, and the domain. This information never leaves the boundaries of your VPC. This information is never shared with the vendor. You can also store those credentials in the secrets manager and instead of providing environment variables here, you can reference them from the agent directly, which is much better for production and when you need to do regular rotation. So now let's test this agent and see if it works. I'm asking it to provide me a summary for the specific account. Let's see. It will take a few more seconds to sync, and here we go.
So here's an example of an agent that can help automate the routine work of your sales teams. There are multiple ways how we can plug in this agent into your own individual workflows with all the data never leaving the boundaries of your VPC with you having full control but at the same time with no building, no coding, and no prompt engineering. The one thing I also want to mention is that this particular agent was built by partners using the Strands framework, which already has Bedrock LLMs baked in. So in the previous example, you saw how we need to pick an LLM and how we need to configure the LLM access. In this case, you don't need to do any of that. It comes prebuilt.
Conclusion: Starting Your Agentic AI Journey with AWS
AWS provides everything you need to succeed with agentic AI: pre-built agents, tools, models, infrastructure, and expertise. Generally, the best way to go about this is to identify a use case and start this flywheel cycle. What is the use case? What is the business pain point or opportunity? Identify whether we should be building, buying, or partnering, and then create this flywheel effect that you can improve over time. There are some resources that are available, and this recording will be available as well. We have information around some of the sample code with AgentCore but also some training material to improve your skills with our agentic AI offerings. Lastly, I want to close off with some of the relevant sessions that are also covering similar topics at re:Invent. Thank you so much for being with us.
; This article is entirely auto-generated using Amazon Bedrock.
Top comments (0)