Multi-turn conversations with Amazon Bedrock Flows

#aws #tutorial #amazonbedrock #ai

The world of agents, agentic flows, agent collaboration etc is as quick to learn about as it is to be overwhelmed by. How many different ways can we say "AI assistant" after all?

As I myself learn about this space I'm taking the time to work through starting to build something from scratch, iterating through documentation and other helpful resources to ideally end up with a working demo on my AWS account. All that to say, the plan is to learn enough along the way that I can save you some of the pain of having to figure it all out yourself. Let's get started 🤘🏼

What even is an Agent?

Before we can dive into console screenshots, click this click that and see the magic happen, we need a foundation from which to start. Why does any of this even matter? Well at the root of this discussion is the "agent", this handy dandy trendy term is nothing but a software program that can interact with users (us), collect data, and use the data to perform tasks to meet a goal. If you're anything like me, this might be conjuring up some unfortunate memories dealing with a customer facing chatbot to engage product support or customer service. Cue the- "raise your hand if you've been personally victimized by a chatbot" ✋🏼

I hear you, but what is now different with this new era of agents is the injection of AI. These agents now have the ability to make rational decisions based on how they interpret input data and are tasked with producing the most optimal results.

I can hear the purists now, yes- this is an oversimplification, but for the purposes of this "learn with me" that's all we need to know to understand what we're building.

Now, how can we use AWS to help with this lift? 🤔

Amazon Bedrock Flows

If you've gotten this far (thank you), I will assume you are familiar with Amazon Bedrock so I won't be discussing the who, what, where- but what I will be diving into here is a feature within Amazon Bedrock known as Flows. The TL;DR on this feature- it has a fun drag/drop GUI to help us create, test and deploy workflows for AI applications. Currently Flows supports elements like linking together prompts, knowledge bases, guardrails, other AWS services and you guessed it, agents. Supports changes often so please reference documentation at the time of reading to confirm any feature changes.

When I'm getting hands on with an AWS service/feature for the first time I prefer the console experience, and Flows is pretty intuitive here reminding me what it feels like to put a LEGO set together (which I love). Don't worry, if you prefer to engage programmatically Flows can also be used via APIs and the AWS CDK.

Multi-turn Conversation

The meat and potatoes for this "learn with me" is exploring the new multi-turn conversation support in agent nodes for Bedrock Flows. So, why does it matter? We mentioned before agents are helpful when they're able to interact dynamically with users to run through tasks and complete a goal. This dynamic interaction with data however, can be rather ineffective if the agent doesn't have all the context it needs to continue through its predefined workflow. With this new feature in preview, an agent node can pause a flow execution and request data it's missing from the user to then continue with full context onto its next step. Ok enough background, let's get building!🔨

Putting it all together

The general design for this sample build is as follows:

A travel agent created through Amazon Bedrock Agents
A travel assistant flow created through Amazon Bedrock Flows

If we correctly set this up we should be able to step through a Bedrock flow asking either general questions about a travel destination, or asking for flight information and a booking pathway. The former part of the flow handles generic information like weather, tourism tricks, etc. The latter engages an agent to book a sample flight, which should check if it has the necessary context or engage with the user in a data gathering exercise before continuing. Both of these user flows work dynamically so questions about a city can be asked throughout the interaction and the same goes for flight booking.

Creating the agent

Head to the Amazon Bedrock Console, and choose Agents from Builder tools in the navigation pane.
Create the agent with the name of your choice and if you'd like, give a description.
In the Agent builder section select a foundation model to power the agent. In this case I'm selecting Anthropic's Claude 3.5 Sonnet v2.
Under Instructions for the Agent we are creating an instruction set that the agent persona will follow. In this sample travel assistant we want to configure an agent that will help a user book a flight when it has context of a users origin, destination and date/time.

a. I use the following instruction: To book a flight, you should know the origin and destination airports and the day and time the flight takes off.
In Additional settings make sure User input is enabled so the agent can prompt user for additional information if needed.
In Memory select Enabled, we do this so the agent can store a session in memory and continue to have user context when a session ends and we reference it again later.
Now we'll move on to create an action group with 2 functions. Action groups help organize what tasks the agent will help with and sets up the logic they'll use. The business logic of an action group lives in a Lambda function which you create and point agents to.

a. Give a name to the action group, I'll use search_book_flights, and select Define with function details.

b. Next we'll select the option which has the wizard Quick create a new Lambda function.

c. For the first action group function we'll define a function that can search for sets of flights based on origin, destination and a date. The three parameters we set are string types, date, origin_airport, and destination_airport with the option of required set to true.

d. The second action group function defines how to book a flight given the origin, destination and the date and time. The four parameters we set are string types, date, time,origin_airport, and destination_airport with the option of required set to true.
Complete the agent creation with Create.
Once complete you can access the Lambda function created by the wizard and edit its contents. I use the Python example described in this blog which simulates flights and times for testing purposes.

Ok take a breath, let's debrief- so far we have created an agent within Amazon Bedrock that will help a user search for flights between two destinations (function 1). It can also book a flight when given a date and time (function 2). Because we're not running our own flight agency or scraping real flight data, we simulate this searching and booking behavior with the Lambda function written by my peer @danilop (👋🏼). With this agent complete, let's now use it in a workflow in Amazon Bedrock Flows.

Creating the Bedrock Flow

Let's head back to the Amazon Bedrock Console and this time select Flows from Builder tools in the navigation pane.
Click Create flow and enter a name, allow for a new service role unless you've done this before and have one ready.
Once created you will be redirected to the flow builder GUI to start stringing together elements.

Let's pause here and quickly discuss the expected outcome. What we want to demonstrate is a travel assistant workflow in which a user can ask generic questions about a city and be given responses powered by an LLM. The user can also inquire about flights to these destinations and if they decide to they can ask the agent to book a particular flight. The following steps outline how we build a Bedrock Flow to implement this functionality.
In the Flow builder panel select and drag the Prompts node to your workspace. When selected the configuration pane opens and we can customize the node.

a. Give the prompt node a name, I use user_intent. Select Define in node to set the logic directly from this pane. Select a model, I use Nova Lite 1.0. This prompt node will be the first entry point for a user using this assistant, it needs to classify whether the user is asking generic questions about a city or looking to book a flight. To do this we give the node a persona and instructions. I use the following: You are a query classifier. Analyze the {{input}} and respond with a single letter: A: Travel planning/booking queries for hotel and flights Example: "Find flights to London" B: Destination information queries Example: "What's the weather in Paris?" Return only 'A' or 'B' based on the primary intent.

b. To finish the prompt node setup we can change the Inference configurations to manipulate the temperature.

The scale is from 0-1 and indicates how much "creativity" or randomness you want the model to take. The closer to 1 the more likely the model is to hallucinate, but depending on the use case such as writing, brainstorming, etc a more random approach can lead to better results.

In this case, I set the temperature to 0.1 so the responses are more focused/deterministic.
Next, I create a condition node that will take as input the category letter the user_intent node generates.

a. Drag and drop the Condition node to the workspace. Name the node, I use Book_or_Questions.

b. Leave the input formats as default. Next, I name the condition and define it using Condition: categoryLetter=="A".

c. At this point the workspace should look like so: flow input -> prompt node -> condition node.

Currently a user interacting with this flow can enter their query and the query is classified into either "A- booking support" or "B- travel destination questions". We now need to build out support for the flow for "B- travel destination questions" which requires we create another prompt node that uses a LLM to respond.
Drag and drop a Prompts node to your workspace.

a. Give the prompt node a name, I use travel_guide. Select Define in node to set the logic directly from this pane. Select a model, I use Nova Lite 1.0. I use the following prompt: You are an enthusiastic and knowledgeable travel guide. Your task is to provide accurate and comprehensive information about travel destinations to users. When answering a user's query, cover the following key aspects: - Weather and best times to visit - Famous local figures and celebrities - Major attractions and landmarks - Local culture and cuisine - Essential travel tips Answer the user's question {{query}}. Present the information in a clear and engaging manner. If you are unsure about specific details, acknowledge this and provide the most reliable information available. Avoid any hallucinations or fabricated content. Provide your response immediately after these instructions, without any preamble or additional text.

b. The input to this Prompts node will be the output of the Flow Input node, aka the users original query.

c. At this point we can terminate the "destination questions" workflow, so we drag and drop a Flow Output node and connect the output of this Prompts node to the input of Flow Output.
Finally we get to the step where we connect the agent created in the previous section. As a reminder my agent was called travel-agent.

a. Drag and drop the Agents node into the workspace. Name it and select the pre-existing agent from the dropdown menu as well as the alias.

b. Disable promptAttributes and sessionAttributes, we won't be using those.

c. Connect the output of the Flow Input node to the input of the Agents node.

d. At this point we can terminate the "book a flight" workflow, so we drag and drop a Flow Output node and connect the output of the Agents node to the input of Flow Output.
If everything is linked up correctly the flow diagram should look similar to this. Double check the input/output connections to confirm everything looks right, I had some lines incorrectly linked which caused the test to fail later on and a debugging nightmare.

Does it work?!

All said and done you can now test the flow by publishing a version 🎉. You can test programmatically or in the console. I opt for the console to keep things more visually interesting and easy to read. You can also look through the query traces to confirm the flow is working as expected.

Overall this process is fairly straightforward as long as you can keep yourself organized through the logical steps (ie- create an agent, create a flow, link both, test, rinse/repeat).

In my test I run through multiple queries with the flow wherein I ask for flight information, ask the agent to book a flight, and ask questions about cities I'm interested in traveling to. Below are examples of how the tests perform and how I confirm the flow performs multi-turn functionality by pausing and asking for additional user input before continuing on its execution path.