With Bedrock you get access a range of different large language models, ( for instance, Claude, Mistral, Llama and Amazon Titan) with new versions becoming available all the time.
Having choice is great, but having to code your requests differently for each model is a pain.
Here’s why the Amazon Bedrock Converse API is going to save you a bunch of time and effort, when comparing the output of different foundation models!
Consistency is key!
The Converse API is a consistent interface that works with all models that support messages / system prompts. This means that you can write your code once, and use it to experiment with different models.
Here’s an example of how it works, and this exercise should cost < $1.
Configure model access
Before you begin, be sure to check that the models you want to use are available in your region, and that you have enabled access to them, here are the ones I'm using, you can select these or choose your own:
anthropic.claude-v2
anthropic.claude-3-haiku
Claude 3.5 Sonnet
Mistral small
1) We can do everything using the CloudShell in the AWS console.
2) When the CloudShell is ready, install boto3 which is the AWS SDK for Python
pip install boto3
3) Download the file named converse_demo.py from GitHub You can do this using wget and providing the raw path to the file:
wget https://raw.githubusercontent.com/fayekins/demos/refs/heads/main/converse_demo.py
converse_demo.py
#first we import boto3 and json
import boto3, json
#create a boto3 session - stores config state and allows you to create service clients
session = boto3.Session()
#create a Bedrock Runtime Client instance - used to send API calls to AI models in Bedrock
bedrock = session.client(service_name='bedrock-runtime')
#here's our prompt telling the model what we want it to do, we can change this later
system_prompts = [{"text": "You are an app that creates reading lists for book groups."}]
#define an empty message list - to be used to pass the messages to the model
message_list = []
#here’s the message that I want to send to the model, we can change this later if we want
initial_message = {
"role": "user",
"content": [{"text": "Create a list of five novels suitable for a book group who are interested in classic novels."}],
}
#the message above is appended to the message_list
message_list.append(initial_message)
#make an API call to the Bedrock Converse API, we define the model to use, the message, and inference parameters to use as well
response = bedrock.converse(
modelId="anthropic.claude-v2",
messages=message_list,
system=system_prompts,
inferenceConfig={
"maxTokens": 2048,
"temperature": 0,
"topP": 1
},
)
#invoke converse with all the parameters we provided above and after that, print the result
response_message = response['output']['message']
print(json.dumps(response_message, indent=4))
4) Run the Python code like this:
python converse_demo.py
It should give you an output similar to this:
5) We can also run this same code using different model, by replacing the model ID in our code as follows:
anthropic.claude-3-haiku-20240307-v1:0
Compare the output from the second model, it is slightly different:
6) We can test again with another version:
anthropic.claude-3-5-sonnet-20240620-v1:0
When a new version of Claude is released, we can request access and then just replace the name of the model in our code!
Access denied error
If you see an error similar to this, it just means you are trying to use a model that you don't have access to yet. Simply request access to th model, and try again after access is granted.
7) I also tried it with a different model provider, by changing the model id to:
mistral.mistral-small-2402-v1:0
So the Converse API gives you a simple, consistent API, that works with all Amazon Bedrock models that support messages. And this means that you can write your code once and use it with different models to compare the results!
So next time you’re working with Bedrock, do yourself a favour, try out the Converse API, and thank me later!
Top comments (0)