Hi Everyone,
For the last few weeks, I have been trying to build a chatbot that suggests movies based on user prompts.
I am not an expert in the AI/ML field. With the knowledge I have in this field and the AWS services I know I tried to build this chatbot by using the following AWS Services
Bedrock(Titan and Claude models)
OpenSearch
AWS Lex
Lambda
Whenever I try to spell AWS Lex it reminds me of the villain Lex Luthor from the Superman series
Let me introduce each service I used and the role of that service in building this chatbot
BedRock: This service will provide the models by Amazon and several third parties. From the available models, we will be using the Claude v2 model for processing the User Prompts and the Titan model for generating the embedding for the movie content we have.
If you don't know what is embedding, think of it as an array with numbers which helps the ML Model to understand the relationship between real-world data.
OpenSearch: We will be using this service to store the embeddings generated by the bedrock Titan model and to query those embeddings to find similar movies
AWS Lex: This service will help us to build a chatbot. This will be a bridge that takes the user prompt and triggers the movie-suggesting logic based on the intent detection and will respond back to the user with movie suggestions.
Lambda: This is where the core logic executes, which will take the user prompt from the Lex Bot, query the OpenSearch index, and respond back to the Lex Bot with movie Suggestions
Let’s dive into the implementation
This is the movie dataset I am using for this task https://www.kaggle.com/datasets/kayscrapes/movie-dataset
Implementation Steps:
Create an OpenSearch Cluster and an index for dumping Embeddings
Generate Embeddings and dump them into the OpenSearch Index
Implement a Lambda function to query the OpenSearch Index
Create a chatbot and connect it to the Lambda
Create an OpenSearch Cluster and an index for dumping Embeddings:
Visit the OpenSearch service in the AWS console and click on the Create Domain button
Provide a name for the domain, Choose the Standard Create option and the dev/test template like below
- Deploying it in a Single AZ without any standby
- Choose General Purpose instances and select t3.medium.search instance for this task and with 1 node. Per node, I gave 10GB of storage
- I am giving public access with IPV4 only for this task
- Enable fine-grained access and create a master user like below
- Change the access control to allow all users to access the endpoint for this task
Keep the rest of the options as it is and click on the create endpoint button. It will take some time for the endpoint to be created
Once the endpoint is created, access the OpenSearch dashboard from the console and provide the master user credentials to access it
Visit the Dev Tools option from the side menu and paste the following code to create the index movies
PUT movies
{
"settings": {
"index.knn": true
},
"mappings": {
"properties": {
"title_org": {
"type": "text"
},
"title": {
"type": "knn_vector",
"dimension": 1536
},
"summary": {
"type": "knn_vector",
"dimension": 1536
},
"short_summary": {
"type": "knn_vector",
"dimension": 1536
},
"director": {
"type": "knn_vector",
"dimension": 1536
},
"writers": {
"type": "knn_vector",
"dimension": 1536
},
"cast": {
"type": "knn_vector",
"dimension": 1536
}
}
}
}
Generate Embeddings and dump them into the OpenSearch Index:
As we have the Open Search cluster and index ready. Let’s generate the embeddings and dump them into the index
Download the movie dataset using this link https://www.kaggle.com/datasets/kayscrapes/movie-dataset
Extract the Zip file and have a look at the CSV file which contains movie data
Create a gen_emb.py with the following code to generate embeddings and dump them into the index. Install Pandas, Boto3, Botocore, and Opensearch-py libraries using PIP
import boto3
import json
import botocore
import pandas as pd
from opensearchpy import OpenSearch
##opensearch configs
host = 'paste open-search-endpoint here'
port = 443
auth = ('username', 'password')
index_name = "movies"
##creating opensearch client
client = OpenSearch(
hosts=[{'host': host, 'port': port}],
http_auth=auth,
use_ssl=True,
verify_certs=True
)
##filterting columns using pandas
df = pd.read_csv('hydra_movies.csv')
refined_df = df[["Title","Summary","Short Summary","Director","Writers","Cast"]]
refined_df.info()
refined_df = refined_df.fillna("Unknown")
##connecting to bedrock runtime
session = boto3.Session(region_name='us-east-1')
bedrock_client = session.client('bedrock-runtime')
##generate embedding using titan model
def generate_embedding(value):
try:
body = json.dumps({"inputText": value})
modelId = "amazon.titan-embed-text-v1"
accept = "application/json"
contentType = "application/json"
response = bedrock_client.invoke_model(
body=body, modelId=modelId, accept=accept, contentType=contentType
)
response_body = json.loads(response.get("body").read())
return response_body
except botocore.exceptions.ClientError as error:
print(error)
##creating a document to insert
def create_document(title,title_emb,summary_emb,short_summary_emb,director_emb,writers_emb,cast_emb):
document = {
'title_org':title,
'title':title_emb['embedding'],
'summary':summary_emb['embedding'],
'short_summary':short_summary_emb['embedding'],
'director':director_emb['embedding'],
'writers':writers_emb['embedding'],
'cast':cast_emb['embedding']
}
insert_document(document)
##inserting document into opensearch
def insert_document(document):
client.index(index=index_name, body=document)
##iterating thorough each row in data frame created through pandas and requesting embedding
for index, row in refined_df.iterrows():
title = row['Title']
summary = row['Summary']
short_summary = row['Short Summary']
director = row['Director']
writers = row["Writers"]
cast = row["Cast"]
title_embedding = generate_embedding(title)
summary_embedding = generate_embedding(summary)
short_summary_embedding = generate_embedding(short_summary)
director_embedding = generate_embedding(director)
writers_embedding = generate_embedding(writers)
cast_embedding = generate_embedding(cast)
create_document(title,title_embedding,summary_embedding,short_summary_embedding,director_embedding,writers_embedding,cast_embedding)
print(f"inserted:{index}")
- Run this file using Python, visit the Query WorkBench from the OpenSearch console side menu, and use the SQL queries to see whether embeddings are inserted or not
Implement a Lambda function to query the OpenSearch Index:
Now that we have the index ready with embeddings, Let’s create a Lambda function to query the index for getting similar movies
Visit the Lambda service from the AWS Console and click on the Create function button
Give a name to the function and choose the runtime as Python 3.9
Paste the following code in the Lambda function code
from opensearchpy import OpenSearch
import json
import boto3
import botocore
##open search configs
host = 'paste-open-search-endpoint-here'
port = 443
auth = ('username', 'password')
index_name = "movies"
##bedrock connection
session = boto3.Session(region_name='us-east-1')
bedrock_client = session.client('bedrock-runtime')
##creating opensearch client
client = OpenSearch(
hosts=[{'host': host, 'port': port}],
http_auth=auth,
use_ssl=True,
verify_certs=True
)
def lambda_handler(event,context):
print("Event: ", event)
prompt = event['inputTranscript']
print(f"received prompt is{prompt}")
generate_embedding(prompt)
embedding = generate_embedding(prompt)
response = run_query(embedding)
return response
#generating embedding for user input
def generate_embedding(value):
try:
body = json.dumps({"inputText": value})
modelId = "amazon.titan-embed-text-v1"
accept = "application/json"
contentType = "application/json"
response = bedrock_client.invoke_model(
body=body, modelId=modelId, accept=accept, contentType=contentType
)
response_body = json.loads(response.get("body").read())
return response_body['embedding']
except botocore.exceptions.ClientError as error:
print(error)
def run_query(query_embedding):
try:
# Define the query
query = {
"query": {
"bool": {
"should": [{
"knn": {
"title": {
"vector": query_embedding,
"k": 100
}
}
},
{
"knn": {
"summary": {
"vector": query_embedding,
"k": 100
}
}
},
{
"knn": {
"short_summary": {
"vector": query_embedding,
"k": 100
}
}
},
{
"knn": {
"cast": {
"vector": query_embedding,
"k": 100
}
}
},
{
"knn": {
"writers": {
"vector": query_embedding,
"k": 100
}
}
},
{
"knn": {
"director": {
"vector": query_embedding,
"k": 100
}
}
}
]
}
}
}
# Run the query
response = client.search(index=index_name, body=query)
# Extract and print the results
hits = response['hits']['hits']
titles = ", ".join(hit['_source']['title_org'] for hit in hits)
return respond_with_message(f"Based on your prompt, I suggest the following movies {titles}")
except Exception as e:
print(f"Error running query: {e}")
return None
def respond_with_message(message):
"""
Helper function to create a response message for Lex V2.
"""
return {
"sessionState": {
"dialogAction": {
"type": "Close"
},
"intent": {
"name": "SuggestMovie",
"state": "Fulfilled"
}
},
"messages": [
{
"contentType": "PlainText",
"content": message
}
]
}
This Lambda function needs the Opensearch-py, Boto3, and Botocore libraries.
-
Install all the libraries in a folder named python and keep that folder in another folder with the name lambda-layer. Use the following command to install libraries in a specific folder
pip install opensearch-py boto3 botocore -t . Zip the entire folder Create a Lambda Layer and attach it to the Lambda function so that the function can access the libraries
Make sure to add necessary permissions to the Lambda role to invoke the bedrock models
Create a chatbot and connect it to the Lambda:
As we have the Lambda and cluster ready with the data and queries, Let’s create the bot
Visit the Lex service from the AWS Console
Click on the Create bot button and Give a name for the bot. Choose the Generative AI Creation Method
- Choose to create an IAM role and choose NO for COOPA for now
- Leave the other fields as it is and click on the Next Button. Keep the Language options as it is and provide the description for the bot to be created.
Click on the Done button and wait for some time for the bot to be created
Once the bot is created visit the Intent section of the bot from the side under All Languages
Click on the Add Intent button and choose Empty Intent from the dropdown
Provide a name for the intent Here I am giving SuggestMovie as the title
Provide the Utterance generation description like below
- Add the sample Utterance like below
- Scroll down to the end of the page Choose the Lambda function trigger
Here what we did is we have added a sample intent to the bot to identify any prompt that is similar to the prompt we provided and trigger the Lambda function we created.
Click on the save intent button and come back to the previous section
Attaching Lambda to the bot
From the side menu under the Deployment section click on Aliases and choose the aliases that are listed
Click on the Language you created under the Language section
- Choose the Lambda we created and Click on Save
- Now Click on the Aliases from the Side Menu and click on the Build Button to build the bot with all the changes
- Once the build is successful click on the Test button and provide the prompts like this
- You can see suggestion prompts like this. Now edit the fallback intent like this so that we will get some good response from the bot when user prompts other than movie suggestions
- After editing this fallback intent also build the bot once and give some random prompt to the bot and see what it is providing
- This is how the bot will respond if the prompt does not belong to a movie suggestion.
That’s it for now. I will come up with more customizations to the bot with different features in my upcoming blogs.
Meanwhile, let me know if you face any issues while trying out this task, and Feel free to comment if there are any suggestions. I am open to suggestions. Have a good day
Thanks
Top comments (2)
Hi bro,
Great job on the progress you've made so far! It's clear you're doing some fantastic work, and these improvements will only make things even better.
Here are a few suggestions to take your project to the next level:
1.Environment Configuration (Most Important): It’s a good idea to secure sensitive information like your OpenSearch credentials using environment variables or AWS Secrets Manager. This keeps things secure and makes it easier to switch between different environments without worrying about hardcoding your credentials.
2.Optimize Error Handling: It’s important to add clear error handling in your functions like generate_embedding and run_query. This way, if there’s an issue with the Bedrock API, OpenSearch, or network problems, the system can handle it gracefully without breaking down, ensuring a smoother experience.
3.Batch Embedding Generation: Instead of processing each row one by one, you can speed things up by processing them in batches. This will reduce the number of API calls and make your system more efficient, especially when dealing with large datasets.
Keep up the excellent work! These tweaks will really help boost the reliability and performance..
Thank you for your suggestions. While working on this POC, I focused primarily on functionality and did not emphasize security, batch processing, or error handling. I will address these aspects in my upcoming articles.