AWS Learning Experience with Streamlit (This Time We Connect To AWS)

In my very last post (click here to read it), I showed you how to create the application using mock AWS using Motto which simulates AWS via an SDK. Now, I'm going to show you how to do it via AWS (yay!)

It uses AWS S3 as the place to store learning resources while using AWS Lambda to process the data coming from the user. The recommender currently applies the TF-IDF vectorization technique and cosine similarity in ranking.

Let's do it, shall we?

1. Set Up The S3 Bucket

Create the S3 Bucket for storing data:

aws s3 mb s3://learning-paths-data

Upload the learning resources JSON:

echo '{
    "resources": [
        {"title": "Introduction to AWS", "tags": "AWS, Cloud Computing"},
        {"title": "Deep Learning on AWS", "tags": "AWS, AI, Deep Learning"}
    ]
}' > resources.json
aws s3 cp resources.json s3://learning-paths-data/

2. Create the lambda function

Write the lambda function in order to process the input:

import json
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

def lambda_handler(event, context):
    user_input = event['query']
    s3 = boto3.client('s3')
    obj = s3.get_object(Bucket='learning-paths-data', Key='resources.json')
    resources = json.loads(obj['Body'].read().decode('utf-8'))['resources']
    titles, tags = zip(*[(r['title'], r['tags']) for r in resources])

    tfidf = TfidfVectorizer().fit_transform(tags + [user_input])
    scores = cosine_similarity(tfidf[-1], tfidf[:-1]).flatten()
    recommendations = [titles[i] for i in scores.argsort()[-3:]]
    return {'recommendations': recommendations}

Use this function and enable its access through API Gateway.

3. Change the Streamlit app accordingly

Change the app based on the query from the lambda function:

import requests

def main():
    user_input = st.text_input('What do you want to learn about AWS?')
    if st.button('Recommend'):
        response = requests.post('<API Gateway URL>', json={'query': user_input})
        recommendations = response.json()['recommendations']
        for rec in recommendations:
            st.write(rec)

if __name__ == '__main__':
    main()

These projects illustrate how to use AWS services to develop resilient and outsized applications. Try them via my Github repo here Thanks for reading!