DEV Community: Awwal

Deploying Text to Image App on ECS

Awwal — Wed, 18 Sep 2024 20:17:19 +0000

Lets start by creating a cluster. Cluster

Go to the Amazon ECS service page. Click clusters to create a new cluster.

Give your cluster what ever name you like
We leave other options as default.
Scroll to the bottom and click create

Creating Task Definition

Click task definitions on the left menu.
On the task definition screen click on create new task definition
We will use an already built docker image on Amazon ECR (Elastic Container Registry) to save time. link to git page of the streamlit app code
Under Container 1 use the public image URI public.ecr.aws/o3s1z9h7/build-on-poster:latest

Give your task definintion a name

Leave the rest of the pages configuration as default, scroll to the bottom and click Next.
Under Environment variables click Add environment variable button.
For Variable Name use
IMAGE_API and TEXT_API

Since we are utilizing code from someone else’s public container image, it is essential that our variable names align with those used in the code. Let’s create two variables: one to call our stable diffusion lambda function and the other for Text Summarization lambda function.

Leave the remaining configurations as default, scroll to the bottom and click Next.

Go to the IAM service page or scroll to Task roles above Task role click the link to the IAM console to navigate to the page (to grants permission to containers enabling them interact with other AWS services).
Select create role on the IAM page
Leave the defaults select elastic container registry for service below. Then scroll down click next

Under services select Elastic container registry

In the Add permissions page search for and select AmazonECSTaskExecutionRolePolicy

Clear filter after selecting AmazonECSTaskExecutionRolePolicy.

Search for lambd_fullAccess and select it. To grant ECS containers unrestricted access to our lambda functions.
Click Next

Give your role any name

Scroll to the bottom click create role.

After successful Role creation we move back to ECS Task definition and select our newly created role under Task role.

You can turn off cloudwatch log collection under Logging to save cost.

Click Next below to proceed

Review

Click create below

Running A Task with Our Task Definition

On the left menu click cluster

Click the cluster you created earlier on the cluster page

Click Tasks tab as shown above, then click the Run new task button

On the create task page.

Under Compute configuration, select Launch type as your compute option to keep things simple.

Under Deployments select the task definition we just created using the drop down as shown below.

Under Networks
We could either edit our default security group to allow inbound traffic or create a new one which does. We would create a new one.
Select create a new security group under security group

Give your security group a name
Give your security group any description.

For inbound rules under Type select HTTP
For Source select "Anywhere"
Also ensure the Public IP button is toggle On, so we know our IP address and connect to it.

Leave Task overrides as default

Edit container overrides
We copy the output url of the lambda functions we created in the previous post to summarize text and the function to convert text to image as values of our container environment variables.

Copy the Function url of both functions and paste them into container IMAGE_API and TEXT_API values.
One at a time

Use function url of diffusion function as value of IMAGE_API and the function url of text_summary function as value of TEXT_API.
Click save button to run task. After pasting function url your enviroment variables should like the image below.

Test your site
Under Tasks click your task when it gets to the Running state

Scroll down on the task page
Under Configuration setting
Click the public url

It displays loads our site on a new page

Lets test it
paste or write an article to be summarized and converted to text

You can use an article here

Comprehensive Tutorial: Deploying SageMaker Endpoint on Lambda for AI-powered Text Summarization and Image Generation

Awwal — Thu, 13 Jul 2023 10:14:35 +0000

Deploying Sagemaker Endpoint on Lambda

To deploy a SageMaker endpoint on AWS Lambda, follow these steps:

Log in to your Amazon Web Services (AWS) account and search for "Lambda" in the search bar. Click on the "Lambda" service to access the Lambda console.
On the Lambda console, click on the "Create function" button to create a new function.
Create two functions as follows:

a. Function 1: This function will interact with your stable diffusion endpoint by sending text and receiving image outputs. You can choose any name for this function, such as "create_image_func".

b. Function 2: This function will interact with the flan-T5 XL model endpoint to produce a summarized text of the initial input text. Name this function "create_summary_func" or any other name you prefer.

On the "Create function" page, leave the defaults as they are.
Under the "Basic information" section, provide a name for your function.
Next, select the runtime for your function. In this case, choose Python 3.9.
Proceed with the remaining steps and configurations required to complete the creation of your Lambda function.

Creating an IAM Role for Lambda

To allow the Lambda function to interact with other services, you need to create an IAM role. Follow these steps:

Go to the AWS Identity and Access Management (IAM) page. You can find it in the left navigation pane under "Access Management." Click on "Roles" to access the roles page.
On the roles page, click the "Create role" button.
Leave the Trusted entity type as "AWS service" and select "Lambda" under common use cases.
Click the "Next" button.
On the next page, add permissions to the role. Perform the following steps:

a. In the search bar, type "SageMakerFullAccess" and add the "SageMakerFullAccess" policy.

b. Click "Clear filters."

c. Search for "S3FullAccess" and select the "S3FullAccess" policy from the results.

d. Click "Clear filters."

e. Search for "IAMFullAccess" and select the "IAMFullAccess" policy.

f. Search for "AWSLambdaBasicExecutionRole" and select the "AWSLambdaBasicExecutionRole" policy.

g. Click the "Next" button.

Give the role a name in the "Role name" text box under Role details.
Review the policies you have added to the role.
Click the "Create role" button to create the role.

You will have successfully created an IAM role to be used by your Lambda function for interacting with other services.

Returning to the Lambda Function

Scroll to the bottom of the page and select the "Enable function URL" option. This will generate a unique URL for your Lambda function to invoke it later.
Click the "Create function" button located below.
You will be redirected to the Lambda function page. On the left navigation pane, click on the "Code" tab.
In the "Code" tab, locate and click on lambda_function.py to open the file.
Erase the existing code in lambda_function.py.
Copy the code from the provided GitHub link GitHub link and paste it into the lambda_function.py file.



import json
import boto3

def parse_response(query_response):
    """Parse response and return generated image and the prompt"""

    response_dict = json.loads(query_response)
    return response_dict["generated_image"], response_dict["prompt"]


def lambda_handler(event, context):

    client = boto3.client('sagemaker-runtime')

    data_string = event["body"]
    text = json.loads(data_string)["text"]

    encoded_text = text.encode("utf-8")


    CONTENT_TYPE = 'application/x-text'
    endpoint_name = # "YOUR_MODEL_ENDPOINT" # Replace with your model endpoint

    response = client.invoke_endpoint(EndpointName=endpoint_name, 
                                  ContentType=CONTENT_TYPE, 
                                  Body=encoded_text)

    print(response)

    response_payload = json.loads(response['Body'].read().decode("utf-8"))
    print(text)

    resp = json.dumps(response_payload)

    return {
        'statusCode': 200,
        'body': resp
    }

Once you have completed these steps, you will have returned to the Lambda function and updated the code with the new content.

Copying the Stable Diffusion Endpoint Name

Go to the Amazon SageMaker console to find the name of the stable diffusion endpoint you want to use.
Scroll down to "Inference" on the left menu and click on "Endpoints".
Copy the endpoint name of the stable diffusion jumpstart (pretrained) model that was deployed in the previous tutorial.
(the image highlights flan 5 but copy the name of the stable diffusion image function)
After Copying the endpoint name from the SageMaker console. You will need to paste it into your create_image_func code around line 22.
Click the "Deploy" button to save the changes in the function.

Testing the Function

Click the "Test" button to create a test for the function.
Give your test a name.
Paste the following JSON into the JSON box for the test:

{
"body": "{\"text\":\"clouds over the ocean\"}"
}

![Image description: Configuring test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sk26j973i9aqx1o4en16.png)

4. Click the "Save" button to create the test event.

5. Click the "Deploy" button to save the code changes in the function and then click the "Test" button to run the test. Note that the test may fail due to a timeout error in the function. We will address this issue in the next steps.

## Configuring Timeout

1. Under the Configurations tab, click on "General configurations".

2. Click the "Edit" button.
![Image description: Edit General Configurations](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/puw5qi9z7w4u04mwpyft.png)

3. Increase the timeout value to a higher value to prevent the timeout error. Adjust it according to the requirements of your function.
![Image description: Increase Timeout](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3jbixh8ky2iw7cenka0p.png)

4. Click "Save" to save the changes.


## Testing the Function Again

1. Proceed to test the function by clicking the "Test" button either on the "Test" tab or the "Code" tab. The output of the test should now run successfully without the timeout error.
![Image description: Image Test output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vh8c0xhaesn53kkvdq30.png)

## Configuring CORS (Cross-Origin Resource Sharing)

If your function doesn't have a URL. Under "Configuration", follow these additional steps:

1. Select "Function URL" from the left menu.

2. Click on "Create Function URL".

3. Under Additional Settings, select "Configure cross-origin resource sharing (CORS)".

4. Scroll down and click the "Save" button.

The Function URL should now appear under "Configuration".

Once you have completed these steps, you will have copied the stable diffusion endpoint name, updated the code in the Lambda function, tested the function, configured the timeout, and configured CORS.

## Creating the Summary Function

1. On the Lambda landing page, click on "Create function" to start creating a new function.
![Image description creation lambda function for flan model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ce1iqhlyvjozystzorvf.png)

2. Repeat the steps used to create the image function (create_image_func) above to create the summary function. Give the function a name of your choice.
![Image description: Creating a Lambda function for the Flan model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ce1iqhlyvjozystzorvf.png)

3. Under Permissions, select the execution role that was created for the Lambda function.
![Image description: Selecting the Lambda execution role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6irnuvkjgin0fuh3hpe4.png)

4. Under "Advanced settings," select "Enable function URL" to enable invoking the function using a URL. Also, select "Configure CORS" to configure cross-origin resource sharing for the function.
![Image description: Enabling function URL and configuring CORS](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/97dnwxfw92wkkl0syslu.png)

5. In the Code section, click on `lambda_function.py` in the left menu. Replace the code in `lambda_function.py` with the code from the provided GitHub [link](https://github.com/build-on-aws/building-gen-ai-apps-with-found-models-workshop/blob/main/fm_create_summary.py) or below.
![Image description: Replacing the code with the Flan Lambda code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wz8amvahhl3vc9mx9adb.png)

import json
import boto3

def lambda_handler(event, context):

client = boto3.client('sagemaker-runtime')

data_string = event["body"]
text = json.loads(data_string)["text"]

query = 'write a summary'

prompt = f'{text}\n{query}'

MAX_LENGTH = 150 # 256
NUM_RETURN_SEQUENCES = 1
TOP_K = 0
TOP_P = 0.7
DO_SAMPLE = True 
CONTENT_TYPE = 'application/json'


payload = {'text_inputs': prompt, 
      'max_length': MAX_LENGTH, 
      'num_return_sequences': NUM_RETURN_SEQUENCES,
      'top_k': TOP_K,
      'top_p': TOP_P,
      'do_sample': DO_SAMPLE}


payload = json.dumps(payload).encode('utf-8')

endpoint_name = # "YOUR_MODEL_ENDPOINT" # Replace with your model endpoint

response = client.invoke_endpoint(EndpointName=endpoint_name, 
                              ContentType=CONTENT_TYPE, 
                              Body=payload)
model_predictions = json.loads(response['Body'].read())
generated_text = model_predictions['generated_texts'][0]
print(f'Response: {generated_text}')

resp = json.dumps({"text":generated_text})


return {
    'statusCode': 200,
    'body': resp
}



6. Replace line 33 in the code with the endpoint name of your Text Model endpoint. To find the endpoint name, follow these steps:

   a. Go to the SageMaker console.
![Image Sagemaker console endpoint nav pane](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/njjgfdrwk267fcq3ozh0.png)

   b. Scroll down to "Inference" on the left menu and click on "Endpoints".

   c. Copy the endpoint name of the Text-to-text jumpstart (pretrained) model that was deployed in the previous tutorial.
![Image description coping highlighted endpoint name from SageMaker console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l35s1x7frd1eu5sd65vc.png)

7. After pasting the endpoint name into your code, it should look like the code snippet provided.
![Image description: Replacing the endpoint name in the code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pus6vgs8kje60hbt2p0a.png)

By following these steps, you will have created the summary function in AWS Lambda, updated the code with the provided GitHub link, and replaced the endpoint name with the one from your Text Model endpoint in SageMaker.


## Testing the Function

1. Click the blue "Test" button above to create a new test function.
![Image description: Test button](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x1tern25oq19xb0li3a3.png)

2. Give your test a name.

3. Replace the default test values with the following JSON text (you can change the text to anything you want to summarize):

{
"body": "{\"text\":\"The internet fundamentally changed things. We weren’t only sending messages back and forth anymore; we were visiting digital locations called websites. These websites needed a place to store their text and images, and eventually they started doing more than displaying stored information; they began to facilitate sales, coordinate deliveries, and much more. All the storage and computing for these countless websites had to come from somewhere, and that somewhere was a server.\"}"
}

![Image description: flan-5 test event configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xj6u5iw3y7v4kz6rxb9w.png)


4. Click the "Save" button to create the test event.

## Deploying and Testing the Code

1. Click the "Deploy" button to save the code changes.

2. Click the "Test" button to test the function with the created test event.
![Image description: Deploying and testing the function](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8qb5s8jiudiarpspjcpr.png)

Additionally, to avoid timeout errors, you can increase the timeout value to about 30 seconds. Configure the timeout as shown in the previous instructions.

After testing the function, the expected result should indicate successful execution, similar to the following:
![Image description: Successful execution of the Flan test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8hao0sjvscxk8a7z4u1f.png)

Please note that the actual output may vary based on the specifics of your Lambda function and the code you have implemented.

If you encounter a timeout error, it may be due to various factors such as the complexity of the task or insufficient resources. Increasing the timeout and configuring it appropriately can help resolve such issues.

Comprehensive Guide: Deploying Prebuilt Models using Amazon SageMaker

Awwal — Thu, 13 Jul 2023 10:13:53 +0000

Objective

The objective of this article is to provide a step-by-step guide on deploying an end-to-end machine learning model using a prebuilt AWS Jump Start model. The focus will be on leveraging low-code and no-code options provided by AWS. The article covers various aspects of the deployment process, including the creation of a SageMaker domain, deploying Jump Start models to SageMaker Inference Endpoint, and interacting with the deployed model in an open SageMaker notebook.

Introduction

Overview:

Source

This article is part of a three-part series that aims to demonstrate how to deploy machine learning models using AWS's low-code and no-code options. Specifically, this article focuses on deploying a prebuilt AWS Jump Start model and hosting it on a Streamlit server.

In this part of the article series, you will learn the following based on SageMaker:

How to create a SageMaker domain.
How to deploy Jump Start models to the SageMaker Inference Endpoint.
How to interact with the deployed model in an open SageMaker notebook (optional).

Assumptions: You are familiar with navigating the AWS console and have an AWS account with SageMaker Full Access. If you don't have an account, you can sign up for one using the following link

Cost:
Please note that the labs from the workshop page costs around $10 if completed within 2 hours.

How to Get Free Access SageMaker (Optional):

Utilize the AWS Free Tier.
For students, consider joining the AWS DeepRacer Challenge or AWS Educate.
Participate in AWS events to receive free credits.
Attend AWS workshops to gain access to temporary accounts.

Jump Start

SageMaker Jump Start is a repository of pretrained end-to-end models that serve as baseline models for building upon. This low-code, no-code solution allows you to quickly leverage ready-to-use models for various use cases, including tabular regression, classification, and state-of-the-art models from Amazon and its partners like Stability AI, Cohere, AI21, and AlexTM. You can also enhance these models by performing additional training on them. Examples of JumpStart models can be found on the JumpStart landing page, which showcases available end-to-end machine learning solutions, pretrained models, and example notebooks.

Domain

In SageMaker, a domain refers to a logical grouping of files and resources within a workspace. It provides a structured environment for managing machine learning deployments. Each domain in SageMaker consists of a collection of resources specifically tailored to facilitate machine learning workflows.

Model Endpoint

A SageMaker Endpoint is an API that allows you to access a model for inference. It serves as the interface for making predictions using your trained model. When deploying a model, SageMaker provides the necessary infrastructure based on your chosen deployment type. For prebuilt models, the infrastructure specifications are already defined. However, if you are creating your own model, you will need to specify the infrastructure type required to host your model. SageMaker leverages Docker containers for hosting the models, ensuring scalability and flexibility.

Model Parsing

Model parsing is the process of interpreting and analyzing the structure and content of a model. It involves understanding the input and output formats, data transformations, and any pre- or post-processing steps required. Accurate model parsing is essential for seamless integration and efficient utilization of the model.

Stable Diffusion

Stable Diffusion is a text-to-image model that utilizes generative modeling techniques to generate photorealistic images based on text prompts. It offers a powerful solution for transforming textual descriptions into visually compelling images. To learn more about Stable Diffusion and its capabilities, refer to this blog

Creating a SageMaker Domain

Open the AWS Management Console and enter "Sagemaker" in the search bar. Click on "Amazon SageMaker" to access the SageMaker console.
On the left navigation pane, click on "Domains" to go to the SageMaker Domain configuration page.
Create a new domain by clicking on the "Create domain" button. Then, click on "Quick setup" (default) for easier domain configuration.
In the "Quick Setup" section, leave the default settings as they are.
Enter a name for your domain in the "Domain name" text box.
Under "User profile," you can enter a friendly user name or use the default name provided.
In the "Execution role" section, click on the "Create new role" option.
In the "Create an Execution role" dialog, select "Any S3 bucket" and leave the defaults as they are. Then, click the "Create role" button.
You'll see a message confirming the successful creation of the IAM role.
Scroll to the bottom of the page and click the "Submit" button to create the domain.
Wait for the domain creation process to complete. Once the domain is created, click on the domain you just created.

By following these steps, you will be able to create a SageMaker domain and proceed with configuring and utilizing it for your machine learning workflows.

Launching SageMaker Studio

On the SageMaker Domain page, click on the newly created domain. This will redirect you to the domain details page.
Image description: Domain details page
Look for the "User profiles" section on the page.

Under "User profiles," click on the "Launch" button located on the right side of the page.

From the dropdown list that appears, select "Studio." This will launch a SageMaker Studio notebook.

Please note that launching the SageMaker Studio may take some time to complete.

Once SageMaker Studio has launched successfully, you will be redirected to the SageMaker Studio landing page, as shown below.

Deploying JumpStart Models

On the SageMaker Studio page, navigate to the left navigation pane and click on "SageMaker JumpStart" to expand the menu.
Click on "Models, Notebooks, Solutions" to access the JumpStart models.
On the next screen, select "Stable Diffusion 2.1 base" from the available models, as shown in the image below.
On the Stable Diffusion details page, click the "Deploy" button to deploy the model. This will create an endpoint that can be used to interact with the model. The deployment process will also launch an instance to host the endpoint.
Click "Deploy" to initiate the deployment process. Please note that the deployment may take some time to complete.

By following these steps, you will be able to deploy the JumpStart model, specifically the Stable Diffusion 2.1 base model. Once deployed, you will have an endpoint that can be used to interact with the model for inference tasks.

Setting Up the Model's Notebook

Once the endpoint is deployed and in service. You can click "Open Notebook" button to interact with the model.
After starting the model's new notebook, please allow some time for the notebook to initialize.
Scroll down to the setup section and run the first cells as indicated in the notebook.

Please note: Running the first cells and testing the model on a notebook are optional and can be done if desired.

Deploying the Flan-T5 XL Model

To deploy the Flan model, follow these steps:

Search for "Flan-t5 xl" in the available models.
Click on the "Flan-t5 xl" model to access the Flan-t5 xl model details page.
On the Flan model details page, click on the "Deploy" button to initiate the deployment process.
The flan-t5 xl model is deployed just as the stable diffusion model was deployed.

By following these steps, you will be able to set up the model's notebook, run the necessary setup cells, and deploy the Flan model for further usage.

Viewing Endpoints

On the SageMaker homepage, navigate to the "Inference" section and select "Endpoints". This will take you to the page displaying all the endpoints.
The endpoints page provides an overview of the endpoints that have been created. This will be useful in Part 2 of this tutorial.

By following these steps, you will be able to navigate to the endpoints page on the SageMaker homepage, which will be useful for the subsequent parts of this tutorial.