DEV Community: yusufgbagci

Scalable and Responsive EC2 Management Application

yusufgbagci — Fri, 17 Jan 2025 09:56:43 +0000

Introduction

Managing workloads on high-cost EC2 instances, such as the g5.2xlarge, demands a scalable and responsive approach. This article explores how a Flask web application can efficiently manage these instances, specifically focusing on a project called Metaverse Avatar running on Unreal Engine.

This project showcases a Flask-based web application designed to optimize EC2 instance management. By incorporating a suite of AWS services—DynamoDB for data storage, Cognito for user authentication, Kinesis for event streaming, and EC2 for instance management—the application ensures seamless and efficient handling of EC2 instances that are running the application. This integration not only enhances resource utilization but also ensures cost efficiency, making it a powerful tool for managing high-cost, performance-intensive workloads like in our case Metaverse Avatar.

Key Components

User Authentication
The application employs AWS Cognito for secure user authentication. This ensures that only authorized users can access and manage EC2 instances, enhancing the overall security of the system. Cognito handles the authentication flow, storing user credentials, and managing session tokens.
EC2 Instance Management
The core functionality of the application revolves around managing EC2 instances. Users can start, stop, and terminate instances based on their needs. The application interacts with AWS EC2 to perform these actions, ensuring that the instances are managed efficiently and cost-effectively.
Warm Pool Management
To reduce the startup time of new EC2 instances, the application maintains a warm pool of pre-warmed instances. This pool is managed in a separate thread to ensure it doesn't interfere with other operations. The warm pool ensures that instances are readily available, reducing the wait time for users and optimizing resource usage.
Real-time Event Streaming
For real-time capabilities, the application integrates AWS Kinesis. This service streams events and logs in real-time, allowing users to monitor the state and performance of their EC2 instances continuously. This feature is crucial for maintaining an immersive and responsive user experience.

Setup and Running

To set up and run the project, follow these steps:
Install the required dependencies:

Ensure you have Python 3. x installed.
Also other libraries:

boto3==1.34.36
botocore==1.34.36
cachetools==5.3.2
Flask==3.0.2
isodate==0.6.1
Jinja2==3.1.2
pydantic==1.10.13
python-dateutil==2.8.2
requests==2.31.0

Ensure you have the necessary permissions to access the required AWS services (EC2, the database service, authentication service, and event streaming service).

Run the Flask:

The application will start running on the https://localhost:5000.

** Algorithm Logic:

Step 1: Running the Flask Application
First, you'll need to run the application. You can do this either on an AWS instance or your local machine. Once the script is running, the Flask app will be accessible via the domain in our case:

metaverse.genai-avatar.com.

Step 2: User Authentication with AWS Cognito
To access the application, you need to log in with the credentials created in Amazon Cognito. The login page looks like this:

After logging in, you'll be redirected to the interface where you can connect or initiate an AWS instance.

Step 3: Initiating an Instance
Upon successful login, you'll see an interface like the one below:

Here, you can initiate your application instance. By clicking the 'Start' button, you initiate the instance specified by its Instance ID and AMI ID.

Step 4: Monitoring Instance Logs
While your application instance starts, you can monitor its logs in real time. This is facilitated by AWS Kinesis Data Streams, which sends logs from your application to the web UI, ensuring that the instance is initializing correctly.

This process not only helps in verifying the initialization but also ensures that any issues can be promptly identified and addressed.

Step 5: Automatic Instance Management
During the instance initialization, several key details are saved in DynamoDB table, including:

UserId
InstanceId
CreationTime
ExpireAt
PublicIp
SubnetId

These details are crucial for managing the instance lifecycle. The application provides each user with a one-hour window to use the instance. If there is no activity from the user within this period, the instance is automatically stopped to save costs.

While the logs from the application are visible in the UI, you can also monitor the status of the instances. This ensures that the instances are initializing correctly.

As shown in the screenshot, the application is running correctly so far. The instances are named according to the user authenticated via Cognito, and five warm pool instances are being initialized. This number can be configured in the configuration file. For testing purposes in this blog post, the instance type selected is t2.micro.

Once the instance status checks are healthy, you will see a message indicating that the instance is ready.

Now, to connect to your running on the instance, you can click the 'Connect' button. This will redirect you to a new domain specific to your username, where your application is running. With the this domain in our case we are able our launch the metaverse avatar. (If you're curious about how our metaverse-metahuman avatar looks and operates, please check out my other blog post )

Step 6: Disconnecting and Automatic Shutdown
When you are done using your application, you can log out. Upon logging out, the instance associated with your session will automatically shut down to save resources and costs.

This automatic shutdown ensures that no resources are wasted when the instance is not in use.

Step 7: Reusing Warm Pool Instances
Now that everything is in place and the algorithm works as expected, you might wonder what happens if you want to start the application again on an instance. Thanks to the warm pool of instances, the algorithm will not initialize a completely new instance for you. Instead, it will start one of the warm pool instances. This significantly reduces the starting time, allowing you to use your application quickly with your designated domain.

When you initiate the application again, the process will be as follows:

Starting Warm Pool Instance: The algorithm will select an available instance from the warm pool. Since these instances are pre-initialized, the startup time is minimal.
Domain Assignment: The selected instance will be associated with your domain. This means you can access your application at the same domain as before, ensuring a seamless user experience.
Quick Availability: Since the instance is already part of the warm pool, the application will be ready to use in a short time. You won't have to wait for a new instance to be initialized, which can save several minutes.

This approach optimizes both time and resources, ensuring that you can start your application promptly without unnecessary delays.

Conclusion

This Flask web application demonstrates an efficient approach to managing EC2 instances using AWS services. By leveraging AWS Cognito, DynamoDB, Kinesis, and EC2, the application ensures scalability, responsiveness, and cost-effectiveness. This project serves as a robust solution for handling dynamic workloads and managing cloud resources efficiently. With secure authentication, effective warm pool management, real-time event streaming, and seamless session management, the application offers a comprehensive and efficient tool for EC2 instance management.

Metahuman with AWS

yusufgbagci — Mon, 19 Aug 2024 14:07:31 +0000

Introduction

Integrating AWS services with Unreal Engine can significantly enhance your project's capabilities, particularly if the actors in Unreal Engine utilize AWS AI services. While Amazon GameLift is a common service for deploying, operating, and scaling dedicated, low-cost servers in the cloud for session-based multiplayer games in a UE environment, other AWS services can also be integrated based on the use case. Services like EC2, API Gateway, and S3 can seamlessly integrate with Unreal Engine to provide various functionalities.

In our work, we integrated Amazon Transcribe, Amazon Bedrock, and Amazon Polly services into Unreal Engine using the C++ SDK and Python SDK. This integration enables the creation of advanced interactive experiences within the Unreal Engine environment.

MetaHuman is a comprehensive framework that empowers creators to develop and use fully rigged, photorealistic digital humans in various projects powered by Unreal Engine. The purpose of our work is to integrate a MetaHuman with AWS to create a real-time LLM (Language Learning Model) - RAG (Retrieval-Augmented Generation) powered avatar that can interact and communicate in real time. This post will guide you through the essential steps to set up and run AWS on Unreal Engine 4.27, covering the necessary configurations, modules, and scripts that make up this powerful combination.

By following this guide, you will learn how to leverage AWS AI services to enhance the interactivity and realism of your Unreal Engine projects, making it possible to create lifelike digital avatars capable of real-time communication.

Prerequisites

Before diving into the integration, ensure you have your AWS STS (Security Token Service) token ready. This token is crucial for secure interactions with AWS services. Follow the steps below to generate and configure your AWS STS token:

Generate AWS STS Token

First, you need to configure your .config and .credentials files as explained here.

docs.aws.amazon.com

Obtain the tokens by running the following command:

aws sts get-session-token --profile <profilename> --serial-number <your mfa device in aws> --token-code <MFA_CODE>

Set Environment Variables:
After obtaining your tokens, set your environment variables using the following commands:

setx AWS_ACCESS_KEY_ID ************
setx AWS_SECRET_ACCESS_KEY ***********
setx AWS_SESSION_TOKEN **********

Unreal Engine version

Although Unreal Engine 5.4 has been announced, our integration works only with versions 4.27 or 4.26. The reason for this is the compatibility of the C++ SDK integration with these specific versions of Unreal Engine. Ensure you have one of these versions installed before proceeding with the integration.

Modules Overview

The Metahuman project consists of several essential modules, each serving a specific purpose. Let's break down these modules:

Content

Integrating Unreal Engine with AWS is a complex process that requires multiple steps. Here’s a streamlined approach to get you started:

Cloning the AWS SDK Repository
First, check if the SDK repository directory exists. If it does not, clone the AWS SDK from GitHub:

git clone https://github.com/aws/aws-sdk-cpp

Next, run CMake to configure the build system for Visual Studio:

cd aws-sdk-cpp
mkdir build
cd build
cmake .. -G "Visual Studio 16 2019"

After configuring the build system, build the SDK using MSBuild

msbuild ALL_BUILD.vcxproj /p:Configuration=Release

Finally, install the built SDK:

msbuild INSTALL.vcxproj /p:Configuration=Release

By following these steps, you’ll have the AWS SDK ready for integration with Unreal Engine. This setup ensures you have the necessary tools to proceed with more advanced AWS functionalities within Unreal Engine.

Plugins

Plugins extend the functionality of Unreal Engine, and our project will utilize the following plugin:

CallExePlugin: This custom plugin, created by Jens Krenzin, allows you to run Python scripts within your Unreal Engine project.

The plugin has two types of functionality:

Blocking Mode: While the script is running in Unreal Engine, it blocks other applications. Once the script runtime is over, Unreal Engine resumes normal operation.
Non-blocking Mode: This version does not interrupt other applications while running.

For this project, we use the non-blocking version to ensure that MetaHuman can function without any interruptions.
In this plugin, you need to specify the directory where Python is set up on your local PC as well as the directory of the Python script you want to run. This configuration ensures that the plugin can locate and execute the necessary Python scripts seamlessly within your Unreal Engine project.

Here's a brief overview of how to set it up:

Python Directory: Provide the path to your local Python installation.
Script Directory: Specify the directory where your Python scripts are located.
By configuring these directories, the CallExePlugin will be able to execute your Python scripts, extending the functionality of your Unreal Engine project without interruptions.

Scripts

The Scripts directory houses various Python scripts essential for the project's core features. Here’s an overview of the key script:

main.py: This Python file takes the user's voice input and converts it to text using Amazon Transcribe. The transcribed text is then sent to Amazon Bedrock for further processing.

Voice to Text: Amazon Transcribe's real-time feature is used to convert the user's voice into text. This allows for immediate processing and response.
Text to LLM: The transcribed text is forwarded to Amazon Bedrock. Bedrock is an AWS service that hosts multiple language models (LLMs).

In our project, we use Claude 3 Sonnet, located in the Frankfurt region.
This seamless integration between Amazon Transcribe and Amazon Bedrock enables real-time interaction, allowing the LLM to provide immediate and relevant responses. This setup ensures a smooth and efficient workflow, enhancing the overall user experience.

Blueprint Communication and Polly Integration

Each avatar has its own Blueprint that must connect to the Scene-Blueprints (BP_UIWidget) for user interaction.

The Metahuman Blueprint is the main place to set up all AWS services and plugins. Here, we configure AWS Polly using the C++ SDK.
Amazon Polly provides two important capabilities that make this project work seamlessly. First, Polly can turn text into natural-sounding speech as an audio file. Second, Polly can generate a JSON-formatted list of mouth shapes corresponding to the sounds in that audio file. These mouth shapes are called "visemes." Here are what the visemes for the word "human" look like:

{"time":2,"type":"viseme","value":"o"}
{"time":52,"type":"viseme","value":"k"}
{"time":196,"type":"viseme","value":"p"}

Because this viseme data includes timestamps for each mouth shape, we can use that information to achieve believable lip sync between the audio playback and MetaHuman facial animations.

We created a Generate Speech function, which takes the text from our CallExePlugin and creates speech using Amazon Polly. This function is connected to another function that updates the AnimGraph. Specifically, it updates the "Viseme" variable to drive a Blend Pose node. This node determines which one of the possible viseme animation assets will be applied to the MetaHuman's face.

By leveraging Amazon Polly's capabilities, we ensure that the MetaHuman's lip movements are synchronized with the generated speech, creating a realistic experience.

Conclusion

Integrating AWS with Unreal Engine 4.27 involves several crucial steps, including configuring AWS STS tokens, setting up essential modules, and utilizing plugins and scripts to enhance MetaHuman capabilities.

In our use case, we configured our LLM (Language Learning Model) in a CEO role and created a RAG (Retrieval-Augmented Generation) system with the necessary data. This configuration allows our MetaHuman to act as a company's CEO. Users can interact with the avatar by asking questions about the company via voice, and the avatar responds with natural-sounding speech.

Here's a video showing how we interact with our avatar: https://www.youtube.com/watch?v=URaJv-YHVT8

Although I am publishing this article, there is a great team behind this project. Thanks for the amazing work: Artur Schneider, Lydia Delyova, Jens Krenzin, and Willi Schwarzbach.