DEV Community: Toyyib Muhammad-Jamiu

From Notebook to Production: How Amazon SageMaker Simplifies Machine Learning Deployment

Toyyib Muhammad-Jamiu — Sun, 05 Jan 2025 16:04:21 +0000

INTRODUCTION

Building machine learning models is only part of the journey; deploying them for real-world use is the real challenge.

Many people are familiar with tools like Streamlit, which is great for creating quick prototypes or demos. However, they lack the scalability and features needed for production-level deployments.

This is where Amazon SageMaker stands out. It’s designed for building, training, and deploying machine learning models at scale, providing the infrastructure necessary for real-world applications.

Amazon SageMaker is a fully managed service that helps data scientists, ML practitioners, and AI professionals quickly and easily manage the entire ML workflow; from data preparation and model training to deployment and monitoring. It ensures your models are production-ready, with high availability and scalability built in.

In this blog, we will explore how Amazon SageMaker offers the tools and features needed to take machine learning models from prototype to production, making it the go-to choice for large-scale deployments.

PREREQUISITES

AWS Account
Basic Knowledge of Machine Learning
Data for Training
IAM Roles and Permissions: Create an IAM role that grants SageMaker access to necessary AWS resources like S3, EC2, and CloudWatch.

Optional but Recommended:

Basic Cloud Computing Knowledge: Familiarity with AWS services like S3, EC2, and IAM can make working with SageMaker easier.
Jupyter Notebooks: Experience using Jupyter notebooks is helpful as SageMaker Studio provides an interactive notebook environment.

1. Setting Up Your SageMaker Environment

Navigate to SageMaker Console: Go to the AWS Management Console and search for Amazon SageMaker AI to get started.

Create a New Notebook Instance: SageMaker provides Jupyter Notebooks, making it easy for practitioners to create, train, and evaluate ML models.
Click on Notebook instances, then click Create notebook instance.

Choose an appropriate instance type (e.g., ml.t2.medium for small workloads).

Set IAM roles and permissions to allow SageMaker to access other services (e.g., S3 buckets for data).

For Root Access: enabling or disabling it is a function of your use-case.

a. For development and experimentation, enable root access only if necessary and restrict permissions to trusted users.

b. For production-level notebooks or environments with sensitive data, disable root access to reduce security risks.

c. If unsure, start with root access disabled and enable it later if specific requirements arise.

Launch the notebook and access it through the console for coding.

2. Preparing Data for Training

Machine learning begins with data, and SageMaker makes it easy to prepare, clean, and transform data. Here's how:

a. Data Storage (Amazon S3): Store your data in Amazon S3 and ensure it is accessible by SageMaker. You can use S3 to upload and organize datasets.

b. Data Preprocessing: You can preprocess data using SageMaker's built-in Processing Jobs, which allow you to run data transformation tasks in parallel on managed compute resources.

You can also use pre-built containers or your custom code for tasks like cleaning, normalization, and feature engineering.

Example: Use Pandas or NumPy for cleaning and preprocessing data directly within your SageMaker notebooks.

c. Data Wrangling with SageMaker Data Wrangler: Data Wrangler is an interactive tool that allows you to import, clean, and transform data with just a few clicks, providing an intuitive interface for data manipulation.

It also supports export to S3 for easy integration with the SageMaker training pipeline.

Steps to Access Data Wrangler

Launch from SageMaker Studio

a. Open SageMaker Studio:

Go to the Amazon SageMaker Console.
Under "SageMaker Studio," click "Launch SageMaker Studio".

b. Access Data Wrangler:
In SageMaker Studio, click File > New and select "Data Wrangler Flow".

This will open the Data Wrangler interface where you can prepare, analyze, and visualize your data.

3. Model Building and Training

Once the data is ready, you can use SageMaker to build and train models efficiently:

Built-in Algorithms: SageMaker offers a variety of pre-built, high-performance algorithms (e.g., XGBoost, Linear Learner, K-Means, and Factorization Machines) that are optimized for speed and scalability.
Custom Models: You can bring your own code to train models in TensorFlow, PyTorch, MXNet, and other popular frameworks using SageMaker Script Mode or Estimator API.

SageMaker handles scaling and infrastructure behind the scenes, making it easier to focus on model design.

Distributed Training: For large datasets or deep learning tasks, you can use distributed training on SageMaker’s distributed training infrastructure to speed up training times.
Automatic Model Tuning: SageMaker offers Hyperparameter Optimization (HPO), allowing you to automatically search for the best hyperparameters for your model, helping improve performance without manual tuning.

4. Model Deployment

After training your model, the next step is deployment. SageMaker offers a managed environment for deploying models:

Real-time Inference: Deploy your trained model to SageMaker Endpoints for real-time predictions. You can expose an HTTP API that can be called from web or mobile applications. Real-time Inference is best for low-latency, high-availability scenarios like live applications or APIs.
Batch Transform: For large datasets or non-real-time predictions, use Batch Transform to run inference on large volumes of data efficiently.It is deal for non-real-time predictions, large datasets, or scheduled batch processing.
Multi-Model Endpoints: SageMaker now supports multi-model endpoints, allowing you to deploy multiple models on a single endpoint, optimizing resource usage and reducing deployment costs. It is optimized for scenarios where multiple models are required but need to share resources, such as A/B testing or multi-tenant applications.

5. Model Monitoring and Management

Once deployed, it’s crucial to monitor model performance and manage its lifecycle:

SageMaker Model Monitor: Use Model Monitor to detect data drift, anomalies, and performance degradation. It automatically compares incoming data to the training dataset, alerting you if the model's performance is declining.
SageMaker Debugger: This tool helps track model training metrics and helps debug your model in real-time, allowing you to make adjustments and improve performance before deployment.
SageMaker Pipelines: Automate and manage the end-to-end ML lifecycle, from data preparation to model deployment. This helps maintain consistency in deployment and model training workflows.

Below is a schematic diagram showing the Amazon Sagemaker Workflow that manages the end-to-end ML lifecycle.

6. Collaborating with Teams

Data science and ML projects often involve collaboration. SageMaker enables teams to work together on various stages of the ML lifecycle:

SageMaker Studio: SageMaker Studio is an integrated development environment (IDE) that provides a unified visual interface for data science and machine learning workflows. It allows you to access notebooks, manage code, and track experiments all in one place.
Version Control: You can use Git integration within SageMaker to manage and version control your models and notebooks, allowing easy collaboration.

How to Integrate Version Control for easy collaboration.

a. Git Integration in SageMaker Studio:

i. Open SageMaker Studio.
ii. Navigate to the "File Browser" or "Launcher" and select the terminal.
iii. Configure Git using terminal commands:

git config --global user.name "Your Name"
git config --global user.email "youremail@example.com"

iv. Clone a Repository:

Use the terminal to clone a Git repository:

git clone https://github.com/your-repo.git

You can now work on notebooks or files within the cloned repository.

v. Commit and Push Changes:
After making changes, use standard Git commands:

git add .
git commit -m "Your commit message"
git push origin main

vi. Track Changes:

Collaborators can pull updates or resolve conflicts using Git commands directly within SageMaker Studio.

b. Git Integration in SageMaker Notebook Instances

For notebook instances, you can also manually integrate Git by installing it and running Git commands through the terminal.

How to Enable Git in Notebook Instances:

i. Launch a notebook instance and open the terminal.
ii. Install Git if not already installed:

sudo yum install git -y

iii. Configure your Git credentials:

git config --global user.name "Your Name"
git config --global user.email "youremail@example.com"

iv. Clone a repository and manage version control as you would on any local machine.

c. Using AWS CodeCommit for Managed Git Repositories

If you want to use an AWS-native Git solution, SageMaker can integrate with AWS CodeCommit, a fully managed source control service.

Steps:
i. Create a repository in CodeCommit.
ii. Clone the repository in SageMaker Studio or a notebook instance.
iii. Use Git commands to manage and version control files.

SageMaker Projects: This feature helps you set up repeatable machine learning projects with pre-defined templates, improving collaboration and workflow consistency.

7. Cost Optimization with SageMaker

SageMaker also provides several features to help optimize costs during development and deployment:

Spot Instances: Use SageMaker Managed Spot Training to reduce training costs by up to 90%. Spot instances allow you to take advantage of unused EC2 capacity at a reduced cost, though they can be interrupted.
Instance Types: Choose the right instance size and type based on your workload. For smaller tasks, start with ml.t2.medium, and scale up to more powerful instances (e.g., ml.p3.2xlarge) for larger workloads.
Model Optimization: Use SageMaker’s Neo feature to optimize machine learning models for faster inference at a lower cost, without sacrificing accuracy.

CONCLUSION

Amazon SageMaker is an incredibly powerful and flexible tool for data scientists and AI/ML practitioners, helping you streamline every aspect of the machine learning lifecycle, from data preparation and model building to deployment and management.

It provides a vast range of features, including pre-built algorithms, powerful training environments, and easy-to-use deployment options, making it easier for you to create, optimize, and deploy AI/ML solutions at scale.

With its integrated tools for monitoring, collaboration, and cost optimization, SageMaker is designed to accelerate AI/ML workflows while helping you manage costs efficiently.

Optimizing AWS Costs: Practical Tips for Budget-Conscious Cloud Engineers

Toyyib Muhammad-Jamiu — Sun, 05 Jan 2025 13:09:50 +0000

INTRODUCTION

AWS provides a flexible cloud environment, but this flexibility can lead to significant costs if not carefully managed.

Cost optimization is about aligning resources and architecture with application needs while leveraging AWS' pricing options to minimize expenses.

Cost optimization in AWS requires a strategic approach to resource management, pricing options, and architectural choices.

Below is a list of practical steps with actionable tips to help reduce AWS bills.

1. Use AWS Cost Management Tools

AWS provides a set of cost management tools that help monitor and analyze expenses:

AWS Cost Explorer

AWS Cost Explorer helps visualize AWS costs and usage patterns over time.

Steps involved in using AWS Cost Explorer:

a. Go to AWS Billing Console > Cost Explorer.

b. Set custom date ranges, filter by services or accounts, and visualize costs.

c. Use Cost Explorer’s forecast feature to predict future expenses based on historical usage.

Tip: Regularly review Cost Explorer reports to identify and eliminate unnecessary resources.

AWS Budgets

AWS Budgets allow you to set custom cost and usage budgets.

Steps involved:

a. In the Billing Console, navigate to Budgets and choose Create a Budget.

There are several budget templates here:
i. Zero-Spend Budget: Suitable for users on Free-tier.

It creates a budget that notifies you once your spending exceeds $0.01 which is above the AWS Free Tier limits.

ii. Monthly Cost Budget: It creates a monthly budget that notifies you if you exceed, or are forecasted to exceed, the budget amount.

iii. Daily Savings Plans coverage budget: It creates a coverage budget for your savings plans that notifies you when you fall below the defined target.

iv. Daily reservation utilization budget:
Create a utilization budget for your reservations that notifies you when you fall below the defined target.

You can also customize your budget yourself rather than using these set templates by opting for the Customize option.

b. Set a budget amount, specify cost thresholds, and set up alerts.

Taking Zero-Spend as an example, the budget is already set at $0.00 and the cost threshold is $0.01.

You can give your Zero-spend budget a name and enable an email notification by adding an email address to notify you whenever you go beyond the AWS Free-tier limit.

Then click on Create budget.

Tip: Use AWS Budgets to enforce a spending cap across teams or projects.

2. Rightsize Instances with AWS Compute Optimizer

AWS Compute Optimizer suggests instance sizes based on your workloads, helping you select the optimal EC2 instance type.

Steps to Use Compute Optimizer:

In the AWS Management Console, go to AWS Compute Optimizer.
Select Get Started, Opt in for Only this Account, then EC2 Instances.

Review instance recommendations, which are categorized as under-provisioned, over-provisioned, or optimally provisioned.

Resize or change instance types based on the recommendations.

Tip: Regularly review recommendations to resize instances as needs change. Switching to the recommended instance size can lead to considerable cost savings.

3. Use Reserved Instances and Savings Plans

Reserved Instances (RIs) and Savings Plans provide significant discounts in exchange for a commitment to a specific usage amount or duration.

Reserved Instances
What they are: Purchase commitments for EC2 instances at reduced rates for 1- or 3-year terms.

How to purchase:
In the EC2 Console, navigate to Reserved Instances.

Choose an instance type, region, and term (1 or 3 years).

Select a payment plan (All Upfront, Partial Upfront, or No Upfront).

Tip: Use RIs for predictable, consistent workloads.

4. Savings Plans

Savings Plans offer flexible pricing models with a commitment to specific usage for 1 or 3 years across EC2, Lambda, and Fargate.
It is part of the budget templates that was briefly discussed under the AWS Budget Section above.

How to set up:

a. Go to AWS Billing Console > Savings Plans

b. Choose a Compute Savings Plan (for any region and instance family) or EC2 Instance Savings Plan (specific to instance family in a region).

Set your hourly commitment and term length.

Tip: Choose Savings Plans if you have variable workloads that span multiple services.

5. Implement Auto Scaling to Match Demand

Auto Scaling dynamically adjusts resources based on actual demand, reducing costs by releasing resources during low demand periods.

Setting Up Auto Scaling for EC2

a. In the EC2 Console, go to Auto Scaling Groups > Create Auto Scaling Group.

Under Launch Template > Create Launch Template

Then, while creating the Launch template, select the Operating System of your choice e.g., Ubuntu and instance type.

b. Use the Launch template you just created and configure the minimum, desired, and maximum instance counts.

c. Configure scaling policies to define triggers, such as CPU utilization or network traffic.

d. Set up notifications for scale-in and scale-out events.

Tip: Use Auto Scaling with Spot Instances for even more savings (see below for details).

6. Use Spot Instances for Non-Critical Workloads

AWS Spot Instances offer spare capacity at up to a 90% discount, ideal for flexible, fault-tolerant workloads.

Using Spot Instances:

a. In the EC2 Console, choose Launch Instance.

Select the instance type and configure it as a Spot Instance under Purchase Options.
Set the maximum bid price for the instance.

b. Launch and monitor spot requests in the Spot Requests section.

Tip: Use Spot Instances for batch jobs, data processing, or other non-time-sensitive tasks.

Combine Spot Instances with Auto Scaling to automatically manage instance availability.

7. Optimize Storage Costs with S3 and EBS

S3 Storage Classes

Standard: For frequently accessed data.
Intelligent-Tiering: Automatically moves data to the most cost-effective tier based on access patterns.
Infrequent Access (IA): For less-frequently accessed data.
Glacier and Glacier Deep Archive: For archival data.

Tip: Set up Lifecycle Policies in the S3 Console to transition objects to cheaper storage classes as they age.

EBS Volume Optimization

a. Delete Unused Volumes: Regularly review and delete any unattached EBS volumes.

b. Use Snapshots for Backups: Use snapshots for backup rather than keeping idle volumes.

c. Right-Size EBS Volumes: Choose the storage type and size that align with performance needs, e.g., switching from Provisioned IOPS SSDs to General Purpose SSDs for less intensive workloads.

8. Leverage AWS Lambda for Serverless Architectures

AWS Lambda enables a serverless approach, where you only pay for the compute time you use. This can lead to significant cost savings, especially for intermittent workloads.

Creating a Lambda Function

a. In the Lambda Console, click Create Function.

b. Choose Author from scratch, name the function, and select the runtime.

c. Define the function's Memory and Timeout settings, optimizing for performance and cost.

d. Test and deploy the function.

Tip: Use Provisioned Concurrency only if necessary. Lambda functions billed per millisecond can be more cost-effective for many small tasks.

9. Monitor and Reduce Data Transfer Costs

AWS charges for data transfer between regions and VPCs, which can accumulate quickly.

Tips to Minimize Data Transfer Costs

a. Use Same Region Resources: Whenever possible, keep resources within the same region to minimize data transfer costs.

b. Use Amazon CloudFront: Leverage CloudFront for content delivery to reduce cross-region transfer costs.

c. Implement VPC Peering: If data transfer between VPCs is necessary, use VPC peering or AWS Transit Gateway to lower costs.

10. Regularly Clean Up Idle or Unused Resources

Unused resources can quietly accumulate costs over time, so regularly auditing your AWS account can save money.

Checklist for Resource Cleanup
Delete Unused EC2 Instances and EBS Volumes.
Remove Old Snapshots and Backups.
Terminate Idle RDS Instances.

Delete Unnecessary Elastic IPs: AWS charges for Elastic IPs not attached to running instances.

Tip: Set up automated scripts or use AWS Lambda to periodically check for and delete unused resources.

CONCLUSION

Optimizing AWS costs is an ongoing process that combines strategic use of AWS tools, pricing models, and automated practices.

Implementing the tips and techniques outlined here can help budget-conscious cloud engineers make the most of AWS resources without overspending.

Regularly review your setup to ensure your infrastructure remains optimized as workloads and requirements evolve.

Building AI-Powered Real-Time Object Detection with OpenCV, Flask, Gemini Vision, and GitHub Actions CI/CD on AWS EC2

Toyyib Muhammad-Jamiu — Mon, 16 Dec 2024 21:31:09 +0000

INTRODUCTION:

Imagine a system that can analyze live video feeds in real time, interpret scenes, and respond intelligently to questions about the environment, just like a virtual assistant with eyes.

This is the potential of combining cutting-edge technologies like OpenCV for video processing and Google's Gemini vision model, leveraging its latest "gemini-1.5-flash-latest" model.

In this article, I will guide you through building a Real-Time Object Detection System that uses live video streaming and AI-powered scene analysis to deliver insightful, context-aware responses.

We'll deploy the application on AWS EC2, setting the stage for scalability and real-world use while employing Github Actions for automated CI/CD, ensuring a seamless update pipeline.

By the end of this tutorial, you'll have a fully functional AI-powered system ready for deployment, with the confidence to expand and customize it for various use cases.

PROJECT STRUCTURE

project/
├── app.py              # Flask application code
├── requirements.txt    # Python dependencies
├── templates/
│   └── index.html     # Frontend UI
└── .env               # Environment variables (API keys, etc.)

Core Components

A. Real-Time Video Capture (OpenCV)
The WebcamCapture class in app.py handles video streaming:

self.stream = cv2.VideoCapture(0)  # Open the default webcam

This ensures efficient, thread-safe frame capture and processing.

B. AI-Powered Object Detection (Google Gemini)
Using the Gemini model, we analyze frames for real-time scene understanding:

self.model = ChatGoogleGenerativeAI(model="gemini-1.5-flash-latest")
response = self.chain.invoke({"prompt": prompt, "image_base64": image_base64})

C. Flask Backend
The Flask application provides endpoints for video streaming, AI queries, and system status checks:

/video_feed: Streams live video.

/process_query: Handles AI-powered analysis based on user input and video frames.

D. Frontend UI
The index.html file provides a responsive web interface for interacting with the system. It captures user queries and displays real-time AI responses.

PREREQUISITES

An AWS account.
A registered domain name (e.g., example.com).
A Google Cloud Account or Open AI account
GitHub actions configured in your repository.
Basic knowledge of SSH and Linux command-line tools.

APPLICATION DEPLOYMENT

Step 1: Clone the Repository, Generate the Google Gemini API & Push the application files to Github

A. Clone the repository

$ git clone https://github.com/Abunuman/Real-Time-ODS.git
$ cd Real-Time-ODS

B. Generate your API key and add to a .env file

i. Create a .env file either manually from the options available on the left-hand side of your text editor (I used VScode)
OR
On the terminal, run:

$ touch .env

Then add these in the .env

GOOGLE_API_KEY=your_google_api_key
OPENAI_API_KEY=your_openai_api_key
FLASK_DEBUG=True

ii. Log into Google Cloud and follow these steps to generate your API key.

a. Navigate to the API & Services Section

b. Click on Credentials then follow the other steps below

Create Credentials > API Key , then the API Key is generated.

Remember to note the name of your API key. You can also give it a name during the process.

Copy the API Key generated, go back to your .env file and replace your_google_api_key with the key you just copied.

c. Enable Gemini API
Search for Gemini API and click on ENABLE

Confirm that your API Key is under the METRICS and Credentials section under the Enabled Gemini API.

iii. Create a .gitignore file and add .env to the file so that it is not pushed to github.

N.B.: Standard practice is to ensure that secrets and environment variables are not exposed to the public. Hence the need for a .gitignore to ignore files added therein while pushing to Github.

C. Push to Repository.
i. Create a Github repository with the application name and follow the commands below to push to github

$ git init
$ git add .
$ git commit -m "first commit"
$ git branch -M main
$ git remote add origin https://github.com/Abunuman/repository-name.git
git push -u origin main

N.B: change repository-name to your repository name

Step 2: Setting Up AWS EC2 Instance

i. Launch an EC2 Instance

Use the AWS Management Console to launch an EC2 instance (e.g., Ubuntu 22.04).

Select an instance type (e.g., t2.micro for free tier users).

Create and download a key pair (.pem file) for SSH access.

Create a new key pair or use an existing one.

If you are creating a new key pair, click on create key pair and give it a name of your choice.

Select Key Pair type as RSA

File format as .pem

Then Create the key

The key pair is automatically downloaded to your system.

Configure Security Groups

Allow the following inbound rules:

a. HTTP (port 80): For serving your application.
b. HTTPS (port 443): For secure access.
c. SSH (port 22): For access management.

Click on Launch instance and allow the instance to be fully launched.

Now your instance is ready to use once the status shows “Running”.

ii. Configure the key pair (.pem key) for SSH access

For Mac book users or Linux users with bash terminal, configure your key pair for SSH access thus:

a. Open the downloaded .pem key using VScode or Xcode

b. On your terminal, navigate to the .ssh directory from the root directory(~)

$ cd .ssh

c. Create a .pem file in the .ssh directory using nano or vim text editors; I will be using nano in this tutorial.

Install nano if you don't have it installed.

For macbook users

$ brew install nano

For linux users

$ sudo apt install nano

Having installed it, create the .pem file in the .ssh directory using nano.

Ensure the file to be created bears the exact name of your .pem file.

$ sudo nano name_of_pem.pem

Then copy the already opened .pem file and paste in the .pem to be created in the .ssh directory.

Press Ctrl X, then Y , then Enter to save.

d. Change the .pem file permission

$ chmod 400 name_of_pem.pem

iii. Access the Instance - SSH into your EC2 instance:

Click on the Instance ID . Once the instance is in the running state, select on the connect option

Once you are at the Connect page , Go to SSH Client

Then copy the last command on the page that looks like this:

ssh -i path/to/key.pem ubuntu@<ec2-public-ip>

Paste this on your terminal and press enter. You should connect seamlessly.

For Windows Users

Windows Setup

Open CMD on your windows machine
Locate desired directory where .pem file is stored

Ideally from this directory , we can run the copied ssh command and we should be able to connect to EC2.

However, sometimes we get a security permissions error when we run the ssh command.

We have to change the permissions to the .pem file.

For that follow the steps below:

Locate the .pem file folder , right click on the file and select properties
Go to Security tab
Go to Advanced tab
Click Disable inheritance
This Advance options also shows other user having all permissions to .pem file. Remove permission for all other users
Add the user with which you are trying to connect to EC2 if not already present in the user list.
Enable all permissions for this user.

Ideally with these steps, you should not encounter an error.

Run the SSH command from CMD prompt
Once the permissions are fixed , prompt will successfully connect to EC2

Now, you have successfully completed the steps and you can run commands from windows CMD on EC2 instance.

iv.Install Dependencies - Update the package list and install necessary packages:

Having connected to your EC2 instance via SSH, install dependencies on EC2.

On your connected terminal, run the following commands:

$ sudo apt update
$ sudo apt install -y python3 python3-pip nginx

Check the version of python3 installed, ensure its 3.12

python3 --version

Step 3: Configuring GitHub Actions for CI/CD

A. Set up Github Actions Environment Secrets

Configure your AWS IAM user secrets and environment variables needed for the project.

Set GitHub Secrets in your repository, navigate to Settings > Secrets and variables > Actions, and add:

EC2_USERNAME (e.g., ubuntu).
EC2_HOST (your EC2 public IP or domain).
EC2_SSH_KEY (path to your private key file).

i. Navigate to Settings in your repository

ii. Click on Secrets and Variables, then Actions

iii. Add your Secrets and Variables like below

B. Create a Workflow File in your repository, add a .github/workflows/main.yml file:

Navigate to Actions under your repository and click on set up a workflow yourself

Then add this to the space provided

name: Deploy to AWS
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      # Check out the repository
      - uses: actions/checkout@v4

      # Set up Python environment
      - name: Set up Python 3.12  
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      # Configure AWS credentials
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}

      # Copy files to EC2 instance
      - name: Transfer files to EC2
        uses: appleboy/scp-action@master
        with:
          host: ${{ secrets.EC2_HOST }}
          username: ${{ secrets.EC2_USERNAME }}
          key: ${{ secrets.EC2_SSH_KEY }}
          source: "app.py,requirements.txt,templates/index.html"
          target: "/home/${{ secrets.EC2_USERNAME }}/app"

      # SSH to EC2 and set up the app
      - name: Deploy to EC2
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.EC2_HOST }}
          username: ${{ secrets.EC2_USERNAME }}
          key: ${{ secrets.EC2_SSH_KEY }}
          script: |
            # Create and navigate to the app directory
            cd /home/${{ secrets.EC2_USERNAME }}/app

            # Ensure system dependencies are installed
            sudo apt-get update
            sudo apt-get install -y python3-venv python3-pip git libopencv-dev python3-opencv portaudio19-dev ffmpeg

            # Setup Python virtual environment
            if [ ! -d "venv" ]; then
              python3 -m venv venv
            fi

            # Activate virtual environment and install dependencies
            source venv/bin/activate
            pip install --upgrade pip
            pip install -r requirements.txt

            # Ensure templates directory exists and move files if needed
            mkdir -p templates

            # Create or update systemd service
            sudo tee /etc/systemd/system/flask_app.service << EOF
            [Unit]
            Description=Flask App
            After=network.target

            [Service]
            User=${{ secrets.EC2_USERNAME }}
            WorkingDirectory=/home/${{ secrets.EC2_USERNAME }}/app
            Environment="PATH=/home/${{ secrets.EC2_USERNAME }}/app/venv/bin"
            ExecStart=/home/${{ secrets.EC2_USERNAME }}/app/venv/bin/python app.py

            [Install]
            WantedBy=multi-user.target
            EOF

            # Reload and restart the service
            sudo systemctl daemon-reload
            sudo systemctl enable flask_app
            sudo systemctl restart flask_app

Having done this, run the CI/CD and check your terminal connected to EC2 to check if the files are now moved to EC2.

Step 4: Acquiring an SSL Certificate with AWS ACM

A. Request a Certificate

Navigate to AWS Certificate Manager in the AWS Console.
Click Request a certificate and choose Request a public certificate.

B. Enter your domain name (e.g., example.com)

C. Choose DNS validation and complete the request.

D. Validate the Certificate

ACM will provide a CNAME record.

Add this record in your domain’s Route 53 hosted zone.

E. Verify the Status

Once validated, the certificate status will change to Issued.

Step 5: Managing a Custom Domain with AWS Route 53

A. Navigate to AWS Route 53 in the AWS Console - Create a Hosted Zone for your domain

B. Update Domain Name Servers

Copy the nameservers (NS) from the hosted zone and update the NS records in your domain registrar’s settings.

C. Add an A Record

Create an A record pointing to your EC2 instance's public IP.

e.g realtime.example.com (realtime as sub-domain)

D. Add a CNAME for SSL

Add a CNAME record from ACM for DNS validation if not already done.

This can be created during your SSL certificate generation by clicking on the CNAME option and it redirects you to AWS Route53 and its created for the domain.

You can also create it under AWS Route 53 by going through similar steps followed in creating an A record.

Step 6: Configure NGINX for reverse proxy

Configure NGINX for reverse proxy

A. Edit nginx sites available for flask app

Navigate to /etc/nginx/sites-available/flask_app:

$ sudo nano /etc/nginx/sites-available/flask_app

B. Add this configuration

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}

# HTTP redirect server block
server {
    listen 80;
    server_name your_a_record_domain.com;
    # Only redirect if not already HTTPS
    if ($http_x_forwarded_proto != 'https') {
        return 301 https://$server_name$request_uri;
    }

    location / {
        proxy_pass http://localhost:5000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Increased timeouts for streaming
        proxy_read_timeout 3600s;
        proxy_send_timeout 3600s;
    }
}

# Main HTTPS server block
server {
    listen 443;
    server_name your_a_record_domain.com.com;

    # Increase timeout settings
    proxy_connect_timeout 600;
    proxy_send_timeout 600;
    proxy_read_timeout 600;
    send_timeout 600;

    # Prevent timeout for video streaming
    keepalive_timeout 650;
    keepalive_requests 10000;
    client_max_body_size 50M;

    location /video_feed {
        proxy_pass http://localhost:5000;
        proxy_buffering off;
        proxy_cache off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_set_header Connection '';
        proxy_read_timeout 24h;
        proxy_send_timeout 24h;
        proxy_connect_timeout 24h;
    }

    location /process_frame {
        proxy_pass http://localhost:5000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_cache off;
        proxy_buffering off;
    }

    location / {
        proxy_pass http://localhost:5000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

    }
}

C. Restart nginx and flask app

Run:

$ sudo systemctl restart nginx
$ sudo systemctl restart flask_app

Step 7: Create an Application Load Balancer

To secure your application with HTTPS using an SSL certificate from AWS Certificate Manager (ACM), you need to link the SSL certificate to your domain via an Application Load Balancer (ALB) and configure your domain in Route 53.

You can create the Load Balancer thus:

A. Set Up an Application Load Balancer (ALB)
Go to the AWS Management Console and navigate to EC2 > Load Balancers.

B. Click Create Load Balancer and choose Application Load Balancer.

Step-by-Step Configuration:

Name: Provide a name (e.g., realtime-alb).
Scheme: Choose Internet-facing.

C. Listeners:

Add two listeners:

Port 80 (HTTP) → Target Group (Redirect to HTTPS).
Port 443 (HTTPS) → SSL Certificate.

Availability Zones: Select the VPC and all subnets for redundancy.

Configure HTTPS Listener and Attach SSL Certificate

In the Listeners section during ALB creation:

For Port 443 (HTTPS):
Choose SSL Certificate from AWS Certificate Manager (ACM).
Select the validated certificate for your domain (e.g., example.com).

Forwarding Rules:

Under Actions, ensure the HTTPS listener forwards traffic to a Target Group that includes your EC2 instance.

D. Set Up a Target Group

Go to Target Groups in the EC2 dashboard.

Create a target group:

Name: e.g, realtime-tg.
Target Type: Instances.
Protocol: HTTP (port 80).

Select your running EC2 instance and add it to the target group.

Verify the health checks:

ALB will perform periodic checks to ensure your application is running.
Use HTTP and / for health checks.

E. Redirect HTTP to HTTPS (Optional but Recommended)

In the Load Balancer Listeners section:

Add a rule for Port 80 to redirect all traffic to HTTPS (Port 443).

Set the redirection response to 301 Moved Permanently.

Example Rule:

Condition: All requests
Action: Redirect to HTTPS (port 443).

F. Update Route 53 to Point to the Load Balancer

Go to Route 53 and navigate to your hosted zone.

Click on your A Record and edit thus:

Alias: Enable Alias.

Target: Select your Application Load Balancer from the dropdown list.

Save the record.

Finally, you can restart the flask app and nginx on your connected terminal, and check the status afterwards

$ sudo systemctl restart nginx
$ sudo systemctl restart flask_app
$ sudo systemctl status nginx
$ sudo systemctl status flask_app

Final Test

Visit your domain e.g https://real.example.com to access your deployed application securely. You should see the real-time object detection system up and running!

N.B: Note that Gemini API free tier for the gemini-1.5-flash-latest model has a limit that if exceeded will start throwing a process query error. Hence, ensure to get a paid version to forestall this.

CONCLUSION

In this tutorial, we embarked on a comprehensive journey to build and deploy a real-time object detection system that seamlessly integrates OpenCV for live video capture and Google's Gemini vision model for intelligent scene analysis.

From configuring the application locally to deploying it securely on AWS EC2 with a custom domain and SSL, we covered every essential step to transform your idea into a functional and scalable solution.

This project highlights the power of combining cutting-edge technologies like Flask, OpenCV, and AI to solve real-world problems while ensuring best practices for cloud deployment.

By following these steps, you've not only deployed a robust AI-powered system but also ensured scalability, security, and efficient CI/CD pipelines.