Deploying a Flask-based Intrusion Detection System to AWS ECS

Introduction

In this post, I’ll Walk you through the process of deploying an intrusion detection system on AWS

NOTE: This project assumes you already have an active AWS account and configured your account credentials (access keys) to your code editor and this project will incur some costs in your console

Project Summary

Deploying a Flask-based Intrusion Detection System to AWS ECS (Project Summary)

Victor ・ Nov 16

#devops #aws #docker #python

Project Overview

Objectives

The objectives of this project is as follows:

• Containerize the application with Docker
• Push the container image to ECR
• Create a VPC, two private subnets and two public subnets
• Create VPC endpoints for the private subnets to access ECR
• Deploy an Application Load Balancer and target group in the public subnets for the ECS Service
• Create a Task definition for the ECS service
• Create an ECS cluster and an ECS service with fargate launch type in the private subnets.
• Create a hosted zone in route 53 and point it to the ALB's DNS name

Project Architecture

Clone Repository

The first step is to clone the project repo.
You can find the project source code in my Github Repo

VSI12 / IDS-Project

A Flask-based Intrusion Detection System web-application deployed to AWS ECS

Deploying an Intrusion Detection System to AWS

Project Overview

This Project demonstrates how to build and containerize a flask web-application with docker and deploy it to AWS. This architecture ensures a secure, highly available, fault tolerant and scalable build by leveraging various AWS architectures.

Architecture

• Virtual Private Cloud (VPC): Configured with public and private subnets across two availability zones for high availability and security.
• Interget Gateway: Enables communication between the VPC and the internet
• VPC Endpoints: The VPC endpoints enable the ECS tasks in the private subnet to access certain resources.
• Application Load Balancer(ALB): Configured to forward traffic to the ECS tasks, through listeners and target groups and distributes the incoming traffic across the ECS tasks.
• Elastic Container Registry: Host the docker image
• Elastic Container Service(ECS): creates the ECS cluster that hosts the Fargate service tasks
• Code Pipeline: Creates a CI/CD pipeline using…

View on GitHub

• Run the following commands in your terminal

git clone https://github.com/yourusername/IDS-ECS.git
cd IDS-ECS

Containerize the application with docker

This section will show the steps involved in creating a docker image if the web app

Step 1: Dockerize The Flask Application

Create a Dockerfile in the project directory in order to package the flask app.

FROM python:3.12

#set the working dir
WORKDIR /usr/src/app

#copy the requirements and install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy all the files to the container
COPY . .

#Expose the port
EXPOSE 5000

#run the app
CMD ["gunicorn","-b","0.0.0.0:5000", "app:app"]

Step 2: Build and test Docker Image

It is important to build and test the docker image locally to ensure that it is working as intended

docker build -t image-name .
docker run -p 5000:5000 image-name

Push Docker Image to ECR

Step 1. Create an Elastic Container Repository (ECR)

• Go to the AWS ECR console and create a repository and take a note of the URI (e.g., 123456789012.dkr.ecr.region.amazonaws.com/repo-name)
• NOTE: Enter your ECR repo and select view push commands in order to see the commands to push the image to your ECR repo. Its those commands that will be used here.

Step 2. Authenticate Docker to ECR

Run the following command to authenticate Docker with ECR (replace your-region and your-account-id):

aws ecr get-login-password --region your-region | docker login --username AWS --password-stdin your-account-id.dkr.ecr.your-region.amazonaws.com

Step 3. Tag and Push the Image

Tag your local Docker image to match the ECR repository, then push it:

NOTE: Ensure it is your accounts respective region and account-id

docker tag image-name:latest your-account-id.dkr.ecr.your-region.amazonaws.com/image-name:latest
docker push your-account-id.dkr.ecr.your-region.amazonaws.com/image-name:latest

Your ECR repo should look just like this is the image is pushed successfully

Create a VPC and its Subnets

Take a note of the Region, as this is where all the resources will be deployed

Step 1: Create a new VPC

• Go to the VPC console and create a new VPC
• Specify CIDR block (e.g. 10.0.0.0/16)

• NOTE: Ensure the Enable DNS hostnames setting is checked when creating the VPC

Step 2: Create an Internet Gateway (IGW) for the VPC

In the VPC console, select the Intergate Gateway tab and create the internet gateway. Once the IGW is created attach it to your VPC

Step 3: Create Subnets

• Create two subnets in one availability zone (e.g. us-east-1)
• Create another set of two subnets in different availability zone (AZ) (e.g. us-east-2)
• NOTE: In each AZ, the subnets will serve as private and public subnets respectively.

NOTE: the ECS cluster will be deployed on the private subnet and the Application Load Balancer will be in the public subnet and will access the ECS cluster in the private subnet

Step 4: Update the Route tables

• Create route tables for the public and private subnets and associate the route tables to the public and private subnets respectively. (Ensure the VPC in use is selected for the both of them.)
• For the public subnets route table, add to route to direct all outbound traffic 0.0.0.0/0 through the internet Gateway.
• The private subnet will not route outbound traffic for now.

Create VPC endpoints for ECR

This will enable the ECS cluster to have access to the Elastic Container Registry (ECR).
NOTE: Four Endpoints will be made for S3, ECR, DOCKER and CloudWatch.

• select Endpoints and click on create endpoint.

• Name the endpoint and search for the ECR api endpoint under services com.amazonaws.us-east-1.ecr.api
• Select the VPC we've been using, this will bring up the subnets option where the availability zones our private subnets are in will be selected and then finally select our private subnets and the default security group.

• Select the default and leave the policy as is and then create the VPC.

*Now create the remaining Endpoints with changing the services for docker com.amazonaws.us-east-1.ecr.dkr, for CloudWatch logs com.amazonaws.us-east-1.logs and for S3 com.amazonaws.us-east-1.s3 respectively and follow everything else exactly expect for those changes.

• NOTE: For the S3 endpoint, select the gateway. This is called the S3 Gateway Endpoint and will prompt you to connect it to your private subnet route table, which will create a route in the route table for it.

Create Application Load Balancer and Target Group

This is a very important step as its the ALB that
will route traffic to the private ECS service.

Firstly, we need to create a security group for the ALB.

• Go to the security group on the left and select create security group. give the security a name and description.
• Add an inbound rule on port range 80 and source 0.0.0.0/0 and add a second one on port range 443 and source 0.0.0.0/24 for HTTP and HTTPS traffic respectively

Step 1: Create a Target Group

• Go to the EC2 console and on the left, under Load Balancing select target group.
• Under basic configuration select IP addresses and name the Target group
• Leave the Protocol and port as HTTP:80
• Select the VPC we are deploying the load balancer in and scroll and click next.
• Here remove the IP address that is there, all IP's will be automatically added. Next, specify the port which was exposed, Port:5000 and Create the Target group.

Create Application Load Balancer

• On the left of the EC2 console, select the load balancer and select create load balancer.
• Select Create Application Load Balancer, this will be an internet-facing load balancer. give the ALB a name.
• In the network mapping section select the Created VPC and the public subnets in the two availability zones.
• In the Listeners and Routing section, select the created target group. The ALB will listen on port:80 and forward to the target group.
• Create the ALB.

Create Task definitions

Go to the ECS console and select task definitions.

Step 1:

• Name the task definition family
• Leave the default infrastructure requirements as is, but you can change the vCPU and memory based on your descretion.

Step 2: For Container 1

• Give a name
• Go to your ECR and copy the URI of the docker image we pushed in earlier sections of this project and come back to the container 1 section of the task definitions we're creating and paste it in the Image URI for the container.
• For the Port mappings set it to 5000, as this was the port we exposed in our docker container and give it a name. NOTE: It is very important the port mappings is the same as the exposed docker port

• Add environment variables
  This will be important in the CI/CD section of this project. But will be skipped for now.

• Go ahead and skip the remaining sections and create the task definition

Create a Fargate cluster and service

Create security group for the ECS service

• Go to the security group on the left and select create security group. give the security a name and description.
• Add an inbound rule on port range 5000 and source will be the Application Load Balancers security group.

Step 1: Create Cluster

• Go to the clusters tab and select create cluster, this will bring you to the cluster configuration page. Name the cluster and ensure only the AWS Fargate(Serverless) is selected under the infrastructure tab and then create the cluster.

Step 2: Create a service

• Select the created cluster and click on the create under service.

This will lead you to a new page where you'll specify the configuration of the Fargate service.

• Scroll past the Environment section and move to the Deployment Configuration. Here, specify the task definition family, which will automatically select the revision as well.

• Select the Desired number of Tasks to launch. One is fine for this project.

• Scroll to the Networking tab and select the VPC that was made and then the private subnet(s)

• In the Load balancer section, select the load balancer type as Application Load Balancer, and select use an existing load balancer. This will bring up the ALB that was created in previous sections. select it.

• Scroll and you'll see, listener. select use an existing listener and select the port 80 listener that is there and under target group do the same, selecting the existing target group that we made.

• Next is Service Auto Scaling. This is optional but is a good addition to have to scale out to app based on defined metrics. Enable this and specify the minimum and maximum number of tasks you want running. Next add a scaling policy. For this project a Target tracking policy is used alongside the ALBrequestCountPerTarget ECS service metric, with the target value 50, scale-out cooldown period and Scale-in cooldown period as 60s

• Create the Service.

• Once the Service is created, the desired number of tasks will be created.

Create a Hosted Zone in route 53

If the above instructions were followed t the T, you should have a fully functioning web app, to access it go to your load balancer, copy the DNS name and paste in your browser. But that's tedious and not using best practices. Ideally, there should be a Web application firewall in front of the ALB or CloudFront, but for simplicity we will be using only Route 53.

NOTE: This section requires you have a registered domain name either with AWS or any other provider

Step 1: Create a hosted in

• Go to the Route 53 console.
• On the left tab, select Hosted Zones. you should have this.

• Select Create Hosted Zone
• Enter your domain name and give a description. Leave the type as Public Hosted Zone and select create Hosted Zone.

Step 2: Add ALB DNS name to hosted zone records

• Go to your ALB console and copy its DNS name.
• Come back to the hosted zone and select create record.

• The default record type that is there is the A record, which will be used.
• Toggle the alias switch

• Under choose endpoint, select the Alias to Application and Classical Load Balancer
• Next, choose the region you launched your Load Balancer in. In this case its us-east-1, Ensure it's your on region.
• Choose your load balancer from the drop-down menu.
• Create record.

This assumes you have your domain name with AWS

CONGRATULATIONS!!

If you followed the Steps to the T, you should have a full functioning web-app that is accessible through your domain-name.

This was a very exciting project as I worked with VPC's, private and public subnets, VPC endpoints, ECS services, ECR, Target groups, security groups and Application load Balancer as they all came together to create this web-app.

Relevant links

• IDS-Project(GitHub Repo)

  VSI12 / IDS-Project

  A Flask-based Intrusion Detection System web-application deployed to AWS ECS

  Deploying an Intrusion Detection System to AWS

  Project Overview

  This Project demonstrates how to build and containerize a flask web-application with docker and deploy it to AWS. This architecture ensures a secure, highly available, fault tolerant and scalable build by leveraging various AWS architectures.

  Architecture

  • Virtual Private Cloud (VPC): Configured with public and private subnets across two availability zones for high availability and security.
  • Interget Gateway: Enables communication between the VPC and the internet
  • VPC Endpoints: The VPC endpoints enable the ECS tasks in the private subnet to access certain resources.
  • Application Load Balancer(ALB): Configured to forward traffic to the ECS tasks, through listeners and target groups and distributes the incoming traffic across the ECS tasks.
  • Elastic Container Registry: Host the docker image
  • Elastic Container Service(ECS): creates the ECS cluster that hosts the Fargate service tasks
  • Code Pipeline: Creates a CI/CD pipeline using…

  View on GitHub

Connect with me

• https://linkedin.com/in/victor-iliya

Resources Used

• https://www.youtube.com/watch?v=AyFiJqoulpY
• https://youtube.com/playlist?list=PLqoUmUbJ_zDHPwK-ZWATXiYrUXwWkLY65&si=yqc6s5NUlKlCaNQe