Building Cloud Native Monitoring Application on Kubernetes and AWS ECR 🚀 🚀 🚀

In the ever-evolving world of DevOps and cloud computing, the need for robust and scalable monitoring solutions is paramount. As applications grow more complex and distributed, having insights into their performance and resource utilization becomes essential. In this tutorial, we’ll take you through the journey of creating a Cloud Native Monitoring Application using Flask, Docker, Kubernetes (K8s), and Amazon EKS.

GitHub

Linkdin

Prerequisites !
(Things to have before starting the projects)

 AWS Account.
 Programmatic access and AWS configured with CLI.
 Python3 Installed.
 Docker and Kubectl installed.
 Code editor (Vscode)

Part 1: Deploying the Flask application locally
Step 1: Clone the code

Start by cloning the project repository to your local machine.

git clone https://github.com/SwapnilM24/cloud-native-monitoring-app.git

Step 2: Install dependencies

The application utilizes Python libraries like psutil, Flask, Plotly, and boto3. Install them using pip:

pip3 install -r requirements.txt

You can also go ahead with pip install -r requirements.txt in the whole project

Step 3: Run the application

Navigate to the project’s root directory and execute the following command:

python3 app.py

This will start the Flask server on localhost:5000. You can access the application by opening your browser and going to http://localhost:5000/.

Part 2: Dockerizing the Flask application
Step 1: Create a Dockerfile

In the project’s root directory, create a Dockerfile with the following contents:

Use the official Python image as the base image

FROM python:3.9-slim-buster

# Set the working directory in the container
WORKDIR /app
# Copy the requirements file to the working directory
COPY requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt
# Copy the application code to the working directory
COPY . .
# Set the environment variables for the Flask app
ENV FLASK_RUN_HOST=0.0.0.0
# Expose the port on which the Flask app will run
EXPOSE 5000
# Start the Flask app when the container is run
CMD ["flask", "run"]

Step 2: Build the Docker image

To build the Docker image, execute the following command:

docker build -t <image_name> .

Step 3: Run the Docker container

Run the Docker container with the following command:

docker run -p 5000:5000 <image_name>

This will start the Flask server in a Docker container on localhost:5000. You can access the application the same way as before.

Part 3: Pushing the Docker image to ECR
Step 1: Create an ECR repository

Using Python and Boto3, create an Amazon ECR repository:

import boto3

# Create an ECR client
ecr_client = boto3.client('ecr')
# Create a new ECR repository
repository_name = 'my-ecr-repo'
response = ecr_client.create_repository(repositoryName=repository_name)
# Print the repository URI
repository_uri = response['repository']['repositoryUri']
print(repository_uri)

Step 2: Push the Docker image to ECR

Push the Docker image to the created ECR repository using the following commands:

docker push <ecr_repo_uri>:<tag>

Part 4: Creating an EKS cluster and deploying the app using Python
Step 1: Create an EKS cluster

Create an EKS cluster and add a node group.

Step 2: Create a node group

Create a node group in the EKS cluster. Remember to create security groups and roles in IAM for specific clusters. Use t2.micro to stay in free tier and create 2 nodes in a node group while creating cluster.

Step 3: Create deployment and service

Using Python and the Kubernetes client library, create a deployment and service for your Flask application. Be sure to replace <image_uri> with the URI of your Docker image.

from kubernetes import client, config

# Load Kubernetes configuration
config.load_kube_config()
# Create a Kubernetes API client
api_client = client.ApiClient()
# Define the deployment
deployment = client.V1Deployment(
    metadata=client.V1ObjectMeta(name="my-flask-app"),
    spec=client.V1DeploymentSpec(
        replicas=1,
        selector=client.V1LabelSelector(
            match_labels={"app": "my-flask-app"}
        ),
        template=client.V1PodTemplateSpec(
            metadata=client.V1ObjectMeta(
                labels={"app": "my-flask-app"}
            ),
            spec=client.V1PodSpec(
                containers=[
                    client.V1Container(
                        name="my-flask-container",
                        image="<image_uri>",
                        ports=[client.V1ContainerPort(container_port=5000)]
                    )
                ]
            )
        )
    )
)
# Create the deployment
api_instance = client.AppsV1Api(api_client)
api_instance.create_namespaced_deployment(
    namespace="default",
    body=deployment
)
# Define the service
service = client.V1Service(
    metadata=client.V1ObjectMeta(name="my-flask-service"),
    spec=client.V1ServiceSpec(
        selector={"app": "my-flask-app"},
        ports=[client.V1ServicePort(port=5000)]
    )
)
# Create the service
api_instance = client.CoreV1Api(api_client)
api_instance.create_namespaced_service(
    namespace="default",
    body=service
)

Ensure that you replace <image_uri>on line 25 with the actual URI of your Docker image.

After running the Python script eks.py, your deployment and service will be created. You can check their status using the following commands:

kubectl get deployment -n default  # Check deployments
kubectl get service -n default    # Check service
kubectl get pods -n default       # Check pods

Once your pod is up and running, run the port-forward command to expose the service:

kubectl port-forward service/<service_name> 5000:5000

Congratulations! You’ve successfully created a Cloud Native Monitoring Application and deployed it on Kubernetes with AWS EKS. This application can be extended to monitor various resources and provide valuable insights into your cloud infrastructure. Explore further and enhance your DevOps skills! 🚀