DEV Community: Shahin Sheikh

How I deployed my first project for my devops portfolio: EC2 health check and monitoring and Conclusion

Shahin Sheikh — Wed, 13 Aug 2025 07:05:21 +0000

So this is the final part where I implement the monitoring of my instance.

Aim of my monitoring

I wanted to monitor my instances disk usage, cpu and ram.

Thoughts and implementations of monitoring

At first I thought of using garafana and prometheus for monitoring as I implemented it in my company. Even though I don't have much idea but very basics of how to collect metrics and view it in grafana dashboard. Then expose the service via guest user to the www. I prepared node exporter into my dockerfiles in Database pod and both app pods and even one for the EC2 but then something struck my mind. WHY??? Why do I have to go this far to monitor. All I want is simple monitor the aim CPU, RAM and Disk usage that is all. Then I rolled back to unix commands and that was the answer.

Commands I settled for

df -h | tee test.txt
top -bn 1 | awk 'NR >= 1 && NR < 6 {print}' | tee test.txt
mpstat -T | tee test.txt

Yes I know top command shows me the cpu but I just kept the mpstat just because I felt like having 2 cpu metrics that I can make an estimated average of it.

Challenge is to send myself the metrics

I choose one by EMail, SMS, and in Discord channel. First is I made a separate python script exclusively for discord. I made a virtual env using venv module and pip installed discord-webhook and created a webhook in my server and used the url and successfully sent the message.

When I first sent the message it was not exactly like this as I improved it once time went by.

Then I came across second challenge SMS and EMAIL

At first I thought of having a personal smtp server and send it but then I found out it is blocked by all ISPS across the world to prevent spams then I thought of a service that does the very same thing SNS. I watched the web and created a basic SNS topic from dashboard and created a mail subscription and an SMS and published messages and wallah as it is working I quickly terraformed it.
Second challenge is to now publish a message from the EC2 instance.
I came across this website and wrote the initial script in python with the access keys in it for the testing and it was working fine. Then I created a role manually and assigned it to the instance and then tried it without the access keys and it was working fine again. The Snag is that when I did it using terraform it wasn't reflecting in the EC2 instance. Thing is after making the role I couldn't assign the role in EC2 from the console. So I went on the hunt why it was not working via terraform and working if created manually via dashboard. Upon inspection of both the roles side by side I saw something different from the dashboard created one with terraform.

I saw instance-profile/RoleName and terrafrom created role/RoleName at the ARN. Now I understood that I merely created normal role profile, but I need to assign an instance-profile to it and hence the hunt begun and I modified my terraform for it and it worked like a charm this time.

I made a custom policy using aws official policy generator.

main.tf

# ----------------------IAM ROLES & POLICIES ----------------

# AWS IAM role creation
data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["ec2.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

# AWS IAM role creation completed
resource "aws_iam_role" "sns-trigger-ec2" {
  name               = "EC2snsPublish"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

# Policy creation in json
data "aws_iam_policy_document" "sns-publish-policy" {
  statement {
    effect = "Allow"
    actions = ["sns:Publish"]
    resources = [
          "arn:aws:sns:ap-south-1:058264131778:ec2-health-topic-email",
          "arn:aws:sns:ap-south-1:058264131778:ec2-health-topic-sms"
        ]
    }
}

# Policy creation getting in json
resource "aws_iam_policy" "sns-publish-policy" {
  name        = "sns-publish-policy"
  description = "SNS Publish only policy"
  policy      = data.aws_iam_policy_document.sns-publish-policy.json
}

# Attach the newly created policy to the newly created IAM role
resource "aws_iam_role_policy_attachment" "sns-policy-attach" {
  role       = aws_iam_role.sns-trigger-ec2.name
  policy_arn = aws_iam_policy.sns-publish-policy.arn
}

# Create an instance profile for the IAM role
resource "aws_iam_instance_profile" "sns-policy-attach" {
  name = "ec2_instance_profile"
  role = aws_iam_role.sns-trigger-ec2.name
}

# -----------------------------------------------------------

# SNS Topic creation and customization
resource "aws_sns_topic" "user_updates_email" {
  name = "ec2-health-topic-email"
  display_name = "EC2 Health topic email"
}

#resource "aws_sns_topic" "user_updates_sms" {
#  name = "ec2-health-topic-sms"
#  display_name = "EC2 Health topic sms"
#}

# Creating subscription
resource "aws_sns_topic_subscription" "user_update_email"{
    topic_arn = aws_sns_topic.user_updates_email.arn
    protocol = "email"
    endpoint = "------@gmail.com"
}

#resource "aws_sns_topic_subscription" "user_update_sms"{
#    topic_arn = aws_sns_topic.user_updates_sms.arn
#    protocol = "sms"
#    endpoint = "+91-------"
#}

Final part

I created a python script which creates the body and using webhook to send the message to discord channel of mine and one EMAIL to me via SNS and all run using virtual env and cronjobed it.

healthupdate.py

from os import system as sys
import boto3
from discord_webhook import DiscordWebhook

# Utility class
class Utility:

    commands = {
      "diskformat" : "df -h | tee test.txt",
      "cpuram" : "top -bn 1 | awk 'NR >= 1 && NR < 6 {print}' | tee test.txt",
      "cpu" : "mpstat -T | tee test.txt",
      "removetemp" : "rm temp.txt test.txt"
    };


    filename = [
      "temp.txt",
      "test.txt"
    ];

    def getmessage(command):
      sys(command);

      with open(Utility.filename[1], 'r') as file:
        data = file.read();

      with open(Utility.filename[0], 'w') as file:
        file.write("```

");
        file.write(data);
        file.write("

```");

      with open(Utility.filename[0], 'r') as file:
        data = file.read();

      sys(Utility.commands["removetemp"]);

      return data;

    def __init__(self):
      pass;

# aws SNS
class SNS:
    def __init__ (self, region):
      self.sns_client = boto3.client('sns', region_name = region);

    def publishmail(self, message, subject):
      self.sns_client.publish(TopicArn="arn:aws:sns:ap-south-1:058264131778:ec2-health-topic-email", Message = message, Subject = subject);

    def publishsms(self, message):
      self.sns_client.publish(TopicArn="arn:aws:sns:ap-south-1:058264131778:ec2-health-topic-sms", Message = message);

# Discord Webhook
class Discord:
    def __init__ (self, DWebhook):
      self.webhookurl = DWebhook;

    def trigger(self, message, uname):
      self.uname = uname;

      self.webhook = DiscordWebhook(url = self.webhookurl, username = self.uname, content = message)
      self.webhook.execute();


body = "";
subject = "Daily instance health check";

message = Utility.getmessage(Utility.commands["diskformat"]);
body += "**Disk Space data**\n" + message + '\n';

message = Utility.getmessage(Utility.commands["cpuram"]);
body += "**Top command output**\n" + message + '\n';

message = Utility.getmessage(Utility.commands["cpu"]);
body += "**CPU data mpstat output**\n" + message + '\n';


# Discord webhook triggers
webhook = Discord("https://discord.com/api/webhooks/1400674337966522399/gfdgdfgdgdfg----------yjfLwc");
webhook.trigger("# Daily instance health check\n\n" + body, "Doctor CPU");

# aws sns triggers
sns = SNS('ap-south-1');
sns.publishmail(body, subject);

I ended up removing SMS as EMAIL and discord is enough to have my eyes on it.

FAQ

In this project of mine I used DB as pod not RDS. That must have been question of many.
I used it as a pod because I wanted to understand cross pod communication as a developers perspective and also RDS is expensive for me currently. That is the reason why I didn't have the DB running in the Instance itself and choose the pod.

My Architecture evolution screenshots

Conclusion

Past 1.5 years of me planning and developing the project and its architectures was a journey to me. My experience across projects in my office, different architectures I changed my deployment architecture multiple times and each iteration was more improved one and many times I scrapped old architectures as I aimed to be able to set up the production as fast as possible and as automated as possible with minimal intervention of mine and if want new addition I can expand it without any problem.
I understood one thing while deploying and creating architecture. It is never about what top end tools we use or what top trending services we need. It all boils down to needs. What and how the budget or the application will be running maybe a few change in app to be compatible with this style of deployment hence a lot of communication between teams. Sometimes a legacy tool can help a lot with automation.
I could be wrong about many things here since this is my first company and my first experience in this field but after all I went through in office and here I came to this conclusion for now.

How I deployed my first project for my devops portfolio: Minikube and Deployment

Shahin Sheikh — Tue, 05 Aug 2025 06:29:29 +0000

OK, first before I begin, a lot of you may have a question in your mind. Why did I choose the minikube instead of EKS or just spin up the container using docker compose and be done with it. Well I have thought of very same thing at the very beginning. Hell I even thought of like just running my projects in simple container in EC2 and host a static S3 website. Then questions came to my mind like what if due to a bug in programming (Which there is since I just wanted to host project so I skipped the code safety) which can break the code and stop the container? Then I have to manually ssh it to start it and I wont be home forever to do it. Sure I can have a periodic health check for the responses using cronjob and if not found then spin it up again. It will take a bit of scripting but what if I have the ability like kubernetes which just spins it up as it goes down. There I decided with minikube and it will help me with learning of kubernetes to.

The quest for running minikube successfully in EC2

In my home environment I have it installed and ran it but there was a snag I saw. minikube tunnel is the way to make the site accessible through any other device in my private network else it was only responding to localhost:port. I thought of minikube tunnel in background, but what if it stops suddenly? I wanted something reliable.

There I was thinking of how the communications are to be meant if it was a kubernetes. Client --> EC2 ingress port ---> ingress routes to service ---> service routes to pods. In case of simple containers Client ---> EC2 port ----> Container port binding there.
In case of minikube with minikube tunnel active what I guess I figured it out like Client ---> EC2 ---> Minikube LoadBalancer IP <Minikube ip> ---> Ingress ---> Service ---> Pods.

Looking at this I figured if tunnel does is bridges gap between the ingress (The minikube running on the mini ip what I say it) and localhost. So as I now had a good guess of the working I wanted to test it by needing a way to route traffic coming from outside to the minikube ip. As I thought of this I remembered there is a term already for this thing, Reverse Proxy. I immediately went for the nginx and made this configurations in /etc/nginx/nginx.conf.

    upstream backend_servers {
        #server 10.110.163.66:8001;
        #server 10.100.252.56:8002;
        server 192.168.49.2:80;
    }

    server {
        listen 80;

        location / {
            #proxy_pass http://192.168.49.2:80;
            #proxy_pass http://192.168.49.2:8080;
            #proxy_pass http://localhost:8000;

             #proxy_pass http://10.110.163.66:8001;
             #proxy_pass http://192.168.49.2:80;

            # load balancing method
             proxy_pass http://backend_servers;
}
}

I wont be removing others, just keeping it raw just to show how I tested this by changing service ports in yaml files.

However This very same configurations were not working when being deployed to EC2. I tried to narrow the problem down by using same method of communications simulations in brain and paper and even using AI. I came to a conclusion that maybe traffic routing is not working properly as curl localhost was not working to. For this I went on a quest to learn iptables, and I found this incredible article here in dev and tried all shorts of stuff even though I understood little but yeah was going somewhere except the fact I was going nowhere. At that time I was reading about VPC, more accurately about NAT, and I kind of got the idea that iptables or netfilter type stuff is implemented in it that's why it was once an EC2 before becoming fully managed service. Anyway coming back to the topic I went on nearly 2 weeks for this solution and later thought of changing to other reverse proxy service and decided to try on apache2 and wallah it worked in an instant in EC2, so I settled on it.
/etc/apache2/sites-available/000-default.conf

<VirtualHost *:80>

    ProxyPreserveHost On

    # Reverse proxy for the application running on port 3000 on the same server
    ProxyPass / http://192.168.49.2:80/
    ProxyPassReverse / http://192.168.49.2:80/
</VirtualHost>

Before doing this we need to run the two commands and need a systemctl restart.

sudo a2enmod proxy
sudo a2enmod proxy_http

Ingress snag

This one is small but it took me 2 days to pin it down. When I had got my application running in minikube in EC2 I saw that two different pods taking same css path and this kind of making confusion as I had made file structure nearly same for html css but are in isolated environment only thing is ingress was routing wrong and later after one or two days I found out that in ingress we need to have the path written to the ingress else no connections will be formed, same thing happened to my /api path in my main application.

Soo many times destroying and restarting EC2

I have lost count on how many times I messed things up. Destroyed and recreated EC2 a lot many times. IaC is really something.
My main.tf

module "key_pair" {
  source             = "terraform-aws-modules/key-pair/aws"
  key_name           = "project-solo"
  create_private_key = true
}

# store 
resource "local_file" "private_key_pem" {
  content  = module.key_pair.private_key_pem
  filename = "${path.module}/project-solo.pem"
  file_permission = "0600"
}

# id would be aws_instance.my-first-terraform-ec2
resource "aws_instance" "my-first-terraform-ec2" {
  ami           = "ami-0131b8f4c937c332f"           # debian arm64 ami
  instance_type = "t4g.medium"
#  instance_type = "t4g.small"
  subnet_id               = aws_default_subnet.default.id
  vpc_security_group_ids = [aws_security_group.allow_tcp.id]

  key_name = module.key_pair.key_pair_name

  iam_instance_profile = "ec2_instance_profile"

  user_data = base64encode(<<-EOF
              #!/bin/bash
              apt-get update

              # install kubectl
              curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/arm64/kubectl"
              sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

              # install minikube
              curl -LO https://github.com/kubernetes/minikube/releases/latest/download/minikube-linux-arm64
              sudo install minikube-linux-arm64 /usr/local/bin/minikube && rm minikube-linux-arm64

              # install docker
              sudo apt-get install ca-certificates curl
              sudo install -m 0755 -d /etc/apt/keyrings
              sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
              sudo chmod a+r /etc/apt/keyrings/docker.asc

              # Add the repository to Apt sources:
              echo \
                "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
                $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
                sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
              apt-get update

              # install docker
              sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

              sudo apt install apache2 -y
              sudo systemctl enable apache2
              sudo systemctl start apache2

              alias k="kubectl"
              EOF
  )

  tags = {
    Name = "Portfolio EC2"
  }

# ebs root volume
  root_block_device {
    delete_on_termination = true
    volume_size = 30
    volume_type = "gp3"
  }
}


# going to use default vpc id
# The ID here becomes data.aws_vpc.default.id
data "aws_vpc" "default"{
  default = true
}

# default subnet az1 i choose for my ec2
resource "aws_default_subnet" "default" {
  availability_zone = local.aws_az["indiaAZ1"]
}


# create a security group with default vpc ID
resource "aws_security_group" "allow_tcp" {
  name = "allow_tcp"
  description = "Allow tcp inbound traffic"
  vpc_id = data.aws_vpc.default.id

  tags = {
    Name = "allow_tcp"
  }
}

# ingress rule for tcp to port 22 from anywhere in the world for ssh
resource "aws_vpc_security_group_ingress_rule" "allow_tcp_22_ipv4" {
  security_group_id = aws_security_group.allow_tcp.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 22
  ip_protocol       = "tcp"
  to_port           = 22
}

# ingress rule for tcp to port 80 from anywhere in the world for nginx httpd 80
resource "aws_vpc_security_group_ingress_rule" "allow_tcp_80_ipv4" {
  security_group_id = aws_security_group.allow_tcp.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 80
  ip_protocol       = "tcp"
  to_port           = 80
}

# an eggress rule to allow tcp from vm to the www
resource "aws_vpc_security_group_egress_rule" "allow_all_traffic_ipv4" {
  security_group_id = aws_security_group.allow_tcp.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1" # semantically equivalent to all ports
}

Note: Ignore the VPC CIDR part as I was practicing VPC stuff using terraform and same vars file I used in my main.tf file

My vars.tf

variable "aws_regions" {
    type = map(string)
    default = {
        india = "ap-south-1"
    }
}

locals {
    aws_az = {
        indiaAZ1 = "${var.aws_regions["india"]}a"
        indiaAZ2 = "${var.aws_regions["india"]}b"
        indiaAZ3 = "${var.aws_regions["india"]}c"
    }
}

# CIDR array for vpc to be created for individual use
variable "vpc_cidr_individual" {
    type = map(string)
    default = {
        cidr1 = "10.1.0.0/16"
        cidr0 = "10.0.0.0/16"
    }

    description = "CIDR blocks of the vpc for individual use"
}

# CIDR array for subnet
variable "vpc_cidr_subnet" {
    type = map(string)
    default = {
        publicA     = "10.0.0.0/24"
        publicB     = "10.0.1.0/24"
        privateA    = "10.0.16.0/20"
        privateB    = "10.0.32.0/20"
        peerpublicA = "10.1.0.0/24"
    }
}


# environment
variable "environment" {
    default = "Shahin"
}

One more thing when my EC2 starts up I need to use command sudo usermod -aG docker $USER && newgrp docker as I dunno why using it in user data script didn't seem to work. If any of you know the reason please let me know in comments below, thank you.

And after this step I run this command minikube start --driver=docker &&kubectl create namespace static &&kubectl create namespace sololeveling &&minikube addons enable ingress and apache2 config for reverse prxy and my EC2 becomes ready for the github actions to ssh and apply the yaml and start the application. All that is left to update the secrets variable in my actions with the newly generated .pem file.

How I deployed my first project for my devops portfolio: CI/CD during development vs CI/CD while live

Shahin Sheikh — Fri, 25 Jul 2025 08:41:20 +0000

CI/CD during development and CI/CD now during live

CI/CD setup during development

First I setup my Jenkins. I didn't want to manually run jenkins every time I boot up my raspberry pi so I made a Custom Daemon Service for it.

[Unit]
Description=Jenkins
After=network.target

[Service]
Type=simple
User=<username>
Group=users
ExecStart=/home/<username>/jenkins/jdk-21.0.5/bin/java -Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true -DJENKINS_HOME=/home/<username>/jenkins/.jenkins-config-new -jar /home/<username>/jenkins/jenkins.war
Restart=always

[Install]
WantedBy=multi-user.target

Then I systemctl enabled and it was working fine. The journalctl -xeu jenkins.service helped me to get the initial password as jenkins was starting the first time.

Second I have installed docker and docker-compose in my pi and using it I used to spin up the containers.

Third At first I used to checkout to local repo that I created using git init --bare at my HDD but later shifted to github. This reason will explain the Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true in the daemon service.

Fourth I used custom database that I got from this mysql generic bianries instead of pulling the already present image at the dockerhub. Reason behind is that I wont be changing anything later and also I don't want to use the env line in compose at that time. All I wanted is just let the DB container get spunned up and all get ready so that is the reason why I customized it to my own liking.
Here is the entrypoint script for the custom DB image I made.

! /bin/bash

bin/mysqld --initialize-insecure --user=mysql --bind-address=0.0.0.0
bin/mysqld_safe --user=mysql &

# waiting time for the mysql_safeto start
for((i=20; ; --i)); do
    if((i == 0)); then
        echo "Starting Database...";
        break;
    fi

    echo -ne "Starting Database in ...${i} sec [/] \r";
    sleep 0.2;
    echo -ne "Starting Database in ...${i} sec [-] \r";
    sleep 0.2;
    echo -ne "Starting Database in ...${i} sec [|] \r";
    sleep 0.2;
    echo -ne "Starting Database in ...${i} sec [-] \r";
    sleep 0.2;
    echo -ne "Starting Database in ...${i} sec [\\] \r";
    sleep 0.2;


done

bin/mysql -u root --skip-password  -e "ALTER USER 'root'@'localhost' IDENTIFIED BY '<password>';"
bin/mysql -u root -p<inline_password> -e "UPDATE mysql.user SET Host = '%' WHERE User = 'root' AND Host = 'localhost';"
bin/mysql -u root -p<inline_password> -e "FLUSH PRIVILEGES;"
bin/mysql -u root -p<inline_password> -e "create database Users;"

echo "Creating importance filler table"
bin/mysql -u root -p<inline_password> -D Users -e "create table if not exists importance(class varchar(1) primary key not null, xp_gain double not null);"
echo "Filling importance table"
bin/mysql -u root -p<inline_password> -D Users -e "insert into importance values('S', 10);"
bin/mysql -u root -p<inline_password> -D Users -e "insert into importance values('A', 8);"
bin/mysql -u root -p<inline_password> -D Users -e "insert into importance values('B', 6);"
bin/mysql -u root -p<inline_password> -D Users -e "insert into importance values('C', 4);"
bin/mysql -u root -p<inline_password> -D Users -e "insert into importance values('D', 2);"
bin/mysql -u root -p<inline_password> -D Users -e "insert into importance values('E', 1);"


# start node exporter
#./node_exporter --web.listen-address=:5000

# to keep the container up and alive
echo "DB server online..." | tee stay.txt
tail -f stay.txt

Here I got the snag of DB not connecting at localhost so upon scraping the web I found out that in mysql.user table in HOST there is something denied so that is the reason why I used this query below and it started working.

bin/mysql -u root -p<inline_password> -e "UPDATE mysql.user SET Host = '%' WHERE User = 'root' AND Host = 'localhost';"
bin/mysql -u root -p<inline_password> -e "FLUSH PRIVILEGES;"

CI/CD Workflow during development

I made/added in my code then I pushed it to repo and using the jenkins pipeline I created it builds the images and then docker compose up it in the pipeline itself. Then I checked and repeated this entire process.

CI/CD Setup and Workflow as it is live

As I was planning to set up some kind of CI/CD during this live as I want to put off some updates there and my plan was very tedious and time consuming like setting up jenkins there then after code push hit jenkins build and yada yada... I got introduced to github actions when I was searching for things like "How to trigger jenkins job as I push code to github repo". Man I loved the actions and implemented it in my git repo.

I currently have two branches, one main and one website. Main has my project and website has my personal website http://shahin.zita.click/Shahin with both having .github/workflows/.yaml each. I did used the if condition if: github.ref == 'refs/heads/<branch_name>' to run only specific yaml for the branch.

Snag: As I saw that actions has many vm and so I choose the ubuntu-24.04-arm since my project is built on ARM but I found out that for some reason when I was compiling the code in actions, building and pushing the image to my dockerhub repo and when I was trying to pull it back into my EC2 or my rasp it was not working. Upon some search I came to conclusion that its nothing but environment inconsistency, so I came up with a utility container idea which solely compiles the code and outputs the generated bianry for the next step where new image is built with the generated binaries and push it to dockerhub for the pull and it started working fine.

docker run --rm -v $PWD:/build-app:rw schd1337/portfolioapp:compiler

I used volume mount along with uses: actions/checkout@v3 with rw permissions and it works like a charm.

So now if I want a small change I just make the change in local and then push it to github and in the actions it compiles and pushes the image and then in next job I used a simple ssh into the EC2 using the secrets in actions to apply the yaml file.

    - name: SSH into EC2 and deploy
      run: |
            echo "${{ secrets.EC2_KEY }}" | tee $PWD/ec2.pem
            chmod 600 $PWD/ec2.pem
            ssh -o "StrictHostKeyChecking=no" -i "ec2.pem" admin@ec2-13-235-247-44.ap-south-1.compute.amazonaws.com "git clone https://github.com/AstroKabutar/SoloLeveling.git"
            ssh -o "StrictHostKeyChecking=no" -i "ec2.pem" admin@ec2-13-235-247-44.ap-south-1.compute.amazonaws.com "kubectl apply -f SoloLeveling/kubernetes/dbdeploy.yaml"
            ssh -o "StrictHostKeyChecking=no" -i "ec2.pem" admin@ec2-13-235-247-44.ap-south-1.compute.amazonaws.com "kubectl apply -f SoloLeveling/kubernetes/project-upskill.yaml"
            ssh -o "StrictHostKeyChecking=no" -i "ec2.pem" admin@ec2-13-235-247-44.ap-south-1.compute.amazonaws.com "kubectl apply -f SoloLeveling/kubernetes/ingress.yaml"
            ssh -o "StrictHostKeyChecking=no" -i "ec2.pem" admin@ec2-13-235-247-44.ap-south-1.compute.amazonaws.com "rm -rf SoloLeveling"
      shell: bash

Problem: Even if this works good I have to manually ssh into the EC2 to delete pods so I am thinking of should I go for simple kubectl delete then apply again or even better using some simple bash script using awk to delete the pods from the actions ssh itself. Either way I will update the post as I come to any of the conclusions and why.

EDIT : I updated my workflow to run additional bash script that gets the pod names and deletes it and all I have to do is wait and its done.

kubectl get pods -n<namespace> | awk 'NR == 2 {print $1}' | xargs kubectl delete pods -n<namespace>

Conclusion

From what I have seen in companies and my own project, what I found out is that on offshore deployment scenarios it's best to have jenkins to do the stuff and with a separate repo or branch would suffice and when that code is approved for next stage like prod or pre-prod the code must be merged to main and let actions take over for the EKS cluster or EC2 or whatever. It makes work easy and using terraform (IaC) is the best way to keep track and also make changes without manually clicking across the console as its far too much prone to human error.

How I deployed my first project for my devops portfolio: The HUNT (Infra and budget)

Shahin Sheikh — Fri, 25 Jul 2025 06:04:00 +0000

The hunt for the cheapest infra without loosing performance

After I successfully made my project, I proceeded for my plans on infra on where to host it. I made it on my raspberry pi 4b and also did it in CI/CD way (More on the CI/CD details in later posts). Which means I need an ARM64 architecture for my target infra on which I need to host my project.
Since I got introduced to AWS in my job so I thought of going with AWS here so I looked in this aws pricing calculator.

At the very first glance x86-64 costing way too much compared to arm64. I didn't know why buy I do know seeing all my life home labbing and using pi I do get a general idea of performance and power consumption does play some role. I found this article as the time of writing, the question of arm vs x86 was intriguing to me. Though I not fully understood it but it did gave me the idea.

Continuing to my budget setting I at first selected t2.small which has 2vCPUs and 2GB RAM. Since I am using minikube for my deployment which requires 2CPUs and 2GB RAM which is exactly what the instance type t4g.small has. I knew that even if I start the minikube in that then deployments will fail and most probably will hang up the system. So using my terraform I spun up this t4g.small and as I thought after enabling ingress and then deploying my project and it started lagging a bit. But when I tried deleting a pod to see how the system reacts and it became disastrous for me. I cant even ssh back into my instance it lagged so much. I scraped the plan and went for the better which takes a bit more money but sure even have the space for some upgrades with no performance sacrifices. t4g.medium which has 2vCPUs and 4G RAM, and as expected it works fine. I even tested by deleting pods repeatedly and saw no problem with deployments. I saw 100% utilization of this instance will cost me Monthly: 16.35/Month which in INR 1,414.66/Month. After this I bought a cheap domain for 2 years zita.click which costed me 6/Month which converts to INR 519.14/Month and also keeping one Hosted Zone alive costs 0.50/Month -- INR 43/Month. All this totals to 1414.66 + 43 = 1457.66/Monthly with 519.14 as setup cost (domain cost for 2 years). Now seeing that it's just a portfolio so less queries and less data outflow it might increase to i guess <= 1650. Seeing this I set my budget to INR 2000/Monthly. Then I decided to Reserve it EC2 Instance Savings Plans which trickles the cost down to 6.94/Month -- INR 600.47/Month reserved for 3 years. Since I already got the domain for 2 years, I have plans on extending it for one more year before changing the domain itself.

Snag - Upon trying to reserve instance t4g.medium only one instance for 3 years showed me instance quota full even if I had got none running. I raised the quota increase limit and wallah, thanks aws from default 20 I got 30 now. As time of writing I am currently running on On-Demand but will purchase the RI soon.

Aaah, as I uploaded the pic above i saw this https://dev-to-uploads.s3.amazonaws.com/uploads/articles/abool2qqcayntzfeh7ju.PNG
Bucket name - dev-to-uploads
Object key - uploads/articles/abool2qqcayntzfeh7ju.PNG
Object - abool2qqcayntzfeh7ju.PNG
We can only imagine the power of cloud here and how much freedom and flexibility it brings to us.

With this the hunt for my own budget which is INR 2000/Monthly and infrastructure selection which is t4g.medium is completed.

How I deployed my first project for my devops portfolio: Project Architecture

Shahin Sheikh — Sat, 19 Jul 2025 09:43:02 +0000

Project Architecture

This project (Github branch main) I built this app entirely in CPP and I used the CrowCpp framework to run it.

I created some custom includes with all the custom libraries for each one.

Mysql.h MySQL Connector/C++ to grant the ability to my application to be able to connect to my mysql database. I got the sql cpp con from this package libmysqlcppconn-dev.
Player.h which comes in as a new player gets created. It handles the initialization stuff like initial xp date of creation xp required. Basically sets the player up.
PlayerGrowth.h comes in when player sets or completes tasks. The calculation of rewarding xp by fetching from database and setting new ones back.
Setup.h is the bootstrap for the application that creates the tables required for the application in the database.

I used CMAKE as my compiling tool followed by make.

So in summery as new player is created Player.h comes in to do the formalities with giving back the UID generated to the player. As in LoadGame section when tasks created or completed and displaying of stats is handled by PlayerGrowth.h and as the app starts the Setup.h bootstraps it. Mysql.h works behind for the communication from app to database. The Auxiliary.h is just another custom header I made to enforce DRY(Don't Repeat Yourself)

How I deployed my first project for my devops portfolio: The Project

Shahin Sheikh — Sat, 19 Jul 2025 08:07:52 +0000

About the project

This project that I made is just a simple task completion, where you insert a task and rate its importance (S,A,B,C,D,E) and based on its importance you will get rewarded with suitable XP as the task gets completed.

Simple working of the project

Project SoloLeveling is the landing page where you will be greeted with name, dob for new player and LoadGame for existing player.

As the new player inserts their name and DOB, a ID is created and must be saved.

As an existing player we greeted with a new page as we hit the LoadGame.

--> To check the XP of the player: At the end of page there is Player name filed and an ID field. Insert your name and the ID got during creation and submit. You will get the current player XP and Level.

--> To add new task: Enter the task under New task and then select the importance from drop down menu beside it. Then again at the bottom Player Name and ID needs to be entered before submit to save it in the database.

--> To complete a task: Your created tasks will be shown at the very top with the id of the task. Insert the task id which is completed in the field below the New Task section and yes again with player name and ID before submitting the task.

XP calculation and importance

First with importance: Below are the importance characters and its corresponding value I decided on.

('S', 10)
('A', 8)
('B', 6)
('C', 4)
('D', 2)
('E', 1)

XP calculation: On task completion, based on the importance value I settled on a simple formula.

XP GAINED = (reward * current XP) / 100

For leveling up as in games an xp needed to be filled. So behind that I have made another simple formula.

XP REQUIRED += (30 * required XP) / 100 + 8

FAQ

Question: xp gained = (reward * current xp) / 100. New player has 0 XP so anything x 0 is 0, so why that?
Answer: When new players signs up, the player gets an xp for that, hence every new player starts with 3.48 XP.

How I deployed my first project for my devops portfolio: Introduction

Shahin Sheikh — Sat, 19 Jul 2025 08:04:41 +0000

INTRODUCTION

Hi all, this is my first time writing anything like this in my life. I am an aspiring cloud engineer/architect whatever you like to call it, as I love cloud tech and actively looking for opportunities.

Last year as I joined my first company and got introduced to this concept of Devops I was in awe of this tech. I started to get ideas as how much we can do stuff by leveraging the cloud tech. And I have always been homelabbing before. Joining my first company and getting oppertunity in devops and cloud which is why I am much more inclined to this technology. I see many opportunities like getting a ML model or like realtime data from IoT, possibilities are endless.

As to get recognized by companies in this tough competitive job market I got an idea of why not just diving into it and try making and deploying a real life simple project and see how messy the actual ground is. The project idea I got from watching an anime and yes that project type exists already everywhere app to web but my aim is to understand deployment and leverage the cloud with less cost and optimum performance. So a simple project and a good start as a beginner. I aim to learn how, what, where to use what services and what not. After all cloud is just another peoples computer out for rent in a nutshell.

For the past 1.5 years as the time of writing this article, I successfully hosted my website and my project in devops style using IaaC.
My website here

I hope you have a good time reading and learning and potentially advising improvement to my steps from that I can learn from you all to.
Thank you.