DEV Community: bhaktraj

How to Compile and Install Python 3.12+ on Amazon Linux 2

bhaktraj — Sat, 03 Jan 2026 02:41:25 +0000

🧠 Introduction

Amazon Linux 2 does not provide Python 3.12 via default repositories.
To use the latest Python features, better performance, and modern TLS security, we must compile Python from source.

This script:

Builds OpenSSL 1.1.1 manually
Compiles Python 3.12 against it
Installs Python without breaking system Python
Creates a virtual environment safely

🔹 Script Overview – What This Script Does

✔ Updates the system
✔ Installs development tools
✔ Builds OpenSSL from source
✔ Sets environment variables for secure compilation
✔ Builds Python 3.12 from source
✔ Installs Python safely using altinstall
✔ Creates a Python virtual environment

#!/bin/bash
sudo yum update
sudo yum groupinstall -y "Development Tools"
# sudo yum install -y openssl-devel bzip2-devel libffi-devel zlib-devel readline-devel sqlite-devel ncurses-devel gdbm-devel db4-devel expat-devel
sudo yum install -y openssl-devel bzip2-devel libffi-devel zlib-devel \
    readline-devel sqlite-devel ncurses-devel gdbm-devel \
    db4-devel expat-devel


sudo yum install -y gcc gcc-c++ make perl-core zlib-devel
cd /tmp/
wget https://www.openssl.org/source/openssl-1.1.1w.tar.gz
tar xzf openssl-1.1.1w.tar.gz
cd openssl-1.1.1w
./config --prefix=/usr/local/openssl --openssldir=/usr/local/openssl shared zlib
make -j $(nproc)
sudo make install
echo '/usr/local/openssl/lib' | sudo tee /etc/ld.so.conf.d/openssl.conf
sudo ldconfig
export PATH="/usr/local/openssl/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/openssl/lib:$LD_LIBRARY_PATH"
export PKG_CONFIG_PATH="/usr/local/openssl/lib/pkgconfig:$PKG_CONFIG_PATH"
export CPPFLAGS="-I/usr/local/openssl/include"
export LDFLAGS="-L/usr/local/openssl/lib"
sudo yum install -y xz-devel libuuid-devel tk-devel tcl-devel
openssl version
cd /tmp/
wget https://www.python.org/ftp/python/3.12.0/Python-3.12.0.tgz
tar xzf Python-3.12.0.tgz
cd Python-3.12.0

#  Configure the Python build.
# --enable-optimizations: improves performance by running multiple tests during build (optional but recommended)
# --with-openssl=/usr/include/openssl: ensures Python is compiled with the system's OpenSSL library for secure connections

./configure --prefix=/usr/local --enable-optimizations --with-ensurepip=install

make -j $(nproc)
sudo make altinstall

# Compile and install Python using 'make altinstall'.
sudo make altinstall
/usr/local/bin/python3.12 --version
/usr/local/bin/pip3.12 --version



#sudo update-alternatives --install /usr/bin/python3 python3 /usr/local/bin/python3.12 2
#sudo update-alternatives --config python3

/usr/local/bin/python3.12 -m venv venv
source venv/bin/activate

✅ Recommended & SAFE Symlink (Best Practice)
🔹 Create symlink for python3.12

sudo ln -s /usr/local/bin/python3.12 /usr/bin/python3.12

🔹 Create symlink for pip3.12

sudo ln -s /usr/local/bin/pip3.12 /usr/bin/pip3.12

🔹 Verify

python3.12 --version
pip3.12 --version

Zero-Downtime Application Deployment Using Blue-Green Architecture

bhaktraj — Tue, 15 Apr 2025 09:45:45 +0000

Continuous Integration and Continuous Deployment (CI/CD) have become the backbone of DevOps, allowing faster delivery of applications. However, deployment failures can introduce downtime and affect user experience. To combat this, Blue-Green Deployment offers a mechanism to deploy new versions with zero downtime by running two environments simultaneously — one active (Green) and one standby (Blue). This project implements an automated CI/CD pipeline with integrated security and quality tools, along with containerization and Kubernetes-based orchestration to achieve safe and efficient deployments.

Literature Review
CI/CD Pipelines: CI/CD automates the process of software delivery and deployment. Jenkins is a widely adopted open-source automation tool for CI/CD.

Code Quality Tools: Tools like SonarQube evaluate code quality by checking for bugs, vulnerabilities, and code smells.

Security Scanning: OWASP Dependency Check and Trivy help identify security flaws in dependencies and container images respectively.

Blue-Green Deployment: This deployment technique provides a safe way to roll out changes without affecting live traffic.

Amazon EKS: A managed Kubernetes service that simplifies deploying, managing, and scaling containerized applications.

Methodology

The project methodology is based on an agile and iterative approach where each phase of the CI/CD pipeline is independently developed, tested, and integrated. Key tools and technologies used include:

Jenkins for pipeline orchestration.

SonarQube for static code analysis.

OWASP Dependency Check for third-party vulnerability scanning.

Trivy for Docker image vulnerability scanning.

Docker Hub for image storage.

Amazon EKS for orchestrating Blue-Green deployments.

Setting Up the Foundation

Spinning up EC2 instances (t2.large for Jenkins, t2.small for EKS management).

Download Following tools in Jenkins Server (t2.large)

Jenkins : Install Jenkins https://dev.to/bhaktraj/how-to-install-jenkins-in-ubuntu-2735
Docker : Install Docker https://dev.to/bhaktraj/docker-install-in-ubuntu-or-on-cloud-aws-provisioning-mfi
SonarQube : After the docker installation, we create a sonarqube container (Remember to add 9000 ports in the security group).

docker run -d --name sonarqube \
  --restart always \
  -p 9000:9000 \
  -v sonarqube_conf:/opt/sonarqube/conf \
  -v sonarqube_data:/opt/sonarqube/data \
  -v sonarqube_extensions:/opt/sonarqube/extensions \
  -v sonarqube_logs:/opt/sonarqube/logs \
  sonarqube:lts-community

Now our sonarqube is up and running on 9000 port
Enter username and password, click on login and change password

username admin
password admin

Update New password, This is Sonar Dashboard.

Trivy : Install Trivy

vim trivy.sh

sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y

AWS CLI : Configure AWS With the help of IAM user Create a IAM User and and access with the command line

IAM
_Create a user “eks-admin” with AdministratorAccess
Create Security Credentials Access Key and Secret access key _

Install AWS CLI v2

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip
unzip awscliv2.zip
sudo ./aws/install -i /usr/local/aws-cli -b /usr/local/bin --update

Setup your access by

aws configure

kubectl : Install

curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin
kubectl version --short --client

Next, we will log in to Jenkins and start to configure our Pipeline in Jenkins its running on 8080 ports

http://ip_address:8080

Password is available on

sudo cat sudo cat /var/lib/jenkins/secrets/initialAdminPassword

copy it and paste to jenkins login

and then configure it
First some basic configure you have to configue

and then after
Install Plugins like

Goto Manage Jenkins →Plugins → Available Plugins
search these plugin and install it

Sonarqube Scanner,
OWASP Dependency Check,
Docker, Docker Commons, Docker Pipeline, Docker API, docker-build-step
Slack Notification
Kubernetes CLI Plugin, Kubernetes Client API Plugin, Kubernetes Credentials Plugin, Kubernetes plugin

Configure Sonar Server in Manage Jenkins:

Grab the Public IP Address of your EC2 Instance, Sonarqube works on Port 9000,
so :9000.
Goto your Sonarqube Server. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and click on Generate Token

copy Token

Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this

You will this page once you click on create

Now, go to Dashboard → Manage Jenkins → System and Add like the below image.

you can leave the server url because sonarqube is running in local host or you can paste the public ip
like
http://ip_address:9000

Click on Apply and Save

The Configure System option is used in Jenkins to configure different server

Global Tool Configuration is used to configure different tools that we install using Plugins

We will install a sonar scanner in the tools.

Configure OWASP Dependency Check Tool:

Plugin is install now we had to configure the Tool
Goto Dashboard → Manage Jenkins → Tools →

Click on Apply and Save here.

Configure Docker Tools:

Add DockerHub Username and Password under Global Credentials

Goto Jenkins Dashboard → Manage Jenkins → Credentials → set username and password It should look like this

For more Security
Create a token on docker hub and paste it on password

Goto DockerHub Dashboard → Account Setting → Personal access tokens → It should look like this

Password Prompt paste it to password section of jenkins credential

Now configure Slack For notification

goto https://slack.com/marketplace/A0F7VRFKN-jenkins-ci

Click on add to slack

Select the channel and add jenkins ci integration

Follow the steps that are mention or simply copy the token and paste to jenkins Credential

and update the slack setting in jenkins
by
go to Dashboard → Manage Jenkins → System and Add like the below image.

and details are taken by slack dashboard of that channel check it is it wokking properly or not if success message show that means slack is connected

Now Setting a EKS Management server

Launch a EC2 Instance with t2small instance type

Configure the following things

IAM
_Create a user “eks-admin” with AdministratorAccess
Create Security Credentials Access Key and Secret access key _

Install AWS CLI v2

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip
unzip awscliv2.zip
sudo ./aws/install -i /usr/local/aws-cli -b /usr/local/bin --update

Setup your access by

aws configure

curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin
kubectl version --short --client

Install eksctl

curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version

Setup EKS Cluster

eksctl create cluster --name blue-green-deployment --region us-east-1 --node-type t2.medium --nodes-min 2 --nodes-max 2 --node-volume-size 10
aws eks update-kubeconfig --region us-east-1 --name blue-green-deployment
kubectl get nodes

Kubernetes Configuration for Jenkins Deployment Access

Run Kubernetes manifest files to:

Create a Service Account for Jenkins.

vi serviceaccount.yml

apiVersion: v1 
kind: ServiceAccount 
metadata: 
  name: jenkins 
  namespace: webapps

note: edit the intent

Create a Role.

vi role.yml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: app-role
  namespace: webapps
rules:
  - apiGroups:
        - ""
        - apps
        - autoscaling
        - batch
        - extensions
        - policy
        - rbac.authorization.k8s.io
    resources:
      - pods
      - componentstatuses
      - configmaps
      - daemonsets
      - deployments
      - events
      - endpoints
      - horizontalpodautoscalers
      - ingress
      - jobs
      - limitranges
      - namespaces
      - nodes
      - secrets
      - pods
      - persistentvolumes
      - persistentvolumeclaims
      - resourcequotas
      - replicasets
      - replicationcontrollers
      - serviceaccounts
      - services
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Create a Role and bind it to the service account.

vi rolebindservice.yml

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: app-rolebinding
  namespace: webapps 
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: app-role 
subjects:
- namespace: webapps 
  kind: ServiceAccount
  name: jenkins

Create a ClusterRole and bind it as well.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: jenkins-cluster-role
rules:
- apiGroups: [""]
  resources: ["persistentvolumes"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-cluster-role-binding
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: webapps
roleRef:
  kind: ClusterRole
  name: jenkins-cluster-role
  apiGroup: rbac.authorization.k8s.io

Generate token using service account in the namespace

create token

vi token.yml

apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  name: mysecretname
  annotations:
    kubernetes.io/service-account.name: jenkins

Run Manifests

kubectl create namespace webapps
kubectl apply -f . -n webapps

Copy the token and add it to Jenkins credentials to allow Jenkins to deploy applications onto EKS securely.

kubectl describe secret mysecretname -n webapps

Paste this token to jenkns credential

Goto Jenkins Dashboard → Manage Jenkins → Credentials → set Secret key
It should look like this

Create a Job on Jenkins
Dashboard and then New Item name the job "Blue-green-Deployment"(What every you want) and then click on ok

Upload the Script And Click on OK

def COLOR_MAP = [
    'success': 'good',
    'FAILURE': 'danger',
]
pipeline {

    agent any


    parameters {
        choice(name: 'DEPLOY_ENV', choices: ['blue', 'green'], description: 'Choose which environment to deploy: Blue or Green')
        booleanParam(name: 'SWITCH_TRAFFIC', defaultValue: false, description: 'Switch traffic between Blue and Green')
    }

    stages{
        stage('clean workspace'){
            steps{
                cleanWs()
            }
        }

        stage('Fetch the code'){
            steps{
                git url: 'https://github.com/bhaktraj/vkonsec.git', branch: 'kubernetes'

            }    
        }

        stage('TRIVY FS SCAN') {
            steps {
                sh "trivy fs . > trivyfs.txt"
            }
        }

        stage('sonarqube scan'){
            environment {
             scannerHome = tool 'Sonarscanner'
          }

          steps {
            withSonarQubeEnv('Sonarscanner') {
               sh '''${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=vkonsec \
                   -Dsonar.projectName=vkonsec \
                   -Dsonar.projectVersion=1.0 \
                   -Dsonar.sources=. \
                   '''
            }
        }
        }
        stage('OWASP Dependency Check') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --format XML --out dependency-check-report --project vkonsec-django --enableExperimental', odcInstallation: 'Owasp'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'

            }
        }
        stage('Build docker images'){
            steps{
                script{
                    dockerimage = docker.build('vkonsec' + ":$BUILD_NUMBER", ".")
                    dockerimage = docker.build('nginx' + ":$BUILD_NUMBER", "./nginx")

                }

            }
        }
        stage("TRIVY"){
            steps{
                sh "trivy image vkonsec:$BUILD_NUMBER > trivy.txt" 
            }
        }
        stage("Docker Push"){
            steps{
                script{
                   withDockerRegistry(credentialsId: 'dockercred', toolName: 'docker'){   
                       sh "docker tag vkonsec:$BUILD_NUMBER bhaktraj/vkonsec:$BUILD_NUMBER "
                       sh "docker tag nginx:$BUILD_NUMBER bhaktraj/nginx:$BUILD_NUMBER "
                       sh "docker push bhaktraj/vkonsec:$BUILD_NUMBER "
                       sh "docker push bhaktraj/nginx:$BUILD_NUMBER "
                    }
                }
            }
        }
        stage('Persistent Volume Claim for MySQL') {
            steps {
                script {
                    withKubeConfig(caCertificate: '', clusterName: 'blue-green-deployment', contextName: '', credentialsId: 'kubecred', namespace: 'webapps', restrictKubeConfigAccess: false, serverUrl: 'https://3CA6E485D43C0E5E4EB7B7108ECACEA2.gr7.us-east-1.eks.amazonaws.com') {
                        sh """ if ! kubectl get pvc mysql-pvc -n webapps; then
                                kubectl apply -f k8/mysql-pvc.yaml -n webapps
                            fi
                        """  
                    }
                }
            }
        }
        stage('Deploying MySQL') {
            steps {
                script {
                    withKubeConfig(caCertificate: '', clusterName: 'blue-green-deployment', contextName: '', credentialsId: 'kubecred', namespace: 'webapps', restrictKubeConfigAccess: false, serverUrl: 'https://3CA6E485D43C0E5E4EB7B7108ECACEA2.gr7.us-east-1.eks.amazonaws.com') {
                        sh """ 
                        kubectl apply -f k8/mysql-deployment.yaml -n webapps
                        kubectl apply -f k8/mysql-service.yaml -n webapps
                        """  
                    }
                }
            }
        }

        stage('Update K8s Manifest') {
            steps {
                sh " sed -i 's/buildid/$BUILD_NUMBER/g' k8/django-blue-deployment.yaml "
                sh " sed -i 's/buildid/$BUILD_NUMBER/g' k8/django-green-deployment.yaml "
                sh " sed -i 's/buildid/$BUILD_NUMBER/g' k8/nginx-deployment.yaml "
            }
        }

        stage('Application deployment') {
            steps {
                script {
                    def deploymentFile = ""
                    if (params.DEPLOY_ENV == 'blue') {
                        deploymentFile = 'k8/django-blue-deployment.yaml'
                    } else {
                        deploymentFile = 'k8/django-green-deployment.yaml'
                    }

                    withKubeConfig(caCertificate: '', clusterName: 'blue-green-deployment', contextName: '', credentialsId: 'kubecred', namespace: 'webapps', restrictKubeConfigAccess: false, serverUrl: 'https://3CA6E485D43C0E5E4EB7B7108ECACEA2.gr7.us-east-1.eks.amazonaws.com') 
                    {
                        sh "kubectl apply -f ${deploymentFile} -n webapps"
                        sh "kubectl apply -f k8/django-service.yaml -n webapps"
                    }
                }
            }
        }

        stage('Switch Traffic Between Blue & Green Environment') {
            when {
                expression { return params.SWITCH_TRAFFIC }
            }
            steps {
                script {
                    def newEnv = params.DEPLOY_ENV

                    // Always switch traffic based on DEPLOY_ENV
                    withKubeConfig(caCertificate: '', clusterName: 'blue-green-deployment', contextName: '', credentialsId: 'kubecred', namespace: 'webapps', restrictKubeConfigAccess: false, serverUrl: 'https://3CA6E485D43C0E5E4EB7B7108ECACEA2.gr7.us-east-1.eks.amazonaws.com') {
                        sh '''
                            kubectl patch service djangoapp -p "{\\"spec\\": {\\"selector\\": {\\"app\\": \\"djangoapp\\", \\"version\\": \\"''' + newEnv + '''\\"}}}" -n webapps
                        '''
                    }
                    echo "Traffic has been switched to the ${newEnv} environment."
                }
            }
        }
        stage('Deploy Nginx') {
            steps {
                script {
                    withKubeConfig(caCertificate: '', clusterName: 'blue-green-deployment', contextName: '', credentialsId: 'kubecred', namespace: 'webapps', restrictKubeConfigAccess: false, serverUrl: 'https://3CA6E485D43C0E5E4EB7B7108ECACEA2.gr7.us-east-1.eks.amazonaws.com') {
                        sh """ 
                        kubectl apply -f k8/nginx-deployment.yaml -n webapps
                        kubectl apply -f k8/nginx-service.yaml -n webapps
                        """  
                    }
                }
            }
        }

        stage('Verify Deployment') {
            steps {
                script {
                    def verifyEnv = params.DEPLOY_ENV
                    withKubeConfig(caCertificate: '', clusterName: 'blue-green-deployment', contextName: '', credentialsId: 'kubecred', namespace: 'webapps', restrictKubeConfigAccess: false, serverUrl: 'https://3CA6E485D43C0E5E4EB7B7108ECACEA2.gr7.us-east-1.eks.amazonaws.com') {
                        sh """
                        kubectl get pods -l version=${verifyEnv} -n webapps
                        kubectl get svc nginx -n webapps
                        """
                    }
                }
            }
        }



    }
    post {
        always {
            echo 'slack Notification'
            slackSend channel: '#jenkins',
                color:  COLOR_MAP[currentBuild.currentResult],
                message:"*${currentBuild.currentResult}:* Job ${env.JOB_NAME} build ${env.BUILD_NUMBER} \n more info at : ${env.BUILD_URL}"
            slackUploadFile channel: '#jenkins',
                filePath: '/var/lib/jenkins/workspace/pipeline/trivy.txt',
                initialComment: "🛡️ Trivy image scan report for build #${env.BUILD_NUMBER}"
            slackUploadFile channel: '#jenkins',
                filePath: '/var/lib/jenkins/workspace/pipeline/trivyfs.txt',
                initialComment: "🛡️ Trivy file scan report for build #${env.BUILD_NUMBER}"
        }
    }
}

And Build this
First time it doesn't look like that because its default blue but from second time its show that

Success full build

SonarQube report
on
http://public_ip:9000

Dependency Report:

Slack Notification :

For Monitoring

We Use Grafana And Prometheus

Configuration
Login into
EKS management Server that we are Created previous:

Now Install Grafana And Prometheus on cluster
With the Help of Helm Chart

To install Prometheus + Grafana stack on Amazon EKS, you should slightly modify your command to align with EKS best practices (like using LoadBalancer instead of NodePort, and ensuring correct IAM roles and security groups are in place).

Add Helm repositories

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

Create monitoring namespace

kubectl create namespace monitoring

Install kube-prometheus-stack with LoadBalancer services (recommended for EKS)

helm install eks-prometheus prometheus-community/kube-prometheus-stack \
--namespace monitoring \
--set prometheus.service.type=LoadBalancer \
--set grafana.service.type=LoadBalancer \
--set alertmanager.service.type=LoadBalancer \
--set prometheus-node-exporter.service.type=ClusterIP

Check services to get LoadBalancer DNS/External IPs

kubectl get svc -n monitoring

You get the External IP in the name of elb url copy it and use it for monitoring

i have import a grafana dashboard for monitoring cluster

For Any Question Comment it Down

CI/CD Pipeline for Node.js App Deployment on AWS ECS

bhaktraj — Wed, 26 Feb 2025 07:36:58 +0000

Overview:
This project implements a fully automated CI/CD pipeline using Jenkins to streamline the deployment of a Node.js application on AWS ECS. The pipeline ensures code quality, security, and high availability by integrating SonarQube, OWASP Dependency-Check, Docker, AWS ECR, and ECS. Additionally, Elastic Load Balancer (ELB) is used to distribute traffic efficiently.

Key Features & Workflow:
✅ Code Fetching: Jenkins pulls the latest source code from GitHub.
✅ Code Quality Analysis: SonarQube scans the code for vulnerabilities & maintainability issues.
✅ Security Scan: OWASP Dependency-Check detects security vulnerabilities in dependencies.
✅ Build & Packaging: Dependencies are installed using npm.
Docker image is built and tagged with a unique build number.
✅ Docker Image Push: The image is securely pushed to AWS Elastic Container Registry (ECR).
✅ AWS ECS Deployment: The application is deployed as a container on AWS ECS (Fargate/EC2).The service is restarted using force-new-deployment to fetch the latest image.
✅ Traffic Management: The ECS service is connected to an Elastic Load Balancer (ELB) for high availability & traffic distribution.
✅ Automated Rollouts: If a deployment fails at any stage, Jenkins stops the pipeline to prevent bad code from reaching production.

Steps:
Step 1 : Launch 2 instance on AWS EC2 of t2medium type

one for Jenkins
another for SonarQube

Step 2 : install docker in both of them

#!/bin/bash
# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
sudo usermod -aG docker $USER && newgrp docker

sudo apt update
sudo apt install docker.io -y
sudo usermod -aG docker $USER && newgrp docker

Step 3 : Install Jenkins in one Inststance
for installing Jenkins
https://dev.to/bhaktraj/how-to-install-jenkins-in-ubuntu-2735

#!/bin/bash
sudo apt update 
sudo apt install openjdk-21-jre-headless -y
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins -y
sudo systemctl enable jenkins
sudo systemctl start jenkins

Once Jenkins is installed, you will need to go to your AWS EC2 Security Group and open Inbound Port 8080, since Jenkins works on Port 8080.

Now, grab your Public IP Address
url = http://ec2_public_IPaddress:8080

for password

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

after that set the jenkins

Step 4 : Run SonarQube container on another Instance

docker run -d --name sonar -p 9000:9000 --restart always sonarqube:lts-community

once SonarQube is running you will need to allow inbound rule for port 9000 in EC2 security group, since SonarQube running on 9000 port

url = http://ec2_public_IPaddress:9000

username = admin
password = admin

Step 5 : Install Plugins like JDK, Sonarqube Scanner, NodeJs, OWASP Dependency Check, docker AWS on Jenkins
Goto Manage Jenkins →Plugins → Available Plugins →

Install Plugin

SonarQube Scanner
NodeJs Plugin
Install OWASP Dependency Check
Docker
Docker Commons
Docker Pipeline
Docker API
docker-build-step
AWS Steps
AWS Credentials
Amazon Web Services SDK :: All

5.1 Configure Java and Nodejs in Global Tool Configuration

Goto Manage Jenkins → Tools → Install JDK(17) and NodeJs(16)→ Click on Apply and Save

Step 6 : Configure Sonar Server in Manage Jenkins

Grab the Public IP Address of your EC2 Instance, Sonarqube works on Port 9000, so :9000. Goto your Sonarqube Server. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and click on Generate Token

click on update Token

Create a token with a name and generate

copy Token

Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text.

Now, go to Dashboard → Manage Jenkins → System and Add Server url , name and and authentication credation

The Configure System option is used in Jenkins to configure different server

Global Tool Configuration is used to configure different tools that we install using Plugins

We will install a sonar scanner in the tools.

Step 7 : we had to configure the Tool OWASP Dependency Check

Goto Dashboard → Manage Jenkins → Tools →

Step 8 : Now go configure → Pipeline and add this stage to your pipeline and build.

pipeline {
    agent any
    tools {
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        scannerhome = tool 'sonarserver'
        imagename = '058264453864.dkr.ecr.us-east-1.amazonaws.com/nodejsapp'
        awscred = 'ecr:us-east-1:awscred'
        registeryurl='https://058264453864.dkr.ecr.us-east-1.amazonaws.com/nodejsapp'
        cluster='nojejsproject'
        service='nodejsprojectservice'
    }
    stages{
        stage('clean workspace'){
            steps{
                cleanWs()
            }
        }
        stage("Fetch Code"){
            steps{
                git url: 'https://github.com/bhaktraj/zomatocicd.git', branch:'main'
            }
        }
        stage("Sonarqube analyse "){
            steps{
                withSonarQubeEnv('sonarserver') {
                   sh '''$scannerhome/bin/sonar-scanner -Dsonar.projectName=zomato \
                    -Dsonar.projectKey=zomato '''
              }
            }
        }
        stage("Quality Gate") {
            steps {
                timeout(time: 1, unit: 'MINUTES') {
                    // Parameter indicates whether to set pipeline to UNSTABLE if Quality Gate fails
                    // true = set pipeline to UNSTABLE, false = don't
                    waitForQualityGate abortPipeline: true
                }
            }
        }
        stage("install Dependences"){
            steps{
                sh 'npm install'
            }
        }
        stage('OWASP Dependency-Check') {
            steps {
                    dependencyCheck additionalArguments: ''' 
                    -o './'
                    -s './'
                    -f 'ALL' 
                    --prettyPrint''', odcInstallation: 'owasp'

                    dependencyCheckPublisher pattern: 'dependency-check-report.xml'
                    }
        }
}
}

Step 9 : Create IAM User on aws cloud and provide permission of awsECSFullaccess
and create access key

Step 10 : Configure AWS Credential

Goto Jenkins Dashboard → Manage Jenkins → Credentials

select aws Credential and then and fill out access key and access id

also install aws cli in server by cmd

sudo apt update
sudo snap install aws-cli --classic

Step 11 : Configure ECR On AWS cloud

Step 12 : Configure ECS On AWS Cloud

Step 13 : now set Pipeline or update it

pipeline {
    agent any
    tools {
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        scannerhome = tool 'sonarserver'
        imagename = '058264453864.dkr.ecr.us-east-1.amazonaws.com/nodejsapp'
        awscred = 'ecr:us-east-1:awscred'
        registeryurl='https://058264453864.dkr.ecr.us-east-1.amazonaws.com/nodejsapp'
        cluster='nojejsproject'
        service='nodejsprojectservice'
    }
    stages{
        stage('clean workspace'){
            steps{
                cleanWs()
            }
        }
        stage("Fetch Code"){
            steps{
                git url: 'https://github.com/bhaktraj/zomatocicd.git', branch:'main'
            }
        }
        stage("Sonarqube analyse "){
            steps{
                withSonarQubeEnv('sonarserver') {
                   sh '''$scannerhome/bin/sonar-scanner -Dsonar.projectName=zomato \
                    -Dsonar.projectKey=zomato '''
              }
            }
        }
        stage("Quality Gate") {
            steps {
                timeout(time: 1, unit: 'MINUTES') {
                    // Parameter indicates whether to set pipeline to UNSTABLE if Quality Gate fails
                    // true = set pipeline to UNSTABLE, false = don't
                    waitForQualityGate abortPipeline: true
                }
            }
        }
        stage("install Dependences"){
            steps{
                sh 'npm install'
            }
        }
        stage('OWASP Dependency-Check') {
            steps {
                    dependencyCheck additionalArguments: ''' 
                    -o './'
                    -s './'
                    -f 'ALL' 
                    --prettyPrint''', odcInstallation: 'owasp'

                    dependencyCheckPublisher pattern: 'dependency-check-report.xml'
                    }
        }
        stage('Build docker images'){
            steps{
                script{
                    dockerimage = docker.build(imagename + ":$BUILD_NUMBER", ".")

                }

            }
        }
        stage('Upload to ECR'){
            steps{
            script{
                docker.withRegistry(registeryurl, awscred){
                    dockerimage.push("$BUILD_NUMBER")
                    dockerimage.push("latest")
                }
            }
        }
        }
        stage('deploy to ecs'){
        steps{
            withAWS(credentials:'awscred', region:'us-east-1'){
                sh 'aws ecs update-service --cluster ${cluster} --service ${service} --force-new-deployment'
            }
        }
    }
    }
}

OWASP report

Stage View

Deploying Java Applications on AWS ECS and ECR with a Jenkins CI/CD Pipeline

bhaktraj — Sat, 25 Jan 2025 08:58:39 +0000

Modern software development demands speed, reliability, and scalability. To meet these challenges, I recently built a CI/CD pipeline that seamlessly deploys a Java application on AWS ECS (Elastic Container Service) and ECR (Elastic Container Registry) using Jenkins. Here’s a detailed walkthrough of the project and the technologies that made it possible.

Project Overview

The goal was to automate the entire software delivery process—from code integration to deployment—ensuring fast, reliable, and scalable application delivery.

Key Objectives:

Automate builds and testing for the Java application.
Perform static code analysis and quality checks.
Dockerize the application for consistent deployments.
Push container images to AWS ECR for versioning and storage.
Deploy the application on AWS ECS with zero downtime.
Notify the team about build and deployment statuses via Slack.

Technologies Used

Jenkins: CI/CD pipeline automation.
Maven: Build and dependency management.
SonarQube: Code quality analysis.
Docker: Containerization of the Java application.
AWS ECS/ECR: Container orchestration and storage.
Slack: Real-time team communication.

Pipeline Workflow

The Jenkins pipeline was structured into several stages:

1. Fetch the Code

The pipeline fetches the latest code from the Git repository.

stage("Fetch the code") {
    steps {
        git url: 'https://github.com/bhaktraj/deploy_javaapp_on_aws_ecs.git', branch: 'main'
    }
}

2. Build Stage

The application is built using Maven, ensuring that all dependencies are resolved.

stage('Build') {
    steps {
        sh 'mvn clean install -DskipTests'
    }
}

3. Testing Stage

Automated tests are executed to validate the application.

stage('Test') {
    steps {
        sh 'mvn test'
    }
}

4. Code Quality Analysis

SonarQube is integrated to perform static code analysis and ensure adherence to quality standards.

stage('Code analysis with checkstyle') {
    environment {
        scannerHome = tool 'sonarserver'
    }
    steps {
        withSonarQubeEnv('sonarserver') {
            sh '''${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=vprofile \
                -Dsonar.projectName=vprofile-repo \
                -Dsonar.projectVersion=1.0 \
                -Dsonar.sources=src/ \
                -Dsonar.java.binaries=target/test-classes/com/visualpathit/account/controllerTest/ \
                -Dsonar.junit.reportsPath=target/surefire-reports/ \
                -Dsonar.jacoco.reportsPath=target/jacoco.exec \
                -Dsonar.java.checkstyle.reportPaths=target/checkstyle-result.xml'''
        }
    }
}

5. Docker Image Build

The Java application is packaged into a Docker image using a multi-stage Dockerfile.

stage('image_build') {
    steps {
        script {
            dockerimage = docker.build(imagename + ":$BUILD_NUMBER", "./Docker-files/app/multistage/")
        }
    }
}

6. Push to AWS ECR

The Docker image is pushed to AWS ECR, enabling version control and easy access for deployment.

stage('upload image to ecr') {
    steps {
        script {
            docker.withRegistry(registeryurl, awscred) {
                dockerimage.push("$BUILD_NUMBER")
                dockerimage.push("latest")
            }
        }
    }
}

7. Deployment to AWS ECS

The application is deployed on AWS ECS using the update-service command, ensuring zero downtime.

stage('deploy to ecs') {
    steps {
        withAWS(credentials: 'awscred', region: 'us-east-1') {
            sh 'aws ecs update-service --cluster ${cluster} --service ${service} --force-new-deployment'
        }
    }
}

Post-Build Notifications

To keep the team informed, Slack notifications are sent at each build stage—success, failure, or completion.

post {
    success {
        slackSend(channel: '#all-javacicdproject', color: 'good', message: "Build #${env.BUILD_NUMBER} succeeded!")
    }
    failure {
        slackSend(channel: '#all-javacicdproject', color: 'danger', message: "Build #${env.BUILD_NUMBER} failed.")
    }
    always {
        slackSend(channel: '#all-javacicdproject', color: 'warning', message: "Build #${env.BUILD_NUMBER} completed.")
    }
}

Key Benefits

Automation Saves Time: Eliminates manual intervention, speeding up the delivery process.
Enhanced Code Quality: SonarQube ensures adherence to coding standards.
Scalable Deployments: AWS ECS allows for easy scaling as application traffic grows.
Consistent Environments: Docker ensures the same runtime environment across all stages.
Real-Time Feedback: Slack notifications provide instant updates to the team.

Conclusion

This project highlights the power of combining Jenkins, Docker, and AWS to create a seamless CI/CD pipeline for modern Java applications. The result is a reliable, scalable, and automated deployment process that aligns with DevOps best practices.

Have you worked on a similar project or have questions about setting up a CI/CD pipeline? Let’s discuss in the comments!

DevOps #JenkinsPipeline #AWS #Java #Docker #CICD #CloudComputing #SonarQube #Automation #SoftwareEngineering

Automating CI/CD for a Java Application with Jenkins: A Complete Pipeline Guide

bhaktraj — Tue, 21 Jan 2025 20:10:44 +0000

Introduction

in modern software development, Continuous Integration and Continuous Deployment (CI/CD) pipelines play a critical role in automating code integration, testing, artifact generation, and deployment. In this blog, we’ll walk through building a CI/CD pipeline for a Java-based web application using Jenkins, SonarQube, and Nexus.

Pipeline Overview
The Java application, vProfile, is a web-based project developed using Maven. The pipeline automates the following tasks:

Fetching code from a Git repository.
Building the project with Maven.
Running automated tests.
Performing code quality analysis with SonarQube.
Uploading the generated artifact to Nexus.
Sending Slack notifications about the build status.

Prerequisites

Before setting up the pipeline, ensure the following tools and configurations are in place:

Jenkins: Installed and running with appropriate plugins:
Git Plugin
Maven Integration Plugin
SonarQube Scanner Plugin
Nexus Artifact Uploader Plugin
Slack Notification Plugin
SonarQube: Configured with a project
Nexus: Running and hosting a repository for the artifacts.
Slack: A workspace with an incoming webhook URL for notifications.
Source Code: Hosted in a Git repository.

Pipeline Script
Below is the complete Jenkins pipeline script:

pipeline {
    agent {
        label 'agent47'
    }
    tools {
        maven 'maven3'
        jdk 'jdk17'
    }
    stages{
        stage("Fetch the code"){
            steps{
                git url: 'https://github.com/hkhcoder/vprofile-project.git', branch: 'atom'
            }
        }
        stage('Build'){
            steps{
                sh 'mvn clean install -DskipTests'
            }
        }
        stage('Test'){
            steps{
                sh 'mvn test'
            }
        }
        stage('Code analysis with checkstyle'){

          environment {
             scannerHome = tool 'sonarserver'
          }

          steps {
            withSonarQubeEnv('sonarserver') {
               sh '''${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=vprofile \
                   -Dsonar.projectName=vprofile-repo \
                   -Dsonar.projectVersion=1.0 \
                   -Dsonar.sources=src/ \
                   -Dsonar.java.binaries=target/test-classes/com/visualpathit/account/controllerTest/ \
                   -Dsonar.junit.reportsPath=target/surefire-reports/ \
                   -Dsonar.jacoco.reportsPath=target/jacoco.exec \
                   -Dsonar.java.checkstyle.reportPaths=target/checkstyle-result.xml'''
            }
        }
    }
    stage('upload artifact'){
        steps {
            nexusArtifactUploader(
                nexusVersion: 'nexus3',
                protocol: 'http',
                nexusUrl: '172.31.18.61:8081',
                groupId: 'QA',
                version: '${env.BUILD_ID}.${env.BUILD_TIMESTAMP}',
                repository: 'projectrepo',
                credentialsId: 'nexuscred',
                artifacts: [
                    [artifactId: 'vproapp',
                    classifier: '',
                    file: 'target/vprofile-v2.war',
                    type: 'war']
        ]
     )

        }
    }

    }

    post {
        success {
            slackSend(channel: '#all-javacicdproject', color: 'good', message: "Build #${env.BUILD_NUMBER} succeeded!")
        }
        failure {
            slackSend(channel: '#all-javacicdproject', color: 'danger', message: "Build #${env.BUILD_NUMBER} failed.")
        }
        always {
            slackSend(channel: '#all-javacicdproject', color: 'warning', message: "Build #${env.BUILD_NUMBER} completed.")
        }
    }
}

Agent Section The agent block determines where the pipeline will run. It could be a specific machine, a Docker container, or any available node in the Jenkins environment.

In our pipeline:

agent {
    label 'agent47'
}

Label (agent47): Refers to a specific Jenkins node (agent) where the pipeline tasks will execute. This label helps Jenkins assign the job to a machine with the required configurations and tools installed.

Why Use Agents?

Distributed Builds: Agents allow Jenkins to run jobs on multiple machines, distributing the workload.
Resource Optimization: Specific agents can be optimized for tasks like builds, testing, or deployments.
Environment Isolation: Different agents can have unique environments for different project requirements.

If no specific label is mentioned, Jenkins uses the default agent available.

Tools Section The tools block in the pipeline script specifies the required software versions that Jenkins needs to run the build and analysis processes. This ensures that the right versions of tools are available on the agent where the pipeline executes.

In our pipeline:

tools {
    maven 'maven3'
    jdk 'jdk17'
}

By declaring tools, Jenkins ensures that the appropriate versions are installed and configured on the agent before executing the pipeline stages.

Key Pipeline Stages

Fetch the Code

The pipeline begins by pulling the source code from the Git repository.

Build

Using Maven, the project is built with the mvn clean install command. Tests are skipped in this stage to save time.

Test

Automated unit tests are executed to validate the codebase.

Code Analysis

SonarQube analyzes the code for potential issues and technical debt. Metrics include code smells, security vulnerabilities, and maintainability.

Upload Artifact

The generated .war file is uploaded to a Nexus repository for future deployment.

Post-Build Notifications

Slack notifications are sent to a designated channel to inform the team about the build status.

Step-by-Step Guide to Building a Containerized Microservices Project

bhaktraj — Mon, 20 Jan 2025 18:33:04 +0000

This blog will walk you through the steps to create a containerized microservices project involving Angular, Node.js, and Java applications. We'll use Docker to containerize each application and NGINX to manage traffic routing.

Project Overview

Frontend: Angular application for the user interface.
Backend 1: Node.js application connected to a MongoDB database for NoSQL operations.
Backend 2: Java application built with Maven and connected to a MySQL database.
Traffic Management: NGINX reverse proxy routes requests to the appropriate service.
Containerization: Docker is used to containerize all components.

Prerequisites

Docker and Docker Compose installed.
Basic knowledge of Angular, Node.js, Java (with Maven), and databases (MySQL, MongoDB).
NGINX configuration knowledge.

now I have a demo microservice project so i use this

Demo Project link https://github.com/bhaktraj/microservice_containerize.git

Create Dockerfile Now you have to create Dockerfile of each and ever services

So for Angular

Dockerfile is

FROM node:14 AS first_image
WORKDIR /app
COPY ./ /app/client/
RUN cd client && npm install && npm run build --prod

#second image
FROM nginx:latest
COPY --from=first_image /app/client/dist/client /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.config
EXPOSE 4200

For Java Web application

Dockerfile

FROM openjdk:8 AS first_image
WORKDIR /app
RUN apt update && apt install maven -y
COPY ./ /app/
RUN mvn install -DskipTests

FROM openjdk:8
WORKDIR /app
COPY --from=first_image /app/target/book-work-0.0.1-SNAPSHOT.jar /app/book-work.jar
EXPOSE 9000
ENTRYPOINT [ "java","-jar","book-work.jar" ]

For Node js
Dockerfile

FROM node:14 AS FIRST_IMAGE
WORKDIR /app
COPY ./ ./nodeapi/
RUN cd nodeapi && npm install

FROM node:14
WORKDIR /app


COPY --from=FIRST_IMAGE /app/nodeapi/ /app/
EXPOSE 5000
CMD [ "/bin/sh", "-c", "cd /app/ && npm start" ]

Configure NGINX as a Reverse Proxy

upstream client {
    server client:4200;
}
server {
    listen 80;
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme; 

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_pass http://client/;
    }
    location /api {

        proxy_pass http://api:5000;
    }
    location /webapi {
        proxy_pass http://webapi:9000;
    }
}

Nginx DockerFile

FROM nginx 
COPY default.conf /etc/nginx/conf.d/default.conf

Docker Compose Setup Create a docker-compose.yml file to orchestrate the containers:

version: '3.8'
services:
  mysql:
    image: mysql
    container_name: emartdb
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: emartdbpass
      MYSQL_DATABASE: books
    volumes:
      - msqldat:/var/lib/mysql

  mongodb:
    image: mongo
    container_name: emongo
    environment:
      MONGO_INITDB_DATABASE: epoc
    volumes:
      - mogoda:/data/db
    ports:
      - "27017:27017"
  nginx:
    build:
      context: ./nginx
    container_name: nginx
    ports:
      - "80:80"
    restart: always

  clientapp:
    build:
      context: ./client
    container_name: client
    ports:
      - "4200:4200"
    depends_on:
      - javaapi
      - nodeapi

  javaapi:
    build:
      context: ./javaapi
    container_name: webapi
    ports:
      - "9000:9000"
    restart: always
    depends_on:
      - mysql

  nodeapi:
    build:
      context: ./nodeapi
    container_name: api
    ports:
      - "5000:5000"
    restart: always
    depends_on:
      - mongodb
volumes:
  msqldat:
  mogoda:

Build and Run the Project

Navigate to the root directory of the project.
Run

docker-compose up --build.

Refrence:
TechWorld with Nana
Youtube channel :https://www.youtube.com/@TechWorldwithNana
Imran Teli
A Udemy Course https://www.udemy.com/share/104Tz63@VxTegKgnIJACN30HBKGjPCEVnfF_0bPs_LpTow0FKaCoI8D6yhr4i5MK4ggS1-R6Dw==/

How to install Jenkins in ubuntu

bhaktraj — Fri, 10 Jan 2025 05:59:20 +0000

What is Jenkins ?
Jenkins is an open source automation server which is used for continuous integration and continuous delivery in software development its help to automate the parts of software development related like building , testing, deploying and delivery the software

how you can install Jenkins on Ubuntu:
Step 1:
You need to read documentation of Jenkins that what are the minimum requirement for installing Jenkins
documentation link : https://www.jenkins.io/doc/book/installing/linux/

The point that are being in the knowledge.

Which java version is required for which jenkins version first install that version of java

(source: Jenkins Documentation)

Step 2:
First Update your System:

sudo apt update

Step 3;
install Java as per the requirement of Jenkins version
observe the picture

(source: Jenkins Documentation)

sudo apt install openjdk-21-jre-headless

Step 4:
Now follow the documentation command

(source: Jenkins Documentation)

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

Step 5:
update it again

sudo apt-get update

Step 6:
Install Jenkins

sudo apt-get install jenkins

Step 7:
enable means start the service when machine is rebooted

sudo systemctl enable jenkins

Step 8:

sudo systemctl start jenkins

Now
Jenkins is install so open it with port 8080

and the password is at
/var/lib/jenkins/secrets/initialAdminPassword
it's mention in the browser

So use Cmd

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

copy the password and paste it there in browser
and configure the setting that you need.

Script For Provisioning in cloud

#!/bin/bash
sudo apt update 
sudo apt install openjdk-21-jre-headless -y
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins -y
sudo systemctl enable jenkins
sudo systemctl start jenkins

Thanks

Docker Install In Ubuntu or on Cloud (AWS provisioning)

bhaktraj — Tue, 07 Jan 2025 07:12:34 +0000

Here’s a guide for installing Docker on Ubuntu or provisioning it on AWS:

Install Docker on Ubuntu

Update Package:

sudo apt update

Install Required Packages:

sudo apt-get install ca-certificates curl

Add Docker's official GPG key:

sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

Add Docker Repository:

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Update Package:

sudo apt update

To install the latest version, run:

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

Verify that the installation is successful or not

docker --version

Or try By run Some Docker Cmd

sudo docker run hello-world

Another Way for installing Docker
only two command need to install docker

sudo apt update
sudo apt install docker.io -y

Script For Install docker in ubuntu
also copy from https://github.com/bhaktraj/dockerinstall/tree/main

#!/bin/bash
# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

after that verify docker is install or not by run following command:

docker --version

sudo docker run hello-world

Docker Install on AWS Cloud By Provisioning

Provision EC2 Instance:

Launch an EC2 instance with the following:
OS:
Ubuntu (latest LTS version recommended)
Instance Type: t2.micro (for learning) or more for production
Configure security group to allow port 22 (SSH) for remote login

Paste the following Script in user data in advance details block:

#!/bin/bash
# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

And Launch the instance wait for 2 to 5 min after launching then login into instance and verify the docker is install properly or not by command

docker --version

sudo docker run hello-world

Comment for any query

Auto scaling Multi-tier Web Application deployment architecture on Cloud(AWS)

bhaktraj — Fri, 03 Jan 2025 09:52:40 +0000

Introduction

The primary goal of this project is to design, deploy, and manage a robust multi-service application on Amazon Web Services (AWS). The application architecture leverages various AWS resources to ensure scalability, availability, and optimal performance. The solution is built around four EC2 instances, each hosting a dedicated service critical to the functionality of the overall system. Additionally, it incorporates an Elastic Load Balancer (ELB) to manage incoming traffic efficiently and an Auto Scaling group to handle dynamic demand.

This architecture provides a well-structured foundation for a cloud-based application, allowing seamless communication between services and offering flexibility to scale up or down as needed.

System Design/Architecture:
The architecture of this project is built on the AWS cloud infrastructure to deploy a multi-service application with high availability, scalability, and security. The system design includes various components that are integrated to ensure efficient resource usage, fault tolerance, and secure communication between services.
High-Level Architecture Overview:

1.Virtual Private Cloud (VPC):

The system is deployed within a Virtual Private Cloud (VPC) to provide a secure, isolated network environment. The VPC is divided into public and private subnets:
o Public Subnets: These host the Elastic Load Balancer (ELB), which is responsible for handling incoming traffic and distributing it across the application servers.
o Private Subnets: These contain the EC2 instances that host various services like Memcached, MariaDB, and RabbitMQ, which need to be isolated for security reasons.
2.Elastic Load Balancer (ELB):

The ELB sits in the public subnet and distributes incoming traffic across the EC2 instances in the private subnets. It helps ensure that the application remains highly available and fault-tolerant, routing traffic to healthy instances automatically.
3.EC2 Instances:

app01:
Hosts the core application logic that handles user requests and serves the application. It is part of an Auto Scaling group to automatically scale based on demand.
mc01:
Hosts Memcached, a caching layer that helps reduce database load by caching frequently accessed data.
db01:
Hosts MariaDB, which provides relational database management for storing application data securely.
rmq01:
Hosts RabbitMQ, a messaging broker that facilitates communication between services asynchronously.

4.Auto Scaling Group:

The Auto Scaling group ensures that the number of EC2 instances in the app01 tier automatically adjusts based on defined metrics such as CPU utilization or incoming traffic. This allows the application to scale in or out to handle varying load efficiently.
5.Security Groups and Network ACLs:

Security groups are configured to allow traffic only from trusted sources, such as the ELB to the application instances, and other necessary communication between services. Network ACLs add an additional layer of security by controlling inbound and outbound traffic at the subnet level.
6.CloudWatch for Monitoring:

AWS CloudWatch is used to monitor the health of EC2 instances, Auto Scaling events, and ELB metrics. Alarms are set to automatically scale resources and notify administrators in case of any issues or performance bottlenecks.

Diagram

Implementation: Methods, Algorithms, and Processes

Security Group Configuration Security Groups are configured to act as virtual firewalls for EC2 instances. Each service (app01, mc01, db01, rmq01) has its own security group with specific inbound and outbound rules to control network traffic

ELBSG (Security Group): The Elastic Load Balancer is set up in the public subnet to distribute traffic to the app01 instances in private subnets. The ELB is configured to automatically route traffic to healthy instances.

allow inbouund rule in which https allow for public IP means traffic is allowed

appSG (Application server): Only allows incoming traffic from the ELB (to handle HTTP/HTTPS requests) and from other trusted services (e.g., RabbitMQ, Memcached, MariaDB).

Allow HTTPS traffic comes from ELB to application server
and allow ssh to your Ip only to enter in the server for installing application or monitoring etc.

BackendSG : Accepts inbound traffic only from app01 and other trusted services in the private subnet like rabbitmq , database,and memecached server, ensuring that database connections are not exposed to the internet.

Allow port 3306,11211,5672 for the traffic that comes from application server
allow all traffic for all rabbitMQ, Memcache, Database server to each other means allow traffic from same security group

2.Key Pair Management:

During EC2 instance creation, a Key Pair is generated (or an existing one can be used) for secure SSH access to instances. This ensures that only authorized personnel or systems with the corresponding private key can connect to the EC2 instances, maintaining the security of the instances from unauthorized access.

3.Provisioning EC2 Instances:

EC2 instances are provisioned using Amazon Machine Images (AMIs) based on the service requirements:

app01:
EC2 instance is configured with a Tomcat server to host the application. Tomcat is installed on the EC2 instance, and the necessary Java Runtime Environment (JRE) is configured to run the application.
The Tomcat web server is configured to handle HTTP requests (typically on port 8080 by default) and route them through to the appropriate application components. If required, Apache HTTPD can be used as a reverse proxy in front of Tomcat to handle requests on port 80 or 443 and forward them to Tomcat on port 8080.
mc01:
Configured with Memcached to provide an in-memory caching layer, improving application performance.
db01:
Set up with MariaDB as the database service for storing application data securely.
rmq01:
Configured with RabbitMQ for handling messaging between different services in an asynchronous manner.

4.Configuring ELB:

The Elastic Load Balancer is set up in the public subnet to distribute traffic to the app01 instances in private subnets. The ELB is configured to automatically route traffic to healthy instances.

5.Auto Scaling Configuration:

An Auto Scaling group is created for the app01 EC2 instances, with scaling policies based on CPU utilization (e.g., scale out when CPU utilization exceeds 80%). This ensures that the application automatically scales based on user traffic