JAVA APP CICD USING GITHUB ACTIONS

Project Title: JAVA APP CICD USING GITHUB ACTIONS
ProjectRepo:https://github.com/robudexIT/shopping_cart

Notes:The Java application use in this project is clone from this github.
repository: https://github.com/shashirajraja/shopping-cart.git.

Project Architecture:

Kubernetes Cluster Architecture:

Project Overview:

1. Workflow Structure: The project utilizes two GitHub workflows or
   pipelines: one dedicated to the development/test environment and the
   other to the production environment.

2. Development/Test Pipeline:

   • This pipeline is triggered by pull requests to the main branch.
   • It executes the following steps:
     1. Running tests using Maven.
     2. Performing additional code analysis using SonarQube.
     3. Building a Docker image for the test environment.
     4. Pushing the Docker image to DockerHub.
     5. Deploying the Docker image to the test environment.

3. Production Pipeline:

   • This pipeline is triggered by merges or pushes to the main branch.
   • It includes a manual approval step.
   • Upon approval, the pipeline:
     1. Builds and pushes the production Docker image with prod tag.
     2. Deploy the production image to the production environment.

4. Branch Protection: Direct pushes to the main branch are prohibited
   to maintain code integrity and ensure changes go through the proper
   workflow.

5. Team Structure:

   • The project team consists of three members:
     1. Dev01: Responsible for updating the code by initiating pull requests.
     2. Dev02:Authorized to review and merge code changes.
     3. Dev03: Responsible for granting manual approval in the CI/CD production pipeline.

6. Test Environment

• Hosted on a single server running Docker Engine.

• Infrastructure provided by Digital Ocean.

7.Production Environment:

• Utilizes a Kubernetes cluster.
• Infrastructure provided by Digital Ocean.

Project Flow of Execution:

1. Create 3 github Users:

• robudex17 - Dev01:

  1. Role: Initiates pull requests.
  2. Responsibilities: Updating code and initiating pull requests for review.

• robudexIT - Dev02:

  1. Role: Responsible for merging and pushing changes to the main branch.
  2. Responsibilities: Reviews pull requests and merges approved changes into the main branch.

• robudex2023 - Dev03:

  1. Role: Responsible for approving or rejecting changes in the production workflow.
  2. Responsibilities: Manually approves changes in the CI/CD production pipeline or requests adjustments if necessary.

Then I clone https://github.com/shashirajraja/shopping-cart.git
java webapps project, create DockerFile and Kubernetes manifiests
and push it to my Dev02 users.

Current project tree. Shown below:

2. On the test server, I created the latest MySQL Docker container and restored the database of the project using these

commands.

 cd /root
 git clone https://github.com/robudexIT/shopping_cart.git 
 mkdir /root/mysqldb

docker run --name shopping-cart-db -v /root/mysqldb:/var/lib/mysql
-e MYSQL_ROOT_PASSWORD=password123 -e MYSQL_DATABASE=shopping-cart -
d mysql:latest

docker exec -i shopping-cart-db sh -c 'exec mysql -uroot -
ppassword123 shopping-cart' <
/root/shopping_cart/databases/mysql_query.sql

Then, verify if the database was properly restored using the following commands:

docker exec -it shopping-cart-db bash
mysql -uroot -ppassword123
show databases;
use shopping-cart;
show tables;

Figure shown that database is successfully Added:

3. On My Kubernetes Cluster I Create Mysql Deployment and Service

Note: I will utilize one of my Kubernetes nodes' storage in a
PersistentVolume, ensuring that MySQL deploys on that node. This will be
achieved by adding a label to the node and using this label in the
deployment. Here are the commands:

  kubectl get nodes

 kubectl label nodes mynode-pool-o6fg9 nodetype=database
 kubectl get node mynode-pool-o6fg9 --show-labels

I like the separation between the app and the database so I created
namespace

  kubectl create namespace database-namespace
  kubectl create namespace app-namespace

Clone git_ https://github.com/robudexIT/shopping_cart.git_ and
Navigate (cd) to s*hopping-cart/kubernetes/database* and run the command in
order

  kubectl apply -f storage-class.yaml -n database-namespace
  kubectl apply -f shopping-cart-pv.yaml -n database-namespace
  kubectl apply -f shopping-cart-pvc.yaml -n database-namespace
  kubectl apply -f shopping-cart-db-deploment.yaml -n database-
namespace
 kubectl apply -f shopping-cart-db-service.yaml -n database-
namespace

Verify Kubernetes Objects

 kubectl get namespace
 kubectl get sc -n database-namespace
 kubectl get pv -n database-namespace
 kubectl get pvc -n database-namespace
 kubectl get deployment -n database-namespace
 kubectl get service -n database-namespace
 kubectl get pods -n database-namespace

4. Restore shopping-cart database to shopping-cart-mysql pod with commands

kubectl get pods  -n database-namespace

kubectl cp shopping_cart/databases/mysql_query.sql shopping-cart-mysql-5b666ff5b5-994pm:/tmp --namespace database-namespace

kubectl exec -it shopping-cart-mysql-5b666ff5b5-994pm -n database-namespace -- bash -c 'mysql -u root -ppassword123 shopping-cart < /tmp/mysql_query.sql'

kubectl exec -it shopping-cart-mysql-5b666ff5b5-994pm  -n database-namespace – bash

mysql -uroot -ppassword123

show databases;

use shopping-cart;

show tables;

5. Now that the MySQL databases are installed on the test server and in the** Kubernetes cluster*, it's time to set up the DockerHub repository, **SonarQube* (organization, project, quality gates and token).

DockerHub:

Sonarqube Organization and Project:

Quality Gates have been added with the name "shop_cart_gate". I have configured it to be less restrictive setting coverage to 0.0% and duplicated lines to 10.0%. Please note that this configuration is not recommended and is solely for demonstration purposes.

Add Token:

6. Configure repository settings by adding production environment, secrets, and GitHub workflows/pipelines

Add Dev01 and Dev03 as Collaborators:

Next, add the secrets and production environment variables. Navigate to Settings -> Security -> Secrets and Variables -> Actions.

These are secrets that I added:

DB_TEST_IP - mysql docker container IP Address running on Test Server.

You can get docker containter ip address by (docker inspect /container-id)

DB_TEST_PWD - mysql docker container password running on Test Server

DB_TEST_USERNAME - mysql docker container root user running on Test Server

DOCKERHUB_USERNAME – dockerhub username

DOCKERHUB_TOKEN – dockerhub password

KUBE_CONFIG – kubernetes cluster configuration you can get this in .kube/config

SONAR_ORGANIZATION – your sonarqube organization on this project
SONAR_PROJECT_KEY – your sonarqube project key
SONAR_TOKEN - your sonarqube token
SONAR_URL – sonarqube url (https://sonarcloud.io)

TEST_HOST_IP **– IP address of the Test Server
**TEST_HOST_PORT – Test Server SSH port

TEST_HOST_PRIVATE_KEY - Test Server Private Key
You can generate the key using ssh-keygen.
TEST_HOST_USER – Test Server User (root)

Create the Production Environment and add Dev03 as a reviewer. The Production pipeline will not run without Dev03's approval.

Production Environment Secrets

DB_PROD_IP – mysql Pod IP address in kubernets cluster
DB_PROD_USERNAME – mysql Pod username in kubernetes cluster
DB_PROD_PWD– mysql Pod password in kubernetes cluster

Creating Test/Dev Pipelines:
Now that all secrets and environments are set up, it's time to create pipelines.

On my Dev02 shopping_cart repository, I click on Actions and create a new blank workflow named shopping_cart_cicd.yml.

  name: Shopping Cart Action 

on: workflow_dispatch #manual trigger
# on: 
#  pull_request:
#    branches: main

permissions:
  issues: write
jobs:        
  TestingCode: 
    runs-on: ubuntu-latest  #this runner has install maven by default
    steps: 

      - name: Code Checkout 
        uses: actions/checkout@v4 

      - name: Maven Test 
        run: mvn test 

      - name: Maven Checkstyle
        run: mvn checkstyle:checkstyle 

      - name: Install Java 11 
        uses: actions/setup-java@v4 
        with: 
          distribution: 'temurin' 
          java-version: '11'

      - name: Setup SonarQube 
        uses: warchant/setup-sonar-scanner@v7 

      - name: SonarQube Scan 
        run: sonar-scanner 
            -Dsonar.host.url=${{ secrets.SONAR_URL }}
            -Dsonar.token=${{ secrets.SONAR_TOKEN }}
            -Dsonar.organization=${{ secrets.SONAR_ORGANIZATION }}
            -Dsonar.projectKey=${{ secrets.SONAR_PROJECT_KEY }}
            -Dsonar.sources=src/ 
            -Dsonar.java.checkstyle.reportPaths=target/checkstyle-result.xml
            -Dsonar.java.binaries=target/classes/com/shashi/

      - name: SonarQube Quality Gate Check 
        id: sonarqube-quality-gate-check 
        uses: sonarsource/sonarqube-quality-gate-action@master
        timeout-minutes: 5 
        env: 
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
          SONAR_HOST_URL: ${{ secrets.SONAR_URLL }}

  BuildAndPushForTesting:
    runs-on: ubuntu-latest 
    needs: TestingCode
    env: 
      DOCKER_REPO: shopping-cart 
    steps: 
      - name: Code Checkout 
        uses: actions/checkout@v4 

      - name: Update application.properties file
        env:
           DB_TEST_IP: ${{ secrets.DB_TEST_IP }}
           DB_TEST_USERNAME: ${{ secrets.DB_TEST_USERNAME }}
           DB_TEST_PWD: ${{ secrets.DB_TEST_PWD }}
        run: |
          sed -i "s|db.connectionString =.*|db.connectionString = jdbc:mysql://$DB_TEST_IP:3306/shopping-cart|"  src/application.properties
          sed -i "s|db.username =.*|db.username = $DB_TEST_USERNAME|" src/application.properties
          sed -i "s|db.password =.*|db.password = $DB_TEST_PWD|" src/application.properties

      - name: Login to Docker Hub 
        uses: docker/login-action@v3 
        with: 
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Build and Push docker image to Dockerhub 
        uses: docker/build-push-action@v5
        with: 
          context: ./
          push: true 
          tags: ${{ secrets.DOCKERHUB_USERNAME }}/${{env.DOCKER_REPO}}:test
           # - ${{ secrets.DOCKERHUB_USERNAME }}/${{ DOCKER_REPO }}:${{GITHUB_RUN_NUMBER}}

  DeployToTestEnv:
    runs-on: ubuntu-latest 
    needs: BuildAndPushForTesting 
    env: 
      DOCKER_USER: ${{ secrets.DOCKERHUB_USERNAME }}
      DOCKER_REPO: shopping-cart
      APP: shopping-cart-app 
      APP_PORT: 8081 

    steps:
      - name: Execute SSH commmands on remote server
        uses: JimCronqvist/action-ssh@master
        with:
          hosts: ${{ secrets.TEST_HOST_USER }}@${{secrets.TEST_HOST_IP}}
          privateKey: ${{ secrets.TEST_HOST_PRIVATE_KEY }}
          command: |
            docker stop ${{env.APP}} 
            sleep 2
            docker pull ${{env.DOCKER_USER }}/${{env.DOCKER_REPO}}:test
            sleep 2
            docker run --name ${{env.APP}} -d --rm -p ${{env.APP_PORT}}:8080 ${{env.DOCKER_USER }}/${{env.DOCKER_REPO}}:test

Creating Production Pipelines:On My Dev02 shopping_cart repo, I created new blank workflow name it shopping_cart_cicd_production.yml

  name: Shopping Cart Action Production

on: workflow_dispatch #manual trigger
# on: 
#  push:
#    branches: main

permissions:
  issues: write
jobs: 
  BuildAndPushForProduction:
    runs-on: ubuntu-latest 
    environment: production
    env: 
      DOCKER_REPO: shopping-cart 
    steps: 
      - name: Code Checkout 
        uses: actions/checkout@v4 

      - name: Update application.properties file
        env:
           DB_PROD_IP: ${{ secrets.DB_PROD_IP }}
           DB_PROD_USERNAME: ${{ secrets.DB_PROD_USERNAME }}
           DB_PROD_PWD: ${{ secrets.DB_PROD_PWD }}      

        run: |
          sed -i "s|db.connectionString =.*|db.connectionString = jdbc:mysql://$DB_PROD_IP:3306/shopping-cart|"  src/application.properties
          sed -i "s|db.username =.*|db.username = $DB_PROD_USERNAME|" src/application.properties
          sed -i "s|db.password =.*|db.password = $DB_PROD_PWD|" src/application.properties

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Build and Push docker image to Dockerhub 
        uses: docker/build-push-action@v5
        with: 
          context: ./
          push: true 
          tags: ${{ secrets.DOCKERHUB_USERNAME }}/${{env.DOCKER_REPO}}:prod
           # - ${{ secrets.DOCKERHUB_USERNAME }}/${{ DOCKER_REPO }}:${{GITHUB_RUN_NUMBER}}

  DeployToKubernetes:
    runs-on: ubuntu-latest 
    needs: BuildAndPushForProduction
    steps:
      - name: Code Checkout 
        uses: actions/checkout@v4  
      - name: Setup Kubernetes Configuration
        uses: tale/kubectl-action@v1
        with:
          base64-kube-config: ${{ secrets.KUBE_CONFIG }} 
      - name: Check for existing deployment
        run: |
          if kubectl get deployment shopping-cart-java-app -n app-namespace >/dev/null 2>&1; then
            echo "Deployment exists, rolling out update"
            kubectl rollout restart deployment shopping-cart-java-app -n app-namespace
          else
            echo "Deployment not found, applying new resources"
            kubectl apply -f kubernetes/application/shopping-cart-app-deployment.yaml -n app-namespace
          fi

As you notice, I temporarily set the two pipelines to manual trigger (workflow_dispatch) to avoid running the pipeline because it is not ready yet.

Create branch rules for the main branch by navigating to Settings -> Branches. Check the option as seen below:

On Settings-> General-> Pull Requests

7. Now that everything is set up, it's time to proceed with testing.

On Dev01-> I Update shopping_cart_cicd.yml from workflow_dispatch to pull_request

As you can see, it is not possible to directly push changes to the main branch. Please click 'Propose changes', add a description, and create a pull request.

Goto Dev02 Account and check the Pull Request and Approved it

Go to Actions and verify that the shopping_cart_cicd has executed successfully.

Paste Test Server Public Ip and 8081 port

After ensuring the successful deployment on the Test Env, it's time to merge the changes into the main branch. To do so,** Dev02** should navigate to the Pull Requests section, select the relevant pull request, and opt for the "Squash and Merge" option. This action condenses all commits from the feature branch into a single commit before merging them into the main branch.

Notice that the change to .github/workflows/shopping_cart_cicd.yml has been merged into the main branch.

Now it's time to update the .github/workflows/shopping_cart_cici_production.yaml as well, changing 'workflow_dispatch' to 'push'. On Dev01, modify the file and create a pull request. Then, approve the pull request on the Dev02 account.

Note that the shopping_cart_cicd pipeline executes because another pull request has been initiated. Please wait for the pipeline execution to finish before merging **it into the **main branch.

Now because there is a merge or a push in the main branch production pipeline execute

Click the workflow and notice it requires approval

Given that I've configured Dev03 to solely approve or reject the pipeline execution, it won't initiate or conclude without Dev03's approval. Please proceed to Dev03 and provide approval.

Production pipeline Start executing...

Production pipeline Done.. Check Kubernetes cluster

Paste the EXTERNAL-IP to Browser

The production environment in the Kubernetes cluster is operational. Now, it's time to implement some code changes, create a pull request, merge it into the main branch, and ultimately, approve the production pipeline.

The flow will start in..

Dev01 --> make code change then make pull request
Dev02 ** --> Approve the pull request and then merge to main
**Dev03 --> Approve the production pipeline execution.

Since I am not a Java developer, I will only change the header background color to purple in the WebContent/header.jsp file."

Now, let's check our SonarQube account. Under 'Your Organization,' click on the 'shop-cart' project, then click on the 'Pull Request' tab, and see that the result has passed.

Let's adjust the Quality Gate and rerun the Test Pipeline. (Note that we expect the execution to fail this time.)

Click on the latest workflow of the TEST Pipeline and rerun all jobs.

And indeed the pipeline fails.

On SonarQube:

Let's revert the SonarQube metrics for 'sonar_cart_gate' to be less restrictive again and rerun the Test Pipeline to ensure it passes once more

That concludes the documentation. Thank you.