My School DevOps Project [part 1] - Jenkins

Introduction

Hello, so for my DevOps module in my school we add a really complete DevOps Project using AWS EC2 options and I founded it fascinating, so I decided to publish how i made it, with some alternative using vagrant instead of AWS. This presentation is separated in different part. This is the first part talking about Jenkins

Simple Jenkins CI pipeline, the mission was to :

• Pull a git repo
• Start Unitary and integration test
• Drop the jar file on the Jenkins workspace

Constraints :
The pipeline should have the following parameter :

• BRANCH: String to choose the repo branch
• SKIP_TESTS: A boolean to control the execution of some test
• VERSION_TYPE: SNAPSHOT or RELEASE (rename jar file )
• VERSION: a string for the jar version (ex: SB3T-1.0-SNAPSHOT)

Deploy Jenkins

Why dockerize my project ?

I want my project easy to build and destroy so i can work whenever i want and easily , i was using also vscode with the plugin for launching vscode in wsl on my computer . WSL with docker make possibility to change file while docker is running to test new elements

Tools to make it :

• Docker Desktop
• WSL
• plugin 'remote wsl' for vs code

Docker Compose and Dockerfile

So for making this project work we need to set up a Dev Environment, I was using docker-compose and dockerfile, so i put my docker compose and Dockerfile in the root of my project for allowing me to import future feature easily.

├── Docker-compose.yml
├── Dockerfile
├── jenkins
│   ├── conf
│   │   ├── plugins.txt
│   │   ├── jenkins.yaml
│   │   └── jobs
│   │       ├── sb3t_wrapper.groovy
│   │       ├── jobs.groovy
│   │       ├── folders.groovy
│   └── jenkins.md

Docker-compose.yml

Inside my docker file I create a Jenkins service, this service is running on port 80 and 50000 on my localhost computer port
This creates different folder in /var directory in docker, importing some file like ./jenkins/conf in /var/jenking_config
This Jenkins services as an admin user because it's only for personal use do not do that on production ◑____________◑ , So this is bind with just admin admin and run on, HTTP protocol on the port 8081 on docker environment .

---
version: '3.8'

services:
  jenkins:
    build:
      context: .
    ports:
      - 80:8081
      - 50000:50000
    volumes:
      - jenkins_home:/var/jenkins_home
      - ./jenkins/conf:/var/jenkins_config/
      - #./terraform:/var/terraform usefull later 
      - #./ansible/:/var/ansible  usefull later 

    environment:
      - CASC_JENKINS_CONFIG=/var/jenkins_config/jenkins.yaml
      - JENKINS_ADMIN_ID=admin
      - JENKINS_ADMIN_PASSWORD=admin
      - JENKINS_OPTS="--httpPort=8081"

volumes:
  jenkins_home:

The Docker File

The purpose of this docker file is to import/create the images and the volume for my Jenkins to run properly, it installs all the necessary features and the future one like Ansible .

FROM jenkins/jenkins:lts-jdk11

# Disable setup wizard
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false -Dpermissive-script-security.enabled=true

USER root
RUN apt-get update && \
    apt-get -y install apt-transport-https \
      ca-certificates \
      curl \
      gnupg2 \
      software-properties-common && \
    curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
    add-apt-repository \
      "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
      $(lsb_release -cs) \
      stable" && \
   apt-get update && \
   apt-get -y install docker-ce
#installation terraform
RUN curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add - && apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main" && apt-get update && apt-get install terraform

#install ansible
RUN apt-get install python3-pip -y && \
  pip3 install ansible 
#install boto3 for inventory dynamic
RUN pip3 install boto3
#Install LVM for lvol and lvg volumes
RUN ansible-galaxy collection install community.general
# Load plugins w jenkins-plugin-cli
COPY --chown=jenkins:jenkins jenkins/conf/plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt

USER jenkins

Jenkins pipeline

Using Declarative Jenkins

Declarative pipeline is a relatively new feature that supports the pipeline as code concept. It makes the pipeline code easier to read and write. This code is written in a Jenkinsfile which can be checked into a source control management system such as Git. pipelines are based on the groovy DSL, the scripted pipeline uses stricter groovy based syntaxes because it was the first pipeline to be built on the groovy foundation. Since this Groovy script was not typically desirable to all the users, the declarative pipeline was introduced to offer a simpler and more optioned Groovy syntax. more on This site

Jenkins.Yaml

here you can fin the origin of this jenkins.yaml

This setup Your Jenkins with some option, with this jenkins yaml you install some tools usefull to test the code and respond to the project.

It's installing :

• Git
• Java
• Maven with a specific version

---

jenkins:
  numExecutors: 1
  systemMessage: "Hi, jenkins setup with jcasc"
  securityRealm:
    local:
      allowsSignup: false # prevents anonymous users from creating an account through the web interface
      users:
        - id: ${JENKINS_ADMIN_ID}
          password: ${JENKINS_ADMIN_PASSWORD}

  authorizationStrategy:
    globalMatrix:
      permissions:
        #- "View/Delete:authenticated"
        #- "View/Read:authenticated"
        #- "View/Configure:authenticated"
        #- "View/Create:authenticated"
        - "Job/Read:authenticated"
        - "Job/Build:authenticated"
        - "Job/Configure:authenticated"
        - "Job/Create:authenticated"
        - "Job/Delete:authenticated"
        - "Job/Discover:authenticated"
        - "Job/Move:authenticated"
        - "Job/Workspace:authenticated"
        - "Job/Cancel:authenticated"
        - "Run/Delete:authenticated"
        - "Run/Replay:authenticated"
        - "Run/Update:authenticated"
        - "Overall/Read:admin"
        - "Overall/Administer:admin"

  crumbIssuer: "standard"

  # Docker: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos/docker
  clouds:
    - docker:
        name: "docker"
        dockerApi:
          dockerHost:
            uri: "unix:///var/run/docker.sock"
        templates:
          - labelString: "docker-agent"
            dockerTemplateBase:
              # TODO: pin sha256 or versions when using in production
              image: "jenkins/agent"
            remoteFs: "/home/jenkins/agent"
            connector:
              attach:
                user: "jenkins"
            instanceCapStr: "10"
            retentionStrategy:
              idleMinutes: 1

security:
  scriptApproval:
    approvedSignatures:
      - method java.lang.String indexOf int
      - method io.jenkins.plugins.casc.ConfigurationAsCode configure
      - staticMethod io.jenkins.plugins.casc.ConfigurationAsCode get
tool:
  git:
    installations:
      - home: "git"
        name: "Default"
  jdk:
    installations:
      - home: "/opt/java/openjdk"
        name: "java"
  maven:
    installations:
      - name: "maven"
        properties:
          - installSource:
              installers:
                - maven:
                    id: "3.6.2"


jobs:
  - file: /var/jenkins_config/jobs/folders.groovy
  - file: /var/jenkins_config/jobs/jobs.groovy

You can import some jobs , by configuring git in your jenkins yaml file , here i specify 2 file a specific directory

How to import Jobs in Docker / Jenkins

The first file i import setup two different directory in my jenkins app , the CI and the CD directory

#!groovy
/**
 * Setup folders
 */

folder('CI')
folder('CD')

After The file setups , the jenkins yaml file import the second file , println is used to print out some information , for importing i need to define my pipeline and specify the root to this pipeline .

Once I have define my pipeline , i add the different required parameters wich gonna have there important role on how gonna work this piepline

#!groovy
println('---------------------------------------------------------------------------Import Job CI/sb3t')
// define first pipeline 
def pipelineScript = new File('/var/jenkins_config/jobs/sb3t_wrapper.groovy').getText("UTF-8")
//Adding pipeline to the right path 
pipelineJob('CI/sb3t_wrapper'){
    description("Build .jar file from sb3t repository , from master branch")
//Importing parameter 
    parameters {
        stringParam {
            name('BRANCH')
            defaultValue('master')
            description("branch to pull")
            trim(false)
        }
        booleanParam {
            name('SKIP_TEST')
            defaultValue(true)
            description("Skip test")
        }
        choice {
            name('VERSION_TYPE')
            choices(['SNAPSHOT', 'RELEASE'])
            description('Version type between snapshot and release')
        }
        stringParam {
            name('VERSION')
            defaultValue('1.0')
            description("version of the project")
            trim(false)
        }
    }
//Importing the script in sandbox 
    definition{
        cps {
            script(pipelineScript)
            sandbox()
        }
    }
}

Usage of Jenkins pipeline

with the parameters define in my jobs.groovy file , i can script what am i importing in my jenkins pipeline . By using the declarative pipeline method i know and i can use all the parameters i have defined earliers .

#!groovy

pipeline {
//use any agent installed 
    agent any
    tools {
        // Install the Maven version configured as "M3" and add it to the path.
        maven 'maven'
    }
    environment {
        TEST = 'TEST'
    }
    options {
        buildDiscarder(logRotator(numToKeepStr: '100'))
    }

    stages {
        stage('Git clone') {
            steps {
//choosing the branch
                git branch: "${params.BRANCH}", url: '<sping boot project on github>'
            }
        }


        stage('Compiling') {
            steps {
                    sh 'mvn compile'
            }
        }
        stage('Testing'){
            when {
//using skip parameter

            expression { params.SKIP_TESTS == false }
            }
            steps {
                sh 'mvn test'
            }
        }

        stage('Package') {
            steps {
                script {
                    sh 'mvn package'
                }
            }
        }

        stage('Verify') {
               when {
//using skip parameter
            expression { params.SKIP_TESTS == false }
            }
            steps {
                sh 'mvn verify'
            }
        }


        stage('Package version ') {
            steps {
//using version parameter
                sh "mvn versions:set -DnewVersion=${params.VERSION}-${params.VERSION_TYPE}"
//using skip parameter

                sh "mvn clean package  -DskipTests=${params.SKIP_TESTS}"
            }
        }

        stage('archive') {
            steps {
//create archive in a artifacts with .jar

                archiveArtifacts(artifacts: '**/*.jar', followSymlinks: false)
            }
        }

    }
}

Run

For running the project i can type docker-compose up --build or docker-compose up -d to create my service and go to http://localhost/ page for accessing my jenkins pipeline

You want to try the project, get it on my GitHub page (Posted after my notation)

PS:

if you see any spelling or grammar mistakes, can you notify me (I am still improving my English) ? thank you !
if you have any tips for improving my post feel free to comment on the post
and i hope you enjoy reading this

nb :
plungin.txt for jenkins

bouncycastle-api:2.23
github:1.34.0
ws-cleanup:0.39
branch-api:2.6.5
pipeline-build-step:2.15
trilead-api:1.0.13
popper-api:1.16.1-2
pipeline-rest-api:2.19
credentials-binding:1.27
git-client:3.9.0
build-timeout:1.20
resource-disposer:0.16
oauth-credentials:0.4
email-ext:2.83
pipeline-model-extensions:1.9.1
ssh-credentials:1.19
handlebars:3.0.8
durable-task:1.39
lockable-resources:2.11
github-branch-source:2.11.2
pipeline-stage-tags-metadata:1.9.1
jquery3-api:3.6.0-2
jsch:0.1.55.2
git-server:1.10
ace-editor:1.1
junit:1.52
workflow-api:2.46
cloudbees-folder:6.16
bootstrap4-api:4.6.0-3
font-awesome-api:5.15.4-1
jackson2-api:2.12.4
pipeline-graph-analysis:1.11
credentials:2.5
snakeyaml-api:1.29.1
pipeline-input-step:2.12
git:4.8.2
dashboard-view:2.17
docker-java-api:3.1.5.2
pam-auth:1.6
structs:1.23
async-http-client:1.9.40.0
docker-commons:1.17
pipeline-model-definition:1.9.1
dockerhub-notification:2.5.2
github-api:1.123
token-macro:266.v44a80cf277fd
command-launcher:1.6
workflow-durable-task-step:2.39
workflow-basic-steps:2.24
popper2-api:2.9.3-1
pipeline-milestone-step:1.3.2
plain-credentials:1.7
apache-httpcomponents-client-4-api:4.5.13-1.0
workflow-cps:2.93
ldap:2.7
docker-plugin:1.2.3
ssh-slaves:1.33.0
ssh-agent:1.23
okhttp-api:3.14.9
caffeine-api:2.9.2-29.v717aac953ff3
antisamy-markup-formatter:2.1
jaxb:2.3.0.1
momentjs:1.1.1
docker-build-step:2.8
jdk-tool:1.5
workflow-support:3.8
matrix-auth:2.6.8
pipeline-stage-step:2.5
plugin-util-api:2.4.0
build-name-setter:2.2.0
pipeline-stage-view:2.19
pipeline-github-lib:1.0
workflow-aggregator:2.6
gitlab-plugin:1.5.20
echarts-api:5.1.2-11
workflow-cps-global-lib:2.21
workflow-multibranch:2.26
timestamper:1.13
workflow-step-api:2.24
sshd:3.1.0
checks-api:1.7.2
workflow-job:2.41
authentication-tokens:1.4
pipeline-model-api:1.9.1
mailer:1.34
copyartifact:1.46.1
workflow-scm-step:2.13
jjwt-api:0.11.2-9.c8b45b8bb173
matrix-project:1.19
scm-api:2.6.5
bootstrap5-api:5.1.0-3
ssh:2.6.1
display-url-api:2.3.5
configuration-as-code:1.53
job-dsl:1.77
ansicolor:1.0.0