KALPESH

Posted on Mar 5

CI/CD & Jenkins

#cicd #githubactions #jenkins

Stages that an Organization Follows in CI/CD:

Jenkins Folder Structure

/var/lib/jenkins/
│
├── 📦 plugins/              → Installed plugins (.jpi files)
│   ├── git.jpi
│   ├── docker.jpi
│   └── slack.jpi
│
├── 💼 jobs/                 → Job configurations only
│   ├── my-app/
│   │   ├── builds/          → Build history
│   │   │   ├── 1/           → Build #1
│   │   │   │   ├── log      → Console output
│   │   │   │   └── build.xml
│   │   │   ├── 2/           → Build #2
│   │   │   └── 3/           → Build #3
│   │   └── config.xml       → Job configuration
│   │
│   └── another-job/
│       ├── builds/
│       └── config.xml
│
├── 🔨 workspace/            → ALL job workspaces
│   ├── my-app/              → my-app workspace
│   │   ├── src/
│   │   ├── package.json
│   │   └── node_modules/
│   │
│   ├── another-job/         → another-job workspace
│   │   ├── src/
│   │   └── pom.xml
│   │
│   └── pipeline-1/          → pipeline-1 workspace
│       └── Dockerfile
│
├── 👥 users/                → User accounts & settings
│   ├── admin/
│   │   └── config.xml
│   └── developer/
│       └── config.xml
│
├── 📝 logs/                 → Jenkins system logs
│
├── 🔐 secrets/              → Encryption keys & secrets
│   ├── master.key
│   └── hudson.util.Secret
│
├── ⚙️ config.xml            → Main Jenkins configuration
│
├── 📊 fingerprints/         → File tracking (artifacts)
│
└── 🗂️ updates/              → Plugin update cache

Jenkins: Plugins vs Tools vs Direct Installation

Aspect	Plugins	Tools (Configured)	Direct Installation
Installed Where	Inside Jenkins	On Jenkins server (Inside Jenkins Directory)	On Jenkins server
Managed By	Jenkins Plugin Manager	Jenkins Tool Config	System admin (apt/yum)
Purpose	Add features/integrations	Version management	Direct execution
UI Integration	✅ Yes	✅ Yes	❌ No
Examples	Git, Slack, Docker Plugin	Maven, JDK, Node.js	Docker, kubectl, AWS CLI
Usage	Special syntax in pipeline	`tools {}` block	`sh 'command'`

CI/CD Tools terminology

Concept	Jenkins	GitLab	GitHub	Azure
Top level	Pipeline	Pipeline	Workflow	Pipeline
Phase grouping	Stages	Stages	-	Stages
Execution unit	Stage	Job	Job	Job
Commands	Steps	Script	Steps	Steps
Runs on (where)	Agent	Runner	Runner	Agent/Pool
Keyword	`agent`	`tags`	`runs-on`	`pool`
Defined at level	Pipeline / Stage	Job	Job (required)	Pipeline / Stage / Job

Installation & Configuration of Jenkins on EC2 Instance

Go to AWS Console
Launch instances

Installation of Jenkins on EC2 Instance

Jenkins is Java based application

Pre-Requisites:

Install Java

sudo apt update
sudo apt install openjdk-11-jre

Verify Java is Installed

java -version

Now, you can proceed with installing Jenkins

curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins

Changing EC2 inbound traffic rules to expose Jenkins

Note: By default, Jenkins will not be accessible to the external world due to the inbound traffic restriction by AWS. Open port 8080 in the inbound traffic rules as show below.

EC2 > Instances > Click on
In the bottom tabs -> Click on Security
Security groups
Add inbound traffic rules as shown in the image (you can just allow TCP 8080 as well, in my case, I allowed All traffic).

Login to Jenkins using the below URL:

http://[public IP of ec2]:8080

Note: If you are not interested in allowing All Traffic to your EC2 instance

1. Delete the inbound traffic rule for your instance

2. Edit the inbound traffic rule to only allow custom TCP port 8080

After you login to Jenkins,

Run the command to copy the Jenkins Admin Password - sudo cat /var/lib/jenkins/secrets/initialAdminPassword
Enter the Administrator password

Click on Install suggested plugins

Wait for the Jenkins to Install suggested plugins

Create First Admin User or Skip the step [If you want to use this Jenkins instance for future use-cases as well, better to create admin user]

Jenkins Installation is Successful. You can now starting using the Jenkins

Install the Docker Pipeline plugin in Jenkins:

Log in to Jenkins.
Go to Manage Jenkins > Manage Plugins.
In the Available tab, search for "Docker Pipeline".
Select the plugin and click the Install button.
Restart Jenkins after the plugin is installed.

Wait for the Jenkins to be restarted.

Docker Slave Configuration

Run the below command to Install Docker

sudo apt update
sudo apt install docker.io

Grant Jenkins user and Ubuntu user permission to docker deamon.

sudo su - 
usermod -aG docker jenkins
usermod -aG docker ubuntu
systemctl restart docker

Once you are done with the above steps, it is better to restart Jenkins.

http://<ec2-instance-public-ip>:8080/restart

The docker agent configuration is now successful.

Sample Jenkinsfile- Docker as Agent

jenkinsfile:

pipeline {
  agent {
    docker { image 'node:16-alpine' }
  }
  stages {
    stage('Test') {
      steps {
        sh 'node --version'
      }
    }
  }
}