DEV Community

nithinalias
nithinalias

Posted on • Edited on

Devops Project - CI/CD Jenkins Ansible Docker

CI/CD pipeline using Git,Jenkins and Maven

Install jenkins using vagrant file

# -*- mode: ruby -*-
# vi: set ft=ruby :


Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu/focal64"

   config.vm.network "private_network", ip: "192.168.33.10"


  config.vm.provider "virtualbox" do |vb|
  vb.memory = "1536"
  end

   config.vm.provision "shell", inline: <<-SHELL
   apt update
   apt install -y openjdk-11-jre
   wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | apt-key add -
   sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
   apt update
   apt install -y jenkins
   SHELL
   end
Enter fullscreen mode Exit fullscreen mode

Now login to jenkin http://ip-address:8080 and enter the password
that you get from /var/lib/jenkins/secrets/initialAdminPassword.You can change the password from Manage jenkins - Manage users.

Integrate Git with Jenkins

Manage Jenkins - Manage plugins - Available - Select Github - Install without restart - Go back to the top page - Manage Jenkins - Global Tool Configuration - Add git name,Add Path to Git executable(This we get from linux terminal by using command 'whereis git' and you get ouput '/usr/bin/git'.If it won't work you can use 'git'.) - Apply - save

New Item - Add item name(PullcodefromGithub),select Freestyle - OK - Add description - Select Git and add Repositories URL,Branch - Apply - Save - Build Now - Console output.You can see the output
/var/lib/jenkins/workspace.

Integrate Maven with Jenkins

Install maven

Download the Binary tar.gz archive from https://maven.apache.org/download.cgi

sudo su -
cd /opt
wget https://dlcdn.apache.org/maven/maven-3/3.8.5/binaries/apache-maven-3.8.5-bin.tar.gz
tar -xvzf apache-maven-3.8.5-bin.tar.gz
cd /opt/apache-maven-3.8.5/bin
./mvn -v
Enter fullscreen mode Exit fullscreen mode

Outside this directory 'mvn -v' won't work.we need to set up the environment variables(path of maven,java).

find / -name jvm
export PATH=$PATH:/opt/apache-maven-3.8.5:/opt/apache-maven-3.8.5/bin:/usr/lib/jvm/java-11-openjdk-amd64
echo $PATH
Enter fullscreen mode Exit fullscreen mode

To change enviornment variables permanentely.

vim /root/.profile

# ~/.profile: executed by Bourne-compatible login shells.

if [ "$BASH" ]; then
  if [ -f ~/.bashrc ]; then
    . ~/.bashrc
  fi
fi

M2_HOME=/opt/apache-maven-3.8.5
M2=/opt/apache-maven-3.8.5/bin
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
PATH=$PATH:$HOME/bin:$JAVA_HOME:$M2_HOME:$M2
export PATH
mesg n 2> /dev/null || true

echo $PATH
source .profile
echo $PATH
Enter fullscreen mode Exit fullscreen mode

Manage Jenkins - Manage plugins - Available - Select Maven Integration - Install without restart - Go back to the top page - Manage Jenkins - Global Tool Configuration - Add JDK name,JAVA_HOME(/usr/lib/jvm/java-11-openjdk-amd64),Maven name,MAVEN_HOME(/opt/apache-maven-3.8.5) - Apply - save

New Item - Add item name(Mavenproject),select Maven project - OK - Add description - Select Git and add Repositories URL,Branch - Build(Add Root POM=pom.xml,Goals and options=clean install) - Apply - Save - Build Now - Console output.You can see all build outcomes in /var/lib/jenkins/workspace/Mavenproject/webapp/target/.surefire contain the reports of build and webapp.war is called artifacts.

Integrating Tomcat server in CI/CD pipeline




       Pull Code        Copy Artifacts & Deploy Code
Github---------->Jenkin----------------------------->Tomcat server
                    |  
                    | Build Code
                   \|/
                  Maven
Enter fullscreen mode Exit fullscreen mode

Setup a Tomcat server
create a virtual machine and install JDK,Tomcat server.
Download tar.gz tomcat packages from https://tomcat.apache.org/download-80.cgi

sudo su -
apt update
apt install -y openjdk-11-jre
cd /opt
wget https://dlcdn.apache.org/tomcat/tomcat-8/v8.5.77/bin/apache-tomcat-8.5.77.tar.gz
tar -xvzf apache-tomcat-8.5.77.tar.gz
cd /opt/apache-tomcat-8.5.77/bin
./startup.sh
Enter fullscreen mode Exit fullscreen mode

Now login to tomcat http://ip-address:8080 - Manage App - 403 Access Denied
By default the Manager is only accessible from a browser running on the same machine as Tomcat. If you wish to modify this restriction, you'll need to edit the Manager's context.xml file.

find / -name context.xml
vim /opt/apache-tomcat-8.5.77/webapps/host-manager/META-INF/context.xml
vim /opt/apache-tomcat-8.5.77/webapps/manager/META-INF/context.xml
Enter fullscreen mode Exit fullscreen mode

Disable the line shown below by using <!-- --> in both context.xml file.

<!--  <Valve className="org.apache.catalina.valves.RemoteAddrValve"
allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" />  -->
Enter fullscreen mode Exit fullscreen mode

Now restart tomcat server.

cd /opt/apache-tomcat-8.5.77/bin
./shutdown.sh
./startup.sh
Enter fullscreen mode Exit fullscreen mode

Login to tomcat http://ip-address:8080 - Manage App.Now it will ask for credentials.

create link files for tomcat startup.sh and shutdown.sh

ln -s /opt/apache-tomcat-8.5.77/bin/startup.sh /usr/local/bin/tomcatup
ln -s /opt/apache-tomcat-8.5.77/bin/shutdown.sh /usr/local/bin/tomcatdown
tomcatup
Enter fullscreen mode Exit fullscreen mode

Update users information in the tomcat-users.xml file goto tomcat home directory and Add below users to /opt/apache-tomcat-8.5.77/conf/tomcat-users.xml file.

<role rolename="manager-gui"/>
<role rolename="manager-script"/>
<role rolename="manager-jmx"/>
<role rolename="manager-status"/>
<user username="admin" password="admin" roles="manager-gui, manager-script, manager-jmx, manager-status"/>
<user username="deployer" password="deployer" roles="manager-script"/>
<user username="tomcat" password="s3cret" roles="manager-gui"/>
Enter fullscreen mode Exit fullscreen mode

Now restart tomcat server and using credential login to tomcat http://ip-address:8080 - Manage App

tomcatdown
tomcatup
Enter fullscreen mode Exit fullscreen mode

Integrate Tomcat with jenkin

Manage Jenkins - Manage plugins - Available - Select Deploy to container(This plugin allows you to deploy a war to a container after a successful build) - Install without restart - Go back to the top page - Manage Jenkins - Manage Credentials - Jenkins - Global credentials - Add credentials - kind(username with password) - username="deployer",password="deployer"(If one system want to access another system we need roles="manager-script"),ID="tomcat_deployer",Description="tomcat_deployer" - Apply - save

New Item - Add item name(BuildAndDeploy),select Maven project - OK - Add description - Select Git and add Repositories URL,Branch - Build Triggers(Poll SCM=* * * * * means every minute check the repository and if there any update it will trigger build - Build periodically means it will trigger build periodically even if repository not updated) - Build(Add Root POM=pom.xml,Goals and options=clean install) - Add post-build Action - Deploy war/ear to a container - Add WAR/EAR files=*/.war,Add Containers=Tomcat 8.x Remote,select the deployer credentail that we created,Tomcat URL=http://ip-address:8080 - Apply - Save - Build Now - Console output.You can see all build outcomes in /var/lib/jenkins/workspace/BuildAndDeploy/webapp/target.
surefire contain the reports of build and webapp.war is called artifacts.This webapp.war is deployed inside /opt/apache-tomcat-8.5.77/webapps of tomcat server by authencating the credentials.

Using credential login to tomcat http://ip-address:8080 - Manage App - select the path webapp - Now you get the required output.

Integrate Docker in CI/CD Pipeline



       Pull Code        Copy Artifacts & Deploy Code
Github---------->Jenkin-----------------------------> Docker
                    |  
                    | Build Code
                   \|/
                  Maven
Enter fullscreen mode Exit fullscreen mode

Create a docker virtual machine using Vagrantfile.

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu/focal64"

  config.vm.network "private_network", ip: "192.168.33.20"

  config.vm.provider "virtualbox" do |vb|
  vb.memory = "1024"
  end

  config.vm.provision "shell", inline: <<-SHELL
     apt update -y
     apt install apt-transport-https ca-certificates -y
     curl software-properties-common
     curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
     add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
     apt-cache policy docker-ce
     apt install docker-ce -y
  SHELL
end
Enter fullscreen mode Exit fullscreen mode

Pull and run tomcat image.

sudo su -
docker pull tomcat
docker images
docker run -d --name tomcat-container 8081:8080 tomcat
docker ps -a
Enter fullscreen mode Exit fullscreen mode

Login to tomcat http://ip-address:8081.It will show an error like HTTP Status 404 – Not Found.This is because when we browse http://ip-address:8081 it will look into /usr/local/tomcat/webapps
directory.By default this directory will be empty and contents will be present inside /usr/local/tomcat/webapps.dist.so we need to copy this contents to webapp directory.

docker exec -it tomcat-container /bin/bash
cd /usr/local/tomcat/webapps
ls
cd /usr/local/tomcat/webapps.dist
ls
cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
exit
Enter fullscreen mode Exit fullscreen mode

Login to tomcat http://ip-address:8081.

Create tomcat using Dockerfile

sudo su -
vim Dockerfile

FROM ubuntu
RUN apt-get -y update
RUN apt-get -y install openjdk-11-jre
RUN mkdir /opt/tomcat
WORKDIR /opt/tomcat
ADD https://dlcdn.apache.org/tomcat/tomcat-9/v9.0.60/bin/apache-tomcat-9.0.60.tar.gz /opt/tomcat
RUN tar -xvzf apache-tomcat-9.0.60.tar.gz
RUN mv apache-tomcat-9.0.60/* /opt/tomcat
EXPOSE 8080
CMD [ "/opt/tomcat/bin/catalina.sh", "run"]


docker build -t mytomcatserver .
docker run -d --name mytomcat-server -p 8083:8080 mytomcatserver
Enter fullscreen mode Exit fullscreen mode

Login to tomcat http://ip-address:8083.

Create customized Dockerfile for tomcat

sudo su -
vim Dockerfile

FROM tomcat
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps


docker build -t mytomcat .
docker run -d --name mytomcat-container -p 8085:8080 mytomcat
Enter fullscreen mode Exit fullscreen mode

Login to tomcat http://ip-address:8085.

Integrate docker with Jenkins and Update tomcat dockerfile to automate deployment process
Enable password authentication in docker machine.

sudo su -
vim /etc/ssh/sshd_config

PasswordAuthentication yes
#PasswordAuthentication no

systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Create a user and add to group 'docker'.

adduser dockeradmin
cat /etc/group
usermod -aG docker dockeradmin
id dockeradmin
Enter fullscreen mode Exit fullscreen mode

Create tomcat dockerfile

mkdir /opt/docker
vim /opt/docker/Dockerfile

FROM tomcat
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
COPY ./*.war /usr/local/tomcat/webapps
Enter fullscreen mode Exit fullscreen mode

we need to give ownership of Dockerfile to dockeradmin.Then only we can automate the deployment using jenkin server.After building, webapp.war inside jenkin server workspace(/var/lib/jenkins/workspace/BuildAndDeploy/webapp/target) will be move to /opt/docker of docker machine.

chown -R dockeradmin:dockeradmin /opt/docker
Enter fullscreen mode Exit fullscreen mode

Go to Jenkin Server

Manage Jenkins - Manage plugins - Available - Select Publish Over SSH(Send build artifacts over SSH) - Install without restart - Go back to the top page - Manage Jenkins - Configure System - SSH servers - Add - Add name(dockerhost),Hostname(ip-of-dockermachine),username(dockeradmin) - Advanced - Enable Use password authentication or use a different key - Add password of user dockeradmin(instead you can use ssh-key/path of ssh-key if present) - Test configuration - Apply - save.

New Item - Add item name(BuildAndDeploy),select Maven project - OK - Add description - Select Git and add Repositories URL,Branch - Build Triggers(Poll SCM=* * * * * means every minute check the repository and if there any update it will trigger build - Build periodically means it will trigger build periodically even if repository not updated) - Build(Add Root POM=pom.xml,Goals and options=clean install) - Add post-build Action - Send build artifacts over SSH - Select SSH server name(dockerhost),Add sourcefile(webapp/target/*.war),Add Remove prefix(webapp/target),Add Remote directory="//opt//docker"(if it is blank webapp.war will be in /home/dockeradmin),
Exec command ="cd /opt/docker;
docker build -t mytomcat .;
docker container stop tomcat-container;
docker rm tomcat-container;
docker run -d --name tomcat-container -p 8081:8080 mytomcat" - Apply - Save - Build Now - Console output.You can see all build outcomes in /var/lib/jenkins/workspace/BuildAndDeploy/webapp/target.
surefire contain the reports of build and webapp.war is called
artifacts.This webapp.war will be in docker machine /opt/docker directory and copy this to tomcat image for tomcat container creation.

Now you can browse http://ip-adress:8081/webapp/

Integrating ansible in CI/CD pipeline





                                               Dockerhub 
                                              /        \
                                            /            \                          
                                      Push/Image       Pull\Image        
                                        /                    \
      Pull Code       Copy Artifacts  /     Deploy Container   \
Github--------->Jenkin-------------->Ansible--------------->Docker
                    |  
                    | Build Code
                   \|/
                  Maven
Enter fullscreen mode Exit fullscreen mode

Create a ansible machine having docker using vagrant file.

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu/focal64"

  config.vm.network "private_network", ip: "192.168.33.25"

  config.vm.provider "virtualbox" do |vb|
  vb.memory = "1536"
  end

  config.vm.provision "shell", inline: <<-SHELL
     apt update -y
     apt install ansible -y
     apt install apt-transport-https ca-certificates -y
     curl software-properties-common
     curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
     add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
     apt-cache policy docker-ce
     apt install docker-ce -y
  SHELL
end
Enter fullscreen mode Exit fullscreen mode

create a user and allow password authentication.

sudo su -
adduser ansadmin
id ansadmin

vim /etc/ssh/sshd_config

PasswordAuthentication yes
#PasswordAuthentication no

systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Modify visudo for created user.

vim /etc/sudoers

# User privilege specification
root    ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
Enter fullscreen mode Exit fullscreen mode

User Privilege Lines

root ALL=(ALL:ALL) ALL The first field indicates the username that the rule will apply to (root).

root ALL=(ALL:ALL) ALL The first “ALL” indicates that this rule applies to all hosts.

root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can run commands as all users.

root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can 
run commands as all groups.

root ALL=(ALL:ALL) ALL The last “ALL” indicates these rules apply to all commands.
Enter fullscreen mode Exit fullscreen mode

Generate ssh-key.

su - ansadmin
ssh-keygen
Enter fullscreen mode Exit fullscreen mode

Go to docker machine and create user,add to group docker,docker.sock permissions to all users

sudo su -
adduser ansadmin
usermod -aG docker ansadmin
id ansadmin
chmod 777 /var/run/docker.sock
systemctl restart docker
docker login
Enter fullscreen mode Exit fullscreen mode

Allow password authentication.

vim /etc/ssh/sshd_config

PasswordAuthentication yes
#PasswordAuthentication no

systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Modify visudo for created user.

vim /etc/sudoers

# User privilege specification
root    ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
Enter fullscreen mode Exit fullscreen mode

Go to ansible machine and add ip address of docker machine and ansible machine(then only jenkin server can access ansible and docker machine) inside ansible host file.

vim /etc/ansible/hosts

[docker]
192.168.33.20
[ansible]
192.168.33.25
Enter fullscreen mode Exit fullscreen mode

copy the ssh public key to docker machine and ansible machine itself(localhost).

su - ansadmin
ssh-copy-id ansadmin@ip-of-ansiblemachine
ssh-copy-id ansadmin@ip-of-dockermachine
ansible all -m ping
Enter fullscreen mode Exit fullscreen mode

Now authorized key will be generated inside /home/ansadmin/.ssh/authorized_keys of docker and ansible machine.

create a directory,give ownership of ansadmin,add ansadmin to docker group and docker.sock permissions to all users.

sudo su -
mkdir /opt/docker
chown ansadmin:ansadmin /opt/docker
usermod -aG docker ansadmin
id ansadmin
chmod 777 /var/run/docker.sock
systemctl restart docker
su - ansadmin
docker login
Enter fullscreen mode Exit fullscreen mode

Create dockerfile for tomcat image creation.

vim /opt/docker/dockerfile

FROM tomcat
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
COPY ./*.war /usr/local/tomcat/webapps
Enter fullscreen mode Exit fullscreen mode

Generate ansible-playbook for building tomcat image.

vim /opt/docker/build_tomcatimage.yml

---

- hosts: ansible

  tasks:
  - name: create docker image
    command: docker build -t mytomcat .
    args:
     chdir: /opt/docker

  - name: create tag to push image onto dockerhub
    command: docker tag mytomcat nithinalias/mytomcat

  - name: push docker image
    command: docker push nithinalias/mytomcat
Enter fullscreen mode Exit fullscreen mode

Generate ansible-playbook for deploy tomcat.

vim /opt/docker/deploy_tomcat.yml

---
- hosts: docker

  tasks:
  - name: stop existing container
    command: docker stop tomcat-container
    ignore_errors: yes

  - name: remove the container
    command: docker rm tomcat-container
    ignore_errors: yes

  - name: remove image
    command: docker rmi nitinalias/mytomcat
    ignore_errors: yes

  - name: create container
    command: docker run -d --name tomcat-container -p 8082:8080 nithinalias/mytomcat


# If there is no container it will show error.To avoid this we use ignore errors: yes
~
Enter fullscreen mode Exit fullscreen mode

Integrate Ansible with Jenkins

Already we installed Publish Over SSH plugin and added dockeradmin
ssh connection of docker machine.Now we need to add ansadmin
ssh connection of ansible machine.

Manage Jenkins - Configure System - SSH servers - Add - Add name(ansible-server),Hostname(ip-of-ansible machine),username
(ansadmin) - Advanced - Enable Use password authentication or use a different key - Add password of user ansadmin(instead you can use ssh-key/path of ssh-key if present) - Test configuration - Apply - save.

New Item - Add item name(BuildAndDeployUsingAnsible),select Maven project - OK - Add description - Select Git and add Repositories URL,Branch - Build Triggers(Poll SCM=* * * * * means every minute check the repository and if there any update it will trigger build - Build periodically means it will trigger build periodically even if repository not updated) - Build(Add Root POM=pom.xml,Goals and options=clean install) - Add post-build Action - Send build artifacts over SSH - Select SSH server name(ansible-server),Add sourcefile(webapp/target/*.war),Add Remove prefix(webapp/target),Add Remote directory="//opt//docker"(if it is blank webapp.war will be in /home/ansadmin),
Exec command ="ansible-playbook /opt/docker/build_tomcatimage.yml;
sleep 10;
ansible-playbook /opt/docker/deploy_tomcat.yml" - Apply - Save - Build Now - Console output.You can see all build outcomes in /var/lib/jenkins/workspace/BuildAndDeployUsingAnsible/
webapp/target.surefire contain the reports of build and webapp.war is called artifacts.This webapp.war will be in ansible machine /opt/docker directory and copy this to tomcat image.The image created inside ansible machine is pushed to dockerhub.Pull this image from dockerhub to create tomcat-container inside docker
machine.

Now you can browse http://ip-adress:8082/webapp/

Note: To check ansible-playbook and run ansible-playbook on particular host you can use the command shown below.

ansible-playbook /opt/docker/deploy_tomcat.yml --check
ansible-playbook /opt/docker/deploy_tomcat.yml --limit ip-adress/groupname
Enter fullscreen mode Exit fullscreen mode

Source: https://github.com/yankils

Top comments (0)

Read next

beansharon1 profile image

A Handbook to Grasping Contemporary Mobile Device Technology

Kara Zhao -

manashjb profile image

Stop Trying to Learn Everything -Focus on These 5 Key Skills Every Developer Needs

Manash Jyoti Baruah -

xxzeroxx profile image

PHP Design Patterns: Front Controller

Antonio Silva -

ayoola_damilare_212d5bde0 profile image

My React Journey: Day 23

Ayoola Damilare -