Automated CI/CD Pipeline for Web Applications Using Jenkins and GitHub

Emmanuel Madubuike — Thu, 15 Jun 2023 15:14:38 +0000

This is my portfolio project for the foundations year at ALX Africa and Holberton School.
It is an Automated CI/CD Pipeline built with Jenkins integrating with GitHub using web-hooks and integrating with Docker.
The main purpose of the project is to build an automated seamless pipeline for web applications using Continuous Integration or Continuous Delivery for web applications. Code changes and updates can be made for scalable applications using the pipeline without in any sort running the application down and also be tested and deployed using Docker.

Team Members, Roles and Timeline:

Members: This is a sole project built by Emmanuel Madubuike.
Roles:
- Project Manager: This project was supervised by me.
- Developer: This project was developed also by me.
- Tester: All tests(Integration) was done by me.
Timeline: This project was built on a 3 weeks time-frame.

This project was built for Software Engineers and DevOps Engineers to provide Continuous Integration or Continuous Delivery of scalable applications to users. Personal focus for this project was configuring the Jenkins pipeline, integrating with GitHub with web-hooks and also deploying on a Docker server.
I have always been a candid lover of DevOps so i thought it will be a great idea to work on a project relating to the field. I had earlier decided that I will work on a CI/CD Pipeline for my portfolio project but I had no prior idea as at then. So I made a through research on projects I can work on with a short time-frame on the related field and I came up with this. Throughout the timeline for this project, I came across some technologies I haven't used or heard before and it helped me deepen my understanding on them. I would have never guessed that 9 months later since my first day at ALX Africa that I will work on a sort of a project as this.

Architecture

Attached above is the architecture of the project. Code updates for the application will be pushed to GitHub which will automatically trigger a build process on Jenkins using web-hooks. Upon successful build process, the Jenkins server is configured to test and deploy to Docker either using the web interface or writing a Jenkinsfile for build, test and deploy operations.

Technologies

The technologies used for this project are as follows:

Jenkins: This is used to create the CI/CD Pipeline for the project.
GitHub: This is the version control for the source code. Written code and code changes are pushed to it.
Docker: Deployment of the application in containers for test and deployment purposes.
Flask: This was used to write a sample web application with a few endpoints to test the pipeline.
AWS web servers (EC2): Jenkins server and Docker server were installed on it to provide public internet access.

Key Features

Some of the key features in this project includes:

Webhooks: Webhooks serves as a trigger between Jenkins and GitHub, keeps track of code changes in the version control and aids in seamless Continuous Delivery.

Docker Integration: Direct integration of Docker from Jenkins to test and deploy applications through containerization. On the image below, it shows the configuration method in which Docker can be set for use.

Jenkins Configuration: Jenkins can be configured for certain tasks and certain builds depending on the job given. It is a nice feature as it enables to build, test and deploy on the pipeline.

Technical Challenges

On course of the project, I faced two major technical challenges of which I will list below:

Webhooks: This was the first major challenge I faced. At the beginning of the project, I had initially installed Jenkins server on my local machine which means the Jenkins server will be running on localhost. As it worked on localhost, I tried to configure webhooks for Jenkins but it wasn't working. I made a research and discovered Ngrok, which allows localhost to be seen on the public internet but Ngrok came with a difficulty, the timeout session. Figuring out this won't work efficiently, i made more extensive research and found the solution by making use of web servers for installation of Jenkins and Docker. After configuration and removal of firewall for the port, it started to work out well.
Docker Integration: This was also a major challenge i faced. I had initially installed Docker on my local machine so I could integrate it with my Jenkins server. On course of integrating, each deployment process I tried to run kept failing. I had to install it on my second web server and made it possible so I can ssh into the web server from my first web server. With the above method, it began to work perfectly.

Conclusion

As an engineer, so far on this project, I became more familiar with the above-mentioned technologies, of which I had no idea about. From integration to configuration of the pipeline. This project deepened my understanding of DevOps, CI/CD, Docker containers and Pipelines.

About the Author

Emmanuel Madubuike is a Software Engineer specializing with BackEnd and DevOps Engineering. Currently a student of ALX Africa and Holberton School. You can connect with me via the following handles:

LinkedIn: https://www.linkedin.com/in/madubuike-emmanuel
GitHub Project Repository: https://www.github.com/NuelUzoma/Portfolio-Project
Project Landing Page: https://emcool17.wixsite.com/portfolio
Jenkins server web interface: http://107.23.96.11:8080/job/Portfolio-Project

Continuous Delivery vs Continuous Deployment.

Emmanuel Madubuike — Wed, 17 May 2023 17:39:38 +0000

Continuous Delivery and Continuous Deployment represent two distinct phases in DevOps. Both steps are used in the deployment of software changes into production, but both aren't the same.

Continuous Delivery (CD)
This stage majorly concerns the written code and any new changes made to the code. Continuous Delivery can be defined as the deployment of software(code) changes into production environments. Code changes made comprised of

New features
Bug fixes
Refactoring
Performance enhancements
Configuration Changes etc.

CD allows for the seamless, frequent, and rapid deployment of code changes into production environments without running them down. It is an essential stage for any deployment process. The CD provides a flexible manual way of releasing the changes deployed for end users. To enhance the CD process, a set of appropriate tools are implemented, and they are:

Version Control Systems: This is an essential part because it manages source code and enables team members to collaborate and make changes to the source code. An example is Git.
Continuous Integration (CI) tools: A notable example, Jenkins, helps in validating the code changes made to the version control and automates the build and testing process.
QA Testing tools: Another notable framework here is Selenium, which helps developers execute unit tests, integration tests, and performance tests.
Configuration Management tools: Configuration Management tool like Puppet helps in facilitating consistent and flexible deployments across various production environments.

The aforementioned tools serve as a limited selection of the various tools utilized in Continuous Delivery. To ensure a complete process of CD, the right combination of toolsets must be used depending on the technology stack and project requirements.

Continuous Deployment (CDep)
Continuous Deployment also can be referred to as the automation of code changes released to production environments. But take note of the keyword here, AUTOMATION.
It can be referred to as a step further to CD because it deals with the automated deployments of the software changes into production after all the necessary criteria have been met and all tests have been passed. Manual approval is eliminated in this process because CDep deals solely with automation.
The main aim of CDep is to minimize the time the end users can be able to access the deployed software changes and updates. By automating the release process, software updates, new features, and improvements can be rapidly delivered to the end users, providing a seamless, and rapid software delivery pipeline.

The toolsets utilized in CDep are similar to CD but the emphasis of the tools may vary to accommodate the automation needs of CDep.

Containerization tools: Notable technologies, such as Docker and Kubernetes, to run applications and their dependencies in portable containers.
Monitoring and Alerting tools: Monitoring tools such as Datadog, are essential for monitoring software applications and infrastructure. These tools provide real-time visibility into availability, performance, and various metrics, enabling rapid issue detection and swift response.

Conclusion
The primary distinction between Continuous Delivery (CD) and Continuous Deployment (CDep) lies in how the release decisions are made. In CD, teams manually decide when to release the software changes whereas CDep automates the release process, once required tests are successfully passed, the changes are promptly deployed to the production environment.

DEV Community: Emmanuel Madubuike