Table of Contents
- Introduction
- What is CI/CD?
- Benefits of CI/CD
- How to Create Your First CI/CD Pipeline
- Best Practices for CI/CD Pipelines
- Conclusion
Introduction
Continuous Integration/Continuous Deployment (CI/CD) pipelines are at the heart of modern software development. They allow teams to automate key steps in the development lifecycle, from integrating new code changes, running automated tests, and deploying applications to various environments. As DevOps practices become more widespread, mastering CI/CD pipelines is essential for developers, DevOps engineers, and IT teams.
In this guide, we will focus on helping you get started with Jenkins, one of the most popular CI/CD tools. Jenkins' flexibility and rich ecosystem of plugins make it a powerful solution for automating a variety of workflows, from simple projects to complex microservices architectures. This comprehensive guide will take you through the entire process of setting up a CI/CD pipeline with Jenkins — from installation to creating your first pipeline, and beyond.
What is CI/CD?
Before we dive into Jenkins, let's take a moment to understand the key components of CI/CD.
Continuous Integration (CI)
Continuous Integration (CI) is a software development practice where developers frequently commit code changes to a shared repository, ideally multiple times a day. Each time new code is committed, an automated process is triggered that builds the code, runs tests, and identifies bugs early. The core purpose of CI is to improve collaboration, detect integration problems quickly, and reduce integration issues that can arise when multiple developers work on the same project.
- Frequent Commits: Developers commit code changes more frequently to ensure the repository always has the latest working code.
- Automated Build and Test: After every commit, automated builds and tests run, giving developers quick feedback on their changes.
- Fail Early: CI allows teams to detect issues early, preventing small bugs from snowballing into more significant problems later in the process.
Continuous Delivery (CD)
Continuous Delivery (CD) builds on top of CI, automating the process of delivering code changes to a staging environment for further testing and validation. The goal of CD is to ensure that the codebase is always in a deployable state and that new features, bug fixes, and updates can be delivered to users as soon as they're ready.
- Automated Testing and Packaging: CD ensures that code changes are always tested and packaged for release.
- Manual Approval for Production: With Continuous Delivery, deployment to production is manual but prepared automatically. Human intervention is required for final approval.
- Release Anytime: By having a deployable product at any given time, teams can release new features faster and more frequently.
Continuous Deployment
Continuous Deployment takes Continuous Delivery a step further by automating the deployment of every change that passes through all stages of the pipeline directly to production. Continuous Deployment requires a high degree of confidence in the automated testing process because there’s no manual approval before deploying to production.
Benefits of CI/CD
Faster Time to Market
CI/CD pipelines allow teams to deliver software updates to users much faster. By automating the integration, testing, and deployment processes, teams can release features, improvements, and bug fixes in a continuous flow rather than waiting for a major release cycle.
- Shorter Feedback Loop: Developers get faster feedback on their code.
- Faster Deployments: Automated pipelines significantly reduce the time it takes to go from code commit to production.
Improved Code Quality
With CI/CD, code is continuously tested throughout the development process. This leads to higher quality code being integrated into the main repository, as issues are caught early through automated testing.
- Automated Tests: Unit, integration, and end-to-end tests can run automatically every time code is committed, ensuring that the code meets quality standards.
- Frequent Testing: Regular testing reduces the number of bugs that slip into production.
Efficient Collaboration
CI/CD pipelines promote better collaboration between team members. Since code is integrated frequently and tested in a shared environment, teams can collaborate more effectively, and any integration issues are identified and resolved early in the process.
- Shared Codebase: Developers always work with the latest code, which reduces the chances of merge conflicts or broken builds.
- Fewer Surprises: The team is more likely to identify breaking changes earlier in the development cycle.
Increased Automation and Consistency
By automating repetitive and manual tasks such as building, testing, and deployment, CI/CD pipelines reduce human error, improve consistency, and free up developer time to focus on more complex tasks.
- Consistency: Automated pipelines ensure that every build follows the same steps and produces reliable results.
- Reduced Errors: Automation removes the possibility of human error in deployment processes.
How to Create Your First CI/CD Pipeline
Now that we understand the benefits of CI/CD, let’s dive into the practical steps of creating your first CI/CD pipeline using Jenkins.
Step 1: Set Up Version Control (GitHub)
Before setting up Jenkins, we need a version-controlled repository for your project code. We’ll use GitHub as the version control platform in this example.
Instructions:
-
Create a GitHub Repository:
- Go to GitHub and create a new repository.
- Name the repository (e.g.,
my-first-pipeline), provide a description, and initialize the repository with aREADME.mdfile. - Choose whether the repository will be public or private.
-
Clone the Repository to Your Local Machine:
- Open a terminal and clone the repository using the following command:
git clone https://github.com/your-username/my-first-pipeline.git
-
Navigate to the project folder:
cd my-first-pipeline
-
Add Project Files:
- Add your application files (for example, a simple web app) to the repository.
- For a Node.js application, this might include files such as
app.js,package.json, and atestsdirectory.
-
Commit and Push the Code:
- Use the following commands to stage, commit, and push your changes:
git add . git commit -m "Initial commit" git push origin main
Your repository now contains the codebase for your application, and it’s ready for Jenkins to automate the build and deployment processes.
Step 2: Choose a CI/CD Tool
There are various CI/CD tools available such as GitHub Actions, CircleCI, GitLab CI, and Jenkins. In this guide, we will focus on Jenkins due to its flexibility, robust community support, and extensive plugin ecosystem.
Step 3: Jenkins Pipeline Setup
Step 3.1: Installing Jenkins
The first step in setting up Jenkins is installing it on your local machine or server. Jenkins can run on various platforms like Windows, macOS, and Linux, and it can also be installed in a Docker container. Here, we will cover the installation process for a Linux machine (Ubuntu/Debian) and Docker.
Installing Jenkins on Linux (Ubuntu/Debian)
- Install Java: Jenkins requires Java to run. Install OpenJDK:
sudo apt update
sudo apt install openjdk-11-jdk
-
Add Jenkins Repository:
- Add the Jenkins repository key and add it to your sources:
wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add - sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/
/etc/apt/sources.list.d/jenkins.list'
```
- Install Jenkins:
sudo apt update
sudo apt install jenkins
- Start and Enable Jenkins:
sudo systemctl start jenkins
sudo systemctl enable jenkins
-
Access Jenkins:
- Open a browser and go to
http://localhost:8080(replacelocalhostwith the server IP if you're using a remote server). - You will be prompted to unlock Jenkins. The initial password can be found in the following file:
cat /var/lib/jenkins/secrets/initialAdminPassword - Open a browser and go to
Install Suggested Plugins: Jenkins will guide you through the installation of essential plugins like Git, Pipeline, and Docker. Click Install Suggested Plugins to set up the most commonly used plugins.
Installing Jenkins using Docker
Alternatively, you can install Jenkins using Docker with the following commands:
- Install Docker:
sudo apt update
sudo apt install docker.io
- Run Jenkins in Docker:
docker run -p 8080:8080 -p 50000:50000 jenkins/jenkins:lts
You can now access Jenkins in your browser by navigating to http://localhost:8080.
Step 3.2: Jenkins Configuration
Once Jenkins is installed, some basic configuration is necessary to integrate it with GitHub and to prepare it for pipeline creation.
-
Install Git:
- Make sure Git is installed on your Jenkins server:
sudo apt install git -
Configure Git in Jenkins:
- Go to Manage Jenkins > Global Tool Configuration and configure the Git executable path. This allows Jenkins to use Git for cloning repositories.
-
Set Up Credentials for GitHub:
- Jenkins will need access to your GitHub repository. To set up credentials:
- Go to Manage Jenkins > Manage Credentials > Global.
- Add credentials such as a GitHub personal access token or SSH key to allow Jenkins to authenticate with GitHub.
- If you use HTTPS for GitHub, create a Username with password type credential, where the username is your GitHub username, and the password is your personal access token (PAT). If using SSH, add your SSH private key as a SSH Username with private key credential.
- Jenkins will need access to your GitHub repository. To set up credentials:
Step 3.3: Creating a Jenkins Pipeline
Jenkins offers a powerful pipeline feature that allows you to define the stages of your build, test, and deployment processes using code. This is done by creating a Jenkinsfile in your repository, which Jenkins will automatically detect and use to execute your pipeline.
-
Create a New Pipeline Job:
- Go to Jenkins Dashboard > New Item.
- Enter a name for your pipeline (e.g.,
my-first-pipeline) and select Pipeline from the job types. - Click OK to create the job.
-
Configure Pipeline Script from SCM:
- Scroll down to the Pipeline section in the job configuration page.
- Under Definition, select Pipeline script from SCM.
- Choose Git as the source control system.
- Provide the URL of your GitHub repository and select the appropriate branch (e.g.,
main). - Specify the path to the
Jenkinsfilein your repository (this is usually justJenkinsfileif it’s located in the root of the repository).
At this point, Jenkins is set up to automatically detect the Jenkinsfile in your repository and execute the pipeline defined within it.
Step 4: Write a Basic Pipeline Configuration (Jenkinsfile)
The pipeline's behavior is defined in a Jenkinsfile, which is a text file that contains the pipeline’s stages and steps. Let’s write a simple Jenkinsfile for a basic Node.js project.
Example Jenkinsfile for Node.js:
pipeline {
agent any
stages {
stage('Checkout') {
steps {
git url: 'https://github.com/your-username/my-first-pipeline.git', branch: 'main'
}
}
stage('Build') {
steps {
// Install dependencies
sh 'npm install'
}
}
stage('Test') {
steps {
// Run tests
sh 'npm test'
}
}
stage('Deploy') {
steps {
// Deployment logic goes here (e.g., pushing to a cloud provider)
echo 'Deploying application...'
}
}
}
post {
always {
echo 'Pipeline completed.'
}
failure {
echo 'Pipeline failed!'
}
}
}
Breakdown of the Jenkinsfile:
-
agent any: This means the pipeline can run on any available Jenkins agent (a node or worker). -
Stages: Pipelines consist of multiple stages that represent different parts of the CI/CD process:
- Checkout: This stage checks out the latest code from your GitHub repository.
-
Build: This stage runs the command
npm installto install dependencies for a Node.js application. -
Test: This stage runs
npm testto execute the unit tests. - Deploy: This is where deployment steps are added, such as pushing the built code to a cloud platform like AWS, Azure, or Heroku.
-
Post-actions: The
postsection allows you to specify actions to take after the pipeline completes, regardless of success or failure. In this case, we print a message depending on whether the pipeline succeeded or failed.
To implement this pipeline, simply create a new file in your repository named Jenkinsfile and add the above pipeline script to it. Once committed and pushed to the repository, Jenkins will detect the Jenkinsfile and execute the pipeline.
Step 5: Deploy the Application
Once your pipeline runs successfully and the tests pass, the next logical step is to deploy the application. Jenkins can be integrated with various cloud platforms, such as Heroku, AWS, or Azure, for automated deployments.
Example: Deploying to Heroku
Heroku is a popular cloud platform that can be integrated easily into Jenkins pipelines. To deploy your application to Heroku:
- Install the Heroku CLI on the Jenkins agent:
sudo snap install --classic heroku
-
Add Heroku Deployment Step to the
Jenkinsfile: Modify theDeploystage in yourJenkinsfileas follows:
stage('Deploy') {
steps {
withCredentials([string(credentialsId: 'heroku-api-key', variable: 'HEROKU_API_KEY')]) {
sh 'heroku git:remote -a your-heroku-app'
sh 'git push heroku main'
}
}
}
- This example assumes you’ve stored your Heroku API key in Jenkins credentials (under the ID
heroku-api-key). - This command pushes the latest code to Heroku, which will automatically trigger a deployment to the Heroku cloud.
By adding deployment steps to your pipeline, Jenkins can fully automate the process from code commit to production deployment.
Best Practices for CI/CD Pipelines
While Jenkins is flexible and powerful, it’s essential to follow best practices to ensure your pipelines are efficient, maintainable, and reliable.
Automate Everything
Aim to automate every step of your development workflow, from building the application, running tests, deploying to staging, and deploying to production. The more you automate, the fewer manual interventions are required, and the faster and more reliable your pipeline becomes.
Fail Fast, Fail Often
Set up your pipeline to detect failures early in the process. For example, run unit tests as early as possible so that issues can be identified before they affect other stages. If the pipeline is going to fail, it should fail early so developers can fix problems quickly.
Version Control Best Practices
Using version control best practices is critical for CI/CD success. Create a branch for each feature, bug fix, or improvement, and merge it into the main branch only after it passes all tests and checks. This ensures that the main branch is always deployable.
Keep Pipelines Simple
Start with a simple pipeline and add complexity as your project evolves. Pipelines with too many stages or complex dependencies can become difficult to maintain. Break large pipelines into smaller jobs or microservices to keep them manageable.
Conclusion
This guide provided a comprehensive introduction to setting up your first CI/CD pipeline using Jenkins. From installing Jenkins, setting up version control, writing a Jenkinsfile, and deploying your application, we’ve covered the essential steps to get you started with automation. Jenkins’ flexibility, paired with its extensive plugin ecosystem, makes it a go-to solution for building, testing, and deploying applications.
As you become more comfortable with Jenkins, you can explore more advanced features like parallel builds, distributed builds with Jenkins agents, and integrating other DevOps tools like Docker, Kubernetes, and monitoring systems.
By following the steps in this guide, you’re well on your way to automating your software delivery process and adopting DevOps best practices that lead to faster, higher-quality software releases. Happy automating!
👤 Author
Join Our Telegram Community || Follow me on GitHub for more DevOps content!
Top comments (28)
I take my time to read every single post from you, because it gives me more understanding of DevOps as a beginner, but as said earlier, it would be better if you could make it more practical,with codes and Screenshot. It could even be a YouTube video. If you are recording too, please the time should not be stretched. Say minimum 1hour, maximum 2 hours. My humble suggestion and request
Thanks @innocentia_azal , please read the article now made a few changes
Muchas gracias por la información, estoy iniciando en esto y quiero saber mas, por lo que si alguien tiene mas información al respecto con ejemplos le agradecería mucho que comparta ese conocimiento.
-
Thank you very much for the information, I am starting this and want to know more, so if anyone has more information about it with examples would be very grateful to share that knowledge.
Thanks 👍@mcubico
It was great. another tool for CI/CD is Azure DevOps which is very famous and handy to use
Yess 👍@dot_nettips_a4e90828daa4
@innocentia_azal, @red_ Here are the key changes and improvements made from the old article to the new, more detailed version:
Expanded Explanation of CI/CD:
Step-by-Step Guide for Creating a CI/CD Pipeline:
Jenkinsfile.Jenkinsfileexample, explaining each section (checkout, build, test, deploy).Added Jenkins Pipeline Script:
Jenkinsfileexample has been added, showing how to automate the build, test, and deployment stages for a Node.js application.Heroku Deployment Integration:
Jenkinsfilewith Heroku deployment commands.Best Practices for CI/CD:
Clearer Pipeline Stages Breakdown:
Conclusion Section Expanded:
These changes make the article much more detailed and actionable for beginners by offering practical, hands-on steps while explaining the theory behind CI/CD.
Where is the "Beginner guide"? The steps do not demonstrate anything actionable. This is just abstract ideas and not a guide or how-to.
Thank you for your honest feedback! @red_ 🙏 I appreciate you taking the time to read the article and share your thoughts.
You're absolutely right that actionable steps are key in a beginner's guide, and I realize I could have included more specific, step-by-step instructions to make it clearer and more practical. I’ll update the article soon with detailed examples, commands, and screenshots to help guide beginners through setting up their first CI/CD pipeline.
In the meantime, if you have any particular areas you’d like me to focus on or if you need help getting started, feel free to let me know! I’d love to make the guide as helpful as possible.
Thanks again for your input—feedback like yours helps me improve future content!
Just create an article that does what it says what it will do : "we will take you through a step-by-step approach to understanding CI/CD concepts and setting up your first pipeline from scratch."
I cannot setup a pipeline from scratch with your article.
Thanks @red_ , please read the article now made a few changes
I am extremely excited for this! Thank you!
Cool article!
could be better if it not just all text (little visuals here and there).
Yupp sure 😀 will do it in next article @abdulmuminyqn
Hi keshav Sandhu,
Top, very nice and helpful !
Thanks for sharing.
Thanks for your feedback ☺️ @jangelodev
I just opened the updated article, I have not started practicing but I am moved with the update. Great explanations and details. I will submit my proof of practice after I am done. Thank you Senior
Thanks😀 @innocentia_azal
Wooo! What amazing enlightenment
Kindly thanks
Thanks @kin7777
A comprehensive and introductory guide to the concept of CI/CD process. Hopefully there would be a how to section to setup a CI/CD pipeline with one of your recommended tools as an example.
Thanks @kc900201 , please read the article now made a few changes