I recently struggled a lot to get a self-hosted Jenkins instance and GitHub to work together, at work.
After having finally made it, I thought to myself that maybe, someone else might have the same problems that I faced.
Hope you enjoy and take something useful for yourself.
A Quick Note
In tech, there are still a lot of words and terms that have a somehow negative attitude when used together.
With Jenkins, for example, the terms 'master' and 'slave', together, usually describe a coordinating instance that does management work and worker nodes that the coordinating instance can offload work to.
Due to to the recent events and due to my personal feelings, that those old-school terms (when used together) do not reflect an inclusive and welcoming world, I will use the following words to replace them:
- master -> coordinator
- slave -> worker
Context and Introduction
Some context always helps. Not everyone has the same requirements or the same ability to do as they like.
Doing something for yourself has other requirements than at your job, where you might have security policies that prevent you from doing certain things.
Even companies can differ greatly in those regards. A startup will most likely be able to move faster than an established enterprise organization, where everything has to be requested by ticket, mail, personally, sacrifice, etc.
Please keep this in mind while reading. I hope the context I give you about the company I work for, helps you understand why some things were done as they are.
What We Do
We are still mainly a Java shop (there's more and more Node we use, e.g. and also other languages like Go or Rust), and we do enterprise stuff.
We mostly deploy micro services these days, which are (again) mostly Java-based HTTP-REST services, that communicate with one another and the outside world.
Getting new technology into banks and the financial industry is actually quite difficult sometimes. You have to do a lot of advocating and evangelism to get a bank's IT to agree on deploying something they don't know, yet.
This is why we don't use too much cloud technology, and e.g. still run with Jenkins.
Our Usual Setup
Our usual setup consists of a lot of on-premise hardware, mainly VMs, that are spread across a multitude of machines.
What we use:
- Bitbucket (self-hosted) for our source code, internal network
- Jenkins coordinators, with workers, internal network
- A sonatype Nexus Repository, internal network
- VMs to run our software (some containerized, some not), internal network We sometimes leverage AWS for cloud workflows. Whenever we're mobile (laptop, smartphone), we leverage a VPN to connect to our internal systems.
Our Use Case
We recently started to slowly migrate more of our infrastructure to the cloud and free up internal resources for other work, starting with our source code management.
Our new setup looks like this:
- GitHub for our source code (private repositories within an organzation)
- An IP-range protected instance of a sonatype Nexus Repository to host our deployed packages (Maven, npm, etc.) within our own DMZ
- An IP-range protected Jenkins instance with worker nodes within our own DMZ
- IP-range protected VMs to run our software (for testing purposes) within our own DMZ
What is a DMZ?
"[...]a physical or logical subnetwork that contains and exposes an organization's external-facing services to an untrusted, usually larger, network such as the Internet. The purpose of a DMZ is to add an additional layer of security to an organization's local area network (LAN): an external network node can access only what is exposed in the DMZ, while the rest of the organization's network is firewalled. The DMZ functions as a small, isolated network positioned between the Internet and the private network and, if its design is effective, allows the organization extra time to detect and address breaches before they would further penetrate into the internal networks."
Taken from this Wikipedia article
Our Goal
First of all, we wanted Jenkins to build all of our Pull Requests and GitHub to have checks for those builds enabled.
Whenever a build fails, no one should be able to merge, thus making a successful build a basic requirement, next to the approval of at least one team member.
Setting Up Jenkins (Plugins)
Prerequisites
For this article, let's assume that you already have a working instance of Jenkins installed. Getting a coordinator and some workers up is worth an article for itself.
At the time of writing this, the version of Jenkins used is 2.241
Very Important
Your Jenkins coordinator must be accessible from the internet, as GitHub will use a webhook to trigger the build.
If you are doing this from home, or on some remote machine you own, you will have to expose Jenkins to the internet.
After some googling, I found this article here on Medium which might help you.
We carefully exposed our Jenkins to the internet by:
- Making it authenticated
- Using a reverse proxy to only make the Jenkins webhook accessible without an IP check (we had to do this because GitHub has too many ips to whitelist them all). The web interface itself is unreachable for everyone that doesn't try to access it with a white-listed ip address.
The Plugins You Need
First of all, you will need some plugins to get Jenkins to work with GitHub at all.
Those are (with the versions used):
| Plugin | Version |
|---|---|
| GitHub Pull Request Builder | 1.42.1 |
| Git | 4.2.2 |
| GitHub Plugin | 1.30.0 |
| GitHub API | 1.114.1 |
(Actually, the Plugin defines all its dependencies pretty well, but these are the ones you will definitely need.)
GitHub Pull Request Builder
This is the plugin that handles everything related to your pull request.
For me, all traditional approaches didn't work and this was the only plugin that actually did what I wanted it to do.
Keep in mind: There might be other ways to achieve this!
This plugin will expose a webhook which GitHub can later use to send meta data to.
The webhook is exposed at: <yourJenkins/>/ghprbhook/
A request arriving at the hook is then used to identify a Jenkins build which is actually run then.
The plugin will (when configured):
- add a comment to your PR and ask for a review
- add a merge check which you can add to the branch protection requirements for a review
- start the build and send the result back to GitHub using its API
Please Note
As of now, the plugin is looking for someone to take over maintenance and has a security issue open.
The issue was none for me, as the machine the Jenkins runs on is safe on-premise and only reachable by HTTPS due to firewall settings. Internal access is secured, as well. Not everyone has access to the machine itself.
But this could however be an issue for you:
"GitHub access tokens stored in in build.xml"
The uncertainty regarding the future of the plugin might also be a turn-off for you.
Adding Credentials For Authentication With GitHub
You will need credentials stored in Jenkins for two things:
- Pulling your source code from GitHub
- Using the GitHub API to make comments and push the result of the merge check
First you have to add some credentials to your Jenkins so that it can later authenticate requests to GitHub.
In the main menu of Jenkins, click on "Credentials":
Choose a scope. The global scope is okay. If you have another one you want to use for it, feel free to do so!
When you have chosen, click on the scope.
Click on "Add Credentials", now.
On the following page, choose the kind "Username with password" and fill out all the necessary information.
After this, click on the "OK" button and you are finished here.
You could also work with an API token here, but for the sake of simplicity, username and password is just more straight forward.
Creating a Jenkins Job
After your credentials are all set up, you need a build to actually do some work.
Click on "New Item" on the Jenkins main page, and choose a Freestyle project for the new job.
Setting The GitHub Project URL
Before doing anything more, enter your GitHub project URL.
Many posts or articles do not mention this enough, but the url is part of the routing mechanism, that decides what project to actually build when your webhook receives a message.
If this field is left blank, no build will ever run!
Adding The Repository
In order for your Jenkins to actually build your project, you have to add the git coordinates.
Go to your project and just copy the HTTPS clone link.
After that, choose the credentials you created earlier from the dropdown.
In your case, that red error message should, of course, not pop up.
If it does, there is something wrong with your credentials (check your password again!) or your Jenkins can't reach GitHub.
The goal is to build individual Pull Requests before they are merged into your main branch. As they originate from another branch as your main one, Jenkins somehow needs to get told which specific branch to build.
By adding a refspec and a branch specifier, including variables that are set by the plugin, the job will always build what is specified by the webhook trigger, that comes from GitHub itself.
The refspec used is: +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
Enabling GitHub Pull Request Builder And Setting It Up
You are nearly finished with setting up the Jenkins job now.
It's time to activate the plugin within your job and make all settings necessary for it to work properly.
Enable "GitHub Pull Request Builder" and tick the checkbox "Use github hooks for build triggering".
Those two are the basic settings you need to get everything working.
However, for security reasons, not everyone that creates a PR in your repository can automatically also trigger a build.
That's a great feature to prevent misuse, spam, ddos, etc.
As you can see, you have several options here to enable certain people to being able to trigger a build.
- Admin List
- Adding specific GitHub user names
- White List
- Also adding specific GitHub user names
- List of organizations
- By adding an organization, all members automatically become white listed
You could, however, just tick the option 'Build every pull request automatically without asking (Dangerous!).' and whitelist basically everybody, but I would advice against it.
It's better to add an organization or add people individually, and only yourself as the Admin.
The last things to do are setting up the triggers.
You can find them here, below all the other plugin settings.
Give your build some context. This is a string that will later be shown on your merge checks.
Click on "Add" and choose "Update commit status during build"
And lastly, click on "Add" again and choose "Build Status Messages".
Then add 3 messages for each possible build result.
Save your build, and you are done with Jenkins, for now.
Whenever Jenkins receives the correct payload at /ghprbhook/, your build will get triggered.
What you now have to do is setting up your GitHub project to actually use the webhook.
Setting Up Your GitHub Repository
Adding the Webhook
Navigate to your GitHub repository and choose "Settings" from the repository nav bar and choose "Webhooks" from the new menu appearing.
Add a new Webhook there by entering the full url to your Jenkins instance, including the path of the webhook, and choose application/json.
Finally, under "Which events would you like to trigger this webhook?" choose "Let me select individual events." and select:
- Commit comments
- Pull requests
Interestingly, if you have given the user you use for your Jenkins job admin rights (on GitHub, to the repo), the GitHub Pull Request Builder will add those hook settings itself.
Create A Test Pull Request
Make any commit to your project on any feature branch and just create a Pull Request for testing purposes.
You should now see your Jenkins instance starting the job you created. Let it run through and ensure that it runs successfully.
Adding The PR Check
When you now want to have those checks on your pull requests, there is one last thing left to do.
Navigate to your repository's "Settings" once again and choose "Branches" this time.
If you don't have a protection rule, yet, create a new one.
Target your main/master branch or whatever is your base branch.
The important setting you want to enable is "Require status checks to pass before merging".
Under that, you should see your "Jenkins" context, if the pipe has run at least once.
Check the box and you are finally finished.
After that, your next Pull Request should have checks enabled like this:
Congratulations, you've made it!
Last But Not Least
If you liked this article, let's consider becoming friends on Twitter
Top comments (1)
Hello Author/Team
I am not able to set up the same with mentioned steps. The build is not getting triggered on jenkins.
Please advise