DEV Community

Webhook Relay profile image Karolis
Karolis for Webhook Relay

Posted on • Edited on

Automated Jenkins builds on GitHub pull request

#beginners #productivity #git #jenkins

workflow

In this short guide we will configure Jenkins to start builds on GitHub pull requests. Subsequent builds will be triggered on any new commits and GitHub pull request status will show whether build succeeded or failed. This setup will work without configuring router, firewall or having a public IP. It will also work behind a corporate firewall.

Getting a VM

In my case, I just grabbed a Vagrant box from https://app.vagrantup.com/ubuntu/boxes/xenial64:



Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
  config.vm.network "private_network", type: "dhcp"
end
Enter fullscreen mode Exit fullscreen mode

Then:



vagrant up
vagrant ssh
Enter fullscreen mode Exit fullscreen mode

And we have our VM. You can get the IP address by typing ifconfig in the terminal.

Installing Jenkins

I mostly followed this guide https://linuxize.com/post/how-to-install-jenkins-on-ubuntu-18-04/. The only caveat I encountered this time with Jenkins, was the jdk version mismatch.

Connecting to Jenkins

First, get your Jenkins token:



cat /var/lib/jenkins/secrets/initialAdminPassword
ce04d19270934633a7badcac3cfac316
Enter fullscreen mode Exit fullscreen mode

Then, either open your node firewall (or check Vagrant port forwarding) or do the easy thing: connect with the relay:

Get CLI

To get the CLI, check instructions here. On a 64-bit Linux OS it's:



curl -sSL https://storage.googleapis.com/webhookrelay/downloads/relay-linux-amd64 > relay \
   && chmod +wx relay && sudo mv relay /usr/local/bin
Enter fullscreen mode Exit fullscreen mode

Login

Go to https://my.webhookrelay.com/tokens, click CREATE TOKEN and copy/paste login command into the terminal, it should be something like:



relay login -k <your key> -s <your secret>
Enter fullscreen mode Exit fullscreen mode

Start tunnel 



$ relay connect :8080
Connecting: 
http://lsw7eq49jlhsuldvhpiyku.webrelay.io <----> http://127.0.0.1:8080
https://lsw7eq49jlhsuldvhpiyku.webrelay.io <----> http://127.0.0.1:8080
Enter fullscreen mode Exit fullscreen mode

Now, open the browser. You should see a similar screen:

Jenkins login view

Follow the steps to configure your Jenkins initial admin user.

Installing the plugin

Plugin installation instructions can be found here.

Once you have it, add GitHub credentials - your username and GitHub token.

Forwarding configuration

Go to your bucket configuration and create a bucket called github-webhooks. Configure it to forward all webhooks to http://localhost:8080/. This will ensure that webhooks will reach Jenkins server.

forwarding config

Once you have the relay CLI on the machine where you run Jenkins, type:



relay forward --bucket github-webhooks
Enter fullscreen mode Exit fullscreen mode

This will start forwarding webhooks. There are alternative options to run the forwarding daemon, such as Docker container.

If you are creating webhook configuration manually in GitHub, use http://localhost:8080/ghprbhook destination as it's the endpoint on which the plugin is listening for webhooks. In default case, Jenkins will automatically transform https://my.webhookrelay.com/v1/webhooks/21e13033-bd3d-47a2-bf15-6fd42d4b40a3 endpoints to https://my.webhookrelay.com/v1/webhooks/21e13033-bd3d-47a2-bf15-6fd42d4b40a3/ghprbhook and
Webhook Relay will preserve the extra /ghprbhook path.

Now, configure GitHub Pull Request Builder:

pr builder config

Creating a job

To create a new job, first select "Freestyle project", then:

Add the project's GitHub URL to the "GitHub project" field (the one you can enter into browser. eg: "https://github.com/rusenask/jenkins-test/"):

jenkins set github project

Configure Source Code Management section:

  • Select Git SCM.
  • Add your GitHub "Repository URL".
  • Under Advanced, set "refspec" to +refs/pull/*:refs/remotes/origin/pr/*
  • In "Branch Specifier", enter ${ghprbActualCommit}

jenkins set source code management

Configure Build Triggers with a list of admins and tick the use github hooks for build triggering:

jenkins build triggers

Add your Build step configuration. This can be anything you want, usually people tend to use either a bash script, Makefile target or something specific to your programming language such as go build:

jenkins build triggers

Opening a PR

Now, whenever you open a new pull request in GitHub, you should see a build being triggered:

pr triggering build

You can view build status in your Jenkins instance as well. This build indicator in GitHub will either turn red or green based on the build status.

Conclusion

As we can see, there are several required steps to make sure your PRs get automatically built and tested when using Jenkins. Those can be split into two groups:

  1. System configuration that involves:
  2. Whenever you create a new project in Jenkins, setting up few settings. The first time I did this it took me some time to go through the configuration options, but second and third time didn't take more than 1 minute :)

I hope you will find this guide useful!

P.S. Bonus troubleshooting below:

When we are talking about Jenkins, there are many ways for things to go wrong. Multiple plugin versions, corporate proxies and different operating systems contribute to all of this. I have compiled a short list of items for you to check if you encounter problems.

Ensuring webhook configuration

Make sure there's an automatically created GitHub repository webhook configuration:

GitHub repo webhook configuration

Ensuring Webhook Relay agent can connect

Normally, connected agent should look like this:



relay forward --bucket github-webhooks
Filtering on bucket: github-webhooks
Starting webhook relay agent... 
1.55523552627511e+09    info    using standard transport...
1.5552355264042027e+09  info    webhook relay ready...  {"host": "my.webhookrelay.com:8080"}
Enter fullscreen mode Exit fullscreen mode

If you are behind a corporate proxy, try adding --ws flag to change default transport type from GRPC to WebSocket:



relay forward --ws --bucket github-webhooks
Filtering on bucket: github-webhooks
Starting webhook relay agent... 
1.5552387754607065e+09  info    using websocket based transport...
1.5552387754607568e+09  info    authenticating to 'wss://my.webhookrelay.com:443/v1/socket'...
1.5552387754608495e+09  info    websocket reader process started...
1.555238775470567e+09   info    subscribing to buckets: [github-webhooks
Enter fullscreen mode Exit fullscreen mode

Check logs

There will be several sources of logs you can check out:

  • Jenkins logs under /log/all:

jenkins logs

  • Webhook Relay forwarding logs:

forwarding logs

  • CLI logs:


relay forward --bucket github-webhooks
Filtering on bucket: github-webhooks
Starting webhook relay agent... 
1.55523552627511e+09    info    using standard transport...
1.5552355264042027e+09  info    webhook relay ready...  {"host": "my.webhookrelay.com:8080"}
1.555236773074343e+09   info    webhook request relayed {"destination": "http://localhost:8080/ghprbhook/", "method": "POST", "bucket": "github-webhooks", "status": 200, "retries": 0}
1.5552368184301443e+09  info    webhook request relayed {"destination": "http://localhost:8080/ghprbhook/", "method": "POST", "bucket": "github-webhooks", "status": 200, "retries": 0}
1.5552368215106862e+09  info    webhook request relayed {"destination": "http://localhost:8080/ghprbhook/", "method": "POST", "bucket": "github-webhooks", "status": 200, "retries": 0}
1.555236829308788e+09   info    webhook request relayed {"destination": "http://localhost:8080/ghprbhook/", "method": "POST", "bucket": "github-webhooks", "status": 200, "retries": 0}
1.5552368314174337e+09  info    webhook request relayed {"destination": "http://localhost:8080/ghprbhook/", "method": "POST", "bucket": "github-webhooks", "status": 200, "retries": 0}
1.555236920064973e+09   info    webhook request relayed {"destination": "http://localhost:8080/ghprbhook/", "method": "POST", "bucket": "github-webhooks", "status": 200, "retries": 0}
1.5552369202506151e+09  info    webhook request relayed {"destination": "http://localhost:8080/ghprbhook/", "method": "POST", "bucket": "github-webhooks", "status": 200, "retries": 0}
Enter fullscreen mode Exit fullscreen mode

Originally published on Webhook Relay blog: https://webhookrelay.com/blog/2019/04/17/automated-github-pull-request-builds-on-jenkins/

Top comments (5)

Collapse
 
jeuniiii profile image
Junaid Subhani •

Thank you for the article Karolis. Is a great read. I do have one question though. How to make this all work with a Pipeline job ?

With a regular Freestyle Project, it works like a charm. But the checks that I want to run are a part of a pipeline in a Jenkinsfile. So Ive configured HitHub project and build triggers as you mentioned. In the Pipeline section, i put the Refspec and Branch specifier as you mentioned. (See attached image)

Now when I issue a PR, the job Jenkins Job does get triggered but it errors out stating

GitHub pull request #16 of commit bb5673be3bb220c74edec123e4fa58b43f5ccd97, no merge conflicts.
Setting status of bb5673be3bb220c74edec123e4fa58b43f5ccd97 to PENDING with url <MY-JENKINS-URL>/job/pre-commit-test/4/ and message: 'Build started for merge commit.'
Setting status of bb5673be3bb220c74edec123e4fa58b43f5ccd97 to FAILURE with url https://<MY-JENKINS-URL>/job/pre-commit-test/4/ and message: 'Build finished. '
hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/${ghprbActualCommit}:refs/remotes/origin/${ghprbActualCommit} --prune" returned status code 128:
stdout: 
stderr: fatal: Couldn't find remote ref refs/heads/${ghprbActualCommit}

    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2099)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1791)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:79)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:472)
    at jenkins.plugins.git.GitSCMFileSystem$BuilderImpl.build(GitSCMFileSystem.java:358)
    at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:197)
    at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:173)
    at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:113)
    at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)
    at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:299)
    at hudson.model.ResourceController.execute(ResourceController.java:97)
    at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE

Any idea why this would happen ?

Collapse
 
krusenas profile image
Karolis •

did it fail to use the actual value or you have just changed ref yourself refs/heads/${ghprbActualCommit}? From the past it just seems like you need to find what it really likes to get through that webhook :) Webhook Relay will send whatever github is sending so I guess if you look for similar issues that people face when they have their Jenkins exposed to the public internet, it should be good enough. Haven't tried using pipelines myself :|

Collapse
 
carleskapy profile image
Carles Kapy •

Good post! So interesting!

Collapse
 
skashmeri profile image
Shamil Kashmeri •

Good write up tho I would suggest not using this plugin, check the link you posted:
You should probably migrate to GitHub Branch Source Plugin

Collapse
 
mprasu23 profile image
Mithiridi Prasanth •

The github pull request builder plugin has some security issues. Is there any other alternative way of doing .Can you please help

Read next

maxdavis profile image

The Inside Scoop: Helping My Friend Choose Driving Lessons in Beaconsfield

Jason Byrne -

sergi_web3 profile image

Getting Started with Web3 Development: A Beginner’s Guide to Skills and Tools

Sergi Mamedov -

surajkumar00 profile image

Verified Commits on GitHub from Ubuntu 22.04 LTS (GPG Keys) ✅

Suraj -

04anilr profile image

Main concept of react || React

Anil -