Objective
This guide shows you how to configure an existing Jenkins server to communicate with a private GitHub repo using GitHub Deploy Keys.
My intent is to just be quick and to the point with minimal explanations.
Prerequisites
This guide assumes you have:
- A GitHub account with a private GitHub repo containing some form of source code or file(s).
- An AWS account containing an EC2 instance running with an installation of Jenkins. See my Jenkins - Setup a Build Environment in AWS guide if you don't have this.
- Basic knowledge of Jenkins.
- Basic knowledge of GitHub.
I use macOS, but this guide does cater to Linux and Windows as well.
Overview
Jenkins - Build a Private GitHub Repo
- Create a Simple Jenkins Job
- Generating an SSH Key on the Jenkins Server
- Add the SSH Key as a Jenkins Credential
- Add the SSH Key as a GitHub Deploy Key
- Add the Credential to the Jenkins Job
Jenkins - Build a Private GitHub Repo
Before beginning, make sure your Jenkins EC2 instance is running, log into your Jenkins dashboard, and your GitHub account.
Create a Simple Jenkins Job
- Open the Jenkins dashboard and select New Item from the menu on the left.
- Give the item a name (this can be anything you want but it's representing the job itself)
- Select Freestyle project from the list of options and then click OK
- Once the job configuration view loads, locate the Source Code Management section and tick Git
- Add in the SSH Clone URL of your desired private GitHub repo
- You should see an error appear that reads something like: "Failed to connect to repository...". This is fine for now; it's because we haven't yet set up the SSH keys.
- Scroll down to the Build section, click Add build step and then select *Execute shell from the list.
- In the text area that appears, enter:
echo "Hello, world!" - Scroll to the bottom and click Save
- You'll now be brought to your job overview; click Build Now from the left menu to trigger a build
- This should immediately fail, but that's intended at this point. Click the red dot next to the build number to view the console output.
- Looking into the console output, you should see references relating to: "Permission denied" or "Could not read from remote repository" - this is what we are aiming to solve.
- Return to the Jenkins dashboard by clicking the Dashboard breadcrumb at the top and then continue onto the next section.
Generating an SSH Key on the Jenkins Server
- Open a terminal and SSH into your EC2 Jenkins server:
ssh -i /path/to/<your_key_pair.pem> ec2-user@<your_public_dns>(replacing the<angles>with your info) - Sudo to the jenkins user:
sudo su -s /bin/bash jenkins.
This guide assumes you are using Linux or macOS, but if you are using Windows, you can reference the AWS guide for
Connecting to your Linux instance from Windows using PuTTY
- Generate an SSH key:
ssh-keygen. - Accept the defaults from the prompts and leave the password blank.
- Leave the terminal open and continue onto the next section.
Add the SSH Key as a Jenkins Credential
- From the Jenkins dashboard select Manage Jenkins from the menu on the left.
- Under the Security section, select Manage Credentials.
- In the table labeled Stores scoped to Jenkins, click the Jenkins store.
- In the table labeled System, click the Global credentials domain.
- From the menu on the left, click Add Credentials.
- In the form that appears, apply the following:
- Kind: SSH Username with private key
- Scope: Global
- ID: (blank)
- Description: (blank)
- Username: gh-deploy
- Private Key: tick Enter Directly and then click Add
- Back in your Jenkins terminal session, copy the private key to your clipboard:
cat ~/.ssh/id_rsa - Paste this key into the form making sure not to include any additional preceding/trailing spaces
- Passphrase: (blank)
- Click OK
- We'll apply this credential to your new job in later steps.
- Keep your terminal session open and continue onto the next section.
Add the SSH Key as a GitHub Deploy Key
- Open GitHub and select the repo you specified in your job configuration
- From within the repo view, select the Settings tab
- From the menu on the left, select Deploy keys and then click the button on the right that says Add deploy key
- Give the deploy key a Title (makes sense to match it with the name given to the jenkins credential:
gh-deploy) - Back in your Jenkins terminal session, copy the public key to your clipboard:
cat ~/.ssh/id_rsa.pub - Paste this key into the form making sure not to include any additional preceding/trailing spaces
- Leave Allow write access unchecked
- Click Add Key
- The SSH key is now attached to this specific GitHub repo and is being used as a Deploy Key. For more info on Deploy Keys, checkout the GitHub documentation
Add the Credential to the Jenkins Job
- Return to the Jenkins dashboard and select your new job that we created in the first section.
- From the job overview; click Configure from the left menu
- Scroll to the Source Code Management section and select your new credential from the Credentials select list
- You should now see the errors resolve
- Scroll to the bottom and click Save
- You'll now be brought to your job overview; click Build Now from the left menu to trigger a build
- The job should now succeed
- Click the blue dot next to the build number to view the console output.
- Looking into the console output, you should see the
Hello, world!from the shell command we added and a status ofFinished: SUCCESS- the job is now pulling from the private GitHub Repo!
Next Steps
Per usual, there are multiple ways that this connection can be configured. I've just shown one here. In a team setting, you might encounter a setup where Jenkins has its own GitHub account with SSH keys configured much like you'd do for yourself. That user can then be added as a collaborator on any given repo and will have access to it. Check out this guide if you're interested in that.
But from here, you can work on configuring your particular build for your project. In this example, I showed a Freestyle project, but I really recommend using Pipeline projects where the build specifics live with the source code in a Jenkinsfile. Check out Using a Jenkinsfile from the official Jenkins documentation to get started.
I'm continuing to work on new content showcasing my own Jenkins pipeline, so I'll make sure to add a reference here as it's completed.
Top comments (3)
One note I would add is to make sure your put the SSH url from gitHub ,as the url in the project build. Not the HTTPS url, or else you will still get the same error.
Very true, thank you so much
This was very helpful, thank you.