CI/CD #06. Jenkins: accessing private GitHub repos using SSH keys.

Using a generated public-private key pair, we set up a GitHub SSH key (public key), and Jenkins SSH username with private key credential to enable Jenkins projects to access private GitHub repositories.

The environment of this post is Jenkins 2.401.1 running on an Ubuntu 22.10 machine. It seems that Jenkins UI is slightly different between versions.

We can access GitHub private repositories using SSH (Secure Shell Protocol). Connecting to GitHub with SSH is the official GitHub document which provides instructions on how to do this.

In essence, we a need a public key for GitHub, and a private key to setting up a Jenkins credential with. Once we've done this, we can create Jenkins projects using the credential we've set up.

The issues we're going to address in this post are thus: ❶ generating the public-private key pair, ❷ configure GitHub SSH key using the generated public key, ❸ set up a new Jenkins credential using the generated private key, ❹ create a Jenkins project to access a private repo using the credential we've set up in ❸, and finally ❺ we're testing the new Jenkins project.

❶ Generating the public-private key pair.

We follow these two official documents:

1. Checking for existing SSH keys
2. Generating a new SSH key and adding it to the ssh-agent

I've done this under user jenkins. That is, I connect (via SSH) into the Ubuntu 22.10 machine with user jenkins:

ssh jenkins@192.168.0.17

There're no existing SSH keys: I did not generate one before. And

$ ls -al ~/.ssh

also confirms no existing SSH keys. Follow the official document mentioned above, we generate the keys with:

$ ssh-keygen -t ed25519 -C "[e-mail]"

Whereby [e-mail] is the email I use to log in to GitHub.

I just accepted the suggested defaults. For the passphrase, I use I live in Melbourne. -- including . (a fullstop). A successful key generation looks like the screenshot below:

⓵ The private key is stored in the file /var/lib/jenkins/.ssh/id_ed25519. And the key looks like:

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABBEN+iqTy
/WQQPkwwkhvZzUAAAAEAAAAAEAAAAzAAAAC3NzaC1lZDI1NTE5AAAAIMTGu+OQjJlMk9jN
qAd8a8KwKijgjIJJgQvfiFd2USbTAAAAoM3Cpg7lnvRBLZbPDzWm4QN9LV7VcnXOOUGJYE
1qzASxnK6iaOQr4dLj0zR8Vi6FopELk7qydHgot/DFDnsygXnxrcTamviv0Z6K5KgKTQJT
C1atxPZUpHnOtPqMQkRz6JrOtg7ReecjVKetNn8NYo+zAlStWR7FuM+g07o40ff6sORZwj
+in8c4yfd5sKkc4Ab2dY6S9igTStyZS9gr1Ds=
-----END OPENSSH PRIVATE KEY-----

⓶ The public key is stored in the file /var/lib/jenkins/.ssh/id_ed25519.pub. And the key looks like:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMTGu+OQjJlMk9jNqAd8a8KwKijgjIJJgQvfiFd2USbT [e-mail]

⓷ Note also the key fingerprint of SHA256:rcDT2TK2VGrmYaCgRYe7mwqlOgKKUR9JORq0D8Pw1PA [e-mail].

⓸ Pre-populate the SSH keys for each server. Please note, I documented this step, because I ran into the problem: stderr: No ED25519 host key is known for github.com and you have requested strict checking. You can certainly skip it and see what happens, if you encounter the same problem, then, you can certainly carry it out. Run the following command:

$ ssh-keyscan github.com >> ~/.ssh/known_hosts

The output looks like the below screenshot (I accidentally ran it a second time):

Without this SSH key re-population, Jenkins raises the following error when using the credential:

This less than a month old Ask Ubuntu post is about the exact same problem. And this post suggests the above command -- and it does work for me.

For Jenkins 2.401.1, Known hosts file verification is the default, as can be seen in the screenshot below:

❷ Configure GitHub SSH key using the generated public key.

Go to https://github.com/, click on profile photo, then Settings:

Then click on SSH and GPG keys link on the left hand side bar:

Click on New SSH key button. On the next screen, for Title, enter something meaningful such as ssh-private-repo; for Key, enter the public key we've generated above, see the screenshot below:

Click on Add SSH key, we will be asked for security confirmation. Once completed, the next screen shows our new key added:

Note the value of key fingerprint (discussed above) displayed underneath ssh-private-repo; and other self-explanatory info.

❸ Create a new Jenkins credential using the generated private key.

There is more than one way to configure a new credential. In my thinking, the approach presented in this section is a bit more logical. We'll mention another approach in a later section. Click on the Dashboard link, the Manage Jenkins link, then the Credentials link:

There is no prior credential configured. Click on the (global) link:

Click on the + Add Credentials button. On the next screen, fill in the information as per the following screenshots:

• ID: ssh-private-repo, which is the value of the Title for GitHub SSH key discussed above.
• Description: something meaningful. E.g. SSH credential to access private repos. This will become an entry in the credential selection list later on.
• Username: this is the email we use to log into GitHub with.
• Private Key: check Enter directly, then click on the Add button, copy and paste in the private key we've generated previously.
• Passphrase: this is the passphrase we've specified when generating the key, it's I live in Melbourne. -- including . (a fullstop).

Click on the Create button. Our new credential has been created:

❹ Create a Jenkins project to access a private repo.

The private repo I'm using for this Jenkins project is an old never-completed project of mine. It is in NodeJs and React.

The objective of the Jenkins project is to demonstrate that we can use our SSH public-private key pair to access a private repo. All this Jenkins project does is to clone the target private repo down to local disk. We will use the Freestyle project.

⓵ We need to get the SSH URL for the repo. Select the repo, then click on the <> Code button, then SSH tab, copy the SSH URL: [e-mail]:behai-nguyen/YouTubeManager.git:

⓶ Back to Jenkins, click on the Dashboard link, then + New Item. On the next page, under Enter an item name, enter youtube_mgr -- at runtime, Jenkins will create a directory with this name under Jenkins' work directory, i.e. /var/lib/jenkins/workspace/youtube_mgr.

And /var/lib/jenkins/workspace/youtube_mgr is the value of the WORKSPACE Jenkins environment variable.

⓷ Select Freestyle project, then click on the OK button to move to the Configuration page.

⓸ On the Configuration page, for Description, write something meaningful, e.g. Cloning / downloading a private repo using SSH.

Under the heading Source Code Management, check Git, then under Repositories, for Repository URL, enter the SSH URL above; i.e. [e-mail]:behai-nguyen/YouTubeManager.git. We'll get the error as seen in the screenshot below:

This is normal, it is expected -- we need to select an existing credential or create a new one and use it. Remember earlier on we mentioned that there is more than one way to configure a new credential? Here is the second method, underneath the Credentials drop-down box, click on the Add button, then click on Jenkins drop-down menu item, we'll be taken to a screen similar to the previous one, whereby we can create a new credential:

But since we've done this already, we can now use it. Click the Credentials drop-down box, we should see our entry SSH credential to access private repos in there, select it:

After a few seconds, the error should go away:

Our credential works!

Under Branches to build, for Branch Specifier (blank for 'any'), leave at /master -- as mentioned before, this is an old repo, before GitHub changed the main branch to /main.

Click on the Save button: we're taken to the youtube_mgr project page.

❺ Test youtube_mgr project.

⓵ Ensure there's no youtube_mgr/ directory under
/var/lib/jenkins/workspace/:

Click on the ▷ Build Now link on the left hand side, it should run successfully:

The target private repo should be cloned to the local disk. That is, sub-directory youtube_mgr/ should now exist under /var/lib/jenkins/workspace/:

We've successfully accessed our private repo using SSH.

⓶ In this post CI/CD #05. Jenkins: trigger a Pipeline via Git webhook, we look at automatically trigger a Jenkins Pipeline whenever we push something onto the target public GitHub repo. I've tested an exact similar set up on a private repo, using the same credential, i.e. ssh-private-repo / SSH credential to access private repos, and it also works: repo gets cloned, virtual environment gets created, all Pytest get run and email gets sent.

Other Jenkins Posts Which I've Written

1. CI/CD #01. Jenkins: manually clone a Python GitHub repo and run Pytest. -- I'm describing the steps required to manually get Jenkins to: ⓵ clone a Python project GitHub repository, ⓶ create a virtual environment, ⓷ run editable install, and finally, ⓸ run Pytest for all tests.
2. CI/CD #02. Jenkins: basic email using your Gmail account. -- We look at the most basic approach to send emails in Jenkins. The SMTP server we use is the Gmail SMTP server.
3. CI/CD #03. Jenkins: using Pipeline and proper Bash script to run Pytest. -- We write a proper and generic Bash script file to ⓵ create a virtual environment, ⓶ run editable install, and ⓷ run Pytest for all tests. Then we write a generic Jenkins Pipeline which will ⓵ clone a Python project GitHub repository, ⓶ call the Bash script file to do all the works.
4. CI/CD #04. Jenkins: trigger a Freestyle project via Git webhook. -- We're setting up a Jenkins “Freestyle project”, using “Execute shell” build step, to: ⓵ clone a Git repo, ⓶ create a virtual environment, ⓷ run editable install, and finally, ⓸ run Pytest. And this Jenkins job can be triggered remotely when we push some file(s) onto the target Git repo.
5. CI/CD #05. Jenkins: trigger a Pipeline via Git webhook. -- We’re setting up a Jenkins “Pipeline script from SCM”, which uses a generic Bash script file to ⓵ create a virtual environment, ⓶ run editable install, and ⓷ run Pytest for all tests. And this Jenkins job can be triggered remotely when we push some file(s) onto the target Git repo.

Thank you for reading... And I hope you find this post useful. Stay safe as always.

✿✿✿

Feature image sources:

• https://in.pinterest.com/pin/337277459600111737/
• https://www.omgubuntu.co.uk/2022/09/ubuntu-2210-kinetic-kudu-default-wallpaper
• https://branditechture.agency/brand-logos/download/jenkins/#google_vignette
• https://www.flaticon.com/free-icon/ssh_5261867?k=1687652052317
• https://pngimg.com/image/73422

Comments

[David Perdomo]

Hi! I wanted to thank you for this post. I have been experiencing issues with Jenkins and its credential problems. I managed to correct the issue by following your guide. Thank you very much! Greetings from Argentina!

[Be Hai Nguyen]

Thank you for your kind words David. I am very happy you have found helpful. I am learning it too. Best regards. ...behai.