CI/CD #01. Jenkins: manually clone a Python GitHub repo and run Pytest.

Manual, in the context of this post, means we have to click something on the Jenkins UI to get a build going.

I've installed Jenkins 2.388 LTS on my Ubuntu 22.10. The installation process is pretty straight forward, I'm including the installation description in the last part of this post.

Once successfully installed and running, we can access Jenkins front-end at http://<machine-ip-address><machine-name>:8080. In my case, from Windows 10, http://hp-pavilion-15:8080/, or on Ubuntu 22.10 where Jenkins is installed, http://localhost:8080.

As usual, I had failed several times before getting my first project going. I am documenting what I've learned in this post, so it is by no means a tutorial. I'm planning to write more on it as I go along, so that is why CI/CD #01 is in the title.

I'm going back to this https://github.com/behai-nguyen/app-demo.git repo for this post. We'll set up a Jenkins project to carry out the four (4) steps outlined in the introduction. Let's get to it.

❶ Log into Jenkins using the admin user name and password created as part of the installation. Click on Dashboard located on top left hand side corner; then click on + New Item underneath.

❷ On the next page:

• Enter an item name: app_demo -- this name matches the directory name of the project. (In hindsight, the name of the repo should've been app_demo instead of app-demo!)
• Then select the first option Freestyle project.
• Finally, click on the OK button to move to the Configure page.

❸ On the Configure page:

• Under General, enter something meaningful for Description, e.g.: “Try Jenkins on app-demo repo.”
• Under Source Code Management:
  • Select Git. Then,
  • For Repository URL, enter https://github.com/behai-nguyen/app-demo.git. Since this is a public repo, anybody can clone it, we don't need any credential to access it.
  • For Branch Specifier (blank for 'any'), enter */main -- since we are interested in the main branch of this repo.
• Scrolling down, under Build Steps, drop down Add build step, then select Execute shell. Enter the following content:

  PYENV_HOME=$WORKSPACE/venv
  
  # Delete previously built virtualenv
  if [ -d $PYENV_HOME ]; then
      rm -rf $PYENV_HOME
  fi
  
  # Create virtualenv and install necessary packages
  virtualenv $PYENV_HOME
  . $PYENV_HOME/bin/activate
  $PYENV_HOME/bin/pip install -e .
  $PYENV_HOME/bin/pytest
  

  This script starts off by deleting the existing virtual environment; create a new one; activate it; then editable install all required packages; finally, run Pytest.

  Note, because of how the script works, it can use a lot of data depending on how many packages are in the project.

• Finally, click on the Save button to move to the project page. The breadcrumb on the top left hand corner should now show Dashboard > app_demo >.

❹ Underneath Dashboard, the fourth (4th) item is ▷ Build Now. Click on ▷ Build Now to build!

With a bit of luck 😂, it should “build” successful, the screen should look like:

Underneath Build History, there is a green tick preceding #1, which indicates that this build has been successful. In case of a failure, it's a red x. In either case, clicking on #1, will go to the build detail screen, then on the left hand side, clicking on Console Output will show the full log of the build -- which is very informatively rich. On the failure ones, I have been able to use the information to get rid of the problems.

❺ Let's look at what happens on disk. Jenkins' work directory is
/var/lib/jenkins/workspace/:

app_demo can be seen on top of the list, it has been created by Jenkins during the build. It is a ready to use Python development environment. Let's go to it, and activate the virtual environment:

$ cd /var/lib/jenkins/workspace/app_demo/
$ source venv/bin/activate

The virtual environment was activated successfully:

Let's run Pytest. All tests should just pass:

$ venv/bin/pytest

All tests passed as should be the case. Deactivate the virtual environment with:

$ deactivate

Tutorial References

1. Jenkins and Python.
2. YouTube: How Do I Run a Python Script From Jenkins Pipeline?

Jenkins Installation

I tried installing Jenkins on my Ubuntu 22.10 three (3) times, all went smoothly. But all failed to start. The last installation instruction I have used is from this link https://community.jenkins.io/t/ubuntu-20-04-initial-jenkins-startup-failure/1419 -- the answer by Mr. Mark Waite -- Jenkins Governance Board.

Basically, run these commands one after the other:

$ sudo apt-get install openjdk-11-jdk-headless
$ curl -fsSL https://pkg.jenkins.io/debian/jenkins.io.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null
$ echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
$ sudo apt-get update
$ sudo apt-get install jenkins

But it still failed to start. I had forgotten that I have set port 8080 for Apache2. I free up the port 8080 and start Jenkins with:

$ systemctl start jenkins.service

And it starts with no problem. I am guessing the previous two installations were also successful. But it does not matter now.

After successfully installed, we need to do some initial configurations. I just followed the instructions in this DigitalOcean article How To Install Jenkins on Ubuntu 22.04.

We can access Jenkins front-end at http://<machine-ip-address><machine-name>:8080. In my case, from Windows 10, http://hp-pavilion-15:8080/, or on Ubuntu 22.10 where Jenkins is installed, http://localhost:8080.

✿✿✿

I like Jenkins thus far, it makes sense. I have worked in an environment where the build server and the unit test server are two VMware machines. The build process is written using Windows Powershell script, it also gets source codes from a source code management software installed in-house. Jenkins offers same capability, but the process seems much simpler. I hope you find the information useful. Thank you for reading and stay safe as always.