QA - no excuses!

I made a very interesting journey into the world of Continuous Integration and QA. The result is a project based on Docker, Jenkins and some really interesting tools I've tried to put together.

Not only do I ask you to download and try it, but above all to recommend new tools to integrate to get the best pipeline ever!

This project is a starter kit for setting up a Jenkins docker installation for CI and QA.

I developed on Windows, with Docker Desktop with WSL 2 engine and VS Code, but with minor adjustments (before all slash notation for command line commands) it should run on Linux too.

Tools integrated

Tool	Usage
Docker	Jenkins is run inside Docker and pipelines run inside docker agents
Jenkins	The automation server
Jenkins Configuration as Code	Jenkins plugin for run an image pre-provisioned / pre-configured
Jenkins plugins	Many Jenkins plugins
SonarQube	Static code analyser
PostgreSQL	Database for storing Sonarqube results
xUnit.Net	Testing framework - used for Unit test, UI tests, Coverage
Selenium	Web browser automation for UI tests
k6	Load testing tool
Grafana	Visualization tool for analyzing load test results
influxdb	Database for storing k6 results used by k6 and Grafana

docker compose up

Project ships with some docker compose files for bringing up main parts of the system:

• /docker-compose.yml
  Brings up the core infrastructure:

  • jenkins
  • local docker registry (insecure version)

• /sonarqube/docker-compose.yml
  Brings up sonarqube infrastructure

  • postregs database
  • sonarqube server

• /selenium/docker-compose.yml
  Brings up selenium infrastructure

  • selenium-hub
  • chrome node
  • firefox node
  • edge node

• /k6/docker-compose.yml
  Brings up Grafana infrastructure

  • influxdb:1.8
  • Grafana with pre-provisioned dashboards

If you want the full system up&running with just one command you can issue (from the root of the repo):

docker compose -f .\docker-compose.yaml -f .\sonarqube\docker-compose.yaml -f .\k6\docker-compose.yaml up

You should see all containers coming up (following picture is taken from VS Code Docker extension):

Note: selenium containers are not run with this command because the grid will be automatically run on demand by the Jenkins before running UI tests.

Note: before running Sonarqube, be sure to meet Docker Host Requirements
Usually you should issue these command:

  sysctl -w vm.max_map_count=524288
  sysctl -w fs.file-max=131072`
  ulimit -n 131072
  ulimit -u 8192

If everything went well, you now have many systems ready:

URL	System
http://localhost:8181	Jenkins
http://localhost:9000	Sonarqube server - User: admin / Pasw: admin
http://localhost:4444	Selenium Grid (only available while UI tests runs)
http://localhost:3000	Grafana

⚠️ Security Notes ⚠️
This stack is not production ready:

• For running docker agents from Jenkins, I exposed docker daemon on tcp://localhost:2375 without TLS locally
• I relaxed some Jenkins CSP to correctly visualize k6 HTML reports. If you do not need this feature, feel free to remove hudson.model.DirectoryBrowserSupport.CSP directive inside /jenkins/Dockerfile file.
• Docker registry is just meant for hosting local built images ( used as Jenkins Pipeline Docker agents) so it runs insecure (without certificates)

Jenkins

The first piece of the puzzle is Jenkins, the automation tool.
A custom docker image is built starting from the official one for minor adjustments:

• disable wizard setup
• setting up Jenkins Configuration as Code plugin directory
• setting up pre-provisioned jobs
• modifying Content Security Policy for supporting k6 HTML reports
• adding docker and docker-compose to the image. This is useful when using the default agent, for running docker commands like bringing up a selenium grid
• pre-provisioning the image with a bunch of plugins

Thanks to the JCasC and Seed Job plugins when we start Jenkins we will find all the job folder containing test jobs alredy in place:

Note: all the Jenkins configuration (JCasC and Jobs) is stored into separate files into /jenkins/casc_configs folder. You can add files here to automatically create new jobs or modify Jenkins configuration

Note: remember to approve scripts before runnig them otherwise they'll fail.
You can also edit and re-save job configuration for approving jobs.

Test job folder "docker":

Here you will find some tests for basic usage.

• testing the default agent
• testing a docker agent with an image from docker hub
• locally building a test image and pushing it lo the local Docker registry
• testing a docker agent with a custom local image pulled by the local registry (the one previously built)

All these pieces are used in following pipelines, so it seems a good idea to test the basic blocks

Test job folder "sonarqube":

In this folder you'll find jobs for executing static code analysis.

For this purpose I used Sonarqube. Sonarqube has a client/server architecture:

• server - started with the docker compose command previously described
• client - the scanner - analyses the project and sends data to the server

In Jenkins, I'm running dotnet core static analysis in two falvours:

• with a netcore scanner built on the fly
• with a netcore scanner pre-built, pulled from the local registry. This is meant to speed up this phase of the pipeline, using an agent with java and the netcore scanner already installed

I chose the very interesting pitstop netcore project to be analyzed.

Before running Sonarqube jobs, you need to create a Sonarqube token for the scanner to login into the server.

• In Sonarqube - create a token for admin user
  • http://localhost:9000, user: admin, password: admin
  • Go to: Administration -> Security -> Users -> "Administrator" Tokens
• In jenkins - config Sonarqube credential.
  • JCasC has already provisioned a placeholder Sonarqube token, just update its password with the token just created.
  • Go to Manage Jenkins -> Manage Credentials -> sonarqube-token-admin -> Update -> Change Password

These are the jobs in the Sonarqube folder:

• 01_build_netcore_sonarscanner_image_and_push_to_local_registry: Builds a netcore image with java & sonarscanner already installed and puh it to local registry
• 02_sonarscanner-in-docker-agent: Example of integrating jenkins - netcore - sonarqube After running this job login to Sonarqube http://localhost:9000 and you'll see:
  • Static code analysis
  • Code coverage
  • Test results
• 03_sonarscanner-in-docker-agent-netcore-scanner: Like the job before but faster. This job uses a docker agent pre-configured for scanning net core projects.

In Sonarqube, the result should looks like this:

Test job folder "unit tests":

Here you will find a job example for running netcore unit tests with xUnit.

We again use the pitstop netcore project which ships with xUnit unit tests.

This job collects tests results and publishes results using xUnit Jenkins plugin.

This job also collects cove coverage and publishes results using cobertura Jenkins plugin

Results should look like this after running a couple of builds:

Test job folder "selenium":

The job runs xunit + selenium UI tests against a basic grid (3 browsers)

Note: selenium grid is automatically run on demand by the jenkins job and discarded when the pipeline ends.

You can follow tests progress pointing your browser to http://localhost:4444

Test job folder "k6":

k6 is a wonderful tool for performance and load testing.

In this job folder we try two integrations:

• a pipeline running k6 load test and publishing HTML report directly in Jenkins.
  After job runs, you should see a new report published like this:
  

• a pipeline running k6 load test and publishing k6 metrics to Grafana
  
  If you browse to the dashboard from Grafana ("Dashboards" -> "Browse" -> "k6 Load Testing Results") after you run the load test, you will see the dashboard populated like this:

HTML Results are obtained converting k6 output to HTML.
This is done using the k6-reporter project.
The HTML generated is not compliant with Jenkins Content Security Policy so I modified it for allowing correct visualization.
If you do not need this feature, feel free to remove hudson.model.DirectoryBrowserSupport.CSP directive inside /jenkins/Dockerfile file.

For visualizing k6 results into Grafana I chose influxdb+grafana integration.
Grafana is pre-provisioned with:

• a datasource pointing to influxdb (version 1.8 because version 2.0 is not still supported by k6)
• a k6 dashboard

Due to this issue it is not still possible to provision dashboard with variables in grafana.
So I modified the k6 dashbord replacing the variable "${DS_K6}" with the provisioned datasource UID: "influxdb-k6-uid".

Conclusions

I always take great inspiration from open-source projects.
This time I hope to be able to reciprocate.

This project is still a "work-in-progress",
please let me know which tools can be integrated to release more and more quality software!

Thank you for reading.