DEV Community

Cover image for Adding PHPUnit Test Log and Coverage to GitLab CI/CD Pipeline
Muhammad Hassan
Muhammad Hassan

Posted on • Edited on

Adding PHPUnit Test Log and Coverage to GitLab CI/CD Pipeline

#cicd #php #testing #docker

In this tutorial we are going to setup a GitLab CI/CD job that can run your PHPUnit test suite and extract a testing report plus the overall coverage. First, we need to setup our server and install the required tools. DigitalOcean has many useful tutorials and we are going to use some of them for the initial setup.

Prerequisites

  • An Ubuntu 20.04 server setup by following this tutorial
  • Install Docker following this tutorial
  • Finally, installing Docker Compose by following this tutorial

Server Setup

Now that the server’s initial setup is complete, our next steps will be: installing GitLab runner, register a runner for your project and create a user for deployment.

Installing GitLab Runner

1- Adding the official gitlab-runner repo and inspecting the security of the installing script

   curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh > script.deb.sh
   less script.deb.sh
Enter fullscreen mode Exit fullscreen mode

2- Running the installer

   sudo bash script.deb.sh
Enter fullscreen mode Exit fullscreen mode

3- Install gitlab-runner service

   sudo apt install gitlab-runner
Enter fullscreen mode Exit fullscreen mode

4- Check the service status

   systemctl status gitlab-runner
Enter fullscreen mode Exit fullscreen mode

The output should be something like this

   Output
   ● gitlab-runner.service - GitLab Runner
      Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled; vendor preset: enabled)
      Active: active (running) since Mon 2020-06-01 09:01:49 UTC; 4s ago
    Main PID: 16653 (gitlab-runner)
       Tasks: 6 (limit: 1152)
      CGroup: /system.slice/gitlab-runner.service
              └─16653 /usr/lib/gitlab-runner/gitlab-runner run --working-directory /home/gitlab-runner --config /etc/gitla
Enter fullscreen mode Exit fullscreen mode

Register a Runner for Your GitLab Project

1- In your GitLab project, navigate to Settings > CI/CD > Runners.

2- In the Specific Runners section, you’ll find the registration token and the GitLab URL. Copy both as we’ll need them for the next command.

Image description

3- Now in the terminal, register the runner by running

   sudo gitlab-runner register -n --url https://your_gitlab.com --registration-token project_token --executor shell --description "Staging Shell Runner" --tag-list deployment
Enter fullscreen mode Exit fullscreen mode

This seems like a lot of options but they’re pretty easy to understand:

  • -n executes the register command non-interactively.
  • --url is the GitLab URL from step 2.
  • --registration-token is the token you copied from the runners page in step 2.
  • --executor is the executor type, we are using shell executer here which is a simple executor that you use to execute builds locally on the machine where GitLab Runner is installed.
  • --description is the runner’s description, which will show up in GitLab.
  • --tag-list is a list of tags assigned to the runner. Tags can be used in a pipeline configuration to select specific runners for a CI/CD job. The deployment tag will allow you to refer to this specific runner to execute the deployment job.

4- The output of the above command will return an output like this:

   Output
   Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
Enter fullscreen mode Exit fullscreen mode

And the runner will show up in your project’s Settings > CI/CD > Runners

Image description

Create Deployment User

1- Create a new user

   sudo adduser deployer
Enter fullscreen mode Exit fullscreen mode

2- Add user to docker group to permit deployer to execute the docker command, which is required to perform the deployment

   sudo usermod -aG docker deployer
Enter fullscreen mode Exit fullscreen mode

3- Create SSH key for deployer

Switch to the deployer user

   su deployer
Enter fullscreen mode Exit fullscreen mode

Then generate a 4096-bit SSH key

   ssh-keygen -b 4096
Enter fullscreen mode Exit fullscreen mode 
> ⚠ Do not choose a password for your key or you will have to enter it with each job. This is not possible since our runner is non-interactive

Add the new key to the authorized keys
Enter fullscreen mode Exit fullscreen mode 
```bash
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
```
Enter fullscreen mode Exit fullscreen mode

4- Storing the private key in a GitLab CI/CD variable

First, get the private key be executing

   cat ~/.ssh/id_rsa
Enter fullscreen mode Exit fullscreen mode

Copy the output and navigate to Settings > CI / CD > Variables and click Add Variable

fill the variable form like this

  • Key: ID_RSA
  • Value: Paste your SSH private key from your clipboard (including a line break at the end).
  • Type: File
  • Environment Scope: All (default)
  • Protect variable: Checked
  • Mask variable: Unchecked

Create another variable for your server IP address

  • Key: SERVER_IP
  • Value: your_server_IP
  • Type: Variable
  • Environment scope: All (default)
  • Protect variable: Checked
  • Mask variable: Checked

And the last variable is for the user

  • Key: SERVER_USER
  • Value: deployer
  • Type: Variable
  • Environment scope: All (default)
  • Protect variable: Checked
  • Mask variable: Checked

Configuring .gitlab-ci.yml File

Next, we will be prepare the docker file for testing environment, the application service in docker compose file and finally PHPUnit job in .gitlab-ci.yml

Application Docker file

In your app docker file we will need to install XDebug extension to collect the testing coverage report and create two new files: phpunit-report.xml and phpunit-coverage.xml for testing report and testing coverage.

We will create the file in ./docker/testingApp.dockerfile with minimal configuration. It should be something like this.

FROM php:8.1.3-fpm-alpine

WORKDIR /var/www/

# Install alpine packages
RUN apk add --no-cache --update # Add your packages

# Install php extensions
RUN docker-php-ext-install # Add the PHP extension you need

# Install XDebug
RUN pecl install xdebug \
    && docker-php-ext-enable xdebug 

# Copy existing application directory contents
COPY --chown=www-data:www-data . .

RUN touch phpunit-report.xml phpunit-coverage.xml
RUN chmod 777 phpunit-report.xml phpunit-coverage.xml

USER www-data
Enter fullscreen mode Exit fullscreen mode

Application Service

The app service in ./docker/docker-compose-testing.yml will have two volumes: one for the tests directory and the other is ./docker/php/conf.d/xdebug.ini that will contain the basic XDebug configuration.

The app service should look like this:

version: "3"
services:
  app:
    container_name: "app-testing"
    build:
      context: ../
      dockerfile: ./docker/testingApp.dockerfile
    volumes:
      - ./php/conf.d/xdebug.ini:/usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
    working_dir: /var/www
Enter fullscreen mode Exit fullscreen mode

And xdebug.ini is quite minimal, we just set enable the coverage mode but you can enable multiple modes at the same time. Learn more about XDebug modes from the documentation

zend_extension=xdebug

[xdebug]
xdebug.mode=coverage
Enter fullscreen mode Exit fullscreen mode

PHPUnit Job

In .gitlab-ci.yml we will create a job for PHPUnit that will have the deployment tag so that it uses our runner and extract the required artifacts for GitLab to display test reports.

The job will be

phpunit:
  stage: test
  tags:
    - deployment
  before_script:
    - docker-compose -p my-project -f docker/docker-compose-testing.yml build
    - docker-compose -p my-project -f docker/docker-compose-testing.yml up -d
    - docker exec $CONTAINER_NAME php artisan migrate
    - docker exec $CONTAINER_NAME php artisan db:seed
  script:
    - docker exec -t $CONTAINER_NAME vendor/bin/phpunit --do-not-cache-result --log-junit phpunit-report.xml --coverage-cobertura phpunit-coverage.xml --coverage-text --colors=never
    - docker cp $CONTAINER_NAME:/var/www/phpunit-report.xml ./
    - docker cp $CONTAINER_NAME:/var/www/phpunit-coverage.xml ./
  after_script:
    - docker-compose -p my-project -f docker/docker-compose-testing.yml down
  artifacts:
    when: always
    reports:
      junit: phpunit-report.xml
      coverage_report:
        coverage_format: cobertura
        path: phpunit-coverage.xml
  coverage: '/^\s*Lines:\s*\d+.\d+\%/'
  only:
    - merge_requests
    - main
    - develop
Enter fullscreen mode Exit fullscreen mode

You should notice that we build our containers using docker-compose-testing.yml, run the DB migrations and seeder then run our test suit with a few options:

docker exec -t $CONTAINER_NAME vendor/bin/phpunit --do-not-cache-result --log-junit phpunit-report.xml --coverage-cobertura phpunit-coverage.xml --coverage-text --colors=never
Enter fullscreen mode Exit fullscreen mode
  • --do-not-cache-result to disable test result caching.
  • --log-junit phpunit-report.xml specify the test log format and the output file.
  • --coverage-cobertura phpunit-coverage.xml specify the coverage full report format and the output file.
  • --coverage-text generate code coverage report in text format.
  • --colors=never disable colors in the output to make it easier to extract the coverage percentage from the command output.

Then we copy the generated reports from inside the container to the pipeline namespace to be used as job artifacts

docker cp $CONTAINER_NAME:/var/www/phpunit-report.xml ./
docker cp $CONTAINER_NAME:/var/www/phpunit-coverage.xml ./
Enter fullscreen mode Exit fullscreen mode

The job will have two artifacts of type reports the first is the a junit for test log and a cobertura report for the test coverage.

artifacts:
    when: always
    reports:
      junit: phpunit-report.xml
      coverage_report:
        coverage_format: cobertura
        path: phpunit-coverage.xml
Enter fullscreen mode Exit fullscreen mode

ℹ The report formats and files’ extensions are specified by GitLab

Project Configuration

At this point we have created a job for PHPUnit that we run on our Staging Shell Runner that we registered on the staging server. GitLab will be able to display a test report that include some helpful information like the total execution time of the test suite and the success rate. It will also show the test coverage on the jobs and in the merge request overview.

To go one step further, we can track the test coverage history and add coverage badge to the project

Test Coverage History Using Project Settings (Deprecated in GitLab 14.9)

Navigate to Settings > CI/CD > General Pipelines and in the Test Coverage Parsing field add the same regular expression that we used in the job ^\s*Lines:\s*\d+.\d+\%

Test Coverage Badge

Navigate to Settings > General Settings > Badges and click Add Badge and fill the form as follows:

  • Name: PHPUnit Coverage
  • Link: https://gitlab.com/[PROJECT_PATH]/-/commits/[BRANCH_NAME]
  • Badge Image URL: https://gitlab.com/[PROJECT_PATH]/badges/[BRANCH_NAME]/coverage.svg

Result

  • You can see a test report in your pipeline

Image description

  • Navigate to Analytics > Repository > Code Coverage Statistics to see the history of test coverage

Image description

  • The effect of each merge request on the test coverage can be found in the merge request overview page.

Image description

Top comments (5)

Collapse
 
swalbrun profile image
s.walbrun •

If you use Laravel, so accordingly with the command
php artisan test --coverage
the tests, then the regex is not correct. Instead just use this regex
coverage: '/^\s*Total\sCoverage\s\.+\s\d+\.\d+\s\%/'

Collapse
 
dracoblue profile image
DracoBlue •

Navigate to Settings > CI/CD > General Pipelines and in the Test Coverage Parsing field add the same regular expression that we used in the job ^\s*Lines:\s*\d+.\d+\%

this is not possible anymore in gitlab. you just have to use the coverage-property on the job (like described in the blog post!)

Collapse
 
muhamadhhassan profile image
Muhammad Hassan •

You're right. I've updated it thank you!

Collapse
 
eric11111199235 profile image
eric1199235 •

Informative guide — integrating PHPUnit test logs and coverage reports into GitLab CI/CD pipelines is crucial for maintaining code quality. The detailed steps provided make the setup process straightforward.
In my local development on macOS, I use ServBay to manage multiple PHP versions and services like MySQL. It allows me to replicate production environments locally, making it easier to test CI/CD configurations before deploying them.

Collapse
 
wj_lee_3f8589205b03973ae2 profile image
WJ Lee •

Just wondering how to set up gitlab to make every test coverage showing in the merge request overview page?