Add coverage report with pytest and Gitlab CI

Introduction

In this blog, I am going to cover the integration of Pytest with Poetry and then using the coverage to generate a report that is displayed as part of Gitlab file diffs.

I will be using poetry to install and manage my dependencies.

Pre-requisite

1. Python 3.8.x or higher
2. Poetry 1.4
3. Pytest 7.2.x
4. Coverage 7.2.1
5. An empty Gitlab project/repository with the name gitlab-demo-project

Setting up the project

This part assumes that you have Python and Poetry installed and setup in your development environment. Additionally, you have cloned the empty project in your local. All the following instructions are to be done in the root of the cloned project.

1. Initialize the current project with poetry using

poetry init

Follow the on-screen instructions that will allow you to add some necessary information and generate a pyproject.toml file to contain your dependency. Your project structure should look something like this

.
├── README.md
├── pyproject.toml
├── gitlab_demo_project
│   └── __init__.py
└── tests
    └── __init__.py

1. Install pytest and coverage using poetry under a test group

poetry install pytest --group test

poetry install coverage --group test

1. In the pyproject.toml file, add the following lines which are responsible for pytest and coverage configurations while running.

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-s -v --durations=0"
cache_dir = ".cache/pytest_cache"

[tool.coverage.run]
branch = true
source = ["gitlab-demo-project"]
command_line = "-m pytest"

[tool.coverage.report]
show_missing = true

[tool.coverage.xml]
output = "coverage.xml"

1. Create a file called .gitlab-ci.yml in the root of the project and copy the following contents in it

default:
  image: python:3.8

stages:
  - test

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
  POETRY_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pypoetry"
  POETRY_VIRTUALENVS_IN_PROJECT: "true"

cache:
  paths:
    - .cache/pip
    - .cache/pytest_cache
    - .cache/pypoetry
    - .venv

# Install specific version of poetry using pipx
# Poetry installation is recommended through pipx
# https://python-poetry.org/docs/#ci-recommendations
.poetry_setup: &poetry_setup
  before_script:
    - python -V
    - python -m pip install --upgrade pip
    - python -m pip install pipx
    - python -m pipx install poetry==1.4.0
    - export PATH=$PATH:$HOME/.local/bin
    - poetry install

Testing:
  <<: *poetry_setup
  stage: test
  script:
    - poetry run coverage run
    - poetry run coverage report
    - poetry run coverage xml
  coverage: '/TOTAL.*\s+(\d+%)$/'
  artifacts:
    # https://docs.gitlab.com/ee/ci/yaml/index.html#artifactsexpire_in
    expire_in: 1 week

    # https://docs.gitlab.com/ee/ci/testing/test_coverage_visualization.html
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.xml

1. Create a file in your gitlab_demo_project folder called hello.py and add the following lines in it

def call_hello():
    print("hey, its me!")

1. Create a file in your tests folder called test_hello.py and add the following lines in it

from gitlab_demo_project.hello import call_hello

def test_hello():
    assert call_hello() == "hey, its me!"

Verify the test case using pytest tests/ command on the terminal.

1. Checkout a branch using the git bash named testing-gitlab-coverage and commit all the changes in that branch.

2. Once committed, push all the changes to remote and create a merge request. The pipeline should be triggered with the name Testing. After it succeeds, check the diff of files in the merge request. It should have the following lines appearing

References

1: Pytest command-line flags https://docs.pytest.org/en/7.2.x/reference/reference.html#command-line-flags
2: Coverage configurations https://coverage.readthedocs.io/en/7.2.1/config.html
3: Gitlab Artifacts https://docs.gitlab.com/15.9/ee/ci/yaml/
4: Gitlab Artifacts Report https://docs.gitlab.com/15.9/ee/ci/yaml/artifacts_reports.html
5: Gitlab Test Coverage visualization image https://docs.gitlab.com/ee/ci/testing/img/test_coverage_visualization_v12_9.png