Pantelis Vardakas Software Engineer in Test
Test reporting is a crucial task which provides an insight for the quality of the test automation strategy. Teams can receive valuable and timely feedback on their product quality during the development lifecycle.
Allure is a very popular reporting tool nowadays which can produce a variety of graphs and visualizations based on those test reports.
In this article we will have an overview on how to configure a GitLab pipeline to create test results and upload them to Allure Server using Allure CLI. All the modern test frameworks can pull in the required plugins in order to produce test results. For our example, we will use a JavaScript project with Webdriver.io (WDIO).
The project configuration
Initially, we must install @wdio/allure-reporter along with wdio-video-reporter dependencies in order to enhance our test reports with video record of the test executions.
Then, we must configure the output directory (allure-results) and the video reporter in the reporter’s property in the WDIO config file.
For our reports, it would be advisable to include a video only for unsuccessful test cases in order to save disk space on the server.
The GitLab CI configuration
On GitLab CI project we must specify 3 mandatory variables in the Settings menu of each project to configure the Allure Server. Those are:
ALLURE_ENDPOINT: the http address for the Allure Server
ALLURE_PROJECT_ID: the id of project on the Server
ALLURE_TOKEN: the access token of the Server
The desirable values of the aforementioned variables can be found on the user menu of the Allure Server itself:
Further configuration is required to achieve our end goal, but this time in gitlab-ci.yaml file.
A new pipeline stage must be introduced and let’s call this one ‘report’. We have to ensure that the stages are configured in a way that our newly created ‘report’ stage is executed right after the ‘test’ one:
On each test job we must introduce the variable allow_failure and set it as true in order to be sure that a failed job will not cause the pipeline to fail. In addition to that, we must use the artifacts section to specify the files and the directories which will contain the report files:
By taking advantage of the ability to pass artifacts between stages, we must create a new job which will be executed right after the tests and will use Allure CLI to upload the results on Allure Server.
For the job at hand, we can use any Docker image (in our case we used node: fermium) in which we have to download and install the allure command line tool.
Having completed the steps described above and by using the allurectl binary we can upload all the artifacts of the previous executions.
In the image below you can see the complete configuration about report job:
If everything is set up correctly then the stages and the jobs of our pipeline might look like the photo below. As you can see there are three different stages and the last stage is the report job which will eventually upload the test results on our server:
After a complete pipeline execution, we can have an overview of our results on Allure Server. The dashboards can be parameterized according to the needs of each team.
Additional information regarding the Allure Server can be found here and for GitLab yaml file configuration here.
At Agile Actors we are always trying to learn new things, search for new technologies, put them in test, adopt them and share our gained knowledge between our teams. If you like to experiment and adapt new technologies, you can check out our openings here and can be part of our SET Chapter.
Top comments (0)