Gitlab CI for Node Testing and Coverage

Gitlab is a popular open-source version control system which is free to use and can be built on an intranet, and Gitlab has many useful features such as Gitlab CI.

Gitlab has been integrating CI/CD pipelines into Gitlab for a long time, and has evolved the so-called Gitlab Flow. In this article, I won't go through the entire Gitlab CI guide, nor will I explain the CI/CD concept, but will focus on how to make Node testing reports more presentable.

Why this topic? The main reason is because we often use nyc and mocha together to build testing reports for Node, but such a combination needs a little twist in order to fit into the rich functionality of Gitlab. This article is about those approaches, and will use an actual .gitlab-ci.yml as an example.

Please be aware that this article is written based on Gitlab v15.0

Testing Report

In a good testing report we will need several important features.

1. an artifact of the full report.
2. test summary for each Pull Request or Merge Request.
3. the change coverage of each Pull Request or Merge Request.
4. the status of the entire pipeline, including the latest success or failure and its coverage, preferably in the form of a badge.

Report Artifacts

This is the latest pipeline report, to be able to be downloaded here, we need to add a new artifacts field to specify the path we want to export at the desired stage. For example, in the figure above, the setting would be as follows.

test_ci:
  script:
    - npm run test
  artifacts:
    paths:
      - coverage/

This means we will export everything under the coverage folder as a package.

Test Summary

In order to display the results of a test in Merge Request, including how many cases were tested and how many succeeded or failed, and even to see how long each case took, you need to let Gitlab know the format of the testing report and produce the results in the corresponding format.

So let's continue to extend the .gitlab-ci.yml example above.

test_ci:
  script:
    - npm run test
  artifacts:
    paths:
      - coverage/
    reports:
      junit:
        - test-results.xml

In this example, we use the JUnit format to create the testing report and inform Gitlab of the path to the CI report. In this way, Gitlab has the ability to present the correct report content and summary in each Merge Request.

Change Coverage

When doing a code review, we all click into Changes to see what parts have been changed.

It would be more efficient for the reviewer to see the test coverage of the changes here in one place. So, we would like to make it easy for the reviewer to know which code has not been tested.

In this picture, we can see at a glance that line 14 is not covered by the test, while the other lines are tested. It is worth mentioning that even if there is test coverage, it does not mean that the test is complete, for example, here it is impossible to determine the conditions of the boundary test, and we have to rely on the experience of the reviewer.

Then, we continue to extend the original settings.

test_ci:
  script:
    - npm run test
  artifacts:
    paths:
      - coverage/
    reports:
      junit:
        - test-results.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

Pipeline Badges

In popular open source projects nowadays, users are informed of the project's health at the beginning of README.md, which is a useful information for users and a quick way for developers to know the project's health.

If you see the status of the pipeline as a failure, something is wrong. On the other hand, the coverage badge is a great indicator of whether the project's test coverage is complete.

Fortunately, badges are a built-in feature of Gitlab. You can find out the badge location at Gitlab settings.

Settings > CI/CD > General pipelines

There are three types of badges, Pipeline status, Coverage report, and Latest release. You can pick what you want.

Since Gitlab v15.0, we can assign a regular expression in re2 syntax at .gitlab-ci.yml to identify what the coverage digits are.

test_ci:
  script:
    - npm run test
  coverage: '/All files\s+\|\s+\d+\.\d+/'

The rule for this re2 syntax is to find the floating point number that follows All files as the coverage. If you are not using nyc, you have to adjust the rule based on the content.

Detail in package.json

The above example has fully implemented the necessary features for development. But we haven't explained how to generate coverage reports, JUnit reports, and change coverage at the same time.

The key to all of this is in the npm run test, i. e. package.json.

{
  "script": {
    "test": "nyc --reporter=html --reporter=text --reporter=cobertura mocha"
  }
}

As we can see from the above settings, this busy nyc is responsible for generating three types of outputs for the three different features.

• html: Serves as a coverage report for the entire project, and will be used when downloading artifacts.
• text: The console output is required to generate the badges.
• cobertura: As we know from the previous section, the change coverages are presented using the cobertura format.

Wait, there's one missing? Who creates the reports for JUnit? The answer is mocha. But this is not a built-in feature of mocha, so we have to use an additional tool to do it.

First, download the mocha-junit-reporter package.

npm i mocha-junit-reporter --save-dev

Next, create the mocha configuration file, .mocharc.js.

module.exports = {
  reporter: "./junit-spec-reporter.js"
};

In the configuration file we tell mocha to generate the report through another file, which is also the JUnit generator.

The following is the content of junit-spec-reporter.js.

const mocha = require("mocha");
const JUnit = require("mocha-junit-reporter");
const Spec = mocha.reporters.Spec;
const Base = mocha.reporters.Base;

function JunitSpecReporter(runner, options) {
  Base.call(this, runner, options);
  this._junitReporter = new JUnit(runner, options);
  this._specReporter = new Spec(runner, options);
  return this;
}
JunitSpecReporter.prototype.__proto__ = Base.prototype;

module.exports = JunitSpecReporter;

At this point, all the formats we need can be generated correctly, and Gitlab CI will present a rich view based on these outputs, and developers can do most of their routine work on Gitlab's web page without actually building the outputs locally.

Conclusion

CI/CD is a very important software development practice. However, in order for every developer to have the interest and even the confidence to "continue" the practice, people must be able to "see" the change. For engineers, seeing is believing is the belief of most of us. Therefore, these rich features are essential for the pipeline to be effective enough.

The full .gitlab-ci.yml, which includes all mentioned features, is as follows.

test_ci:
  script:
    - npm run test
  artifacts:
    paths:
      - coverage/
    reports:
      junit:
        - test-results.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
  coverage: '/All files\s+\|\s+\d+\.\d+/'

In my experience, when a testing platform is built, not everyone is happy to use it, after all, writing tests is extra work. But when the platform is rich enough, most people will be willing to try it. For a team just starting to establish a development process, it's more important to get people willing to try it than anything else. So this article focuses on the presentation of Gitlab CI and introduces the role of CI through a different perspective in the development process.