Enforce JavaScript Code Coverage with GitHub Actions

I recently migrated a few open-source projects over to GitHub actions for continuous-integration. In the process, I came up with a neat trick that I thought was worth sharing...

nyc's --check-coverage flag

The JavaScript coverage libraries c8 and nyc expose the flag --check-coverage. Used in conjunction with --lines, --branches, --functions, and --statements, --check-coverage will exit with an error if coverage falls below the threshold specified.

c8 is a tool inspired by nyc that uses V8's built-in test coverage, see: Rethinking JavaScript Test Coverage.

As an example,

c8 --check-coverage --lines=95 npm run test

when run on a program with less than 95% line coverage, exits with the following error:

ERROR: Coverage for lines (92%) does not meet global threshold (95%)

Adding thresholds to .nycrc

Thresholds can be specified in the .nycrc configuration file, which is read automatically by c8 and nyc.

Here's an example of the configuration file from yargs:

{
  "reporter": [
    "html",
    "text"
  ],
  "lines": 100,
  "branches": "96",
  "statements": "100"
}

The thresholds in this file can then be treated like a ratchet, i.e., as coverage increases on your project, increase the value of lines, branches, and statements accordingly.

Adding a coverage job to a workflow

Once thresholds are set in a project's configuration, it's easy to add an additional job to a continuous integration workflow that fails if coverage drops below the thresholds.

Here's an example of a coverage job that was added to yargs' ci.yaml workflow:

on:
  push:
    branches:
      - master
  pull_request:
name: ci
jobs:
  ...
  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v1
        with:
          node-version: 13
      - run: npm install
      - run: npm test
      - run: npm run coverage

npm run coverage corresponds to the following npm script:

{
  ...
  "scripts": {
    "coverage": "c8 report --check-coverage"
  }
}

c8 reads the thresholds from .nycrc and exits with 1 if the specified thresholds are not met, failing the workflow's job.

But what about my badges?!

You're probably asking, "this is all well and good, but how can I add a badge to the top of my README"?

Good news, support for a .nycrc coverage badge was just added to shields.io.

see: https://github.com/badges/shields/pull/4759

---

Services like codecov.io and coveralls.io are awesome: providing coverage report merging, beautiful visualizations, historical data, etc.

If you're not feeling you need these features just yet, but would like to enforce coverage on your JavaScript project, consider using c8 or nyc with the --check-coverage flag in a GitHub Action workflow.

Comments

[Charlike Mike Reagent]

Oh please don't promote Coveralls. I even want to puke when I see it in package.json, even more when in npm scripts :D

Codecov is the name of the game. Single line of 20 bytes, period. No tokens, no setup, no "add", no bullshits.