Vincent A. Cicirello

Posted on Nov 29, 2021 • Edited on Jul 14, 2022

JaCoCo coverage badges, PR coverage checks, and PR coverage comments, from GitHub Actions

#actionshackathon21 #github #java #githubactions

My Workflow

This workflow builds a Java project, runs all JUnit tests, runs JaCoCo to generate a test coverage report, uses the cicirello/jacoco-badge-generator GitHub Action to generate instructions coverage and branches coverage badges from the JaCoCo coverage report, uploads the JaCoCo report as a workflow artifact, and if the workflow run is associated with a PR it then comments on the PR with the computed coverage percentages.

The coverage badges are generated with the cicirello/jacoco-badge-generator, which I have been developing and maintaining for a while now. Initially, it was strictly a badge generator, but its functionality has expanded over time to include optional support for PR checks that can be used to fail workflow runs if coverage is below a target threshold or if coverage has decreased. The cicirello/jacoco-badge-generator includes the following functionality:

Generates instructions coverage and/or branches coverage badges entirely within the action, without any external services.
Badge color is based on coverage percentage, and a reasonable set of defaults is provided, but the colors and associated coverage intervals can be customized with action inputs.
Option to either generate the badges directly (which is the default behavior) or to generate Shields JSON endpoints for users who may want to customize badge appearance beyond what is directly supported by the action. I use the default of direct generation of badges in my own workflows.
Option to generate a simple JSON file containing the coverage and branches coverage percentages, either in addition to the badges or instead of the badges.
Option to fail the workflow run if coverage, or branches coverage, or both are below customizable thresholds (this option is disabled by default).
Option to fail the workflow run if coverage, or branches coverage, or both have decreased since the most recent run (this option is disabled by default).

The default behavior of the cicirello/jacoco-badge-generator assumes that the JaCoCo coverage report is in the default location with the default filename when JaCoCo is run with Maven. But, the action has inputs that can be used if either the JaCoCo report is named differently or if it is in a different location, or both, so it is not limited to use with Maven projects. The GitHub repository provides examples with how to configure it for use with Maven as well as Gradle, but it is also applicable if you run JaCoCo with any other tool.

The workflow that I am submitting to the Actions Hackathon is from my project Chips-n-Salsa, a Java library of customizable, hybridizable, iterative, parallel, stochastic, and self-adaptive local search algorithms, specifically, the build.yml workflow from that project. A summary of the steps of the workflow submitted to the Actions Hackathon is as follows (complete workflow later in this post):

Checkout (using the action actions/checkout).
Checkout the badges branch nested within the first checkout in a badges directory. The purpose of this step is multifaceted. First, the default branch of this repository has required checks, so I cannot use the GITHUB_TOKEN to push the generated badges to the default branch. For security reasons, I don't want to use a PAT to override those checks. So I instead have a badges branch without required checks that is dedicated solely to storing the coverage badges that are generated by a later step of the workflow. Since the badges branch has no required checks, the GITHUB_TOKEN is sufficient for pushing to it. The second reason for this second checkout is that an action used later, which generates coverage badges, has additional functionality for checking if coverage has decreased during PRs that uses the coverage as stored in the prior badge to check for a decrease.
Set up the JDK with the actions/setup-java GitHub Action.
Build the library with Maven. This step just uses the shell to run mvn directly, with mvn -B package -Pcoverage. This will build the library and run all JUnit tests. The command line option -Pcoverage enables a profile coverage from the project's pom.xml, which configures JaCoCo to run during the test phase.
Generate JaCoCo coverage badges with the cicirello/jacoco-badge-generator GitHub Action, which I have been developing and maintaining for a while. I'm using the badges-directory input to change the directory for where to store the badges, and I'm enabling generating a branches coverage badge with the generate-branches-badge input. The action generates an instructions coverage badge by default, but the additional branches coverage badge must be enabled. I'm also enabling (with the generate-summary input) the generation of a JSON file containing the coverage percentages. This is actually a new feature of the cicirello/jacoco-badge-generator action that generates a simple JSON file with the computed coverage percentages, which may be useful for consumption by other workflow steps (such as later in this workflow).
Logs the coverage and branches coverage percentages to the workflow output, which may be useful when inspecting logs of workflow runs.
Uses the actions/upload-artifact GitHub Action to upload the complete coverage reports generated by JaCoCo earlier in step 4 above as a workflow artifact. This may be useful if I need to inspect the coverage details provided by JaCoCo for a workflow run.
If this workflow run is not associated with a PR, then commit and push the coverage badges and summary file to the badges branch, using shell commands. Since the workflow runs on pushes, PRs, and workflow_dispatch events, then this step will specifically run on either pushes or workflow_dispatch events. The badges must be in sync with the state of the default branch, so it only makes sense to push them on events against the default branch, which is why PRs are excluded during this step.
If this run is for a PR, then comment on the PR with the coverage percentages. This step is implemented simply with shell commands, and specifically uses the GitHub CLI to comment on the PR. This step is conditional and runs only on PRs. This is actually the newest addition to this workflow, added to it within the last week.

See the Chips-n-Salsa repository's README for live examples of the generated badges. And see this PR comment for an example of the PR coverage comment created by the last step of the workflow.

Submission Category:

Maintainer Must-Haves

Yaml File or Link to Code

The live version of this workflow is found in the build.yml of the Chips-n-Salsa library, which at the time of submission is identical to the workflow found below:

name: build

on:
  push:
    branches: [ master ]
    paths: [ '**.java', '.github/workflows/build.yml' ]
  pull_request:
    branches: [ master ]
  workflow_dispatch:

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Checkout badges branch to a badges directory nested inside first checkout
      uses: actions/checkout@v2
      with:
        ref: badges
        path: badges

    - name: Set up JDK 11
      uses: actions/setup-java@v2
      with:
        distribution: 'adopt'
        java-version: '11'

    - name: Build with Maven
      run: mvn -B package -Pcoverage

    - name: Generate JaCoCo badge
      id: jacoco
      uses: cicirello/jacoco-badge-generator@v2
      with:
        badges-directory: badges
        generate-branches-badge: true
        generate-summary: true

    - name: Log coverage percentages to workflow output
      run: |
        echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
        echo "branches = ${{ steps.jacoco.outputs.branches }}"

    - name: Upload JaCoCo coverage report as a workflow artifact
      uses: actions/upload-artifact@v2
      with:
        name: jacoco-report
        path: target/site/jacoco/

    - name: Commit and push the coverage badges and summary file
      if: ${{ github.event_name != 'pull_request' }}
      run: |
        cd badges
        if [[ `git status --porcelain *.svg *.json` ]]; then
          git config --global user.name 'github-actions'
          git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com'
          git add *.svg *.json
          git commit -m "Autogenerated JaCoCo coverage badges" *.svg *.json
          git push
        fi

    - name: Comment on PR with coverage percentages
      if: ${{ github.event_name == 'pull_request' }}
      run: |
        REPORT=$(<badges/coverage-summary.json)
        COVERAGE=$(jq -r '.coverage' <<< "$REPORT")%
        BRANCHES=$(jq -r '.branches' <<< "$REPORT")%
        NEWLINE=$'\n'
        BODY="## JaCoCo Test Coverage Summary Statistics${NEWLINE}* __Coverage:__ ${COVERAGE}${NEWLINE}* __Branches:__ ${BRANCHES}"
        gh pr comment ${{github.event.pull_request.number}} -b "${BODY}"
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Additional Resources / Info

The repository for the cicirello/jacoco-badge-generator GitHub Action at the heart of the workflow submitted to the GitHub Actions Hackathon, which also has additional sample workflows using the action:

cicirello / jacoco-badge-generator

Coverage badges, and pull request coverage checks, from JaCoCo reports in GitHub Actions

jacoco-badge-generator

Check out all of our GitHub Actions: https://actions.cicirello.org/

About

GitHub Actions
Command-Line Utility
Build Status
Security
Source Info
Support

The jacoco-badge-generator can be used in one of two ways: as a GitHub Action or as a command-line utility (e.g., such as part of a local build script). The jacoco-badge-generator parses a jacoco.csv from a JaCoCo coverage report, computes coverage percentages from JaCoCo's Instructions and Branches counters, and generates badges for one or both of these (user configurable) to provide an easy to read visual summary of the code coverage of your test cases. The default behavior directly generates the badges internally with no external calls, but the action also provides an option to instead generate Shields JSON endpoints. It supports both the basic case of a single jacoco.csv, as well as multi-module projects in which case the action can produce coverage badges from the combination of…

View on GitHub

The repository of Chips-n-Salsa, a Java library of customizable, hybridizable, iterative, parallel, stochastic, and self-adaptive local search algorithms, whose build.yml workflow is the subject of this submission to the GitHub Actions Hackathon:

cicirello / Chips-n-Salsa

A Java library of Customizable, Hybridizable, Iterative, Parallel, Stochastic, and Self-Adaptive Local Search Algorithms

Chips-n-Salsa

Website: https://chips-n-salsa.cicirello.org/

API documentation: https://chips-n-salsa.cicirello.org/api/

Publications About the Library
Packages and Releases
Build Status
JaCoCo Test Coverage
Security
DOI
Other Information
Support

How to Cite

If you use this library in your research, please cite the following paper:

Cicirello, V. A., (2020). Chips-n-Salsa: A Java Library of Customizable, Hybridizable, Iterative, Parallel, Stochastic, and Self-Adaptive Local Search Algorithms. Journal of Open Source Software, 5(52), 2448, https://doi.org/10.21105/joss.02448 .

Overview

Chips-n-Salsa is a Java library of customizable, hybridizable, iterative, parallel, stochastic, and self-adaptive local search algorithms. The library includes implementations of several stochastic local search algorithms, including simulated annealing, hill climbers, as well as constructive search algorithms such as stochastic sampling. Chips-n-Salsa now also includes genetic algorithms as well as evolutionary algorithms more generally. The library very extensively supports simulated annealing. It includes several classes for representing solutions to a variety of optimization problems…

View on GitHub

More information on the cicirello/jacoco-badge-generator GitHub Action, as well as other GitHub Actions that I maintain is found at: https://actions.cicirello.org/.

DEV Community