🧑🏽💻 CMake Coverage Example
In this post, we see a CMake coverage example using GitHub Actions. We build the project and run unit tests on each git push, before uploading the latest code coverage results to codecov.io. This is all working on a CMake C++ project. Using gcov, genhtml and lcov locally, it is possible to generate an HTML coverage report, with graphs. However, here we automate the process, running tests remotely. This has added advantages. For example, you might configure your repo to reject pull requests that cause the test coverage to drop below a target percentage.
🧱 What we are Building
We will use an arkanoid-clone project that I set up for testing. This is a clone of the classic arcade game built using CMake and SFML. The repo itself is based on a YouTube tutorial, and you can find a link to the original tutorial in the arkanoid-clone repo README. You might also want to reference the repo for any extra useful details missing here — do let me know, though, if there is anything, extra, I could add here to make the article clearer.
Let’s start by adding Catch2 tests to the repo. We assume you have a CMake repo ready to add tests, too. If you don’t, there is a fantastic JetBrains tutorial on adding Catch2 unit tests in CMake, which you can set up quickly as a toy project.
☑️ Setting up CMake Unit Tests
I will add Catch2 tests in a new Catch_tests directory, and need to include the new directory at the bottom of the main CMakeLists.txt file:
# include test folder towards the bottom of CMakeLists.txt
option(RUN_UNIT_TESTS "Run Catch2 unit tests" ON)
if(RUN_UNIT_TESTS)
enable_testing()
add_subdirectory(Catch_tests)
endif()
Then, I create Catch_tests/CMakeLists.txt:
include(FetchContent)
FetchContent_Declare(
Catch2
GIT_REPOSITORY https://github.com/catchorg/Catch2.git
GIT_TAG v3.5.0)
FetchContent_MakeAvailable(Catch2)
add_executable(Catch_tests_run BallTests.cpp)
target_link_libraries(
Catch_tests_run
PRIVATE Ball_lib
Brick_lib
Paddle_lib
arkanoid_compiler_flags)
target_link_libraries(Catch_tests_run PRIVATE Catch2::Catch2WithMain)
target_include_directories(Catch_tests_run PUBLIC "${PROJECT_SOURCE_DIR}/src")
include(Catch)
catch_discover_tests(Catch_tests_run)
This fetches the Catch2 dependency from GitHub, adds a test executable and finally, sets up Catch2 to discover the tests.
As an example test, you might have something like src/Catch_tests/BallTests.cpp:
#include "Ball.h"
#include <SFML/Graphics.hpp>
#include <catch2/catch_test_macros.hpp>
TEST_CASE("BallCorrectlyInitialised", "[BallTests]")
{
const Ball ball{256, 512};
const sf::Color ball_colour{ball.shape.getFillColor()};
CHECK(ball_colour == sf::Color::Red);
}
See the tutorial, mentioned above, for more on creating Catch2 tests and setting them up in CMake.
⛺️ Adding Coverage Tests
To start, download the CMake lcov Coverage module and add it to a new cmake folder in the root directory of your project:
function(add_coverage_target exclude)
find_program(GCOV gcov)
if (NOT GCOV)
message(WARNING "program gcov not found")
endif()
find_program(LCOV lcov)
if (NOT LCOV)
message(WARNING "program lcov not found")
endif()
find_program(GENHTML genhtml)
if (NOT GENHTML)
message(WARNING "program genhtml not found")
endif()
if (LCOV AND GCOV AND GENHTML)
set(covname cov.info)
add_compile_options(-fprofile-arcs -ftest-coverage)
add_link_options(--coverage)
add_custom_target(cov DEPENDS ${covname})
add_custom_command(
OUTPUT ${covname}
COMMAND ${LCOV} -c -o ${covname} -d . -b . --gcov-tool ${GCOV}
COMMAND ${LCOV} -r ${covname} -o ${covname} ${exclude}
COMMAND ${LCOV} -l ${covname}
COMMAND ${GENHTML} ${covname} -output coverage
COMMAND ${LCOV} -l ${covname} 2>/dev/null | grep Total | sed 's/|//g' | sed 's/Total://g' | awk '{print $1}' | sed s/%//g > coverage/total
)
set_directory_properties(PROPERTIES
ADDITIONAL_CLEAN_FILES ${covname}
)
else()
message(WARNING "unable to add target `cov`: missing coverage tools")
endif()
endfunction()
Then, update the main CMakeLists.txt to find and use this, adding these lines towards the top of the file:
set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH};${PROJECT_SOURCE_DIR}/cmake")
include(coverage)
add_coverage_target("*/Catch_tests/*")
The argument of the add_coverage_target function is passed to lcov and gcov as an exclude path.
That is all you need to run the coverage tests. We will convert the data to XML for codecov.io later, using gcovr.
CMake Coverage Example: GitHub Test & Coverage Action
Next we need a GitHub action. This runs on every pull request submission. In our case, we use it to build the project, run tests and then generate a coverage report in XML format ready for codecov.io. Create a GitHub workflow flow in .github/workflows/ubuntu.yml to do this:
name: Ubuntu CI Test
on:
push:
branches: [main, master, dev]
pull_request:
branches: [main, master, dev]
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ee0669bd1cc54295c223e0bb666b733df41de1c5 # v2.7.0
- name: install
run: sudo apt-get update && sudo apt-get install lcov #libxrandr-dev libxcursor-dev libudev-dev libopenal-dev libflac-dev libvorbis-dev libgl1-mesa-dev libegl1-mesa-dev freeglut3-dev # OpenGL dependencies only needed for SFML
- name: configure
run: |
cmake -H. -Bbuild -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DENABLE_COVERAGE=On
- name: building
run: |
cmake --build build --config Debug --target Catch_tests_run
- name: run unit tests
working-directory: ./build/bin
run: |
./Catch_tests_run
- name: generate coverage
working-directory: ./build
run: |
make cov
- name: Install gcovr
run: |
pip install -r requirements.txt --require-hashes
- name: Generate JSON coverage report
working-directory: ./build
run: |
gcovr -r .. . --branches --cobertura > coverage.xml
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
I use git commit hashes instead of repo tags or version numbers for added security (in line 13, for example). In line 15 you can install gcovr and lcov as well as any extra dependencies needed to build your project.
Lines 16 to 21 will configure, then build the project in the remote environment, after a pull request. Then, the code in lines 22 to 25 will run the Catch2 unit tests. If we broke something in the pull request, and the unit tests fail, the whole pull request is rejected: exactly what we are looking for.
In lines 26-29, we generate the coverage report in a text format. I had most success with codecov.io working in the XML format. We can convert the report, running:
gcovr -r .. . --branches --cobertura > coverage.xml
Which is exactly what we do in lines 33-36, because our working directory will be build/ we adjust the path passed to gcovr. You can find more details on gcovr commands in the docs.
The final lines are where we upload the XML coverage report to codecov.io. You will need to obtain a codecov token and add this to your GitHub repo. To get a token, go to https://app.codecov.io/gh/YOUR-GITHUB-ACCOUNT, log in with GitHub and select the repo you want to add coverage for. Then, follow instructions on adding a repository secret. That will cover step one, displayed, and we have tackled step 2 above!
Python requirements.txt
Finally, add a requirements.txt file in the project root directory, which will be used in line 32, above, to install gcovr (and the lxml dependency):
gcovr==6.0 \\
--hash=sha256:2e52019fdb76c6e327f48c2a2d8555fb5e362570b79cc74c5498804d1ce54a60
lxml==5.1.0 \\
--hash=sha256:22b7ee4c35f374e2c20337a95502057964d7e35b996b1c667b5c65c567d2252a
💯 CMake Coverage Example: Testing your Work
Next time you merge a pull request, after a short delay, you will see a coverage report on the codecov.io console, for your project.
🙌🏽 CMake Coverage Example: Wrapping Up
In this CMake coverage example post, we saw a complete example of adding coverage tests to a CMake repo. We used GitHub actions to run a test coverage report, and push it to codecov.io. More specifically, we saw:
- how you can use run Catch2 unit test in GitHub actions;
- the codecov.io setup process for working in GitHub; and
- how to run code coverage test in CI and push the result to codecov.io.
I hope you found this useful. If you need additional pointers, take a look at the arkanoid-clone repo mentioned at the start of the post, which fully implements the coverage approach discussed here. Do let me know, though, if anything here is not clear, or would benefit for further clarification.
🙏🏽 CMake Coverage Example: Feedback
If you have found this post useful, see links below for further related content on this site. Let me know if there are any ways I can improve on it. I hope you will use the code or starter in your own projects. Be sure to share your work on X, giving me a mention, so I can see what you did. Finally, be sure to let me know ideas for other short videos you would like to see. Read on to find ways to get in touch, further below. If you have found this post useful, even though you can only afford even a tiny contribution, please consider supporting me through Buy me a Coffee.
Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on X (previously Twitter) and also askRodney on Telegram. Also, see further ways to get in touch with Rodney Lab. I post regularly on Deno as well as Search Engine Optimization among other topics. Also, subscribe to the newsletter to keep up-to-date with our latest projects.
Top comments (0)