.Net Core Unit Test and Code Coverage with Visual Studio Code

There is a bad notion about you can’t make Code Coverage in .Net Core on Mac, because OpenCover tool only works on Windows.

But I had discovered a new NuGet package called Coverlet a cross-platform code coverage library for .NET Core. Then finally we can make code coverage using the same commands no matters if you are working on Mac, Linux or Windows.

Coverlet

Coverlet - Cross platform code coverage tool for .NET Core

It’s unbelievable easy to use, comparing with OpenCover. Add this NuGet package in your test project:

dotnet add package coverlet.msbuild

Now add some parameters when running the test project:

dotnet test /p:CollectCoverage=true

And that’s all. But if you want personalize it according your needs:

dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=\"opencover,lcov\" /p:CoverletOutput=../lcov

I need a code coverage report in opencover and lcov formats. The first one to be uploaded on Sonar Cloud and the second one to be used with Coverage Gutters Visual Studio Code extension.

Visual Studio Code Extensions

If Ash Ketchum were a programmer, surely would say: Visual Studio Code...

It can be extensible according to what you need. And it’s perfect because these two extensions give you unit test superpowers:

• Coverage Gutters - Visual Studio Marketplace
• .NET Core Test Explorer - Visual Studio Marketplace

Coverage Gutter display coverage result with colors in your screen and you can activate or deactivate it. And Test Explorer gives you a visual explorer panel when you can run tests: all of them, a group in context or individual test. Even better lights up code lens style over each test and you can see his result.

Unit Testing Framework

You can found a lot of literature about unit test frameworks (xUnit, nUnit, and MSTest) in .Net, and really no matters what you choose, there are no significant differences between them.

Why I choose xUnit? because its part of .Net Foundations, I like his syntaxis and works like a charm with Test Explorer plugin.

There is a good explanation about xUnit with .Net Core. If you are making baby steeps I highly recommend reading it:

Unit testing C# code in .NET Core using dotnet test and xUnit

Sonar Cloud

I can’t imagine a present without using a tool like SonarQube in a project. I have created two scripts to run Sonar Scanner and upload the results including code coverage.

Install SonarScanner for MSBuild:

dotnet tool install --global dotnet-sonarscanner

Create a sonar login token on SonarCloud on Profile > Security > Generate Token. Save it inside sonar.txt file and add this file to .gitignore. Now you can use it locally and can’t be revealed to curious eyes.

Use this on Windows (Bat):

@echo off
set /p token=<sonar.txt
dotnet sonarscanner begin /k:"company:project" /n:"Project" /v:"#.#.#" /o:"companyname" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="%token%" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.coverage.exclusions="Project.Tests/**,**/*Tests.cs" /d:sonar.cs.opencover.reportsPaths="%cd%\lcov.opencover.xml"
dotnet restore
dotnet build
dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=\"opencover,lcov\" /p:CoverletOutput=../lcov
dotnet sonarscanner end /d:sonar.login="%token%"

sonar.bat

Use this on Mac/Linux (bash) and remember give execution permissions:

#!/bin/bash
token="$(cat sonar.txt)"
dir="$(pwd)"
dotnet sonarscanner begin /k:"company:project" /n:"Project" /v:"#.#.#" /o:"company" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="${token}" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.cs.opencover.reportsPaths="${dir}/lcov.opencover.xml"
dotnet restore
dotnet build
dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=\"opencover,lcov\" /p:CoverletOutput=../lcov
dotnet sonarscanner end /d:sonar.login="${token}"

sonar.sh

Change company , project , #.#.# and Project words with yours.

Analysis Parameters - SonarQube Documentation - Doc SonarQube

Bonus Track

Tasks

Another loved feature is Visual Studio Code is that you can automate and create tasks and can be easily launched.

That’s my configuration task.json file. I have automated build, test, publish, pack and sonar tasks.

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "build",
            "command": "dotnet",
            "type": "process",
            "group": {
                "kind": "build",
                "isDefault": true
            },
            "args": [
                "build",
                "${workspaceFolder}/Project/Project.csproj"
            ],
            "problemMatcher": "$msCompile"
        },
        {
            "label": "build tests",
            "command": "dotnet",
            "type": "process",
            "group": "build",
            "args": [
                "build",
                "${workspaceFolder}/Project.Tests/Project.Tests.csproj"
            ],
            "problemMatcher": "$msCompile"
        },
        {
            "label": "test",
            "command": "dotnet",
            "type": "process",
            "group": {
                "kind": "test",
                "isDefault": true
            },
            "args": [
                "test",
                "${workspaceFolder}/Project.Tests/Project.Tests.csproj",
                "/p:CollectCoverage=true",
                "/p:CoverletOutputFormat=\"opencover,lcov\"",
                "/p:CoverletOutput=../lcov"
            ],
            "problemMatcher": "$msCompile"
        },
        {
            "label": "publish win",
            "command": "dotnet",
            "group": "none",
            "args": [
                "publish",
                "${workspaceRoot}/Project/Project.csproj",
                "-o",
                "${workspaceRoot}/Library/win/",
                "-c",
                "release",
                "-r",
                "win10-x64"
            ],
            "problemMatcher": "$msCompile"
        },
        {
            "label": "publish mac",
            "command": "dotnet",
            "args": [
                "publish",
                "${workspaceRoot}/Project/Project.csproj",
                "-o",
                "${workspaceRoot}/Library/mac/",
                "-c",
                "release",
                "-r",
                "osx.10.12-x64"
            ],
            "problemMatcher": "$msCompile"
        },
        {
            "label": "pack",
            "command": "dotnet",
            "args": [
                "pack",
                "${workspaceRoot}/Project/Project.csproj",
                "/p:NuspecFile=${workspaceRoot}/Project/Project.nuspec",
                "-o",
                "${workspaceRoot}/Package/",
                "-c",
                "release"
            ],
            "problemMatcher": "$msCompile"
        },
        {
            "label": "permissions",
            "type": "shell",
            "osx": {
                "command": "chmod +x ${workspaceRoot}/sonar.sh"
            },
            "presentation": {
                "reveal": "always",
                "panel": "new"
            },
            "problemMatcher": []
        },
        {
            "label": "sonar",
            "type": "shell",
            "windows": {
                "command": "${workspaceRoot}\\sonar.bat"
            },
            "osx": {
                "command": "${workspaceRoot}/sonar.sh"
            },
            "presentation": {
                "reveal": "always",
                "panel": "new"
            },
            "problemMatcher": []
        }
    ]
}

tasks.json

Change company , project , #.#.# and Project words with yours.

Travis CI

Run SonarScanner on Travis it’s a little problematic. The trick is to export the path when it’s installed.

In order to keep secret the sonar key, create an environment variable on Travis as SONAR_KEY.

language: csharp
mono: none
dotnet: 3.1.201

solution: Project/Project.csproj

install:
  - dotnet tool install --global dotnet-sonarscanner
  - dotnet restore Project/Project.csproj
  - dotnet restore Project.Tests/Project.Tests.csproj

before_script:
  - export PATH="$PATH:$HOME/.dotnet/tools"

script:
  - dotnet sonarscanner begin /k:"company:project" /n:"Project" /v:"#.#.#" /o:"companyname" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="$SONAR_KEY" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.cs.opencover.reportsPaths="lcov.opencover.xml" || true
  - dotnet build Project/Project.csproj
  - dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=../lcov
  - dotnet sonarscanner end /d:sonar.login="$SONAR_KEY" || true

cache:
  directories:
    - '$HOME/.nuget/packages'
    - '$HOME/.local/share/NuGet/Cache'
    - '$HOME/.sonar/cache'

Change company , project , #.#.# and Project words with yours.

GitHub Actions

In order to keep secret the sonar key, create an environment variable on Settings -> Secrets called SONAR_KEY.

Create this file /.github/workflows/build.yml on your project:

name: build

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

env:
  dotnet: 3.1.201
  version: #.#.#
  key: company:project
  organization: company:project
  name: Project

jobs:
  build:
    runs-on: ${{ matrix.platform }}
    strategy:
      matrix:
        platform: [ubuntu-latest, macos-latest, windows-latest]
    name: build on ${{ matrix.platform }}
    steps:
      - uses: actions/checkout@v2
      - name: setup .Net Core
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: ${{ env.dotnet }}
      - name: restore
        run: dotnet restore Project/Project.csproj
      - name: build
        run: dotnet build Project/Project.csproj --no-restore
  test:
    runs-on: ubuntu-latest
    name: test
    steps:
      - uses: actions/checkout@v2
      - name: setup .Net Core
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: ${{ env.dotnet }}
      - name: install sonar-scanner
        run: dotnet tool install --global dotnet-sonarscanner
      - name: restore
        run: dotnet restore Project/Project.csproj
      - name: build
        run: dotnet build Project/Project.csproj --no-restore
      - name: restore test
        run: dotnet restore Project.Tests/Project.Tests.csproj
      - name: build test
        run: dotnet build Project.Tests/Project.Tests.csproj --no-restore
      - name: scanner begin
        run: dotnet sonarscanner begin /k:"${{ env.key }}" /n:"${{ env.name }}" /v:"${{ env.version }}" /o:"${{ env.organization }}" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.login="${{ secrets.SONAR_TOKEN }}" /d:sonar.language="cs" /d:sonar.exclusions="**/bin/**/*,**/obj/**/*" /d:sonar.cs.opencover.reportsPaths="lcov.opencover.xml"
      - name: scanner build
        run: dotnet build Project/Project.csproj
      - name: scanner test
        run: dotnet test Project.Tests/Project.Tests.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=../lcov
      - name: scanner end
        run: dotnet sonarscanner end /d:sonar.login="${{ secrets.SONAR_TOKEN }}"
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}  

Change company , project , #.#.# and Project words with yours.

Commit and push the file into the master branch and see this script running on the Actions tab.

---

Test Case

If you want to see all this working together in a project, take a look at this GitHub repo Kata: TDD Arabic to Roman Numbers with C#.

---

That’s All Folks!
Happy Coding 🖖

Comments

[Camilo Martinez]

IDK, because I've used the same configuration and debug tests on the three OS, and works fine.

[Ronaldo Peres]

Nice to know, thanks for sharing!

[Manish Jain]

Nice Article. Is this possible to generate coverlet xml file using test project dll instead of csproj? Also, Is there any way to generate a single coverage file for multiple test case projects?

[Camilo Martinez]

I've never tried scan a dll file and to be honest sound weird to me because this practice it's more to check quality on source code and a dll file, in general, have compiled code.

On the coverage reports that I see on C# and JS, all of them create a report for each file and not for each test case.