Kurt

Posted on Aug 9, 2020 • Edited on Sep 18, 2021

Testing .NET Core Apps with GitHub Actions

#dotnet #github #devops #csharp

In the following post I’m going to cover how you can run and publish results for your .NET Core Tests using GitHub actions.

Updated 2021 to replace deprecated 'set-env' usage with new command

In particular we will be setting up a CI pipeline action to publish test results in Markdown format as both a comment on pull requests (PR), and a build artifact. Running tests automatically on PRs is a good way to make sure you don’t push code that breaks your project.

To make the most of .NET Core’s cross platform builds, and while maximizing GitHub Actions free tier, we will be using the Linux (Ubuntu) runner. At the time of writing this, the GitHub free tier gives you 2000 minutes per month on Linux runners, if you use Windows this drops to 1000 minutes, and macOS only 200 minutes.

Getting Started

In order to run tests on GitHub we will need these prerequisites;

GitHub account You can join here if you don't have an account
GitHub Repository If you don’t have a repo, take a look at the following article to get setup: Create a repo
Repository cloned locally If you don’t already have a setup with favorite git client, try the official docs: Cloning a repository

Setting up your .NET Core test project

The following commands should be ran in your project folder, so make sure to open up the command prompt in the directory you have checked out your project to.

Optional Step. If you already have a test project you can skip to Step 1.
Otherwise, we will create a new test project now with the following command:

dotnet new xunit

You should see a newly created test project

Step 1. Add the Markdown logger to your test project

We will be utilizing the LiquidTestReports.Markdown VS Test logger to generate a test report in the Markdown format (disclaimer: I am the creator of this library)

You can add the library either by getting it from nuget, or running the following command:

dotnet add package LiquidTestReports.Markdown

Step 2. Run the tests locally

Make sure the tests will run on your machine with the test logger attached, by running the following command:

dotnet test -l liquid.md

You output should look similar to the image below, note the “Saved report” line. If you check in your project directory you should have a new TestResults folder with the newly created TestReport.md file (preview shown below).

VS code showing dotnet test console and report preview:

You can delete the report for now, as we want to generate them inside the actions each time.

Step 3. Commit and Push your code

Now that your test project is ready from the previous steps, it’s time to get it onto GitHub itself.

To do this, commit your local changes including the test project referencing the markdown logger. And push your branch to your remote GitHub repository.

Once again, leaning on the official GitHub help docs, for more info, take a look at Committing and reviewing changes to your project

Creating your first PR

With your code now pushed to the remote, jump back on your GitHub repo, and you should now see a panel saying your branch has recent pushes, click the green 'Compare & pull request' button.

You should now see the create pull request page.
Add a comment describing the changes e.g. 'Add new test project' and press the green 'Create pull request' button.

You should now see the pull request page. For simplicity, we will just go straight to pressing the green 'Merge pull request' button
followed by the 'Confirm merge' button.

To keep things clean you can also press the 'Delete branch' button.

With this all done, we are ready to move on to setting up the action.

Setting up the Github Action

In your browser

Go to your GitHub repo and click the ‘Actions’ tab
On the Actions tab, you will see lots of workflow options, at the top click ‘Skip this and set up a workflow yourself’
You will now see the yaml editor with a default CI pipeline. Delete the contents of this, so we can start the pipeline from scratch

The GitHub Action

Starting with the setup, we will add something similar to the default dotnet workflow. Here we are setting up the environment, restoring the nuget dependencies, and building the project.

You may notice the 'on' conditions at the top, there are two scenarios that we want to trigger a build

A PR has been created to merge into the master branch

A push to the master branch, created by a merge commit

name: .NET Core Build with Tests

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    # use ubuntu for more build minutes
    runs-on: ubuntu-latest
    # use release mode for all steps
    env:
      config: 'Release'

    steps:
    - uses: actions/checkout@v2
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.101    

      # restore dependencies  
    - name: Install dependencies
      run: dotnet restore

      # build project
    - name: Build
      run: dotnet build --configuration $config --no-restore

Next, we will be conditionally setting some variables to set the test report name and title, for both cases, we want to include the run number, so we know which build the test results are for. If it is a pull request, then we additionally want to know the pull request number.

      # set pr number, if it's a pr build
    - name: set pr build number
      id: PRNUMBER
      if: ${{ github.event_name == 'pull_request' }}
      uses: kkak10/pr-number-action@v1.3

      # set report file and title 
    - name: Set Test Title
      run: | 
            if ${{ github.event_name == 'pull_request' }}
            then
              echo "title=Test Run for PR #${{steps.PRNUMBER.outputs.pr}} (${{github.run_number}})" >> $GITHUB_ENV
              echo "file_name=TestReport.${{steps.PRNUMBER.outputs.pr}}.${{github.run_number}}.md" >> $GITHUB_ENV              
            else
              echo "title=Test Run ${{github.run_number}}" >> $GITHUB_ENV
              echo "file_name=TestReport.${{github.run_number}}.md" >> $GITHUB_ENV
            fi

With this set, we can now move on to running the tests, this is using the same dotnet test method as earlier in the post, but now passing in some parameters for the title, and file name.

  # run tests with built project
- name: Test PR      
  run: dotnet test --no-restore --no-build --configuration $config --logger:"liquid.md;LogFileName=${{github.workspace}}/${{env.file_name}};Title=${{env.title}};"

Now the report is generated, we want to store it, this next step will attach the markdown file as a build artifact.

  # upload report as build artifact
- name: Upload a Build Artifact
  uses: actions/upload-artifact@v2
  if: ${{always()}}
  with:
    name: 'Test Run'
    path: ${{github.workspace}}/${{env.file_name}}

Lastly, we want to add one final step, for pull requests. This step will attach the contents of the markdown report as a comment to your pull request.

  # add report as PR comment (if PR)
- name: comment PR
  uses: machine-learning-apps/pr-comment@master
  if: ${{ github.event_name == 'pull_request' }}
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  with:
    path: ${{env.file_name}}

A complete copy of the workflow is available here on a gist.

Time to commit
Pressing the green "start commit" button

There's two options here, commit to master branch, or create a branch.

I'm going to:

select the 'new branch' option
set the branch name to ci/test-workflow
press the green 'Propose new file' button

You should now see the familiar 'Open a pull request' page. Once again, press the green button to create the pull request.

With the pull request created, switch over to the 'Actions' tab. You should now see your workflow, and after a minute or so of running, a green tick next to the workflow result.