With the publishing of Orleans.SyncWork, I’ve had the opportunity to explore GitHub actions - which is a way to automate workflows. Here’s some of my first experience into the “action” (groan).
Automated workflow
First things first, what even is a workflow, and what does it mean to automate one? Well dear potential reader, a workflow is nothing more than a set of steps taken to complete a task.
From Wikipedia:
A workflow consists of an orchestrated and repeatable pattern of activity, enabled by the systematic organization of resources into processes that transform materials, provide services, or process information. It can be depicted as a sequence of operations, the work of a person or group, the work of an organization of staff, or one or more simple or complex mechanisms.
You can think of a workflow as the steps taken to accomplish “something”. That “something” can be any number of things, related to any number of subjects. In the context of this post, we’ll be mostly covering workflows as it relates to a build and release pipeline, also commonly referred to as continuous integration (CI) and continuous delivery (CD).
I’d like to cover both the CI and CD aspects of the Orleans.SyncWork, so let’s get started.
What you’ll (probably) need
- Experience working with a CLI
- Unit tests
- An idea of the steps that you need to take in order to build, test, and deploy your code. If these steps are already in your head in the form of CLI commands, then you’re already most of the way there!
- Some patience in getting your workflow properly laid out
Continuous Integration
Before you’re able to deploy code through a work flow (continuous delivery), you need to be able to integrate it safely into your main/trunk. For dotnet, through a handful of CLI commands, the building and testing of code is pretty straight forward. Doing CI has the added benefit of bringing up a brand new environment for builds, each and every time, a similar idea to why I’ve been a proponent of build servers for so long.
Build
dotnet build
The above command is the minimum you need to build either a solution file or project file on the dotnet side of things. From a continuous integration perspective, you may want to throw a few flags onto the command, such as:
dotnet build --configuration release
and things of that nature, see what’s available to you with the dotnet build documentation.
That could look like this from the CLI:
Test
Next is testing. I’ve probably already said it too many times, but test your code! Especially if you’re building libraries! Tests help ensure that the code you’re writing does what you say it does. Additionally tests be used as “documentation” in a way, if the tests are named well, and are invoking the code in a similar manner to how your consumer will use it, they’ll be in a better place to get started using what you’ve delivered.
Like the build command, the test command is quite straightforward as well:
dotnet test
Of course the above is the absolute bare minimum command, there are lots of parameters that can be passed to dotnet test as well.
A test run could look like this from the CLI:
CI Action
With the above dotnet build and dotnet test commands, we have most of what we’ll need to put together an action to build and test our code, automatically!
There is lots of good information, even some specific to .net testing on the documentation. I pretty much used the documentation as a starting point, and ended up with this…
.github/workflows/ci.yml:
name: Build and test
on:
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Setup .NET
uses: actions/setup-dotnet@v1
with:
dotnet-version: 6.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build -c Release --no-restore
- name: Test
run: dotnet test -c Release --no-build --verbosity normal --filter "Category!=LongRunning"
The above file should be mostly straightforward, first we give a name to our workflow with name, specify the triggers for the workflow, in this case “on pull requests against the main branch”. The file then goes on to define the job “build”, which specifies an OS to run on, then steps. The steps do a few things of note:
- Checkout the code, placing it into the ubuntu instance that is being utilized from the step previous
- Setup .NET with a prebuilt action
- restore dependencies
- This is an explicit step, rather than implicit from our previous
dotnet buildcommand, as if this explicit restore step were to fail, we’d more quickly know at what point of the build there is a failure
- This is an explicit step, rather than implicit from our previous
- build the solution file at the project root
- finally test the code
We have a few new flags in our build and test commands, namely specifying a configuration of release, and don’t restore/build on steps after those steps having already occurred. One final note is the --filter "Category!=LongRunning" - I was having trouble with the test runner getting through the tests I had laid out. They took 3 minutes to run locally, but ran for over 25 minutes on the build agent. Due to this fact, I added some classifications of “category” to the longer running tests, and excluded them from the test run in the above ci.yml file.
Continuous Delivery
Continuous delivery is a lot like continuous integration, and builds on top of it. I’m of the thinking that CD should do everything CI should do, or perhaps even better, actually rely on the CI, rather than redefining the steps in your CICD like I ended up doing. That was a bit rambly, but CD should do everything CI should do, except with the additional step of actually delivering (deploying/pushing) the code as a part of its workflow.
Delivery Complexities
That delivery part can have a lot of nuance to it that ups the complexity by a significant amount when compared to just “CI”. What does it mean to actually deliver code? Well, that could depend a lot on what type of code you’re actually delivering. In my case, I’m delivering a NuGet package, which has its own complexities, but what else is there? Well the other obvious thing that comes to mind is a web site / web api, one which could potentially have database changes to roll out in addition to the code. This to me, has the potential to be worlds more complex than just pushing a NuGet package up. How do you not only handle failures, but detect them and roll back, in the case of something going wrong with either you web push or database push? Perhaps I’ll be able to explore that one day, but for now, let’s get back to the NuGet package.
So, is there a complexity with delivering a NuGet package? Yes. NuGet package versioning can be a big undertaking when it comes to manual deployments, much less CD; as there is a requirement of NuGet packages being immutable. Does this mean that for every check in, on every potential branch that will be pushed to NuGet, you need to update some text file or code to indicate the next built version? That was my initial thinking, but thankfully that is not the case with the help of Nerdbank.GitVersioning.
I don’t think I have my CICD set up exactly how I’ll end up having it, but for right now it works. I installed the NerdBank.GitVersioning tool and package, and now for each build, I get a unique version number for the NuGet package upon build. I can toggle between prerelease or release packages, and can even publish “nightly” builds that contain a commit hash on them, all in the name of uniquely identifiable NuGet packages.
There was a fair amount of setup, that in some ways I’m still working through, but this article is already getting long enough, take a look at the PR(s) if you’re curious: https://github.com/OrleansContrib/Orleans.SyncWork/pull/8 and https://github.com/OrleansContrib/Orleans.SyncWork/pull/13. The “tldr” of it is, nbgv tooling uses the git history to rev the version number being used during builds, allowing for unique build numbers each time the CI/CD fires.
CD Action
There’s been a fair amount of information so far, but between our CI action and the information about GitVersioning, we have everything we need to put together a “first pass” cicd.yml. For CI, we were doing builds/tests against PRs to main. For CD, we’ll want to do delivery when code is pushed to main, as well as branches that begin with “RELEASE/v*”. My thinking here is that since we’ll be integrating often into main (theoretically), we don’t necessarily want to create full “new release packages” for every commit to main. We could however, create “prerelease” NuGet packages to main for each commit, making those changes available to the NuGet feed, but them not being labeled as a release version. Otherwise I have Nerdbank.GitVersioning set up to release “release” versions of packages from the “RELEASE/v*” branches.
The CD action file itself will look very similar to the CI one, just with the few additions of dotnet nuget... commands, shown below:
.github/workflows/cicd.yml
name: Build, test, and deploy
on:
push:
branches:
- 'main'
- 'RELEASE/v**'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Setup .NET
uses: actions/setup-dotnet@v1
with:
dotnet-version: 6.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build -c Release --no-restore
- name: Test
run: dotnet test -c Release --no-restore --no-build --verbosity normal --filter "Category!=LongRunning"
- name: Pack
run: dotnet pack src/Orleans.SyncWork/Orleans.SyncWork.csproj -c Release --no-restore --no-build --include-symbols -p:SymbolPackageFormat=snupkg -o .
- name: Push to NuGet
run: dotnet nuget push *.nupkg --skip-duplicate -s https://api.nuget.org/v3/index.json -k ${{secrets.NUGET_API_KEY}}
In the above, you’ll notice that it’s more than 50% “the same file” as CI. I was going to potentially look into composite actions as some point, so see if I could instead “chain” CI and CD, rather than redefining the CI within the CD file; but I’ve not yet had a chance to explore that.
Aside from the change to the “on” event (pull_request -> push), there are two new commands at the bottom dotnet pack and dotnet nuget push. The dotnet pack command is used to “package” up the project specified into a “.nupkg” file (and snupkg in this case for symbols). Finally the dotnet nuget push command is used to push those newly packed NuGet package(s) to the feed specified of nuget.org. On this command you’ll also see the {{secrets.NUGET_API_KEY}} portion of the command, this is defined as a repository secret and it can be used to pass “secret” information to things like workflows, in this case it’s my NuGet API key. These secrets can be set from the repository “Settings” -> “Secrets”:
Still to do
The “putting out a release branch” is still a bit of a manual step for me. I need to run nbgv prepare-release from my local environment, then push up the subsequently created “RELEASE/v*” branch and updated new pre-release version that is created under main.
That may not have made sense.
If I’m working in main with a prerelease version of “1.0-prerelease”, when I nbgv prepare-release, main will be (as an example) updated to “1.1-prerelease” with a branch called “RELEASE/v1.0” created having a release version of “1.0”. The push of these two changes will currently build a new prerelease package of “1.1-prerelease” and release package of “1.0”, both of which will contain “the same content” at the time of being pushed.
I’m not sure how I feel about the above. I like the automatic build and deploying of packages, but I don’t like having to create the release locally. I could conceivably create a manually dispatched workflow that did this release preparation for me, but then there’d still be the slight strangeness around immediately pushing out a prerelease package with no changes in comparison to the “previous” pre-release package built, and the new release package being built. I’m not sure what the “right flow” is quite yet, what I have right now does work, it just seems a bit messy.
Perhaps I’ll eventually look into workflows more like this:
- CI - continues to be run on PRs to main
- CICD - can be executed against the main branch and “RELEASE/v*” branches, but is not done automatically as it currently is
- I’m not sure about this part, main and the release should theoretically always be deployable, but do I really want to deploy every time there’s a change to main…?
- Prepare release workflow - with this workflow, I’d want to do the “local” steps I currently take for preparing a release, but do it through a github action.
My Workflow
I wasn't sure if the literal sections were required, so here they are, "My Workflow" I think was pretty well described above, hopefully!
Submission Category:
- Maintainer Must-Haves
- DIY Deployments
Yaml File or Link to Code
OrleansContrib
/
Orleans.SyncWork
This package's intention is to expose an abstract base class to allow https://github.com/dotnet/orleans/ to work with long running CPU bound synchronous work, without becoming overloaded.
This package's intention is to expose an abstract base class to allow Orleans to work with long running, CPU bound, synchronous work, without becoming overloaded.
Built with an open source 
license, thanks Jetbrains!
Project Overview
There are several projects within this repository, all with the idea of demonstrating and/or testing the claim that the NuGet package https://www.nuget.org/packages/Orleans.SyncWork/ does what it is claimed it does.
The projects in this repository include:
- Orleans.SyncWork
- Orleans.SyncWork.Tests
- Orleans.SyncWork.Demo.Api
- Orleans.SyncWork.Demo.Api.Benchmark
- Orleans.SyncWork.Demo.Services
Orleans.SyncWork
The meat and potatoes of the project. This project contains the abstraction of "Long Running, CPU bound, Synchronous work" in the form of an abstract base class SyncWorker; which implements an interface ISyncWorker.
When long running work is identified, you can extend the base class SyncWorker, providing a TRequest and TResponse unique to the long running work. This allows you to create as many ISyncWork<TRequest, TResponse> implementations as necessary, for all…
Top comments (0)