DEV Community: Kosala (KP) Perera

Migrating Service Fabric apps to .NET 6

Kosala (KP) Perera — Mon, 10 Apr 2023 09:11:46 +0000

Migrating Service Fabric apps to the .NET 6 suck! Not only is there no opensource community engagement, but there's no same-day support for new .NET versions at all. What a SHAM! There's nothing I can do about that.

The Alertbox Inc. (@alertbox) pipelines page is open on my work PC and one of the CI pipelines is broken. For the record, Alertbox has ~60 microservices. They require Service Fabric / .NET Core combo to build and release. I'm going to go ahead and say that they use windows-latest on hosted agents. And the Azure Pipelines team is upgrading the .NET used by agents, so they have no way of running pipelines without adding the UseDotNet task to select what .NET version is to use in the pipeline agents. Maybe that's a plausible fix, but they recommend keeping the source code up-to-date with the current LTS version of .NET. So what does each microservice have that must be upgraded for .NET 6?

Service Fabric app

The first thing we're going to need is the latest update for Service Fabric. And that means runtime, SDK, and the tools to create, build, and run reliable service apps on a local cluster. Unfortunately, the IIS team recently sunset the Web PI, so the Service Fabric team bundled the latest releases with Visual Studio, and I'm going to have to update Visual Studio to download them for me.

The other thing we're going to need is a global.json file since there are several .NET versions installed already in my work PC, so it's definitely necessary to break out that dotnet driver and create a global.json file to select net6.0 SDK version that is in.

#!/bin/zsh
# Show installed SDKs.
dotnet --list-sdks
# Add a global.json file
dotnet new globaljson --sdk-version 6.0.403

I'm a bit of a control freak, not gonna lie. I like the idea of knowing what's happening underneath. It's slightly obnoxious that Service Fabric project templates are not opensourced yet. I can create a new reliable service app on .NET 6 to figure out what's new in the scaffolding project.

The first on the list is the entry project .sfproj. They use a non-SDK-style project template with a bunch of .xml files for configuration but with no C# code. They require Fabric.MSBuild nuget to build and package Service Fabric apps. Unfortunately, the dotnet add package command won't update dependencies since they only support <PackageReference /> and non-SDK-style project template uses the package.config file to manage dependencies.

<!-- packages.config -->
<package id="Microsoft.VisualStudio.Azure.Fabric.MSBuild"
         version="1.7.7"
         targetFramework="net48" />

I can update the package.config to use the latest versions but there's more to do. The .sfproj still points to the older build tools. And you do want to use the latest version to build and package.

Updating the .sfproj is extraordinarily uncomplicated. All we need to do is figure out the references to the Fabric.MSBuild package and replace the version numbers, say, find all and replace the version number to 1.7.7 from the version that is already been used. Then I can replace the <TargetFrameworkVersion /> version with the .NET Framework 4.8 and we're done.

The .xml configurations need no changes, so we can ignore them for now.

Services and dependencies

Next on the list are reliable services. This covers a lot of ground since Alertbox uses multiple stateless services bundled into a single Service Fabric app, say, Web APIs and Console apps, and they target netcoreapp3.1 to build and run, so we're going to have to upgrade them to net6.0. I can replace <DefaultNetCoreTragetFramework /> in the Directory.Build.props file like they do on @dotnet repos on GitHub, but there's more to it. They need to upgrade dependencies.

Figuring out dependencies to upgrade for all the .csproj projects that are in the code repo is basically the same as listing installed SDKs.

#!/bin/zsh
# First restore packages
dotnet restore && \
# Then scan for outdated packages
dotnet list package --outdated

I can check and review each package to decide whether or not to upgrade.

#!/bin/zsh
dotnet add package Azure.Messaging.ServiceBus

It's slightly obnoxious that they only let you check for the status, so I'm going to have to upgrade outdated packages one at a time, say, migrating Web APIs and the Automapper configurations to the latest versions by going thru what's new and migration docs. It's taking a few tries to get the unit tests green since it demands highly intense concentration on breaking changes of each package upgrade.

Troubleshooting

Visual Studio can build from source but it keeps failing to deploy the Service Fabric app onto the local cluster. I'm going to go ahead and say that they use Deploy-FabricApplication.ps1 powershell script in the .sfproj to deploy, so replacing it from the script in that new boilerplate code to take care of it seems like the right answer.

Then I can build and run the app from source and Visual Studio will do the rest for me. But there are more problems.

The Visual Studio debugger won't attach. This may be difficult. I'll come back to this later.

This may not be an issue with Service Fabric since it works on my colleague's PC, so it's definitely necessary to diff both development environments to figure out what SDK version to use.

Service Fabric creates dynamic proxy classes for service remoting, but the .NET runtime team has turned off metadata generation for dynamic classes in .NET 6, so it would be impossible to tell Visual Studio debugger how to load metadata for dynamic proxy classes that Service Fabric creates. The .NET team has a release that always generates metadata, so a little winget install magic is all that's necessary to update the .NET 6 installed locally.

#!/bin/zsh
# Display the status of the installed SDK.
dotnet sdk check
# Install/upgrade the SDK via Windows Package Manager.
winget install Microsoft.DotNet.SDK.6
# Show installed SDKs.
dotnet --list-sdks

But there's more to do. We need to update the global.json with the new SDK version.

// global.json
{
  "sdk": {
    "version": "6.0.407",
    "allowPrerelease": false,
    "rollForward": "latestMinor"
  }
}

Then I can build and run from source and Visual Studio debugger will do the rest for me. And if version >=6.0.407 is not installed locally, nothing gets compiled.

Pipelines

I'm not exactly sure when the Azure Pipelines team is going to remove .NET 6 once it reaches end-of-life support, so I'm going to have to update the build steps to always select the .NET version used in the code repo. All I need to do is push all changes to a feature branch to raise a pull request and the branch policies will trigger the CI pipeline for me. Excellent!

/KP

Selecting .NET version to use

Kosala (KP) Perera — Tue, 04 Apr 2023 06:58:12 +0000

I'm a little procrastinated, not gonna lie. The Alertbox Inc. (@alertbox) pipelines page is open on my work PC and one of the CI pipelines is broken. What a SHAM!

It's a little weird since the pipeline logs show the target .NET version net5.0 isn't available, but the changeset seems to work locally. Not only are there no tests failing, but there are no errors at all. For the record, Alertbox has several hundred git repos and pipelines that they run quite often on hosted agents. The truth is they have not upgraded the .NET used by the code repos to the latest version, so they require several .NET versions installed locally.

Maybe they don't have .NET end-of-life support versions on hosted agents anymore? Yup! The Azure Pipeline team is upgrading the .NET used by pipeline agents. I almost want to upgrade the .NET used by the code repo to the latest LTS version, say, replacing TargetFramework to net6.0 from net5.0 and running dotnet list package --outdated to see what packages are to upgrade. You'd be amazed how problematic is it to have several .NET versions running locally since the dotnet driver selects the latest SDK installed locally for commands and not from the entry project / solution that's in, so it would be difficult to figure out what packages are to upgrade without going thru all of them one at a time completely. And .NET Core 3.1 will be end-of-life support soon, so that means more code repos to upgrade.

It's not such a great idea and probably not even pragmatic to go thru all of that Jazz and upgrade each one since you can only boil so many oceans at the same time, so it's definitely necessary to break out that pipeline and modify the YAML script to install any end-of-life support .NET versions.

#.azuredevops/steps/build.yaml
steps:
# Install .NET from global.json
- task: UseDotNet@2
  inputs:
    packageType: 'sdk'
    useGlobalJson: true

All I need to do is add a UseDotNet task as a prerequisite for the build step and it will install .NET on pipelines for me. But there are more problems. They require a global.json file to figure out what SDK version is to install.

// global.json
{
  "sdk": {
    "version": "5.0.408",
    "allowPrerelease": false,
    "rollForward": "latestMinor"
  }
}

I can run dotnet --list-sdks to get the SDK version that is running on my work PC, and then run dotnet new globaljson to create the global.json file to mention that SDK version. Then I can run the pipeline and the UseDotNet@2 task will take care the rest for me. Excellent! Moving right along.

# Trigger the pipeline
az pipelines run \
   --branch kp/fix-23-dotnet-not-found \
   --folder-path ci \
   --name ca-cocktails \
   --output table \
   --open

# Check for build status
az pipelines runs list \
   --top 3 \
   --status all \
   --branch kp/fix-23-dotnet-not-found \
   --output table

It's taking a few tries to get the pipeline green since some of the dependencies and C# features don't seem to recognize anymore since .NET 5 only supports C# 9 and not latest version. This may be difficult. I'll come back to this later!

/KP

Development containers with Docker

Kosala (KP) Perera — Wed, 22 Mar 2023 08:54:30 +0000

Docker Team has deprecated native dev environments feature. They removed references and archived the @docker/dev-environments repo.

I need to build an eng-practice wiki for Alertbox Inc. (@alertbox) and most of these contents are pretty outdated word docs and slide decks. It's not such a great idea and probably not even fun to begin with since, Alertbox has strict security policies on work PC.

For the record, not only are there no administrative rights for developers, but there's no permission at all. The PC is a Brick, as in barely anything you can do with it. The truth is, I'm way too lazy to go thru their intense approval process to install and configure a dev environment. I need to keep the Hacktivation Effort to minimal.

Dan Wahlin's Docker for Web Developers course is open on my work PC and he is showing how to map the source code into a docker container. Good idea Mr. Wahlin!

I'm not exactly sure how the Alertbox's wiki is going to fit into this whole thingy, but I like the idea of developing inside a container. It gives the whole thing a very Homogenize feel since each teammate's work PC will be more implicit than, say, installing all the prerequisite dev tools on their work PC like we typically do. Easy enuf, except, I need to bake in development environments as part of source repos.

The first thing we're going to need is a markdown parser to generate the site like they do in Microsoft Docs sites. And that means Docsify. They are basically similar to Jekyll, but with no build steps. All I need to do is fork their docsify-template repo and we're good to go!

The other thing we're going to need is a dead simple node container to run docsify locally, so a little docker run magic is all that's necessary to sync the repo contents and spawn the container. Then I can run npx serve to spin up an HTTP server to preview on localhost and docsify will parse the markdown to generate the site on the fly for me. Kids' stuff!

#!/bin/zsh
docker run --name stoic_goldstine \
           --rm \ # clean up
           -w /workspaces \ # working directory
           -v "$(pwd):/workspaces" # sync immediate file changes to container
           -p 3000:3000 \ # expose port 3000
           node:lts-bullseye \ # node image
           npx serve . --listen 3000 # run the http server in port 3000

Unfortunately, docker runs in the foreground and requires another Terminal to stop to and remove the container. It would be impossible to run in detached -d mode without giving away docker clean up --rm option completely. And you do want docker to automatically clean up when the container exit. It's time to switch to Docker Dev.

I prefer docker dev over docker. They're technically the same as docker compose, except docker dev is specifically for dev environments. It also requires a compose YAML file to configure the container, so you don't need to memorize which image to pull, what ports to expose, the working directory to map, args to override, et al. All I need to do is break out the same docker command that I use to spawn the container in detached mode to create a compose-dev.yaml file and we're all set.

# ./compose-dev.yml
services:
  node:
    image: node:lts-bullseye
    container_name: stoic_goldstine
    ports:
      - 3000:3000
    working_dir: /workspaces
    volumes:
      - ./:/workspaces
    command: /bin/sh -c "npx serve . --listen 3000"

Then I can use docker dev commands to spawn the dev environment and docker will do the rest for me. Excellent!

#!/bin/zsh
# Spin up the dev environment
docker dev create $(pwd)
# Stop the dev environment
docker dev stop devcontainers-try-docsify-hungry_elion

Note

Docker Dev CLI is handy if you would want to stick to the Terminal. Use Docker Desktop or the Browser Extension, to create and launch dev environments otherwise.

I use Typora for markdown but we're going to have to change the default theme. They require intellisense and syntax highlighting to work with HTML/CSS and JavaScript. I can use VS Code with a few extensions but there are more problems.

Not only is there no language recognition, but no intellisense at all. I'm going to go ahead and say that they don't have access to the node runtime inside the container, so they have no way of knowing what syntax highlighting and intellisense to show. And you do want to them to work. Time to install the Remote - Containers extnesion and Attach to Running Container. Then I can install the required extensions for node and JavaScript. It's slightly obnoxious that VS Code doesn't install recommended extensions from the .vscode/extensions.json automatically, so I'm going to have to install recommendations manually.

Unfortunately, there's no simpler way of converting word docs and slide decks into markdown so I'm going to have to copy all the contents from the individual directories that are in. And there's no way I'm going to go to 100 pages and slides to create markdown content one at a time, so it's definitely necessary to push this initial content to a git repo and let the teammates contribute.

/KP

PS: For the complete work, see my devcontainers-try-docsify repo on GitHub.

Non-opinionated code review guidelines

Kosala (KP) Perera — Fri, 02 Sep 2022 05:53:17 +0000

I'm not going to lie, opinions are like Arseholes, everyone has one. It would have been acceptable but the objectives of code reviews are simple. In general, not only is to get the best possible pull request, but also improving skills of the fellow contributors. Easy enuf, except we need a set of ground rules.

Etiquette

The first thing we're going to need is a contributor etiquette. I like the idea of Jante Law. It gives everyone a very Egalitarian feel since you are not to think you're anyone special or you're better than everyone else, as they teach in Scandinavian schools.

Workflow

The next is the release engineering strategy. All we need is a version control tool and git is the mainstream thingy for that. But some of the branching strategies are pretty horrendous so you have to keep the branch integrations to minimum.

Trunk-based development seems extraordinarily uncomplicated but there are more to consider.

Branch policies for protecting the default branch.
Naming conventions for branches.
Pipelines for release engineering.

Changesets

Some organizations often call this a Pull request or a Changelist. And it has to be small-enuf to one self-contained change. Unfortunately, no playbook has a recipe for this so I'm going to have to let individual contributor decide the size of the pull request.

Once the pull request is ready to review, they have to describe it, specifically what has been done and why it was made as-is. This can be tricky, so a conventional-based description seems like a great way of adding structure and meaning to it.

Design and Health

This covers a lot of ground and can be challenging enuf. Unfortunately, you can only boil so many oceans at the same time, so what does each reviewer should look for in a pull request? In general, readability and maintainability would hint a few mistakes here and there, but there are more to review.

Intended functionality and the design.
Over-engineering and complexity.
Clear naming and conformance to style guide.
Parallel programming and thread safety.
Sufficient exception handling and logging et al.

Style Guide

Yes, a homegrown standards and conventions isn't such a great idea. The major issue is that it would demand an intense concentration to keep up with each tech stack without giving away the egalitarian culture completely since you would want to Dictate-It-All. And you do want contributors to maintain consistancy of the code. This may be difficult. I need a community-adept style guide to start with.

Faster Reviews

A pull request won't get reviewed itself. And a contributor must intervene at some point. I can review shortly after it comes in. But that seems a little unproductive since it takes time to get back to The Zone after been interrupted. So how fast should code reviews be? One business day or 24 hours sounds reasonable but there is more to this.

Professional courtesy when pointing out problems.
Code analytics for obvious smelly codes, anti-patterns, conformance to standards, et al.

Summary

All we need is non-opinionated guidelines for all kinds of contributors, so in doing a code review, you should value for:

Technical facts and data over opinions and personal preferences.
Style guides over personal preferences.
Software design principles over opinionated-design.
Consistency of the code over personal preferences.
Team velocity over personal velocity.

/KP

PS: You can find more about the topic in my curated list of standards, practices, guides, and discussions on my awesome list #33.

Writing pipes and filters for fluent OR-mapping

Kosala (KP) Perera — Mon, 15 Aug 2022 09:13:37 +0000

I like the idea of fluent interfaces. It gives the whole thing a very Ubiquitous feel since the code becomes more understandable, say, two months from now like Uncle Bob says in the Clean Code.

// An example LINQ expression to filter all active products
// that belongs to a specified supplier.
db.Products
  .Where(p => !p.Discontinued && p.SupplierId == supplierId)
  .Take(10)
  .ToList();

Simplifying query expressions

The first thing we are going to need is a lot of query expressions. I can decompose or consolidate query expressions so a little extract method magic is all that is necessary to get started. Kids' stuff.

// Better readability.
db.Products.Where(p => Active(p) && WithSupplier(p, supplierId));
// Better resuability.
db.Suppliers.Where(s => s.Products.Any(p => Active(p));

That seems a little better since it allows reusing the query expressions. LINQ may look like the best thing that happened for .NET, but it's getting all kinds of help from our friends at C#. It's the extension methods.

Chaining query expressions

The other thing we're going to need is chaining. And that means extension methods. It's slightly obnoxious that extension methods require non-nested non-generic static classes. So what does each predicate have that can be used for chaining that the fluent interface requires? IQueryable?

// Use filters to create the pipes.
db.Products.Active().WithSupplier(supplierId)
  .Take(10)
  .ToList();
// Extension methods for `IQueryable<Product>`.
static class IQueryableOfProduct
{
    static IQueryable<Product> Active(this IQueryable<Product> query)
        => query.Where(p => !p.Discontinued);
    static IQueryable<Product> WithSupplier(
        this IQueryable<Product> query, int supplierId)
        => query.Where(p => p.SupplierId == supplierId)
}

I can pass in the IQueryable<> domain entity type as the first parameter on which the method must operate. All I have to do is add the this modifier to the parameter and the C# compiler will do the rest for me. I'm not going to lie, you'd be amazed how tricky it is and if you return the wrong type, chaining breaks and nothing gets compiled.

Describing pipes and filters

Naming things is hard. But there are more problems. Not only are the extension methods should be reusable, but they must be understandable. And you do want the code to scream what kind of business there is. Using a domain-specific language to describe methods seems like the right answer. Indeed.

/KP

Hello world

Kosala (KP) Perera — Sat, 13 Aug 2022 06:17:39 +0000

GitHub used to be my unicorn and now it is my bitch.

I interact with GitHub more often than Twitter or LinkedIn, so why not use it for everything else, say, todos, bookmarks, links, ahaa thoughts, notes for future me, journaling, and the like? It makes more sense, at least for me. I'm that nerd 🤓

The truth is I tried out all those widespread productivity and notes-taking apps from Evernote to OneNote, Workflowy, Trello, Todoist, Asana, and even Notion. For the record, they all seemed great apps but this is just my personal preference.

About the site

In case you were wondering, these journal notes are:

Written on a Mac in Colombo, Sri Lanka.
Written in Markdown with Typora app.
Served with GitHub in a public repo.
Published on Hashnode.
Synced to DEV.to via RSS
Originally, a flavor of my Developer Journal.

You can find more about my work on my GitHub Page.

For comments or questions, feel free to ask me on GitHub, on Twitter, or via email at kosala dot nuwan at gmail dot com.

/KP