DEV Community: Ghassan Karwchan

Design and implement Git branching strategies for CI/CD integration

Ghassan Karwchan — Sun, 22 Sep 2024 22:57:42 +0000

Establishing a standardized development process that integrates a Git workflow with Continuous Integration/Continuous Deployment (CI/CD) is crucial before writing any code. This post will explore a widely-used approach: the Trunk-Based Development Workflow, which is adopted by many leading companies, including Microsoft.

Choosing a branching strategy:

Every team should adopt a consistent code release process to maintain quality and reduce risks. This starts with selecting a Git workflow, continues through integrating CI/CD, and culminates in defining a release strategy.

In this post, we’ll focus on the Git workflow, starting with the basic steps of a typical release flow: creating branches, pushing changes, submitting pull requests, and merging code.

Choosing the right Git workflow for your team can significantly boost productivity. Consider the following when selecting the best strategy for your team:

Scalability: Will this workflow scale as the team grows?
Error Recovery: How easy is it to recover from mistakes?
Overhead: How much additional work, such as resolving merge conflicts or waiting on CI/CD, will the workflow introduce?

What is Trunk-Based Development?:

The core principle of Trunk-based Development Workflow is that all development work happens directly on the main branch (commonly called “trunk” or “master”). This method encourages continuous integration, with developers frequently committing small, incremental changes. Automated testing and integration play a pivotal role in maintaining high code quality.

Notably, Microsoft uses this workflow internally, and it's a slight variation of the GitHub flow.

Branching:

For every new feature or bug fix, a developer creates a dedicated branch. Once the work is complete, this branch will be merged into the trunk via a pull request.

Pull Request with review policies:

After completing their work, developers submit a pull request (PR) to merge their branch into the trunk. Direct check-ins to the trunk are not permitted—merging must happen through a PR.

To ensure code quality, additional policies can be enforced on PRs:

Mandatory code reviews by other developers

All PR comments must be resolved
CI/CD builds must pass all tests (unit tests, code scanning) before merging
This process safeguards the integrity of the codebase while maintaining a consistent flow of changes.

Releases and Release branches:

When it’s time to release code to production, a Release branch is created. This branch is then deployed to production.

If a bug is found in production, the fix is made on the trunk using the same process: a bug-fix branch is created, a PR is submitted, and after merging into the trunk, the fix is cherry-picked into the release branch for deployment.

(Image above courtesy for Microsoft Learn)

Branche policies and permissions:

The Trunk-Based Development workflow allows for the enforcement of policies to enhance security, code quality, and team efficiency.

* Branch hierarchy and permissions

A structured branch hierarchy can help maintain organization and control. For instance, user-created branches could reside in /users or /features, while release branches live under /releases. Permissions can be set accordingly—only administrators would have the authority to create or modify release branches.

Enforcing Branch policies:

To further ensure code quality, the following policies can be applied to the trunk and release branches:

Mandatory Code Reviews: Every PR must be approved by another developer before it can be merged.
Successful Builds: CI processes ensure that a PR’s build passes unit tests or code scans before allowing a merge.
CD Automation for Releases: Creating a release branch can trigger automated deployment to a pre-production environment, along with performance and stability checks.

Enhancing Your Git Workflow with Git Credential Manager Core

Ghassan Karwchan — Sun, 22 Sep 2024 22:00:16 +0000

Git’s credential helper is a handy tool that saves your credentials, so you won’t need to re-enter your username and password for every git pull or git push. However, depending on the age of the tutorial you're following and the platform you're using, the instructions for setting up credential storage may vary.

In August 2021, GitHub introduced a new cross-platform tool called the Git Credential Manager Core. Let’s explore how this tool can streamline your team’s workflow.

Before Git Credential Manager Core:

Historically, Git offered several options for managing credentials, which you can see their list here, and we’ll briefly review:

Git Credential Store:

Before Git Credential Manager Core, the only cross-platform option was Git Credential Store, which stores the credentials as plain-text.

With the rise of two-factor authentication (2FA), this method became obsolete since it could not handle 2FA.

Platform-Specific Encryption Store:

Credential storage methods that used encryption were platform-specific, relying on the operating system’s native encryption capabilities. These included:

git-credential-osxkeychain for Mac: which use Mac's keyChain to store the credentials.
git-credential-libsecret for Linux: which use Linux secret service.
git-credential-wincred for Windows: which use Windows Credential Manager.
deprecated Git Credential Manager for Windows: This tool was deprecated after the Core one was created, and it is no longer maintained.

Other special stores:

There are two stores that are less used:

git-credential-cache (not for windows): which stores the credential in memory for 15 minuntes.
git-credential-oauth for Linux: which use OAuth authentication.

The arrival of Git Credential Manager Core:

As mentioned above, there wasn’t a single tool that could both encrypt credentials and work across all platforms—until Git Credential Manager Core was introduced. This tool addresses both of these needs: it's cross-platform and encrypts credentials securely.

Although it was created by GitHub, it has since been adopted by the broader Git community and integrated into Git installations for Windows as well.

How to set it up.

If you installed git on your machinge with version 2.39 and later, then you installed that tool by default.

To know if you have that tool, check the following:

git config --global credential.helper
## the output depend on the version you use

# for Git 2.38.1 on Windows , core was called manager-core
credential.helper=manager-core

# Git 2.39+ core replaced Windows manager and it took its name : manager
credential.helper=manager

If the tool you are using is not one of the above, then you can change it as follows:

# For Git versions up to 2.38.1 on Windows, the credential helper was called manager-core
git config --global credential.helper manager-core

# Starting from Git 2.39+, the helper was renamed to 'manager'
git config --global credential.helper manager
credential.helper=manager

By configuring Git Credential Manager Core, you ensure a smoother, more secure authentication experience, freeing up time to focus on what matters most: coding.

Are you still using git checkout for everything? It is time to switch to git switch

Ghassan Karwchan — Sat, 21 Sep 2024 23:23:08 +0000

If you're like me and still rely on the trusty git checkout for a variety of tasks—like switching branches, restoring files, and checking out commits—then you're in for a treat. The Git community has made things easier by introducing two new commands to split the workload: git switch and git restore.

These new commands were created to reduce the overload on git checkout and make Git a bit more intuitive, especially for developers who sometimes found it confusing. Let's break it down!

What git checkout used to do

The git checkout command was a Swiss Army knife in Git. It handled several different jobs, which could sometimes be confusing. With git checkout, you could:

Switching branches:

git checkout <branch-name>

Create a new branch and switch to it:

git checkout -b <new-branch-name>
## or create a branch from a specific commit
git checkout -b <new-branch-name> <commit-hash>

Restoring files from the stage area Files that were modified and added to staging area can be brought back to last

git checkout <file>

Restoring files from specific commit

git checkout <commit-hash> --file

Checking out the code to a specific commit or tag: It is as well called detached HEAD, because the HEAD pointer will point to a specific commit instead of the latst commit in the current branch:

git checkout (<commit-hash> | <tag-name>)

Why `checkout` was so overloaded

In Git, the branches tags and commit SHAs, all are under the hood the same thing, which they are refs (or references) in the log history, so having one command to manage all does make sense technically.
But from a developer workflow viewpoint (where branch is different than tag), bundling them into one command could cause confusion, especially for newer users. You might ask, "Why does git checkout work in some situations and not others?" For instance, sometimes it works with staged files, sometimes with uncommitted changes, and other times it doesn't unless you commit first.

Because git checkout had to handle so many different tasks, the Git community decided to split its functionality. The command still works for backward compatibility, but now we have two specialized commands that simplify common tasks: git switch and git restore.

How to simplify your Git workflow

With Git 2.23, two new commands were introduced to ease the confusion: git switch for handling branches and git restore for managing files. These commands break down checkout's responsibilities and make Git operations clearer.

git switch

git switch is a command for switching branches.

So you have a command to switch to another branch

git switch <branch-name>

Or create a new branch from existing status

git switch -c <new-branch-name>

git restore

git restore is focusing on restoing files from specific commit, or disregard the changes in staging, or working area.

restore will restore the files from HEAD content, but without touching the index/stage area, but you can override that as we will describe down:

git restore <file>
# or restore all files from HEAD content without touching index/stage area
# to compare to checkout, which revert both working tree and stage area
git restore .
# to revert the changes in index/stage area only use the following override argument
git restore . --staged 
# to revert both working tree and stage area use
git restore --source . --staged --worktree # this similar to git checkout .

Or you can restore to a specific commit

git restore --source <commit-hash> <file>

Wrapping up

While git checkout still works, switching to the new commands can make you Git workflow more intutive and less confusion. Branches, files, and commits are all just pointers in your Git history, but by using these specialized commands, you’ll reduce the mental load and avoid mistakes.

Take a few minutes to familiarize yourself with these new commands—you’ll be glad you did!

How Single Page Application call a secure your API with OAuth2.

Ghassan Karwchan — Sat, 06 Jan 2024 04:43:07 +0000

Public SPA challenge when it comes to use OAuth 2.0, is you cannot store any type of secrets of credentials in your code. This is why OAuth provided a special flow to be used by SPA: Authorization Code Grant & PKCE. We are going to describe this flow in details here.

Summary of the process:

To describe the flow briefly, when the SPA requires to get a token it will direct the application to an Authorization Server (AS), which will use that server login page and UI, and the user will run the login process on the AS server website, then that server will return back to the original client website using a callback url passing the token to the SPA.

So the client SPA has no idea about the credentials and the client cannot intercept those credentials because the login run on Authorization Server website.

Let us imagine we have this setup and see how we will do that. I am going to use Auth0 service as an example:

How to configure Auth0

Before we start writing code, we should configure our API and our SPA application on Auth0.

I am not going to show the details on how to do it in Auth0, and assume you know how to do it.

Create an API in Auth0, and configure its audience to be something like : https://myapiserver.com"
Create an SPA application and get its Client Id.

From Auth0 we need two values:

the audience you used to create the API.
the Client Id for the SPA application.

Details of the OAuth Authorization Code flow:

Here are the steps to achieve getting token from Authorization Code flow.

Step 1: call authorize endpoint

The process start when the SPA application will find it needs a token to access the remote resource, so it start the process by navigating to the Authorize end point: https://<your-tenant-id>.us.auth0.com/authorize passing the following information to this end point:

GET /authorize?
response_type=code
& client_id=<client_id>
& state=<state>
& scope=<scope>
& redirect_uri=<callback uri>
& resource=<API identifier>
& code_challenge=<PKCE code_challenge>
& code_challenge_method=S256 HTTP/1.1
Host: authorizationserver.com

The parameters are as follows:

client_id: it is the application (SPA) you create on Auth0.
audience: is the audience used in configuring API in Auth0. Other OAuth providers use the term resource instead of audience, but it play the same function.
redirect_uri: the callback URL to your application. This call back should be one of the allowed url registered in your SPA Auth0 application.
scope: this setting could be use to define the scope or permissions requested by the client. In OpenID Connect it can be only the value openid profile, and in OAuth 2.0 usually it is the list: openid profile email offline_access.
response_type: should be "code" because Authorization Code flow * response_mode: should be "query" which means the code will return as query parameter.
code_challenge: a value generated by PKCE code verifier to protect the code. code_challenge_method: S256, it is the recommended encryption to use.
state: is an encrypted auto-generated string used to track calls during authorization process.

Step 2:

The Authorization Server find there is no valid token for this session, so it redirect it to its own login UI page:

GET /login?state=<state>

Step 3:

You provide your credentials in the login screen, and the Authorization server might take you to a consent page.

After Submit login, the page will do a post call to login as follows:

POST /login?state=<state>

Inside the post there is a form of the credentials:

username
password

Step 4:

The Authorization server will call back your SPA with the callback url you provided with the code as query parameter

HTTP/1.1 302 Found
Location: https://clientapplication.com/callback?
code=<authorization code>
& state=<state>

Last Step Step 5: generate token

Last step is you call the Authorization Server token endpoint with POST with the authorization code that generated from pervious step:

POST /token HTTP/1.1
Host: <auth0 tenant authorization url>
Content-Type: application/x-www-form-urlencoded
grant_type=authorization_code
& code=<authorization_code>
& client_id=<client id>
& code_verifier=<code verifier>
& redirect_uri=<callback URI>

The server will response with json with the token and refresh token, and maybe id_token (in case of OpenID Connect):

"access_token":"<refresh token>",
"token_type": "Bearer",
"expires_in":<token expiration>,
 "refresh_token":"<refresh_token>"
"id_token": "<id token>", 
"scope": "openid profile email"
}

Calling HTTP API and the problem of Socket exhaustion and DNS rotation.

Ghassan Karwchan — Sat, 06 Jan 2024 00:00:46 +0000

You definitely had to write code to communicate to an API service. Regardless of the language you use, or the framework you use, there is a two major problems when you want to do lots of calls. They are:

Socket Exhaustion
DNS Rotation

I am going to use C# and ASP.NET as an example and show you what .NET world has a solution for that, but there are solutions in every language and framework.

Calling an API end point.

Let us start from beginning, how to call an API service.



using (HttpClient client = new HttpClient())
{
     client.BaseAddress = new Uri("<baseAddress>");
     var response = await client.GetAsync("<api endpoint>");
     response.EnsureSuccessStatusCode();
     var result = await response.Content.ReadAsStringAsync();
}

Let's explain what we are doing:

wrap the HttpClient in using statement, so it disposed at the end of the block.
set the base address.
make a call.
Throw an exception of the request is not successful.
extract the result.

How Http communication work?

Let's explore the underlying protocol that implement the HTTP call.

When a computer needs to send a request to HTTP server, it creates a connection between the computer itself and the server.

To open a connection the computer needs to open a port with a random number between 0 and 65,535 and connects to the server's IP address and port.

The combination of the computer IP address and port is called Socket.

The problem with above code is even you dispose the Http client, but the underlying connection won't be disposed immediately, and the port won't be release right away.

In Http protocol, when a connection disposed, it will stays in TIME_WAIT status for 240 seconds (4 minutes).

With lots of calls you might use all the ports on the client computer, which lead to the application will hang waiting for a port to be released.

That situation is called Socket Exhaustion.

Fixing Socket Exhaustion

So what if we introduce one instance of Http Client (that is using only one random port), and keep it alive, and re-use it for all calls?

This will solve the problem of Socket Exhaustion, but we introduce another problem.

When we call an API endpoint which has the address: https://www.myapiservice.com/api/myendpint, the HttpClient will first call the DNS servers to translate "myapiservice.com" to an IP address.

HttpClient will keep the connection open to that IP address.

But what the IP changed in the DNS server?

If that API was hosted in on-promise company computer then this might not happen a lot.

But on the cloud, IP address are not that stable (unless you specifically configure it to be like that).

By default cloud, and Kubernetes deals with the outside world with domain names, and they have their private DNS to resolve the IP resolving. Again the administrators could be using reserved IP address, but by default they are not.
Any scale up on cloud PaaS, or even new deployment might change the IP.

Having one HttpClient, will keep the old IP address that was detected on the first call.

How .NET core fixed the problem?

.NET fixed the problem by introducing a feature called IHttpClientFactory in Core version 2.1.

How IHttpClientFactory solved the problem?

HttpClient internally is creating a connection and doing the calls using a class in .NET called SocketsHttpHandler.

The real work of establishing a connection is happening inside SocketsHttpHandler , and the disposing of this handler that will take Time_WAIT for 4 minutes to be really released.

HttpClientFactory uses an internal class called HttpClientHandler, and it is the same as SocketsHttpHanlder, which might be confusing, so before we talk what HttpClientFactory is doing, let us explain the difference.

HttpClientHandler vs. SocketsHttpHandler:

HttpClientHandler was the old handler that HttpClient uses to do the real http communication. After version Core 2.1, the SocketsHttpHanlder was introduced, and HttpClient started using the new handler, but HttpClientFactory kept using the old handler HttpClientHandler.

But in Core 5.0, HttpClientHandler was changed to use SocketsHttpHanlder internally. So in reality they are the same, and when you are talking about Core version 5.0 and later both are the same which is SocketsHttpHandler.

If you read about IHttpClientFactory you will read that it is using an internal class called HttpClientHandler, and that might create confusion.

So back to our question how HttpClientFactory fixed the problmes?

IHttpClientFactory will create an internal pool of HttpClientHandler and keep them and passing them when create HttpClient. So when create or dispose HttpClient, IHttpClientFactory is using HttpClientHandler from its pool.

IHttpClientFactory rotate between the HttpClientHandler in the pool every 2 minutes (the half time of WAIT_TIME).

For 2 minutes IHttpClientFactory uses one HttpClientHandler to create all HttpClient during that 2 minutes.

And after 2 minutes, it switch to another handler, and dispose the previous handler and recreate it and add it back to the pool.

A new HttpClientHandler will create new connections, which means new DNS calls, and getting it latest IP addresses.

This technique solves both the problems of Socket Exhaustion and DNS rotation.

How to use IHttpClientFactory:

In ASP.NET Core, you need to add IHttpClientFactory as a singleton in configuring services:



public void ConfigureServices(IServiceCollection services)
{
    services.AddHttpClient()
}

And then you can use it to create HttpClient:



[ApiController]
public class MyController : ControllerBase
{
    private readonly IHttpClientFactory _factory;
    public MyController (IHttpClientFactory factory) 
    {
        _factory = factory;
    }
    [HttpGet("getdata")]
    public async Task<string> GetData()
    {
        var client = _factory.CreateClient();
        client.BaseAddress = new uri("https://www.myapiservices.com");
        var response = await client.GetAsync("<end point>");
        response.EnsureSuccessStatusCode();
        return await response.Content.ReadAsStringAsync();
    }
}

Dealing with different Http client configurations

What if you want to use different Http Client configurations with different services, and to re-use those configurations?

ASP.NET gives two options to re-use the same configurations:

Named Http Client
Typed Http Client

Named HttpClient

You to configure Named Http Client, where you can store custom configuration
In ConfigureServices part:



public void ConfigureServices(IServiceCollection services)
{
    services.AddHttpClient("mycustomClient", (HttpClient client) => 
    { 
         client.BaseAddress = new Uri("https://www.myapiservices.com");
         client.DefaultRequestHeaders.Add( HeaderNames.UserAgent, "mycustomagent");
    })
    .ConfigureHttpClient((HttpClient client) => { ...custom configuration ...})
    .ConfigureHttpClient(
       (IServiceProvider provider, HttpClient client) => {});
}

How to use that?




[ApiController]
public class MyController : ControllerBase
{
    private readonly IHttpClientFactory _factory;
    public MyController (IHttpClientFactory factory) 
    {
        _factory = factory;
    }
    [HttpGet("getdata")]
    public async Task<string> GetData()
    {
        var client = _factory.CreateClient("mycustomClient");
        var response = await client.GetAsync("<end point>");
        response.EnsureSuccessStatusCode();
        return await response.Content.ReadAsStringAsync();
    }
}

Typed Http Client

Another way is to use Typed Http Client, which are classed that inject Http Client as follows:

First create a class that inject HttpClient and configure it



public class MyFinancialClient : IMyFinancialClient
{
   private readonly HttpClient _client;
   public MyFinancialClient(HttpClient client)
   {
       _client = client;
       _client.BaseAddress = new Uri("https://myapiservices.com");
   }
   public async Task<string> GetFinancialData() 
   {
       var response = await _client.GetAsync("financialEndPoint");
       response.EnsureSuccessStatusCode();
       return await response.Content.ReadAsString();
   }
}

And you register the class



public void ConfigureServices(IServiceCollection services)
{
    services.AddHttpClient<MyFinancialClient>();
}

And then use it



[ApiController]
public class MyController : ControllerBase
{
    private readonly MyFinancialClient _client;
    public MyController (MyFinancialClient factory) 
    {
        _client = client;
    }
    [HttpGet("getdata")]
    public async Task<string> GetData()
    {
        return await client.GetFinancialData();
    }
}

OAuth, OpenID Connect, SSO and Authentication Flows in plain English.

Ghassan Karwchan — Sat, 28 Oct 2023 04:31:31 +0000

Gone are the days when you force your users to register into your application by creating username/password.
Everyone wants to skip that process when they can identify themselves using other ways (google, facebook, twitter…), and they want to use their single sign on process to identify them across multiple applications and multiple websites.
In this post I am trying to explain the process and its technologies in simple English.

An explanation by example:

So you are a developer who is building an application and want to allow your users to use their Google credentials to authenticate themselves into your account.
I am going to explain to you the concepts and the technologies.

To accomplish that, your application must make a request to some API service on the Google Account website in order to initiate the user authentication process and receive a response that identifies the user.
The Google Account website is the only place where the authentication process should run, and where the user enter their credentials. And Google authentication service should ensures that the credentials are safe and hidden from our application.

The process of using an external server to sign-on and using its returning identity in different applications is called Single Sign-On (SSO).

Google Account website stores information about the user who did previously register into Google Account website. The digital information that uniquely identifies a user on Google Account website and shared on other websites and domains is known as Federated Identity.

The Google Account website has to provide API services that will achieve that process, and these services must use common protocols and policies that are understood by our application and Google Account. These protocols and policies are called Federated Identity Management (FIdM).

Google Account is the provider that provides Authentication service to identify users, and return their Federated Identities, and the technical terms for it is: Federated Identity Provider.

Our application will be the consumer or the client for FIdM.
There are many different Federated Identity Providers: Google, Microsoft, Twitter, Facebook…etc. They all provide these FIdM services for industry standard protocols.

What technologies and protocols are used?

There are three technologies that provide FIdM services:

SAML (Security Assertion Markup Language): It is the oldest technology, and although it is limited used now but still kicking around.
OAuth 2.0: the most common one, because it is implemented by all providers.
OpenID Connect: is it built on top of OAuth but it cut lots of extra stuff and concentrate on authentication.

How they are different?

SAML , and WS-Federation:

SAML was the oldest one, and is used mainly for web applications and mostly internal web applications. It uses XML to communicate to the Identity providers. SAML was developed when calling external services was done using SOAP (an old protocol from web services).
Interesting to note that an approach to make better SAML was initiated and was called WS-Federation but then later REST and HTTP replaced SOAP, and JSON replaced XML, and SAML/WS-Federation became less and less used.

SAML only returns a digital identity called SAML Assertion to identify the user. The SAML Assertion is very simple and it is the simplest representation of the user.

OAuth 2.0:

Although OAuth is used as one of FIdM technologies, but it wasn't mainly built for that purpose.
With the rising of social media and the wealth of its data, social media companies wanted to provide its data and share user data across the applications.
They provided more sophisticated protocols to share user information and allow users to choose what information they want to share, and this is how OAuth 2.0 was built.

Because OAuth 2.0 needed to identify the users on their own social media servers, so it has the SSO features of FIdM, beside it added more features to allow users to chose what information they want to share.

For example on google we all saw this when using google login.

As you see that OAuth has more that SSO and FIdM, but it built very good technologies and protocols, and this is why OpenID came up to life.

OpenID Connect:

Because OAuth had good technologies and standards to achieve SSO and FIdM, but has way more of Authorization and sharing information, the industry built a new standard that use OAuth technologies but concentrate only on the Authentication process and cut out the authorization, and it only share the basic data that uniquely identify the user. This new standard is OpenID Connect 1.0.
OpenID Connect uses lots of common technologies with OAuth, that is sometimes it get confused.

How the process works?

OAuth created many workflows each work better for a specific type of applications. OAuth called these workflows: Flows
OpenID still using these flows but slightly modified them. So understand them with OAuth is the same as OpenID.
We can define these types of applications:

Website application that serve its own UI.
Single Page Application (SPA), where the client is a thick JavaScript application that can be hosted on any place, and it called different API application.
Mobile native APP.
Local desktop App. Each type has its matching OAuth

Application Type	OAuth Flow
Web application	Authorization Flow
SPA	the Authorization Code Flow with Proof Key for Code Exchange (PKCE) Implicit Flow used to be the one but not any more
Local Desktop App	Client Credential Flow
Native mobile APP	the Authorization Code Flow with Proof Key for Code Exchange (PKCE)

Conclusion

Hopefully I was able to explain these technical terms and technologies in simple plain Engligh.

Best way to run background services in ASP.NET.

Ghassan Karwchan — Thu, 13 Jul 2023 01:27:21 +0000

Asp.Net provides many ways to run background processes. They are:

IHostedService interface
BackgroundService abstract class
Worker Service using Microsoft.NET.SDK.Worker

Actually they are just layers on top of each others, each layer provides extra functionality on the one below.

This post is going to describe each layer and what it provides.

Running background tasks using `IHostedService`

Asp.NET provides support for running background tasks in the same process that host Asp.net application.

By implementing IHostedService interface, your background task will start a little bit after Asp.Net application start, and end when Asp.Net application shutdown.

P.S.: Asp.NET core server, Kestrel itself is a hosted service.

IHostedService has two methods:

public interface IHostedService
{
    Task StartAsync(CancellationToken cancellationToken);
    Task StopAsync(CancellationToken cancellationToken);
}

After you implement the interface, you need to register the task, and register it with the DI container. Asp.NET provides AddHostedService extension method on IServiceCollection for that purpose:


public void ConfigureServices(IServiceCollection services)
{
   services.AddHostedService<MyExampleService>();
}

Using scoped services in the background task:

AddHostedService is doing registering the service as Singleton in the DI container.

That leads to a problem if you need to use services registered as scoped-lifetime services.

To fix this problem, create a new container scope anywhere you need access to scoped service, as the following code:

public class MyExampleService : IHostedService
{
   private readonly IServiceProvider _provider;
   public MyExampleService (IServiceProvider provider)
   {
      _provider = provider;
   }

   public Task StartAsync (CancellationToken cancellationToken)
   {
     using (IServiceScope scope = _provider.CreateScope())
     {
         var scopedProvider = scope.ServiceProvider;
         var client = scopeProvider.GetRequiredService<ExampleScopedService>();
         ...
      }
    }
}

Guidelines on IHostedService

There are subtleties to implementing the interface correctly. In particular, the StartAsync method, although asynchronous, runs inline as part of your application startup. Background tasks that are expected to run for the lifetime of your application must return a Task immediately and schedule background work on a different thread. Calling await in the IHostedService.StartAsync method will block your application from starting until the method completes. This can be useful in some cases, but it’s often not the desired behavior for background tasks.

BackgroundService

To make it easier to create background services using best-practice patterns, ASP.NET Core provides the abstract base class BackgroundService, which implements IHostedService and is designed to be used for long-running tasks. To create a background task you must override a single method of this class, ExecuteAsync. You’re free to use async-await inside this method, and you can keep running the method for the lifetime of your app.

Worker Service

.NET provides a special kind of application called Worker Service, which runs IHostedService in a host that is lighter than Asp.NET host.

That lighter host is Microsoft.Extensions.Hosting.Host, which is a general host that is hosting the Asp.Net WebApplication itself.

I wrote before about the relation between Host and WebApplication.

The worker service's Host will run background service, but doesn't handle HTTP request. It has the following:

configuration features.
logging features.
dependency injection features.

To use Worker Service, run the following:

dotnet new worker

This will create a BackgroundService class, that is hosted in Microsoft.Extensions.Hosting.Host, which is an implementation of IHost which what we described as a lightweight host.

Running Worker Service in production

You can run Worker Service as a Windows Service, or Linux's Systemd service.

build resilient applications accessing Azure services with no code.

Ghassan Karwchan — Sun, 12 Mar 2023 02:08:54 +0000

When working on distributed system like systems that are running on the cloud, and microservices, then you should anticipate transient faults like temporary loss of network connection, or temporary unavailability of a service, or a service is busy.

This is why in cloud application, the term Building for resiliency become a pattern in building application, and it means properly handling these transient faults in the code.

There are two common used patterns that we can use in our code, that can help build resilience application.

Retry pattern.
Circuit-breaker Pattern.

The good news is there are libraries that will help you doing that, and even the SDKs for some Azure services have these two patterns built-in, and by just configure the connection properly you will get this benefits without writing your own code for those.

Retry Pattern

The retry pattern means if calling a service failed, you wait for a short time, and then retry again because you expect the problem will be resolved shortly. You try not only one time, but maybe for 3 times, and leave a short waiting time between the retries.

Services that support retry.

Azure SQL Server, and Entity Framework: Entity Framework has a built-in retry technique when it comes to communicate with Azure SQL Server.

services.AddDbContextPool<ConcertDataContext>(options => options.UseSqlServer(sqlDatabaseConnectionString,
    sqlServerOptionsAction: sqlOptions =>
    {
        sqlOptions.EnableRetryOnFailure(
          maxRetryCount: 5,
          maxRetryDelay: TimeSpan.FromSeconds(3),
          errorNumbersToAdd: null);
    }));

Azure Cache for Redis If you are using the most famous Redis .NET library which is StackExchange.Redis, then the library has the mechanism for retry as follows:


var options = new ConfigurationOptions
{
    EndPoints = {"localhost"},
    ConnectRetry = 3,
    ReconnectRetryPolicy = new ExponentialRetry(
TimeSpan.FromSeconds(5).TotalMilliseconds
, TimeSpan.FromSeconds(20).TotalMilliseconds),
    ConnectTimeout = 2000
};
ConnectionMultiplexer redis = ConnectionMultiplexer.Connect(options, writer);

Azure Service Bus The .NET package for the service bus Azure.Messaging.ServiceBus has built-in support for retry using the configuration of its client.


var options = new ServiceBusClientOptions();
options.RetryOptions = new ServiceBusRetryOptions
{
    Delay = TimeSpan.FromSeconds(10),
    MaxDelay = TimeSpan.FromSeconds(30),
    Mode = ServiceBusRetryMode.Exponential,
    MaxRetries = 3,
};
await using var client = new ServiceBusClient(connectionString, options);

More Azure service support retry

I am not going to list the code for all services, but just check the SDK for each service, and you will find how to enable the retry features. But these services will support the feature:

Azure Cosmos DB
Azure Active Directory
Azure Search
Event Hub
IoT Hub
Azure Storage

Retry using Polly

If you need to connect to other API services, or you need to connect to Azure SQL but without Entity Framework, then you need to implement the code yourself.

But there is a .NET package that is already doing that: Polly.

Let us see for example how to connect to SQL using ADO.NET and Polly:

public async static Task<SqlDataReader> ExecuteReaderWithRetryAsync(this SqlCommand command)
{
    GuardConnectionIsNotNull(command);

    var policy = Policy.Handle<Exception>().WaitAndRetryAsync(
        retryCount: 3, // Retry 3 times
        sleepDurationProvider: attempt => TimeSpan.FromMilliseconds(200 * Math.Pow(2, attempt - 1)), // Exponential backoff based on an initial 200 ms delay.
        onRetry: (exception, attempt) =>
        {
            // Capture some information for logging/telemetry.
            logger.LogWarn($"ExecuteReaderWithRetryAsync: Retry {attempt} due to {exception}.");
        });

    // Retry the following call according to the policy.
    await policy.ExecuteAsync<SqlDataReader>(async token =>
    {
        // This code is executed within the Policy

        if (conn.State != System.Data.ConnectionState.Open) await conn.OpenAsync(token);
        return await command.ExecuteReaderAsync(System.Data.CommandBehavior.Default, token);

    }, cancellationToken);
}

Circuit-breaker Pattern

There is a problem with the retry pattern. What if a service was down and stayed down for long time?

Then every time we call this service, we are going to call it three times and wait for a second between these calls.

This will add delay for our application.

The Circuit-breaker pattern will fix this, by assuming the service is down after many failed calls.

If the service marked as down, then code will immediately fail the call without even calling the service.

This state is called Open Circuit.

Then the code will wait for a while before it try again to call the service, and if it is still failing, then the state will still be Open, otherwise it is marked as healthy, and the state will be Closed Circuit.

Circuit breaker should be used in conjunction with the retry pattern.

Using Cicruit-breaker with Polly.

There is built-in implementation of circuit-breaker in SDKs of Azure services, but we can achieve it using Polly package, as follows:

private static IAsyncPolicy<HttpResponseMessage> GetCircuitBreakerPolicy()
{
    return HttpPolicyExtensions
        .HandleTransientHttpError()
        .CircuitBreakerAsync(5, TimeSpan.FromSeconds(30));
}

Buy custom domains with Azure for your App Service.

Ghassan Karwchan — Sun, 05 Mar 2023 04:36:14 +0000

If you googled how to buy a domain name on Azure, you might get answers saying that Azure doesn't provide domain name registrar, like the following answer, or the following discussion.

Not even those answers, even the book Exam Ref AZ-900 which is a Microsoft book says the following about Azure DNS Zone:

If you want to purchase your own domain name, you go to a domain registrar, and they register the domain to you. ....

The book didn't mention a way to buy your domain from Azure itself.

I struggled with that myself, and bought my domain with a third party provider. But it turned out Azure provide this service already.

This is why I hope you will come to my post and find the answer you are looking for.

Custom domains with App Service Domain Service:

To buy a custom domain through Azure, you need to create an App Service Domain resource in Azure.

When you create the resource, you will enter the domain name you want, and Azure will find out if you can use it or not.

Right now, as I am writing this post, only these Top Level Domain (TLD) are available through Azure:

.com
.net
.co.uk
.org
.nl
.in
.biz
.org.uk
col.in

Why you need to buy from Azure?

The obvious answer to this question, is you have one place to manage your site content, and its domain name and its SSL certificate.

But beside that, Azure is cheeeeeap. The pricing right now is only $11 US dollar for the whole year.

I don't know what you are using for as your domain provider, but I am using GoDaddy, and I am paying for the domain and SSL certificate $10/month.

And on top of that the SSL certificate is free (if you are using standard domain like www.somesite.com). You have to pay for wildcard certificates.

Everything you need to know about telemetry for your Asp.NET application on Azure.

Ghassan Karwchan — Sat, 25 Feb 2023 05:10:04 +0000

All of the services that Azure offers for the instrumentation and observability (monitoring) of applications are grouped under Azure Monitor.

Three services fall under the banner of Azure Monitor:

Azure Monitor itself, which offers the observability component to track your apps.
Application Insights, or its substitute OpenTelemetry, which offers the instrumentation and telemetry components
The analytics component is provided by Log Analytics.

We will emphasise the instrumentation component in this post.

Telemetry and instrumentation in Azure.

Telemetry, the data collected to observe your application, can be broken into three types or "pillars":

Distributed Tracing
Metrics
Logs

Azure provides two services that cover this area:

Application Insights
OpenTelemtry

Application Insights is the service we will discuss in this post because it offers thorough and rich capabilities for gathering information about your application. Yet OpenTelemetry deserves a quick mention.

OpenTelemetry on Azure:

Resuming with Application Insights
OpenTelemetry is the new kid in the hood and it is the future of telemetry. This initiative aims to standardize telemetry APIs and SDKs, and establish vendor-neutral SDKs for telemetry.

It is still in early phase of development, and it only cover the Distributed Tracing part of the telemetry.

The technology is widely utilised in the world of Docker and Kubernetes.

Recently, Azure Monitor began implementing OpenTelmetry by reading its data. Nonetheless, in general, we should choose Application Insights because it has far more functionality till this technology advances further, especially if we are working on App Service and Azure functionalities.

Now back to Application Insights.

Application Insights and Auto-Instrumentation:

You can enable Application Insights on an App Service or Azure Function without writing any line of code.
This feature is called Auto-instrumentation or “Run-time instrumentation”
This feature is available because the App Service base image has a built-in agent that will collect the instrumentation and communicate with the Application Insights server.

Asp.net core (on both linux and windows) supports Auto-instrumentation.

To enable Auto-instrumentation on an App Service, on Azure Portal check the App Service, and on the left side there is Application Insigths blade.

Then you enable it on the App Service.

Auto-instrumentation on client side:

If you are using Asp.NET MVC application on Windows hosted App Service, or using Azure Static Web App, you can auto-instrument your JavaScript code, by just doing the following:

Add a new application setting APPINSIGHTS_JAVASCRIPT_ENABLED and set its value to true.

That will inject JavaScript SDK for Application Insights in your JavaScript code.

Client-side instrumentation captures information about the user experience of the app, including page load times, details of browser exceptions, and performance data about AJAX calls.

Application Insights and Manual-Instrumentation:

Although Auto-instremenation will provide good coverage, it is not enough.
We need to track our application manually from inside the code.
The more we track from inside the code the more coverage we will have on the performance and operational issues we encounter in production, which translates to better support.

Manual instrumentation on the Asp.NET server code:

To collect telemetry from the code, we need to include NuGet package:

Microsoft.ApplicationInsights.AspNetCore

Then initialize the service from the code:

var builder = WebApplication.CreateBuilder(args);

// The following line enables Application Insights telemetry collection.
builder.Services.AddApplicationInsightsTelemetry();

You can pass the connection string in the code, but better to provide it from the appsettings.json file as follows:

// in appsettings.json
"ApplicationInsights": {
    "ConnectionString": "Copy connection string from Application Insights Resource Overview"
  }

By just adding these few lines, our SDK will collect lots of metrics from the running application. It will collect:

Application Dependencies: for example communication with databases including sql statements, or with blob storage.
Performance Counters: for example memory and CPU usage.
Events Counter: There are built-in events that the SDK will collect, and you can create your own custom event. For a list of built-in events check this link.

What to capture with the SDK

Many metrics can be collected, and each one has a corresponding config name which we can turned off

Http request calls
Performance counters (memory, CPU usage)
Dependencies: check this for a list of trackable dependencies
Heart Beat feature
Events: this deserve another post

Frequency of metric sampling

How often the SDK will send sampling to Azure service?

We can configure that using sampling configuration. There are two options to define sampling frequency:

Adaptive: which is the default
Fixed-rate: which you can use to reduce the traffic to the server.

Logging to Application Insights

Finally, we need to do our logging in Application Insights.

First add this package:

Microsoft.Extensions.Logging.ApplicationInsights

then we add this to our appsettings.json.

"ApplicationInsights": {
    "InstrumentationKey": "[The instrumentation key]"
  },
"Logging": {
    "PathFormat": "[The path and file format used in file logging, e.g.: c:\\log-{Date}.txt]",
    "LogLevel": {
      "Default": "Information"
    },
    "ApplicationInsightsLoggerProvider": {
      "LogLevel": {
        "Default": "Warning"
      }
    }
  }

and add this when you configure the service collection

// as we discussed before
services.AddApplicationInsightsTelemetry();

    services.AddLogging(logBuilder =>
             {
                 logBuilder.AddApplicationInsights(YourInstrumentationKey)
                     // adding custom filter for specific use case. 
                     .AddFilter("Orleans", (level) => level == LogLevel.Error);
    });

Check performance bottleneck with Azure Application Insights.

Ghassan Karwchan — Thu, 23 Feb 2023 04:43:16 +0000

If you have Application Insights enabled for your azure app service, then it is a piece of cake to get the performance bottleneck in your app, and even it check external dependencies like database queries or any other external dependencies.

I have a Web API application, that is accessing Azure Sql Server, and Azure blob storage.

By going into the performance blade under Investigate:

You will see the average time for each API call, sorted descending by time:

By clicking on any one, you can drill down to where the time consume even by its external dependencies:

As you can see it shows the time it takes to connect to its external resources, and in our case they are:

Sql Server
Blob Storage

You can even drill down to its collected samples:

And you can get a very granular look at the time consume in the whole workflow of that API call:

Wait, there is more

If the app administrator had concerns about the performance, and want to inform the developer team, then they can report the issue from Application Insights.

On the right side, you can create a Work Item to report this issue to the dev team.

You need to have a Work Item Template, which define where you report those issues. There are two venues where you report the issues:

Azure DevOps.
GitHub Repository, where it report it to the Issues tab in the repository.

Creating a template is easy, but I am not going to cover it here, and I am already created a template the report in my github repo.

After you create a Work Item, (in my case I had already template, it will create an issue with the name of the API call

and inside the issue will have a link to the details captured in Application Insights.

Use Azure Key Vault to retrieve secured parameters during Azure deployment.

Ghassan Karwchan — Sat, 04 Feb 2023 00:38:20 +0000

You have an ARM template to deploy, and you need to pass secure parameters. Instead of storing secure values in the parameter file, you can just retrieve these values from Key Vault.

To be able to access the key vault by the resource manager you need to change access policy to allow "Azure Resource Manager for template deployment", as shown here.

Or you can do it from Powershell:

// to update an existing key vault
Set-AzKeyVaultAccessPolicy -VaultName MyVaultName -EnabledForTemplateDeployment

// to create a new key vault with this feature enabled
New-AzKeyVault `
  -VaultName MyVaultName `
  -resourceGroupName myresourcegroup `
  -Location centralus `
  -EnabledForTemplateDeployment

How to use it?

in the deployment parameter file specify the location of the secured string to be the keyvault as follows:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "location": {
            "value": "eastus"
        },

        "adminUsername": {
            "value": "companyAdmin"
        },
        "adminPassword": {
            "reference": {
        "keyVault": {
          "id": "/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.KeyVault/vaults/MyVaultName"
        },
        "secretName": <the name of the secret>
      }
        }
    }
}

Permissions to run deployments:

The beauty of this feature is the user who is doing deployment doesn't need to have access to the secrets, even read access. Just need a special permission called deploy permission, or more specifically this permission

Microsoft.keyVault/Vaults/deploy/action

To assign this permission to the user, it is easier if we create a custom role with this permission and then assign this role to any user want to deploy.

First we create a json to represent the definition of the custom role:

{
  "Name": "TemplateDeploymentForResourceManagerRole",
  "IsCustom": true,
  "Description": "Lets you deploy a resource manager template with the access to the secrets in the Key Vault.",
  "Actions": [
    "Microsoft.KeyVault/vaults/deploy/action"
  ],
  "NotActions": [],
  "DataActions": [],
  "NotDataActions": [],
  "AssignableScopes": [
    "/subscriptions/<your subscription id>"
  ]
}

And then create the role and assign it to the user.

This powershell script does this:

New-AzRoleDefinition -InputFile "<path-to-role-file>"
New-AzRoleAssignment `
  -ResourceGroupName ExampleGroup `
  -RoleDefinitionName "TemplateDeploymentForResourceManagerRole" `
  -SignInName <user-principal-name>