DEV Community: PostSharp Technologies

The Builder Pattern in C# [2024]

Gael Fraiteur — Mon, 21 Oct 2024 07:00:01 +0000

The popularity of immutable objects has made the Builder pattern one of the most essential in C#. In this article, we’ll discuss which use cases are a good fit for the Builder pattern and how to practically implement it in .NET, with real-world examples and source code on GitHub. We’ll see different tools to automatically generate the implementation instead of writing it by hand.

What is the Builder pattern?

The Builder design pattern is a creational design pattern that aims to separate the creation process of a complex object from the object itself.

Instead of creating an object directly using its constructor, you use a Builder class. When you’re done configuring the object, you call a Build() or ToImmutable() method of the Builder class, which returns the constructed object.

When to use a Builder Pattern?

There are basically 3 uses cases for the Builder pattern:

Complex creation of immutable objects
Abstracting the instantiation process
Async creation of objects

Use case 1: Complex creation of immutable objects

The most common use case for the Builder pattern is when you want to use a mutable object during the initialization phase of an object but want the constructed object to be immutable.

To take a real-word example, suppose you’re building a clone of Amazon and want to start selling books. You have an immutable Book class. To make it possible to edit a book, you create a BookBuilder object with mutable properties. In the Build method, you check that all properties have acceptable values before instantiating the immutable object.

public record Book( 
   string Title, 
   ImmutableArray<string> Authors, 
   ImmutableArray<string> Tags );

public class BookBuilder
{
    public string? Title { get; set; }
    public List<string> Authors { get; set; } = new();
    public List<string> Tags { get; set; } = new();
    public virtual Book Build()
    {
        return new Book( this.Title!, [..this.Authors], [..this.Tags] );
    }
}

The full source code of examples in this article is available on GitHub.

You might want to create a Builder class for your immutable objects in the following scenarios:

When building an interactive UI, you must bind the UI controls to writable properties.
When you want to allow for initialization in multiple steps.
When you must validate the object before creating it, and you cannot do it before knowing the value of all properties.
When the optimal data structure of the immutable object differs from the optimal data structure of the builder. A typical example is the System.Collections.Immutable namespace, where immutable collections and their builders have radically different implementations.
When you want to ensure that your callers only modify the objects when it is allowed. A good example of this use case is the .NET Core AppBuilder pattern. When creating an API to build components, you want to allow API users to modify the configuration during the initialization phase by offering an API like AddMyComponent(Action<ComponentMyBuilder>? builder = null). However, you want to forbid them from modifying the configuration after the component has started.

Use case 2: Abstraction of the instantiation process

In the “Gang of Four’s” seminal Design Patterns book, the Builder pattern is described as a way to abstract the process of building objects.

Suppose you want to build an object of type IBook. The IBook interface has two implementations, PaperBook and EBook, which differ in their implementation of the Deliver method. For some reason, you want the implementation classes to remain internal. To make it possible to create IBook instances, you provide a new public interface: IBookBuilder. This interface is meant to be bound to the UI, so its properties are not strongly validated: we only ask the object to be valid when the Build method is called. The IBookBuilderFactory interface supplies IBookBuilder for the required kind of book. To get an IBookBuilderFactory, the caller will use IServiceProvider.

This long chain of invocation (IBookBuilderFactory → IBookBuilder → IBook) can seem cumbersome, but it’s a realistic design if you must pull a dependency, for instance IIdGenerator, from IServiceProvider.

Here is the public API:

public interface IBook
{
    string Title { get; }
    ImmutableArray<string> Authors { get; }
    void Deliver();
}

public interface IBookBuilder
{
    string? Title { get; set; }
    List<string> Authors { get; }
    IBook Build();
}

public enum BookKind
{
    Paper = 1,
    EBook = 2
}

public interface IBookFactory
{
    IBookBuilder CreateBookBuilder( BookKind bookKind );
}

This API is meant to be used as follows:

var factory = serviceProvider.GetRequiredService<IBookFactory>();
var builder = factory.CreateBookBuilder( BookKind.Paper );
builder.Title = "Dix contes de Perrault";
builder.Authors.Add( "Charles Perrault" );
var book = builder.Build();
book.Deliver();

Let’s look at the implementation of immutable classes:

internal abstract record Book( 
   int Id, string Title, ImmutableArray<string> Authors ) : IBook
{
    public abstract void Deliver();
}

internal record PaperBook( 
   int Id, string Title, ImmutableArray<string> Authors )
    : Book( Id, Title, Authors )
{
    public override void Deliver() => throw new NotImplementedException();
}

internal record EBook( 
   int Id, string Title, ImmutableArray<string> Authors )
    : Book( Id, Title, Authors )
{
    public override void Deliver() => throw new NotImplementedException();
}

The builders are almost equally trivial:

internal abstract class BookBuilder : IBookBuilder
{
    public int Id { get; }
    public string? Title { get; set; }
    public List<string> Authors { get; } = new();
    protected BookBuilder( int id )
    {
        this.Id = id;
    }
    public abstract IBook Build();
}

internal class PaperBookBuilder : BookBuilder
{
    public PaperBookBuilder( int id ) : base( id ) { }
    public override IBook Build() 
      => new PaperBook( this.Id, this.Title!, [..this.Authors] );
}

internal class EBookBuilder : BookBuilder
{
    public EBookBuilder( int id ) : base( id ) { }
    public override IBook Build() 
       => new EBook( this.Id, this.Title!, [..this.Authors] );
}

Finally, here is the top-level class, the one added to the IServiceProvider:

internal class BookBuilderFactory( IIdGenerator idGenerator ) 
  : IBookFactory
{
    public IBookBuilder CreateBookBuilder( BookKind kind )
    {
        var id = idGenerator.Next();
        return kind switch
        {
            BookKind.Paper => new PaperBookBuilder( id ),
            BookKind.EBook => new EBookBuilder( id ),
            _ => throw new ArgumentException()
        };
    }
}

Use case 3: Async creation of objects

Another use case is when the creation of the object requires an asynchronous call. Since there is no async constructor in C#, you must use either an async Factory pattern or an async Builder pattern.

As an example, suppose that books have an Id property whose value must be uniquely generated by the database.

public record Book( int Id, string Title );
internal interface IDatabaseIdGenerator
{
    Task<int> NextIdAsync();
}
public class BookBuilder
{
    private readonly IDatabaseIdGenerator _idGenerator;
    internal BookBuilder( IDatabaseIdGenerator idGenerator )
    {
        this._idGenerator = idGenerator;
    }
    public string? Title { get; set; }
    public async Task<Book> BuildAsync()
    {
        var id = await this._idGenerator.NextIdAsync();
        return new Book( id, this.Title! );
    }
}

Alternatives to the Builder pattern

Since C# 9.0, record types make it easier to build immutable objects. Thanks to init fields and properties, you no longer need mammoth constructors, so one of the traditional justifications of the Builder pattern falls. Last but not least, the with keyword allows you to perform incremental modifications of immutable objects. The downside of the with keyword is that each use instantiates a new object, so it might add some performance overhead for initializations that require many steps. This overhead may be negligible when this happens only at application startup.
The Factory pattern and the Abstract Factory pattern can also be used to instantiate immutable objects, but these patterns do not allow for multi-step initializations.
The Freezable pattern combines the benefits of mutability during initialization and immutability after initialization.

How to implement the Builder pattern in C#?

You can either implement the Builder pattern manually or automate its implementation using meta-programming. In both cases, you should first define an implementation strategy and ensure that everyone on the team always uses the same strategy.

Requirements

Most of the time, you will want your Builder pattern to support the following features:

The builder object must always have a Build() method that creates the immutable object.
The builder object may have a Validate() method called by the Build() method to validate the object integrity before creating an instance of the immutable type.
The immutable object may have a ToBuilder() method that creates a new builder initialized with the current values.
Collection properties of the builder type must be mutable. For instance, if the immutable object has a property of type ImmutableArray<string> or IReadOnlyList<string>, the builder object must have a corresponding property of type ImmutableArray<string>.Builder or List<string>.
The implementation pattern must support type inheritance.

Example

Consider the following immutable classes:

public partial record Book( 
   string Title, ImmutableArray<string> Authors );
public partial record EBook( 
   string Title, ImmutableArray<string> Authors, string Url )
    : Book( Title, Authors );

The implementation of the Builder pattern for Book, the base class, is the following:

public partial record Book
{
    public virtual Builder ToBuilder() => new( this );

    public class Builder
    {
        public string? Title { get; set; }
        public ImmutableArray<string>.Builder Authors { get; set; }
        public Builder()
        {
            this.Authors = ImmutableArray.CreateBuilder<string>();
        }
        public Builder( Book prototype )
        {
            this.Title = prototype.Title;
            this.Authors = prototype.Authors.ToBuilder();
        }
        protected virtual void Validate()
        {
            if ( this.Title == null )
            {
                throw new ValidationException( 
                          "The book title cannot be null." );
            }
            if ( this.Authors.Count == 0 )
            {
                throw new ValidationException( 
                           "The book author cannot be null." );
            }
        }
        protected virtual Book Build()
        {
            this.Validate();
            return new Book( this.Title!, 
                             this.Authors.ToImmutableArray() );
        }
    }
}

Note the following points in the above snippet:

We used an ImmutableArray<string>.Builder to represent the collection of authors in the builder class.
We included a copy constructor in the build class, copying values from the Book class into Book.Builder.

Let’s now implement the pattern for the derived class to check that our pattern is sound for class inheritance.

public partial record EBook
{
    public override Builder ToBuilder() => new( this );
    public new class Builder : Book.Builder
    {
        public string? Url { get; set; }
        public Builder() { }
        public Builder( EBook prototype ) : base( prototype )
        {
            this.Url = prototype.Url;
        }
        protected override void Validate()
        {
            base.Validate();
            if ( this.Url == null )
            {
                throw new ValidationException( 
                               "The book Url cannot be null." );
            }
        }
        protected override EBook Build()
        {
            base.Validate();
            return new EBook( this.Title!, 
                      this.Authors.ToImmutableArray(), this.Url! );
        }
    }
}

How to automate its implementation?

If you feel that the Builder pattern requires a lot of boilerplate code, you’re absolutely right. Although the pattern is very convenient for users of your code, it’s a pain for the author. Unless, of course, you automate its implementation.

Two technologies can generate the code for you: Roslyn code generators and Metalama. Metalama is itself based on Roslyn, including code generators. While code generators are string-oriented, Metalama offers a real object model for meta-programming.

With Metalama, you’re done in three steps:

Add a package reference:
Copy into your project the aspect code developed in the article Implementing the Builder pattern with Metalama. The code is available on GitHub. It’s not the simplest aspect, but you can read the source code and adapt it to your needs.
Since the aspect is a custom attribute, you can it to your code like this:

And you’re done! The Builder code is now generated automatically both at build time and as you type in the editor.

What is the Director in the Builder design pattern?

You may have heard of a component of the Builder pattern called the Director, mentioned in the GoF book.

You don’t necessarily need a Director when using the Builder pattern. In this article, we have shown simple examples of single-part objects. The Director is useful when you build a multi-part object and need to orchestrate the work of several builders or build methods.

Conclusion

The Builder pattern is one of the most common in C#, especially in recent years, as the functional style relying on immutability has gained popularity. This pattern has many benefits, effectively decoupling the initialization of an object from its use. It is both more flexible and complex than patterns such as Factory or Abstract Factory. A traditional use case (avoiding mammoth constructors) has been made obsolete by recent C# features like records, init properties, or required members. Yet, it remains very useful.

The problem with the Builder pattern is the sheer amount of boilerplate code it requires. Fortunately, you can use meta-programming to automate the generation of this code. In this article, we mentioned using Metalama. The implementation of the Builder pattern using Metalama is detailed in a separate article.

This article was first published on a https://blog.postsharp.net under the title The Builder Pattern in C# [2024].

Implementing the Builder pattern with Metalama

Gael Fraiteur — Mon, 07 Oct 2024 07:00:01 +0000

The popularity of immutable objects has made the Builder pattern one of the most important in C#. However, implementing the Builder pattern by hand is a tedious and repetitive task. Fortunately, because it is repetitive, it can be automated using a Metalama aspect. This is what we will explore in this article. We will start discussing the implementation strategy, then we will comment the source code of the Metalama aspect.

Components of the Builder pattern

As with any pattern automation, the very first step is to describe how we would implement the process by hand. It’s a good practice to start with a few code snippets before transformation and to handwrite the code we want to generate. Once we think we have covered all the cases we can identify, we can reason about how to turn this into an algorithm.

Features

A proper implementation of the Builder pattern should include the following features:

A Builder constructor accepting all required properties.
A writable property in the Builder type corresponding to each property of the build type. For properties returning an immutable collection, the property of the Builder type should be read-only but return the corresponding mutable collection type.
A Builder.Build method returning the built immutable object.
The ability to call an optional Validate method when an object is built.
In the source type, a ToBuilder method returning a Builder initialized with the current values.

Examples

Let’s start with a simple example:

[GenerateBuilder]
public partial class Song
{
    [Required] public string Artist { get; }

    [Required] public string Title { get; }

    public TimeSpan? Duration { get; }

    public string Genre { get; } = "General";
}

We want the [GenerateBuilder] aspect to generate the following code:

public partial class Song
{
    private Song(string artist, string title, TimeSpan? duration, 
                 string genre)
    {
        Artist = artist;
        Title = title;
        Duration = duration;
        Genre = genre;
    }

    public Builder ToBuilder() => new Builder(this);

    public class Builder
    {
        // Public constructor.
        public Builder(string artist, string title)
        {
            Artist = artist;
            Title = title;
        }

        // Copy constructor.
        internal Builder(Song source)
        {
            Artist = source.Artist;
            Title = source.Title;
            Duration = source.Duration;
            Genre = source.Genre;
        }

        public string Artist { get; set; }
        public TimeSpan? Duration { get; set; }
        public string Genre { get; set; } = "General";
        public string Title { get; set; }

        public Song Build()
        {
            var instance = new Song(Artist, Title, Duration, Genre)!;
            return instance;
        }
    }
}

By the end of this article, we will be able to generate this code automatically on-the-fly during the build.

Here are some use cases for this code:

// Use case 1. Create from scratch.
var songBuilder = new Song.Builder( "Joseph Kabasele", 
                                    "Indépendance Cha Cha" );
songBuilder.Genre = "Congolese rumba";
var song = songBuilder.Build();

// Use case 2. Create builder from existing object.
var songBuilder2 = song.ToBuilder();
songBuilder2.Duration = new TimeSpan(0, 3, 5);
var song2 = songBuilder.Build();

Implementation steps

Our [GenerateBuilder] aspect will need to perform the following steps:

Introduce a nested class named Builder with the following members:
- A copy constructor initializing the Builder class from an instance of the source class.
- A public constructor for users of our class, accepting values for all required properties.
- A writable property for each property of the source type.
- A Build method that instantiates the source type with the values set in the Builder, calling the Validate method if present.
Add the following members to the source type:
- A private constructor called by the Builder.Build method.
- A ToBuilder method returning a new Builder initialized with the current instance.

Implementing the Builder pattern

In this article, I will only outline the major steps of the implementation. For a detailed implementation, see the Builder pattern example in the reference documentation.

Step 1. Create a Metalama aspect

The first step is to add the Metalama.Framework package to your project:

<ItemGroup>
    <PackageReference Include="Metalama.Framework"/>
</ItemGroup>

Then, create an aspect class:

public partial class GenerateBuilderAttribute : TypeAspect

The TypeAspect class is an abstract base class for aspects that can be applied to types.

Step 2. Define some infrastructure

Before adding anything to the aspect, we need a data structure to store references to the declarations we generate. The PropertyMapping type maps a property of the source code to its corresponding property in the Builder type and in constructor parameters.

[CompileTime]
private record Tags(
    INamedType SourceType,
    IReadOnlyList<PropertyMapping> Properties,
    IConstructor SourceConstructor,
    IConstructor BuilderCopyConstructor);

[CompileTime]
private class PropertyMapping
{
    public PropertyMapping(IProperty sourceProperty, bool isRequired)
    {
        this.SourceProperty = sourceProperty;
        this.IsRequired = isRequired;
    }
    public IProperty SourceProperty { get; }
    public bool IsRequired { get; }
    public IProperty? BuilderProperty { get; set; }
    public int? SourceConstructorParameterIndex { get; set; }
    public int? BuilderConstructorParameterIndex { get; set; }
}

Note that we added the [CompileTime] attribute to these classes because they need to be accessible at compile time by the aspect.

The full source code of examples in this article is available on GitHub.

Step 3. Identify the properties to be mapped

We can now start implementing the aspect. Its entry point is the BuildAspect method. The first thing we do is create a list of properties.

public override void BuildAspect(IAspectBuilder<INamedType> builder)
{
    base.BuildAspect(builder);
    var sourceType = builder.Target;

    // Create a list of PropertyMapping items for all properties 
    // that we want to build using the Builder.
    var properties = sourceType.Properties.Where(
            p => p.Writeability != Writeability.None &&
                 !p.IsStatic)
        .Select(
            p => new PropertyMapping(p,
                 required:  p.Attributes.OfAttributeType(
                              typeof(RequiredAttribute)).Any()))
        .ToList();

Step 4. Introduce the Builder type and its properties

Let’s create a nested type using the IntroduceClass method:

// Introduce the Builder nested type.
var builderType = builder.IntroduceClass(
    "Builder",
    buildType: t => t.Accessibility = Accessibility.Public);

Now we can add properties to our new Builder type:

// Add builder properties and update the mapping.
foreach (var property in properties)
{
    property.BuilderProperty =
        builderType.IntroduceAutomaticProperty(
                property.SourceProperty.Name,
                property.SourceProperty.Type,
                buildProperty: p =>
                {
                    p.Accessibility = Accessibility.Public;
                    p.InitializerExpression = 
                        property.SourceProperty.InitializerExpression;
                })
            .Declaration;
}

Step 5. Creating the Builder public constructor

Our next task is to create the public constructor of the Builder nested type, which should have parameters for all required properties. Let’s add this code to the BuildAspect method:

// Add a builder constructor accepting the required properties and update the mapping.
builderType.IntroduceConstructor(
    nameof(this.BuilderConstructorTemplate),
    buildConstructor: c =>
    {
        c.Accessibility = Accessibility.Public;
        foreach (var property in properties.Where(m => m.IsRequired))
        {
            var parameter = c.AddParameter(
                NameHelper.ToParameterName(property.SourceProperty.Name),
                property.SourceProperty.Type);
            property.BuilderConstructorParameterIndex = parameter.Index;
        }
    });

Here is BuilderConstructorTemplate, the template for this constructor. You can see how we use the Tags and PropertyMapping objects. This code iterates through required properties and assigns a property of the Builder type to the value of the corresponding constructor parameter.

[Template]
private void BuilderConstructorTemplate()
{
    var tags = (Tags)meta.Tags.Source!;
    foreach (var property in tags.Properties.Where(p => p.IsRequired))
    {
        property.BuilderProperty!.Value =            
            eta.Target.Parameters[
                property.BuilderConstructorParameterIndex!.Value].Value;
    }
}

Step 6. Adding a constructor to the source type

Before we implement the Build method, we must implement the constructor in the source type. This code snippet from the BuildAspect method creates the constructor and its parameters:

// Add a constructor to the source type with all properties.
var sourceConstructor = builder.IntroduceConstructor(
        nameof(this.SourceConstructorTemplate),
        buildConstructor: c =>
        {
            c.Accessibility = Accessibility.Private;
            foreach (var property in properties)
            {
                var parameter = c.AddParameter(                             
                 NameHelper.ToParameterName(property.SourceProperty.Name),
                 property.SourceProperty.Type);

                 property.SourceConstructorParameterIndex =
                                                    parameter.Index;
            }
        })
    .Declaration;

The template for this constructor is SourceConstructorTemplate. It simply assigns properties based on constructor parameters.

[Template]
private void SourceConstructorTemplate()
{
    var tags = (Tags)meta.Tags.Source!;
    foreach (var property in tags.Properties)
    {
        property.SourceProperty.Value =
               meta.Target
              .Parameters[property.SourceConstructorParameterIndex!.Value]
              .Value;
    }
}

Step 7. Implementing the Build method

The Build method of the Builder type is responsible for creating an instance of the source (immutable) type from the values of the Builder.

// Add a Build method to the builder.
builderType.IntroduceMethod(
    nameof(this.BuildMethodTemplate),
    IntroductionScope.Instance,
    buildMethod: m =>
    {
        m.Name = "Build";
        m.Accessibility = Accessibility.Public;
        m.ReturnType = sourceType;
    });

The T# template for the Build method first invokes the newly introduced constructor, then tries to find and call the optional Validate method before returning the new instance of the source type.

[Template]
private dynamic BuildMethodTemplate()
{
    var tags = (Tags)meta.Tags.Source!;

    // Build the object.
    var instance = tags.SourceConstructor.Invoke(
        tags.Properties.Select(x => x.BuilderProperty!))!;

    // Find and invoke the Validate method, if any.
    var validateMethod = tags.SourceType.AllMethods.OfName("Validate")
        .SingleOrDefault(m => m.Parameters.Count == 0);

    if (validateMethod != null)
    {
        validateMethod.With((IExpression)instance).Invoke();
    }
    // Return the object.
    return instance;
}

Next implementation steps

I hope the previous steps gave you an idea of how Metalama works. Automating the implementation of the Builder pattern requires a few more steps, all covered in the Builder pattern example:

Generating the ToBuilder method
Coping with base and derived types
Handling collection types

Is it worth it?

As you can see, even with Metalama, automating the Builder pattern is not completely trivial. So, is it worth it?

It depends on how often the aspect will be used in your application. Typically, if an aspect is used fewer than a dozen times, automation may not be worthwhile. However, if you’re planning a large project with dozens or even hundreds of classes that would benefit from the builder pattern, then automating it is definitely worth the effort.

Remember, every project will have slightly different requirements for implementing the Builder pattern. To save time, start with the Builder pattern example, understand its principles, and customize it to your needs.

The benefit of automating the pattern implementation as an aspect is that when you want to change the pattern, you only have to edit a single class: the aspect.

Wrapping up

The Builder pattern has become one of the most important patterns in modern .NET, thanks to its focus on immutability. With the Builder pattern, you get the convenience of mutability during the configuration stage of a component, coupled with the safety and simplicity of immutability after the component has been built.

Implementing the Builder pattern traditionally involves a lot of boilerplate code. Fortunately, tools like Metalama make it easier to automate its implementation.

This article was first published on a https://blog.postsharp.net under the title Implementing the Builder pattern with Metalama.

The Memento Design Pattern in C#, Practically With Examples [2024]

Gael Fraiteur — Thu, 25 Jul 2024 07:00:01 +0000

The Memento pattern is one of the foundational design patterns. It helps build UI features like undo/redo or UI transactions. In this article, we will see how to implement the Memento pattern in C#. We use a small WPF-based Fish Tank Manager as a sample application.

What is the Memento design pattern?

The Memento design pattern is a software design pattern that provides the ability to restore an object to its previous state without revealing its internal structure. The aim is to provide standardization in scenarios where the object’s state needs to be saved and restored, such as the GUI undo/redo mechanism we presented earlier, but also for, for example, in-memory transactions.

The Memento pattern consists of three key components:

Originator : The object whose state needs to be saved and restored.
Memento : An object that contains a snapshot of the Originator’s state.
Caretaker : The object that keeps track of multiple mementos, often implementing mechanisms to store and retrieve them.

While using the Memento pattern, two processes occur:

Capture : When a state snapshot is needed, the Originator creates a Memento object and passes it to the Caretaker for safekeeping.
Restore : To restore the state, the Caretaker provides the stored Memento back to the Originator, which rewrites its state.

When to use the Memento pattern?

Here are some situations when you might use the Memento pattern:

Undo/Redo Functionality : When you need to implement undo and redo mechanisms, the Memento pattern is ideal. It allows you to save the state of an object at a certain point and revert back to it when needed. For example, in text editors, graphic editors, or games.
State History or Snapshot Tracking : If an application requires keeping a history of different states of an object, the Memento pattern can be useful. This is common in version control systems or in any system where you need to track changes over time.
Transaction Management : In applications where transactions are used (like database operations), the Memento pattern can help in saving the state before the transaction starts. If the transaction fails, the state can be restored to its original state.

When introduced in an application, the Memento pattern must be exhaustively implemented in the whole object model for the user experience to be consistent. This means significant implementation and maintenance costs.

Example: Fish in a Fish Tank

Imagine you are implementing an application that allows for tracking fish in your home fish tank.

The application consists of a list of fish, allowing you to add a fish, remove a fish, and edit a fish.

Let’s see how it looks, with its beatiful Undo button:

The app, which is implemented in WPF, initially has no functionality for undoing changes. We will add this as the article progresses. In particular, the Fish class, which represents a single fish, looks like this:

public sealed class Fish : ObservableRecipient
{
    private string? _name;
    private string? _species;
    private DateTime _dateAdded;

    public string? Name
    {
        get => this._name;
        set => this.SetProperty( ref this._name, value, true );
    }

    public string? Species
    {
        get => this._species;
        set => this.SetProperty( ref this._species, value, true );
    }

    public DateTime DateAdded
    {
        get => this._dateAdded;
        set => this.SetProperty( ref this._dateAdded, value, true );
    }
}

As you can see, the class is initially pretty simple and contains only the data fields.

Let’s also look at the data fields of the MainViewModel class, which we will change later:

private readonly IFishGenerator _fishGenerator;
private bool _isEditing;
private Fish? _currentFish;

public ObservableCollection<Fish> Fishes { get; } = new();

How to implement the Memento pattern in C#?

The Memento pattern is implemented by defining interfaces for the Memento, Originator, and Caretaker:

The IMementoable interface is implemented by originator objects that hold state needing to be captured and restored. It allows IMementoCaretaker to retrieve memento objects and restore the state of the object.
The IMemento interface is minimalistic: it just exposes an Originator property used by the IMementoCaretaker during the Undo process.
The IMementoCaretaker is a service that manages stored IMemento instances and uses them to restore states of IMementoable objects.

In C#, it’s best to define the implementation of the IMemento interface as a private nested class. This stays true to the primary goal of the Memento pattern to respect encapsulation and allow the memento to access the private state of the originator.

A straightforward approach involves manually implementing the IMementoable and IMemento interfaces. In our example, this involves declaring a Memento nested type in both MainViewModel and Fish.

Step 1: Implement the Caretaker class

First, we implement the Caretaker class that implements the IMementoCaretaker interface. Our implementation is very simple: just a stack of memento objects.

public sealed class MementoCaretaker : IMementoCaretaker
{
    private readonly Stack<IMemento> _mementos = new();

    public void CaptureSnapshot( IMementoable mementoable )
    {
        this._mementos.Push( mementoable.SaveToMemento() );
    }

    public void Undo()
    {
        if ( this._mementos.Count > 0 )
        {
            var memento = this._mementos.Pop();
            memento.Originator.RestoreMemento( memento );
        }
    }

    public bool CanUndo => this._mementos.Count > 0;
}

Step 2: Register the Caretaker service

Conceptually, the Caretaker is often a Singleton. In a modern .NET app, it’s a good practice to use appBuilder.Services.AddSingleton to register it.

services.AddSingleton<IMementoCaretaker, MementoCaretaker>();

Step 3: Implement the IMementoable interface

Here is the most boring and repetitive part. We must implement the IMementoable interface for all classes that hold undoable state. In each of these classes, we create the following artifacts:

A private nested Memento with the following members:
- For each mutable field or property of the originator class, a public field of the same name and type.
- An implementation of the IMemento interface, specifically the Originator property.
- A constructor that stores the fields and properties of the originator class into the properties of the Memento class, as well as the Originator property.
An implementation of the IMementoable interface including:
- The SaveToMemento method, returning an instance of the Memento nested class.
- The RestoreMemento method, copying the memento properties back into the originator’s fields and properties.

Let’s implement this pattern into the Fish class:

public sealed class Fish : ObservableRecipient, IMementoable
{
    private string? _name;
    private string? _species;
    private DateTime _dateAdded;

    public string? Name
    {
        get => this._name;
        set => this.SetProperty( ref this._name, value, true );
    }

    public string? Species
    {
        get => this._species;
        set => this.SetProperty( ref this._species, value, true );
    }

    public DateTime DateAdded
    {
        get => this._dateAdded;
        set => this.SetProperty( ref this._dateAdded, value, true );
    }

    public void RestoreMemento( IMemento memento )
    {
        if ( memento is not Memento s )
        {
            throw new InvalidOperationException( "Invalid snapshot." );
        }

        this.Name = s.Name;
        this.Species = s.Species;
        this.DateAdded = s.DateAdded;
    }

    public IMemento SaveToMemento()
    {
        return new Memento( this, this.Name, 
                            this.Species, this.DateAdded );
    }

    private sealed record Memento(
        Fish Originator,
        string? Name,
        string? Species,
        DateTime DateAdded ) : IMemento
    {
        IMementoable IMemento.Originator => this.Originator;
    }
}

Step 4: Coping with collection types

When implementing the Memento pattern, ensure that all mutable classes implement it. For collections, you have two options: either implement IMementoable collections or use immutable collections. In this example, we use the second approach. In MainViewModel, we trade our ObservableCollection<Fish> for an ImmutableList<Fish>. Using ImmutableList instead of, say, an ImmutableArray, is a good choice because ImmutableList tries to reuse as much memory as possible between different edits of the list, while ImmutableArray would always create a new copy.

Let’s see how the data fields of MainViewModel have changed:

private readonly IMementoCaretaker? _caretaker;
private readonly IFishGenerator _fishGenerator;
private bool _isEditing;
private Fish? _currentFish;
private ImmutableList<Fish> _fishes = ImmutableList<Fish>.Empty;

Also, see the Memento class for MainViewModel:

private class Memento : IMemento
{
    public IMementoable Originator { get; }

    public bool IsEditing { get; }

    public Fish? CurrentItem { get; }

    public ImmutableList<Fish> Items { get; }

    public Memento(
        MainViewModel snapshotable,
        bool isEditing,
        Fish? currentItem,
        ImmutableList<Fish> items )
    {
        this.Originator = snapshotable;
        this.IsEditing = isEditing;
        this.CurrentItem = currentItem;
        this.Items = items;
    }
}

Step 5: Calling CaptureMemento and Undo

So far, we have only implemented the IMementoable pattern, and we should start using it to add functionality to the application.

An inconvenience of the Memento pattern is that the IMementoCaretaker.CaptureMemento method must be called manually. If you’re implementing an undo/redo feature, you would typically call it before any operation.

private void ExecuteNew()
{
    this._caretaker?.CaptureSnapshot( this );

    this.Fishes = this.Fishes.Add(
        new Fish()
        {
            Name = this._fishGenerator.GetNewName(),
            Species = this._fishGenerator.GetNewSpecies(),
            DateAdded = DateTime.Now
        } );
}

If you’re still using modal dialogs with a transactional user interface based on the Ok and Cancel buttons, you can capture a memento when the dialog opens, and call Undo if the user hits Cancel.

For instance, here is the code that opens the UI in edit mode (ExecuteEdit) and handles the OK (ExecuteSave) and Cancel (ExecuteCancel) buttons.

private void ExecuteEdit()
{
    this.IsEditing = true;

    this._caretaker?.CaptureSnapshot( this._currentFish! );
}

private void ExecuteSave()
{
    this.IsEditing = false;
}

private void ExecuteCancel()
{
    this.IsEditing = false;
    this._caretaker?.Undo();
}

As you can see, the Memento pattern can be used to implement UI transactions.

How to implement Memento without boilerplate?

As you can judge from Step 3 above, implementing Memento types follows a very repetitive pattern. Each Memento type must declare fields that correspond to the target class, create a similar constructor, implement the same interface, etc. There is virtually no reason to implement this pattern by hand. Adopting a code generation technology saves both implementation time and maintenance costs. And a lot of gray hairs, because a code generator, unlike a human, won’t forget to add a property to the Memento when one is added to the Originator!

The .NET community offers several code generation technologies which can be grouped into two categories: run-time and compile-time generation. Run-time generation should be avoided because it adds a startup performance overhead and is difficult to debug. Within the second category, you have two options: source generators like Roslyn source generators or Metalama, and, historically, MSIL generators like Fody. Fody and Roslyn source generators are very low-level and tend to be overly complex. Metalama, on the other hand, offers a simpler API and development experience. Metalama is free to use. It is based on Roslyn and actually uses Roslyn source generators under the hood.

With Metalama, you can reduce the pattern to a single attribute, MementoAttribute, called an aspect. This aspect generates the Memento code on-the-fly during compilation, keeping your source code clean and succinct. The aspect automatically performs all the steps we mentioned above. Going through all implementation steps of this aspect is beyond the scope of this article. We refer you to The Memento Pattern in Metalama documentation for details.

Metalama also comes with other open-source aspects, such as Observable, which automatically implements the INotifyPropertyChanged interface, and WPF Command.

When we use these aspects in our project, the Fish class gets reduced to its essence:

[Memento]
[Observable]
public sealed partial class Fish
{
    public string? Name { get; set; }

    public string? Species { get; set; }

    public DateTime DateAdded { get; set; }
}

The code that handles IMementoable and INotifyPropertyChanged is no longer necessary in your source code because it is generated during compilation. So, we’ve saved work and, at the same time, improved the reliability and maintainability of the code. If we have 50 classes implementing the Memento pattern, we still have just one MementoAttribute to maintain.

How to automatically capture snapshots upon changes?

You’re certainly aware that there are two approaches to user interfaces for data entry. One is based on explicit Ok and Cancel buttons. The second auto-saves any change. In this case, a good undo/redo feature is almost mandatory.

If you are building an auto-save user experience, it’s a good idea to trigger the call to CaptureMemento whenever a property changes.

If your object model already implements INotifyPropertyChanged, it’s reasonably easy to subscribe to the PropertyChanged event and call CaptureMemento upon any change. For instance, we added this logic to FishControl.xaml.cs. If you have several windows or controls, you can move this logic to a base class, or even to an aspect if you’re using Metalama.

protected override void OnPropertyChanged( DependencyPropertyChangedEventArgs e )
{
    base.OnPropertyChanged( e );

    if ( e.Property.Name == nameof(this.DataContext) )
    {
        if ( e.OldValue is IMementoable and INotifyPropertyChanged newObservable )
        {
            newObservable.PropertyChanged -= this.OnMementoablePropertyChanged;
        }

        if ( e.NewValue is IMementoable newMementoable and INotifyPropertyChanged newObervable )
        {
            newObervable.PropertyChanged += this.OnMementoablePropertyChanged;

            // Capture _before_ any change.
            this._caretaker?.CaptureMemento( newMementoable );
        }
    }
}

private void OnMementoablePropertyChanged( object? sender, PropertyChangedEventArgs e )
{
    var mementoable = (IMementoable?) this.DataContext;

    if ( mementoable != null )
    {
        this._caretaker?.CaptureMemento( mementoable );
    }
}

In WPF, this approach will save a memento whenever the user focus switches from one control to another.

If your user is expected to type long texts, you might want to save the state more often. In this case, you can cause the binding to be updated on every keystroke. You can do this by hooking the KeyUp event.

<TextBox Grid.Row="0" Grid.Column="1" Margin="2" 
         Text="{Binding Name}" KeyUp="OnTextBoxUpdated" />

private void OnTextBoxUpdated( object sender, KeyEventArgs e )
{
    if ( sender is TextBox textBox )
    {
        var binding = BindingOperations.GetBindingExpression( 
                                  textBox, TextBox.TextProperty );

        if ( binding != null )
        {
            binding.UpdateSource();
        }
    }
}

However, this strategy creates a new snapshot for every keystroke, which is highly inconvenient both for the user and for memory usage.

To mitigate this problem, the Caretaker can ignore a new memento if the last memento is recent and of identical originator.

Modify the IMemento interface to this:
Modify the implementation pattern to set the value of the MementoTime property to DateTime.Now.
Change Caretaker.CaptureMemento to ignore requests that are too close to each other.

Now, you will have a savepoint every 5th second at most for each object.

How to implement IEditableObject using the Memento pattern

The IEditableObject is the system interface for editable objects that support Ok-Cancel semantics. It is used by controls such as DataGrid.

public interface IEditableObject
{
    void BeginEdit();
    void EndEdit();
    void CancelEdit();
}

It’s easy to implement IEditableObject if your class already implements IMementoable. Here is an example.

public abstract partial class EditableObject : IEditableObject, IMementoable
{
    private IMemento? _beforeEditMemento;

    public void BeginEdit()
    {
        if ( this._beforeEditMemento != null )
        {
            throw new InvalidOperationException();
        }

        this._beforeEditMemento = this.SaveToMemento();
    }

    public void CancelEdit()
    {
        if ( this._beforeEditMemento == null )
        {
            throw new InvalidOperationException();
        }

        this.RestoreMemento( this._beforeEditMemento );
    }

    public void EndEdit()
    {
        if ( this._beforeEditMemento == null )
        {
            throw new InvalidOperationException();
        }

        this._beforeEditMemento = null;
    }

    public abstract IMemento SaveToMemento();

    public abstract void RestoreMemento( IMemento memento );
}

What are the common problems of the Memento pattern?

The Memento pattern is very useful. However, when deciding whether to use it and how to implement it in detail, there are some challenges to keep in mind.

Problem 1. Working with object graphs

The Memento pattern has been formalized to take snapshots of a single class instance. However, in most practical applications, we will work not with isolated object instances but with object graphs. And we will want to take a memento of the whole graph.

This problem is not addressed by the classic Memento pattern, and we see that as its major limitation.

If we want to follow the Memento pattern with multi-object operations, we will need to call the CaptureMemento method for all objects in the operation. We will need to improve the Caretaker to support multi-object undo operations.

void CaptureMemento( IEnumerable<IMementoable> originators );

It’s important that all mementos captured in a single call of CaptureMemento are also restored in a single call of Undo.

Problem 2. Memory consumption

Storing a large number of mementos can lead to significant memory usage. This is especially true for collections.

One solution is to selectively save only the necessary information, such as changes or deltas, especially on large objects. This can significantly reduce memory usage but is very challenging to implement. This solution, however, deviates from the pure Memento pattern.

Another approach is limiting the available history to a reasonable number of changes by removing the oldest mementos when the history contains more than this limit.

Problem 3. Working with collections

Dealing with mutable collections within the Originator’s state requires careful handling to prevent inconsistencies. Simply copying the data field will result in a reference to the same collection, which may be shared between the live state and multiple mementos. This violates the purpose of the Memento pattern.

There are two ways to address this issue:

Deep copy : The collection can be copied when the Memento is being captured. The problem with this approach is very high memory consumption.
Immutable collections : The target class uses fully immutable collections, which are remembered by the memento as normal values.

In the sample, we have used the second approach. While there are some performance and memory costs associated with immutable collections, it’s the simplest way of working with collections with the Memento pattern.

Problem 4. State isolation during edits

Last and not least, a major drawback of the Memento pattern is that user edits will affect the “good” instance of the object instead of a “draft” one. If your application has background tasks, they will see unconfirmed changes, that were not yet saved by clicking the Ok button. Writing temporary edits to a “good” instance can create serious problems because neither your background task nor the user expects unconfirmed changes to be considered.

To avoid this scenario, performing edits on a clone of the object is preferable. If edits are canceled, the clone is just discarded. If edits are confirmed, a memento is taken from the clone and restored into the “good” copy.

Conclusion

The Memento pattern is a classic and simple design pattern that has several applications in the UI layer. One of its main drawbacks is that it requires much boilerplate code to implement, but code generators such as Metalama can help reduce repetitive work to almost zero. As always with classic patterns, Memento should not be applied dogmatically. However, it remains a great source of inspiration in your journey to build reliable and user-friendly applications.

This article was first published on a https://blog.postsharp.net under the title The Memento Design Pattern in C#, Practically With Examples [2024].

Adding Serilog to ASP.NET Core: a practical guide

Gael Fraiteur — Thu, 18 Jul 2024 07:00:01 +0000

Serilog is a logging library for .NET. It can be used on its own, but it is also compatible with Microsoft.Extensions.Logging, making it ideal for ASP.NET Core applications. In this article, we’ll explain why you should use Serilog in your ASP.NET Core application and demonstrate how to integrate it into your project. We’ll also cover ASP.NET middleware as a way to enrich your logs with contextual properties, and techniques to add high-verbosity logging without boilerplate code.

Why use Serilog in ASP.NET Core?

By default, ASP.NET Core apps have a built-in logging system thanks to the Microsoft.Extensions.Logging namespace. However, it lacks many useful features. And while it is extensible, someone has to implement and maintain those extensions. Therefore, it often makes sense to use a third-party library like Serilog, which is already feature-rich and well-maintained.

Here are the features that make Serilog a good choice for ASP.NET Core applications:

Contexts and enrichers: While Microsoft.Extensions.Logging supports scopes, they are quite limited and verbose. Serilog contexts make it much easier to add custom contextual information to your logs. Enrichers provide automatic ways of adding commonly used information, such as the correlation ID.
Sinks, formatting, and filtering: Microsoft.Extensions.Logging supports a limited number of logging providers, while Serilog offers about a hundred different sinks, including logging to a file and various cloud services and databases. Serilog also supports advanced formatting and filtering options.
Structured logging: Microsoft.Extensions.Logging has only very limited support for structured logging, while Serilog is built around structured logging. This makes it easier to query and analyze logs, especially when you have a large number of logs.

Using Serilog is not an all-or-nothing decision. As long as you use only Serilog sinks, you can, for example, combine Microsoft.Extensions.Logging scopes and Serilog contexts.

Adding Serilog to your ASP.NET Core project

Step 1. Add the Serilog packages

To add Serilog to your ASP.NET Core project, install the Serilog.AspNetCore NuGet package:

dotnet add package Serilog.AspNetCore

This package includes support for registering Serilog with your application and optionally enabling request logging.

Step 2. Add the Serilog service

To configure Serilog, add code like the following to your Program.cs file:

Log.Logger = new LoggerConfiguration()
    .WriteTo.Console()
    .CreateLogger();

builder.Services.AddSerilog();

You can also delete the now unnecessary Microsoft.Extensions.Logging configuration from your appsettings.json file.

The console output from an ASP.NET Core Web API application will now look something like this:

[15:56:46 INF] Now listening on: http://localhost:5106
[15:56:46 INF] Application started. Press Ctrl+C to shut down.
[15:56:46 INF] Hosting environment: Development
[15:56:46 INF] Content root path: /mnt/c/src/TimelessDotNetEngineer/src/logging/serilog-aspnetcore/SerilogInAspNetCore
[15:56:53 INF] Request starting HTTP/1.1 GET http://localhost:5106/swagger/index.html - null null
[15:56:53 INF] Request finished HTTP/1.1 GET http://localhost:5106/swagger/index.html - 200 null text/html;charset=utf-8 71.1747ms
[15:56:53 INF] Request starting HTTP/1.1 GET http://localhost:5106/swagger/v1/swagger.json - null null
[15:56:54 INF] Request finished HTTP/1.1 GET http://localhost:5106/swagger/v1/swagger.json - 200 null application/json;charset=utf-8 70.2124ms

Step 3. Enable HTTP request logging

Since the default ASP.NET Core logging is very noisy, it’s a good practice to reduce its verbosity (using .MinimumLevel.Override(...)) and replace it with Serilog request logging by adding a call to app.UseSerilogRequestLogging() in your Program.cs:

Log.Logger = new LoggerConfiguration()
    .WriteTo.Console()
    .MinimumLevel.Verbose()
    .MinimumLevel.Override( "Microsoft.AspNetCore", LogEventLevel.Warning )
    .MinimumLevel.Override( "Microsoft.Extensions.Hosting", LogEventLevel.Information )
    .MinimumLevel.Override( "Microsoft.Hosting", LogEventLevel.Information )
    .CreateLogger();

builder.Services.AddSerilog();
var app = builder.Build();
app.UseSerilogRequestLogging();

Using that, the output will be nicer:

[16:18:03 INF] Now listening on: http://localhost:5106
[16:18:03 INF] Application started. Press Ctrl+C to shut down.
[16:18:03 INF] Hosting environment: Development
[16:18:03 INF] Content root path: /mnt/c/src/TimelessDotNetEngineer/src/logging/serilog-aspnetcore/SerilogInAspNetCore
[16:18:07 INF] HTTP GET /swagger/index.html responded 200 in 57.3368 ms
[16:18:07 INF] HTTP GET /swagger/v1/swagger.json responded 200 in 68.1877 ms

Adding logging to your code

Now that Serilog is properly configured, let’s talk about how and when to add logging to your code.

Step 1. Pull the ILogger service

The first step is to pull the logging interface from the dependency injection container. It’s recommended to use the regular Microsoft.Extensions.Logging.ILogger interface instead of the product-specific Serilog.ILogger one because it will make your code more standard and portable. However, there are no significant differences between these interfaces.

Instead of pulling the non-generic ILogger interface, pull the generic ILogger<T> interface where T is the current class. This trick will allow you to adjust the verbosity of the logging for this specific class and easily filter or search messages reported by this specific class.

[ApiController]
[Route( "[controller]" )]
public class WeatherForecastController( ILogger<WeatherForecastController> logger ) : ControllerBase

Step 2. Add logging instructions

The general rules are not specific to Serilog: you should log whenever something happens that might be useful to troubleshoot a problem in production. Use the LogError, LogWarning, LogInformation, and LogVerbose methods of the ILogger interface to write messages of different severities.

By default, Serilog will use the ToString() method to format parameters to text. If you want Serilog to include the JSON representation of the log parameter instead, you can use the @ destructuring operator. This will work even if you use the regular Microsoft.Extensions.Logging.ILogger interface.

logger.LogDebug(
    "Returning weather forecast for the {days} days after today: {@forecast}",
    days,
    forecast );

This will produce a log message like this:

[19:09:56 DBG] Returning weather forecast for the 5 days after today: [{"Date": "2024-07-10", "Temperature": 31, "$type": "WeatherForecast"}, {"Date": "2024-07-11", "Temperature": 26, "$type": "WeatherForecast"}, {"Date": "2024-07-12", "Temperature": 31, "$type": "WeatherForecast"}, {"Date": "2024-07-13", "Temperature": 19, "$type": "WeatherForecast"}, {"Date": "2024-07-14", "Temperature": 25, "$type": "WeatherForecast"}]

Step 3. Optionally, add scope information

In the examples above, we wrote the logs as text to the console. Serilog supports several semantic backends like Elasticsearch or the .NET Aspire dashboard, where Serilog messages are stored as JSON objects. This allows you to easily search and filter messages based on object queries rather than text. This is what is meant by semantic logging.

Serilog supports several mechanisms to add properties to the JSON payload. Often, you’ll want to search according to the execution context (the current username, client IP, operation, etc.), so it’s a good idea to enrich messages with these properties.

To enable this feature, the first thing to do is add a call to .Enrich.FromLogContext() in your Serilog configuration code.

If you want to visualize properties in the console or text output, you also need to override the console message template.

These two steps are achieved by the following snippet:

Log.Logger = new LoggerConfiguration()
    .Enrich.FromLogContext()
    .WriteTo.Console( outputTemplate: "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Properties}{NewLine}{Exception}" )

We can now add contextual information to our logging.

The first and standard way is to use the BeginScope method of the Microsoft.Extensions.Logging.ILogger interface:

using (logger.BeginScope( "Getting weather forecast for {ScopeDays} days", days ))

This approach defines a text property named Scope, which is not very semantically searchable.

Serilog’s proprietary API offers a better solution: adding properties to the current execution context using the LogContext.PushProperty static method. You can use it from anywhere. If you want to enrich the logs with information about the current HTTP request, it’s best to create an ASP.NET Middleware. A middleware is a C# class with a single method InvokeAsync that gets called in the request pipeline, before any controller or API is called. Here is an example that pushes two properties: Client and RequestId.

public sealed class PushPropertiesMiddleware : IMiddleware
{
    public async Task InvokeAsync( HttpContext context, RequestDelegate next )
    {
        var requestId = Guid.NewGuid().ToString();

        using (LogContext.PushProperty( "Host", context.Request.Host ))
        using (LogContext.PushProperty( "RequestId", requestId ))
        {
            await next( context );
        }
    }
}

To add the middleware to the pipeline, you must first add it to the service collection:

builder.Services.AddSingleton<PushPropertiesMiddleware>();

Then, call the UseMiddleware method.

app.UseMiddleware<PushPropertiesMiddleware>();

Keep in mind that using a middleware is just one of the options. You can call LogContext.PushProperty from anywhere.

Automatically adding verbose logging to your code

Adding logging to your code by hand makes sense for low-volume, high-relevance pieces of information such as errors, warnings, and important messages. However, it would be a severe waste of time for more noisy messages. If all you want is better request tracing, you can write messages from your custom middleware. If you want logging deep inside your application and still avoid boilerplate, you can use a free code generation tool like Metalama.

With Metalama, you can create a template (called an aspect) that teaches the compiler how to implement logging, then apply the templates to all required methods.

First, install the Metalama.Extensions.DependencyInjection. Most of the magic is done by the Metalama.Framework package, which is implicitly added.

Then, create an aspect to fit your logging needs. Here’s a simple example:

public class LogAttribute : OverrideMethodAspect
{
    [IntroduceDependency]
    private readonly ILogger _logger;

    public override dynamic? OverrideMethod()
    {
        var formatString = BuildFormatString();

        try
        {
            _logger.LogDebug( formatString + " started.", (object[])meta.Target.Parameters.ToValueArray() );

            return meta.Proceed();
        }
        finally
        {
            _logger.LogDebug( formatString + " finished.", (object[])meta.Target.Parameters.ToValueArray() );
        }
    }

    [CompileTime]
    private static string BuildFormatString()
    {
        var parameters = meta.Target.Parameters
            .Where( x => x.RefKind != RefKind.Out )
            .Select( p => $"{p.Name}: {{{p.Name}}}" );

        return $"{meta.Target.Type}.{meta.Target.Method.Name}({string.Join( ", ", parameters )})";
    }
}

You can now add the aspect to all methods you want to log. You can mark methods one at a time using the [Log] attribute. Another option is to use a Metalama fabric (a commercial feature) to add logging more broadly using a single piece of code.

internal class Fabric : ProjectFabric
{
    public override void AmendProject( IProjectAmender amender )
    {
        amender.Outbound
            .Select( c => c.GlobalNamespace.GetDescendant( "SerilogInAspNetCore.Controllers" )! )
            .SelectMany( c => c.Types )
            .Where( t => t.Accessibility == Accessibility.Public )
            .SelectMany( c => c.Methods )
            .Where( m => m.Accessibility == Accessibility.Public )
            .AddAspectIfEligible<LogAttribute>();
    }
}

When applied to a WeatherForecastController.Get(int days) method (whether using an attribute or a fabric), the aspect generates code equivalent to the following at the start of the method:

this._logger.LogTrace( "WeatherForecastController.Get(days: {days}) started.", days );

Summary

Serilog is an excellent semantic logging provider and it’s fully compatible with ASP.NET Core. In this article, we discussed how to set up Serilog, and how to use it in your code in different scenarios, including the use of an ASP.NET middleware. We mentioned Metalama as a possible tool to add verbose logging to your code without boilerplate.

This article was first published on a https://blog.postsharp.net under the title Adding Serilog to ASP.NET Core: a practical guide.

MemoryCache in C#: A Practical Guide

Gael Fraiteur — Thu, 18 Jul 2024 07:00:01 +0000

MemoryCache is the standard implementation of an in-memory cache for C# applications. It helps boost performance and scalability at the cost of increased memory consumption. This article covers practical questions about when and how to use MemoryCache.

When to Use In-Memory Caching
How fast is MemoryCache
What are the Disadvantages of In-Memory Cache
How to Add MemoryCache to an ASP.NET Core App
How to Expire Cache Entries
How To Reduce Strong Coupling of Your Code With Caching
How To Avoid Boilerplate Code With Caching

When Should We Use In-Memory Caching?

In-memory caching is beneficial for virtually all types of applications: mobile, server- or client-side web, cloud, and desktop. It’s especially useful when a computation or request is resource-intensive or time-consuming and is likely to be reused later. Consider caching as your first line of defense against hefty computation and cloud consumption costs.

As a general guideline, an operation should be cached if it exhibits at least three of the following properties:

Small in size, so it does not consume too much memory.
Frequently used.
Expensive to obtain.
Stable over time (doesn’t change often).

For example:

A product catalog is a great candidate for caching because it does not change often, has a small size, and typically requires dozens of database calls to build.
An order list is large and changes often, so it probably should not be cached.

A Practical Example in ASP.NET Core: Retrieving Currency Data Without Caching

To illustrate the use case for caching, we’ll use a small ASP.NET Core application that displays the prices of two fiat currencies and two cryptocurrencies.

Our goal is to display current Bitcoin and Ethereum price data in USD to the webpage user. So, we query the API that provides live data from CoinCap and render the data for the user.

The GetCurrencyData method from the model class retrieves its data from an HTTP service. The IHttpClientFactory dependency is pulled using the primary constructor.

namespace Sample1.Pages;

public class NoCachingModel( IHttpClientFactory httpClientFactory ) : BaseModel
{
    public async Task<CoinCapData> GetCurrencyData( string id )
    {
        var response = await httpClient.GetFromJsonAsync<CoinCapResponse>(
                          $"https://api.coincap.io/v2/rates/{id}" );

        return response!.Data;
    }
}

The list of currencies is defined in the base model class.

namespace Sample1.Pages;

public class BaseModel : PageModel
{
    public IReadOnlyList<string> Currencies { get; } =
    [
        "bitcoin",
        "ethereum",
        "euro",
        "british-pound-sterling"
    ];

}

And here is the code of the page that displays the currencies:

@page "/"
@using System.Globalization
@model NoCachingModel
@{
    ViewData["Title"] = "Home page";
}

<div class="text-center">
    @{
        foreach (var currency in Model.Currencies)
        {
            var data = await Model.GetCurrencyData( currency );

            // Output a paragraph with symbol and rate.
            @:<p>@( data.Symbol )(@data.Type): 
           @data.RateUsd.ToString("F3", CultureInfo.InvariantCulture )</p>
        }
    }
</div>

As an aside, we’ve included code using ASP.NET middleware to monitor the time it takes to render the page. This will display the internal processing time on the page, which we will reference throughout the article:

processed in 397 ms.

However, every view of our page triggers four data retrievals from the service. This leads to several problems:

Latency. The page load time is around 400 ms in my location. This latency can add up, potentially increasing the load time to several seconds if left unaddressed.
Costs. Our server uses the API, which could result in higher cloud hosting costs.
Reliability. Data providers typically enforce rate limits. In our case, Coincap.io allows roughly 200 requests per second, which may seem high. However, our page makes four requests per page load, and a spike of 50 requests per second is plausible.

We can tackle all these issues by employing MemoryCache.

How Fast is MemoryCache?

MemoryCache is very fast. The cached data resides in the same process that uses them and is stored as regular .NET objects. All MemoryCache does is store a reference to your object, associate it with a key and prevent it from being garbage-collected.

In contrast, with distributed caching servers like Redis, the cache is located on a different machine, often a dedicated one, and is shared among multiple servers. A distributed cache does not store .NET objects but binary or text (often JSON or XML) data. Therefore, .NET objects should be serialized before being stored in the cache, and deserialized after being retrieved. With MemoryCache, serialization and deserialization is not necessary.

So, how fast is MemoryCache? On my development notebook, I measured:

TryGetValue.TryGetValue takes ~50 ns.
TryGetValue.Set takes ~110 ns if the item is overwritten or ~100 ns if the item is new.

That means that you should not bother about the performance of MemoryCache even if your app is serving tens of thousands of requests per second and per instance.

Is MemoryCache the fastest cache for C#?

Even if it’s very fast, MemoryCache is not the fastest cache for C#. However, it provides a good balance between features and performance.

If you never want to remove anything from the cache, ConcurrentDictionary<string,object> is faster. However you risk running out of memory unless you are caching a small dataset.
To cache even more performance-critical data, building the string-based cache key may become the performance bottleneck. Consider using a ConditionalWeakTable, use lazy initialization, or the Memoization pattern.

What are the Disadvantages of In-Memory Cache?

Cache incoherence in distributed apps. If your application is deployed to several instances, you will have several in-memory caches, each in a different machine, and they need to be synchronized. One node may not be aware of the changes done by the second node, and even if the database is consistent, it will not get to the database because of its own local cache. Incoherent caches can result in users seeing different data from different servers. This is one of the problems addressed by distributed caching.
Cache invalidation. As with every caching implementation, adding data to a MemoryCache and getting it back is the easy part, but removing it when it is no longer valid can be notoriously challenging.
Danger of mutable objects. Since MemoryCache does not serialize/deserialize objects, a frequent mistake is adding mutable objects to the cache. You must only add immutable objects to the cache, or you prepare yourself for hours of painful debugging.

How to Add MemoryCache to an ASP.NET Core App?

The MemoryCache class is included in the Microsoft.Extensions.Caching.Memory package. This package is automatically included in ASP.NET Core applications. If you are building a different kind of applications, you may have to add this package to your project. Typically, we would not write code directly against the MemoryCache class but against IMemoryCache, its abstraction.

At the lowest level, the IMemoryCache interface provides the following basic methods:

ICacheEntry CreateEntry(object key) - adds an entry into the cache for the given cache key. If the entry already exists, it’s overwritten.
bool TryGetValue(object key, out object? value) - retrieves the value stored in the cache for the given key, if it exists.
void Remove(object key) - removes an entry for the given key.

Most of the time, we will not use these basic methods in our code, but instead the high-level GetOrCreate or GetOrCreateAsync extension methods. Given a cache key, these methods will:

Check if the cache contains the result we are interested in. If yes, return it to the caller.
Invoke the logic to get the data.
Add the data to the cache.

Step 1. Initializing MemoryCache

To start, we need to add the memory cache component into our service collection.

In Program.cs, add the memory cache to the WebApplicationBuilder as follows:

builder.Services.AddMemoryCache();

Step 2. Injecting IMemoryCache

The next step is to inject the IMemoryCache into your page or model.

If you are adding code directly to the page, inject the IMemoryCache into our page model using the following directive:

@inject IMemoryCache Cache

If you are writing pure C# instead of Razor, pull the IMemoryCache service.

public class WithMemoryCacheModel(
    IHttpClientFactory httpClientFactory, IMemoryCache memoryCache ) 
    : BaseModel

Step 3. Add the caching logic to your methods

Now, you can add the caching to our data-retrieval method by adding a call to IMemoryCache.GetOrCreateAsync.

A convenient pattern is to move the original method logic to a local function, like in the following code snippet.

public async Task<CoinCapData> GetCurrencyData( string id )
{
    return (await memoryCache.GetOrCreateAsync(
        $"{this.GetType().Name}.GetCurrencyData({id})",
        _ => GetData() ))!;

    async Task<CoinCapData> GetData()
    {
        var response = await httpClient.GetFromJsonAsync<CoinCapResponse>(
                 $"https://api.coincap.io/v2/rates/{id}" );

        return response!.Data;
    }
}

Each time the method is called, the cache is checked. If the data is available, it’s retrieved. If not, the local function is invoked, and the result is stored in the cache. When users visit our site, the first request populates the cache with the results of GetData, and all subsequent requests use this data. This approach significantly reduces user latency, lowers CPU usage, and decreases external API usage. As a result, we improve the user experience, save on costs, and enhance reliability, all for a small memory trade-off.

It’s important to note that all method parameters should be added to the caching key. Additionally, we need to ensure that our method’s cache key does not collide with the cache keys of other methods.

Running the app, we can see that the local latency of our example has now dropped to 1 ms, down from its previous 397 ms.

processed in 1 ms

How to Expire Cache Entries?

In the example above, we successfully improved the service’s latency and reliability. However, we inadvertently made it useless because it now displays constant prices. We’ve added data to the cache, but we haven’t implemented any mechanism to refresh the data. This mechanism is referred to as expiration or expiry.

Let’s say we want the data to be at most 30 seconds old.

In MemoryCache, the mechanism we’re after is called absolute expiration. We can set this by calling the SetAbsoluteExpiration method of the ICacheEntry object that GetOrCreateAsync provides.

public async Task<CoinCapData> GetCurrencyData( string id )
{
    return
        (await memoryCache.GetOrCreateAsync(
            $"{this.GetType().Name}.{id}",
            async entry =>
            {
                entry.SetAbsoluteExpiration( TimeSpan.FromSeconds( 30 ) );

                return await GetData();
            } ))!;

    async Task<CoinCapData> GetData()
    {
        var response = await httpClient.GetFromJsonAsync<CoinCapResponse>(
                       $"https://api.coincap.io/v2/rates/{id}" );

        return response!.Data;
    }
}

Now, MemoryCache will delete the entry approximately 30 seconds after its creation. Most requests will still have a latency of 1 ms, but one request every 30 seconds will be slow as it needs to hit the currency service.

How To Reduce Strong Coupling of Your Code With Caching?

Hardcoding your application to use IMemoryCache is not always the best approach. If there is a chance that you might want to change the caching strategy in the future, consider using an intermediate layer between your source code and the caching component.

Polly is a popular .NET package that helps handle transient faults and enhances application resilience. One of the available policies is Polly.Caching.Memory, which internally uses IMemoryCache. Polly offers several other useful policies. For instance, you might want to retry the operation if it fails. With Polly, this is simple. Polly also makes it easier to switch from MemoryCache to a distributed cache.

Let’s see how we can use Polly to add caching to your app.

Step 1. Adding Polly and configuring a caching policy

To set it up, we need to add the following code during application initialization:

builder.Services.AddSingleton<IAsyncCacheProvider, MemoryCacheProvider>();
builder.Services.AddSingleton<IReadOnlyPolicyRegistry<string>, PolicyRegistry>(
    serviceProvider =>
    {
        var cachingPolicy = Policy.CacheAsync(
            serviceProvider.GetRequiredService<IAsyncCacheProvider>(),
            TimeSpan.FromMinutes(0.5));

        var registry = new PolicyRegistry {
                              ["defaultPolicy"] = cachingPolicy };

        return registry;
    });

Step 2. Inject the policy into your page, model or service

Next, we need to incorporate the resilience policy into our model:

private readonly IAsyncPolicy _cachePolicy;
private IHttpClientFactory _httpClientFactory;

public Step4Model( 
   IReadOnlyPolicyRegistry<string> policyRegistry, 
   IHttpClientFactory httpClientFactory )
{
    this._httpClientFactory = httpClientFactory;
    this._cachePolicy = policyRegistry.Get<IAsyncPolicy>( 
                                                     "defaultPolicy" );
}

Step 3. Call the Polly policy from your methods

Lastly, we can use the cache policy in our GetCurrencyData method:

public async Task<CoinCapData> GetCurrencyData( string id )
{
    return await this._cachePolicy.ExecuteAsync(
        async _ => await GetData(),
        new Context( $"{this.GetType().Name}.GetCurrencyData({id})" ) );

    async Task<CoinCapData> GetData()
    {
        var response = await httpClient.GetFromJsonAsync<CoinCapResponse>(
                               $"https://api.coincap.io/v2/rates/{id}" );

        return response!.Data;
    }
}

If, later on, you want to add some resilience feature, you can do it with editing only the initialization code. In the next code snippet, we are adding a retry policy to the caching policy.

builder.Services.AddSingleton<IAsyncCacheProvider, MemoryCacheProvider>();

builder.Services.AddSingleton<IReadOnlyPolicyRegistry<string>, PolicyRegistry>(
    serviceProvider =>
    {
        var cachingPolicy = Policy.CacheAsync(
            serviceProvider.GetRequiredService<IAsyncCacheProvider>(),
            TimeSpan.FromMinutes( 0.5 ) );

        var retryPolicy = Policy.Handle<Exception>().RetryAsync();

        var policy = Policy.WrapAsync( cachingPolicy, retryPolicy );

        var registry = new PolicyRegistry { ["defaultPolicy"] = policy };

        return registry;
    } );

Alternative: EasyCaching

Another alternative to using MemoryCache directly in your code is EasyCaching. The main benefit of EasyCaching is that it provides implementations for several caching servers, making it easier for you to migrate to distributed caching if needed. It also handles the serialization and deserialization of .NET objects from/into binary or text data.

How To Avoid Boilerplate Code With Caching?

You may have noticed that the examples above require a considerable amount of repetitive code, as each method needs to be wrapped in a delegate call and generate the cache key.

This repetitive code can be eliminated by using Metalama, a code generation and validation toolkit for C#. The free edition will suffice if you just want to use it for caching.

There are two ways to implement caching using Metalama.

One is to write a code template (called aspect) manually. This is beyond the scope of this article, and you can check the caching example to learn more.

The second is to use the Metalama.Patterns.Caching.Aspects package that already implements the aspect.

The initial setup is straightforward and requires just two lines in our Program.cs:

builder.Services.AddMetalamaCaching();

You then mark your method with a the [Cache] attribute:

[Cache( AbsoluteExpiration = 0.5 )]
public async Task<CoinCapData> GetCurrencyData( string id )
{
    var response = await httpClient.GetFromJsonAsync<CoinCapResponse>(
                         $"https://api.coincap.io/v2/rates/{id}" );

    return response!.Data;
}

And that’s all! When building the project, Metalama generates all the necessary code without us having to modify anything in the method.

Like EasyCaching, Metalama.Patterns.Caching is agnostic of the caching provider. It also handles serialization and will fail your build if you are trying to cache something uncacheable – for instance, an enumerator. Unlike other solutions, it handles the hassle of generating cache keys.

Summary

MemoryCache is the C# implementation of an in-memory cache. You can use it to improve your application performance and reduce its resource consumption at the cost of an increased memory usage. We discussed situations where using caching is not suitable – either because of the nature of the operation or the topology of your deployment.

Although you can use MemoryCache directly in your code, this might not be the best option if you want to keep your source code robust and maintainable. Using an intermediate layer like Polly, EasyCaching, or Metalama Caching will make it easier to evolve your code in the future. Additionally, Metalama significantly reduces the amount of code you need to write, as it boils down to a single custom attribute.

This article was first published on a https://blog.postsharp.net under the title MemoryCache in C#: A Practical Guide.

Simplify Your .NET Aspire Caching With Metalama

Gael Fraiteur — Mon, 08 Jul 2024 07:00:02 +0000

As you may already know, .NET Aspire makes it very easy to build distributed cloud-native applications. It manages all the wiring between the different services of your application and other components like databases, messaging, and caching. One of the services you can use with .NET Aspire straight out of the box is Redis, a great solution for distributed caching. There are many points in your app where you can add caching (and one of the ways is to use Polly). In this article, we will see how to cache method results without boilerplate code thanks to a special kind of custom attribute called aspect. We’ll see how Metalama can generate the caching boilerplate code for you based on this custom attribute.

Metalama is a free code generation framework for C#. It helps you write and maintain less code by eliminating boilerplate, generating it dynamically during compilation. It is built upon the concept of aspect, which encapsulates a code pattern. Most of the time, aspects are also custom attributes, but special kinds of custom attributes because they act like code templates and modify the code they are applied to.

Thanks to Metalama’s [Cache] aspect, adding caching to a method is as easy as applying the [Cache] custom attribute:

[Cache]
public async Task<IEnumerable<Todo>> GetTodosAsync( CancellationToken cancellationToken = default )
    => await db.Todos.ToListAsync( cancellationToken );

[Cache]
public async Task<Todo?> GetTodoAsync( int id, CancellationToken cancellationToken = default )
    => await db.Todos.FindAsync( id );

Under the hood, the [Cache] aspect:

pulls the necessary services into your class,
intercepts the method calls and adds the caching behavior,
generates cache keys for you,
serializes and deserializes the cached objects,
handles “weird” types to cache like IEnumerable or streams.

In this article, we demonstrate how to configure both .NET Aspire and Metalama to add caching with minimal effort, keeping your code clean of boilerplate. You’ll see, this is really a piece of cake.

The example app

Before we start, a few words about the example app. It is a to-do list app with an ASP.NET Core Minimal API backend and a Blazor front-end, orchestrated using .NET Aspire. The app stores the tasks using SQL Server, with an Entity Framework front-end.

To improve the app’s responsiveness and relieve the load from the database, we’d like to cache the to-do list, as it is more often displayed than updated. However, we can’t use HTTP response caching because we need to invalidate the cache when the data is updated.

Setting up the project

Step 1. Create a solution with the .NET Aspire template

To follow this tutorial, you will need an IDE supporting the .NET Aspire workload and an OCI-compliant container runtime, such as Docker Desktop or Podman.

If you’re not familiar with .NET Aspire, we recommend trying out the .NET Aspire Starter Application first.

The template will create several projects:

The app host project, containing the service orchestration. You can think of this project as the bootstrapper of other projects.
The web API project,
The web front-end project,
A service default project that contains shared service configuration,
A data project shared between the API and front-end projects.

Step 2. Set up .NET Aspire app host

Firstly, we need .NET Aspire to run and route the Redis cache server. To allow the Redis cache server to be orchestrated by .NET Aspire, we need to add it to the app host.

Add Aspire.Hosting.Redis NuGet package to the app host project.
Add the Redis cache to your app host and reference it in the app that will use the caching. In our case, we’ll use the caching in the API app.

var cache = builder
    .AddRedis( "cache" );

var api = builder
    .AddProject<Projects.TodoList_Api>( "todolist-api" )
    .WithReference( db )
    .WithReference( cache );

The name "cache" will be used among the .NET Aspire orchestrated apps to reference the Redis cache added in this step.

Step 3. Set up the app

With the app host prepared, we can proceed to set up the app that’s going to use the cache.

To allow Metalama Caching with Redis to be used, add the following NuGet packages. In our example, we use the cache in the API project, so we add these packages into it.
In the API project, we use the AddRedisClient to add the Redis client, an object of type IConnectionMultiplexer, to our service collection. The "cache" argument is the name of the Redis service previously defined in the app host.
Finally, we call AddMetalamaCaching. By default, an in-memory caching service will be created, so we must not forget calling the Redis method. It implicitly consumes the IConnectionMultiplexer service added by .NET Aspire.

Step 4. Add the caching to your code

You can now add caching to your methods.

Caching a method

As we’ve shown in the introduction, adding caching to a method is now as easy as adding the [Cache] attribute.

A few points will require your attention.

First, you must ensure that the parameters of a cached method generate a meaningful caching key. The ToString() method is used by default, and Metalama offers several strategies to customize it.
Then, you might want to check that all return values are safely JSON-serializable. Metalama transparently handles special types like streams, enumerables, or enumerators.

Invalidating the cache

Now that we’ve set the reading methods to be cached, they will always return the same result even if we add or modify todos. We must now remove items from the cache when the underlying data has changed – a problem called cache invalidation.

The simplest approach is to use the [InvalidateCache] aspect, as you can see in the DeleteTodoAsync() method.

[InvalidateCache( nameof(this.GetTodosAsync), nameof(this.GetTodoAsync) )]
public async Task<bool> DeleteTodoAsync( int id, CancellationToken cancellationToken = default )
{
    var todo = await this.GetTodoAsync( id, cancellationToken );

    if ( todo is null )
    {
        return false;
    }

    db.Todos.Remove( todo );
    await db.SaveChangesAsync( cancellationToken );

    return true;
}

This aspect matches the method arguments with the cache keys and invalidates the proper cache items when a to-do item gets deleted.

As you probably know, cache invalidation is the second hardest problem in computer science, so there’s way more to cache invalidation in Metalama Caching besides the [InvalidateCache] aspect: imperative invalidation, and even cache dependencies.

Summary

.NET Aspire and Metalama make caching a real piece of cake. With just a few method calls and custom attributes, you can improve the responsiveness of your distributed cloud-native application, keeping it robust, scalable, and maintainable at the same time.

This article was first published on a https://blog.postsharp.net under the title Simplify Your .NET Aspire Caching With Metalama.

Improving Your Team's Productivity Through Consistent Code Style

Gael Fraiteur — Mon, 17 Jun 2024 07:00:01 +0000

While it may seem trivial, the first step towards fostering a maintainable, team-friendly codebase is reaching a consensus on code style and ensuring its strict enforcement. This includes code formatting (spaces, blank lines, parentheses, and brackets), naming (casings, prefixes, suffixes, etc.), and usage of this, var and more. The ultimate goal? Make running a full automated clean-up on your codebase a routine task. In this article, I explain the practices we are following on the Metalama team. After all, since we are building tools that help C# developers write better quality code, it’s only logical that our own code is of the highest possible quality. Don’t take my words for granted: our source code is available on GitHub, so you can check for yourself if we live up to our standards.

Code Style As a Philosophy

You might be wondering, why all the fuss about code style? Let’s highlight two main reasons:

A consistent style makes code easier to read. Since developers typically spend 80% of their time reading code, code quality is one of the most important factors for the long-term productivity of your team. There are tools, like ReSharper’s Virtual Formatter, that can help with reading code in a consistent style while you are in the IDE, but that doesn’t solve the issue in other places where code is read, such as your Git repository.
A uniform code style simplifies pull requests and merges. Have you ever been frustrated by irrelevant changes appearing in a PR diff because a developer ran a code cleanup? This annoyance could’ve been avoided if the code style had been adhered to from the start. Consistent code style also minimizes merge conflicts. One of the objectives of the code quality pipeline is that everybody should be able to reformat the entire codebase without causing a revolution in the team – as long as it’s done in a separate PR of course. In other words, complete reformatting should be a non-event.

Step 1. Achieving Consensus on Code Style

While coding style can be a matter of preference, it’s crucial that everyone on the team aligns with a chosen convention. Everyone will have their own opinion, which can easily lead to endless and heated debates. Your main objective here is to get through the consensus-building process as soon and quickly as possible.

I suggest starting with Microsoft’s common coding convention for C#. It’s a solid starting point, widely accepted by the C# community, but it doesn’t cover every detail.
At the beginning of the project, you might still find yourselves in lengthy discussions. Don’t dismiss these debates: it is better to have them sooner than later. Listen to all arguments swiftly, remind everyone that the emphasis is not on the chosen style itself, but on making an unambiguous decision that will be consistently applied. Ensure everyone is aware of the bikeshedding dynamics in collective decision-making. Then, let the most experienced developer make the final call.

Remember, deciding on a coding style is an iterative process. Don’t spend too much time on this step—you’ll revisit it later.

Step 2. Configuring Your IDE

Next, you need to configure your coding style in the IDE.

Configuring Code Formatting

To keep everyone on the same page, store the coding style configuration in the source repository, not in the user profile. This file is typically named .editorconfig. The EditorConfig format is supported by Visual Studio, ReSharper, Rider, and many other editors.

Here’s how you can use the Code Style editor in Visual Studio:

Your code style configuration should be thorough and specific, leaving no room for personal interpretations. I recommend setting the severity to warning whenever practically possible. Some formatting rules are brittle because they can be easily broken if you have #if clauses in your code. On our team, we turn off any brittle rules. We only enforce code formatting in the Debug build configuration because having a zero-warning codebase for all platforms and all configurations is very cumbersome for minimal benefits.

In addition to .editorconfig, we also use JetBrains’ tools due to their superior code formatting and cleanup capabilities. We set up a team-shared layer in Rider to ensure the Rider configuration is stored in the source repository. Instead of the *.sln.DotSettings file, I choose a different layer file that can be imported from all solutions in the repository.

In addition to .editorconfig, we also use JetBrains’ tools due to their superior code formatting and cleanup capabilities. Here’s the Rider settings dialog:

In our team, we set up the same team-shared layer in Rider in each solution to ensure the Rider configuration is stored in the source repository. So instead of the *.sln.DotSettings file, we create a different layer file that can be imported from all solutions in the repository. To access layers, click on the Managed Layers at the left bottom corner of the settings dialog.

When the layer is created, make sure to save your code style changes into the proper layer. It’s a good practice for team members to reset their code style settings in both personal layers.

Feel free to copy our .editorconfig and Rider settings.

Configuring Code Cleanup

Now that you have set up your code formatting preferences, the next step is to configure a code cleanup profile.

I suggest adding all harmless fixers, i.e., those that make minor syntax changes and, most importantly, are 100% bug-free. For example, include Apply parenthesis parameters and Add this qualifications, but not Make field readonly because this analysis sometimes makes mistakes. The ultimate goal is to be able to use your IDE’s clean up feature confidently and without any manual tweaking afterward, so I would avoid any risky fixer.

Unfortunately, Visual Studio does not allow saving cleanup profiles in source control. Ideally, all team members should use the same profile. While Visual Studio offers a cleanup on save option, I am not using it because it sometimes has bugs, and when it does, you have no way at all to fix them except by opening your file in another editor than Visual Studio.

Rider, on the other hand, allows to store the clean-up profile in the team-shared settings layer. Instead of format-on-save, Rider can automatically perform a code cleanup before each commit. I personally don’t use this feature because I always use Rider for git commits. Besides, since some of my teammates prefer Visual Studio, we need a vendor-neutral solution.

Step 3. Reporting Warnings for Style Violations

Remember that our goal is to improve the team’s productivity, both in the short and long terms. If your process is too lax, your code quality will degrade, and you will hamper your long-term productivity. However, if your process is too strict, any build or pull request could become a nightmare.

Therefore, it’s essential to find a good balance. We’ve found the following compromise to work well for us:

As I said before, we set the severity of most code style violations to warning. For instance, our coding style requires all instance members to be qualified with this, which translates to the following lines in .editorconfig. Note the warning setting.
We allow builds with warnings on development machines. Why? Because you certainly don’t want to have a perfectly formatted build every time you want to run your tests or your apps.
However, in CI builds, we treat warnings as errors. This way, code with style issues won’t be merged. To detect continuous integration builds, use the ContinuousIntegrationBuild property or the specific environment variable of your CI pipeline.
While this article focuses on code style and conventions, you should also enable code analysis for various rulesets. Check out Overview of .NET source code analysis for more details. We on the Metalama team are also using StyleCop, which is redundant with Rider/Resharper analysis but integrates directly with the C# compiler.

Avoid being too strict about whitespace violations. Requiring an exact number of whitespaces for a merge could lead to multiple rounds of whitespace fixing commits, which can slow down productivity. Remember, your ultimate goal is to improve productivity, not to achieve a whitespace-perfect codebase.

Step 4. Planning for Periodic Full Cleanups

At this point, your development and build pipeline should ensure that PRs are reasonably well-formatted before they undergo code review. While demanding perfection for each PR is impractical, formatting defects can accumulate over time, necessitating a thorough cleanup.

On the Metalama team, we perform a comprehensive cleanup immediately after the dev freeze milestone of each release, usually every 6 to 12 weeks. The goal is to periodically restore the codebase to perfect order.

Running the cleanup tool should be a non-event. It should be entirely deterministic, non-disruptive, and not cause any frustration.

You have two main options:

dotnet format is a .NET SDK command-line tool that reformats code to align with .editorconfig settings. However, its capabilities are currently limited. For me, “good enough” means being entirely satisfied with the tool’s output, and dotnet format isn’t quite there yet.
JetBrains offers two types of cleanup tools. You can use the interactive tool in ReSharper or Rider, or ReSharper’s CleanupCode command-line tool. The latter is free and doesn’t require a license for the entire team if all you need is code reformatting. This is what our team uses. You will need at least one license to set up the correct configuration.

To ensure all runs produce the same predictable output, create a script that runs the tool with the exact parameters. This output, termed the canonically formatted code, should be your gold standard.

From this point forward, no one should be blamed for reformatting code to its canonical form. Instead, the blame should be placed on the developer merging non-canonical code.

Step 5. Taking Code Validation to the Next Level

If you’ve successfully implemented all the above steps, congratulations! Your processes now ensure consistent adherence to the code style, making team members more confident in reformatting and refactoring code.

To further enhance codebase maintainability and readability, consider validating your codebase with more complex rules. For instance, enforcing that all classes implementing IFactory end with the *Factory suffix, or checking that no one uses the double type in the Billing namespace (you should use int or decimal for any money!). Consider writing architectural unit tests or using our tool Metalama to enforce naming conventions and verify dependencies.

For example, here is how you could enforce a naming convention using Metalama:

[DerivedTypesMustRespectNamingConvention( "*Factory" )]
public interface IFactory<T>
{
   T Create();
}

And here is how you could prohibit internal members of a namespace from being used from a different namespace:

namespace TheNamespace
{
    internal class Fabric : NamespaceFabric
    {
        public override void AmendNamespace( INamespaceAmender amender )
        {
            amender.Verify().InternalsCanOnlyBeUsedFrom( 
                                   r => r.CurrentNamespace() );
        }
    }
}

Conclusion

Mastering code formatting is a crucial step towards creating a maintainable, team-friendly codebase. Adhering to and enforcing a particular style creates an environment that improves code-reading productivity, a critical factor considering that developers spend a whopping 80% of their time reading code. Consistent style also reduces merge issues and simplifies pull requests. A clean, well-maintained codebase is a pleasure to work with and the pride of every team member.

This article was first published on a https://blog.postsharp.net under the title Improving Your Team's Productivity Through Consistent Code Style.

5 Practical Ways to Add Polly to Your C# Application [2024]

Gael Fraiteur — Tue, 11 Jun 2024 07:00:02 +0000

Polly is a .NET library that helps to increase the resiliency of your application. It offers a myriad of strategies such as Retry, Circuit Breaker, Timeout, Rate Limiter, Fallback, and Hedging to manage unexpected behaviors. Polly also offers chaos engineering features, enabling you to introduce unexpected behaviors into your app, allowing you to test your resilience setup without waiting for a real incident. In this article, we will focus on practical implementation strategies to add Polly to your .NET app.

We will describe the following approaches:

Using built-in support in the client API, typically in HttpClient.
Adding Polly to your business code, doing things manually.
Using the Type Decorator pattern to reduce boilerplate.
Reducing boilerplate with an aspect featuring Metalama as another, more general approach to boilerplate reduction.
Adding Polly to ASP.NET inbound requests using ASP.NET middleware.

1. Using built-in support in the client API

Certain components are designed with Polly in mind. One of them is the HttpClient class that comes with .NET. Microsoft provides the Microsoft.Extensions.Http.Resilience and Microsoft.Extensions.Resilience libraries. These libraries are built with Polly and simplify the integration of resilience into the .NET application.

Here’s a straightforward example that retrieves data from an HTTP endpoint and retries when there’s a failure. It calls the AddStandardResilienceHandler method to inject pre-configured resilience policies into the HttpClient.

const string httpClientName = "MyClient";

var services = new ServiceCollection()
    .AddLogging( b => b.AddConsole().SetMinimumLevel( LogLevel.Debug ) )
    .AddHttpClient( httpClientName )
    .AddStandardResilienceHandler()
    .Services
    .BuildServiceProvider();

var clientFactory = services.GetRequiredService<IHttpClientFactory>();
var client = clientFactory.CreateClient( httpClientName );
var response = await client.GetAsync(
 "http://localhost:52394/FailEveryOtherTime" );
Console.WriteLine( await response.Content.ReadAsStringAsync() );

The full source code for this article is available from blog.postsharp.net.

You can customize the resilience policies by passing options as an argument to the AddStandardResilienceHandler method. You can learn more about the resilience strategies in the Polly documentation.

Avoid creating HttpClient instances yourself

The AddStandardResilienceHandler method will work only if you get your HttpClient instances by using the IHttpCientFactory you get from the service provider. It won’t handle the case where an HttpClient instance is created using its constructor. It might be challenging to remember this rule. Luckily, software architecture validation tools, like Metalama, can prevent developers from writing code that breaks this rule. Adding the following code to your project would trigger warnings wherever the HttpClient’s constructor is used explicitly:

internal class AvoidInstantiatingHttpClientFabric : ProjectFabric
{
    public override void AmendProject( IProjectAmender amender )
    {
        amender
            .Verify()
            .SelectTypes( typeof(HttpClient) )
            .SelectMany( t => t.Constructors )
            .CannotBeUsedFrom(
                r => r.Always(),
                $"Use {nameof(IHttpClientFactory)} instead." );
    }
}

With this code in your project, code using the HttpClient constructor will immediately be reported with a warning.

2. Adding Polly to your business code

There are a few cases where you might need to call Polly directly from your business code or data access layer. One of those is when there’s no component-specific API for managing application resilience. The second is when the business logic involves several steps that must be retried as a whole.

Let’s consider a method that executes SQL commands on a cloud database service. This is a money transfer operation, where both account updates must be performed atomically in a transaction.

public async Task TransferAsync(
    int sourceAccountId,
    int targetAccountId,
    int amount,
    CancellationToken cancellationToken = default )
{
    var transaction = await connection.BeginTransactionAsync( cancellationToken );

    try
    {
        await using ( var command = connection.CreateCommand() )
        {
            command.CommandText = 
      "UPDATE accounts SET balance = balance - $amount WHERE id = $id";
            command.AddParameter( "$id", sourceAccountId );
            command.AddParameter( "$amount", amount );
            await command.ExecuteNonQueryAsync( cancellationToken );
        }

        await using ( var command = connection.CreateCommand() )
        {
            command.CommandText = 
      "UPDATE accounts SET balance = balance + $amount WHERE id = $id";
            command.AddParameter( "$id", targetAccountId );
            command.AddParameter( "$amount", amount );
            await command.ExecuteNonQueryAsync( cancellationToken );
        }

        await transaction.CommitAsync( cancellationToken );
    }
    catch
    {
        await transaction.RollbackAsync( cancellationToken );

        throw;
    }
}

Occasionally, an influx of requests may temporarily overload the database. In such cases, we might want to retry the whole transaction.

The best way to add Polly to your services is to use dependency injection and to add a resilience policy to the IServiceCollection. We configure the policy with a Retry Strategy that reacts when the database commands fail on the DbException, waits initially 1 second, followed by an exponential backoff strategy, and retries no more than 3 times:

services.AddResiliencePipeline(
    "db-pipeline",
    pipelineBuilder =>
    {
        pipelineBuilder.AddRetry(
                new RetryStrategyOptions
                {
                    ShouldHandle = new PredicateBuilder()
                                               .Handle<DbException>(),
                    Delay = TimeSpan.FromSeconds( 1 ),
                    MaxRetryAttempts = 3,
                    BackoffType = DelayBackoffType.Exponential
                } )
            .ConfigureTelemetry( LoggerFactory.Create(
                 loggingBuilder => loggingBuilder.AddConsole() ) );
    } );

With the pipeline in place, we can consume it from the Accounts service.

internal class Accounts(
    DbConnection connection,
    [FromKeyedServices( "db-pipeline" )] 
    ResiliencePipeline resiliencePipeline )

We can wrap the method to be retried with a call to the policy’s Execute method.

public async Task TransferAsync(
    int sourceAccountId,
    int targetAccountId,
    int amount,
    CancellationToken cancellationToken = default )
{
    await resiliencePipeline.ExecuteAsync(
        async t =>
        {
            var transaction = await connection.BeginTransactionAsync( t );

            try
            {
                await using ( var command = connection.CreateCommand() )
                {
                    command.CommandText =
       "UPDATE accounts SET balance = balance - $amount WHERE id = $id";
                    command.AddParameter( "$id", sourceAccountId );
                    command.AddParameter( "$amount", amount );
                    await command.ExecuteNonQueryAsync( t );
                }

                await using ( var command = connection.CreateCommand() )
                {
                    command.CommandText =
        "UPDATE accounts SET balance = balance + $amount WHERE id = $id";
                    command.AddParameter( "$id", targetAccountId );
                    command.AddParameter( "$amount", amount );
                    await command.ExecuteNonQueryAsync( t );
                }

                await transaction.CommitAsync( t );
            }
            catch
            {
                await transaction.RollbackAsync( t );

                throw;
            }
        },
        cancellationToken );
}

3. Using the Type Decorator pattern

Instead of editing all code locations that use a DbCommand, an alternative approach is to inject the Polly logic into the DbCommand itself. Since DbCommand is an abstract class, we can implement a Type Decorator pattern and wrap the call to the real database client with a call to Polly.

Some database connectors, like the SqlConnection class, already have their retry mechanism and would not benefit from an additional decorator.

Here is a partial implementation of ResilientDbCommand that follows the Type Decorator pattern:

public partial class ResilientDbCommand( 
    DbCommand underlyingCommand, 
    ResiliencePipeline resiliencePipeline ) : DbCommand
{
    public override int ExecuteNonQuery() 
      => resiliencePipeline.Execute( underlyingCommand.ExecuteNonQuery );

    public override object? ExecuteScalar() 
      => resiliencePipeline.Execute( underlyingCommand.ExecuteScalar );

    protected override DbDataReader ExecuteDbDataReader( 
                                                 CommandBehavior behavior )
        => resiliencePipeline.Execute( () => 
                            underlyingCommand.ExecuteReader( behavior ) );

    public override void Prepare()
       => resiliencePipeline.Execute( underlyingCommand.Prepare );

    public override void Cancel() 
       => resiliencePipeline.Execute( underlyingCommand.Cancel );
}

And here is how the connection is initialized.

await using var connection = new UnreliableDbConnection( new SqliteConnection( "Data Source=:memory:" ) );

var resiliencePipeline = CreateRetryOnDbExceptionPipeline();
var resilientConnection = new ResilientDbConnection( 
   connection, resiliencePipeline );
services.AddSingleton<DbConnection>( resilientConnection );

In this manner, the data layer code remains unchanged.

There is a fundamental difference between the two previous approaches: while the type-decorator approach retries an individual DbCommand, the data-layer approach retries the whole transaction. This difference can be significant if the DbCommand executes a non-transactional, multi-step operation, such as a stored procedure.

4. Reducing boilerplate with an aspect

When the Type Decorator pattern is not possible or convenient, there is still a better approach than using Polly directly in the business code. Imagine that your business code does not have a single method as in this simplistic example, but hundreds or thousands. Do you really want to repeat the Polly boilerplate for each of them? Probably not.

Fortunately, there are tools that allow you to add features to methods without modifying their source code, thus keeping the code readable. One such tool is Metalama. Metalama allows for moving the wrapping logic to a custom attribute, called an aspect. You can compare an aspect to a code template.

Without going into details, here is the source code of the aspect, where OverrideMethod is the template for non-async methods.

public partial class RetryAttribute : OverrideMethodAspect
{
    private readonly string _pipelineName;

    [IntroduceDependency]
    private readonly ResiliencePipelineProvider<string> _resiliencePipelineProvider;

    public RetryAttribute( string pipelineName = "default" )
    {
        this._pipelineName = pipelineName;
    }

    public override dynamic? OverrideMethod()
    {
        var pipeline = this._resiliencePipelineProvider
                                 .GetPipeline( this._pipelineName );

        return pipeline.Execute( Invoke );

        object? Invoke( CancellationToken cancellationToken = default )
        {
            return meta.Proceed();
        }
    }
}

If we add the aspect to our method, the code template will be expanded at compile time. As a bonus, we also implemented transaction handling as an aspect:

[Retry]
[DbTransaction]
public async Task TransferAsync(
    int sourceAccountId,
    int targetAccountId,
    int amount,
    CancellationToken cancellationToken = default )
{
    await using ( var command = this._connection.CreateCommand() )
    {
        command.CommandText =
         "UPDATE accounts SET balance = balance - $amount WHERE id = $id";
        command.AddParameter( "$id", sourceAccountId );
        command.AddParameter( "$amount", amount );
        await command.ExecuteNonQueryAsync( cancellationToken );
    }

    await using ( var command = this._connection.CreateCommand() )
    {
        command.CommandText =
         "UPDATE accounts SET balance = balance + $amount WHERE id = $id";
        command.AddParameter( "$id", targetAccountId );
        command.AddParameter( "$amount", amount );
        await command.ExecuteNonQueryAsync( cancellationToken );
    }
}

As you can see, we got rid of most of the boilerplate code in this code.

5. Adding Polly to ASP.NET inbound requests

We have seen approaches to add Polly to client endpoints (both HTTP and database), and approaches to add Polly in your business code. A third possibility is to add Polly at your server endpoint, that is, to wrap your entire request processing in a Polly policy.

Indeed, in ASP.NET Core apps, Polly can be easily introduced as a middleware. To illustrate this approach, let’s consider a microservice that processes data of a public API endpoint. Since we have no control over the endpoint, we need to handle any transient failures on our app’s side.

Any class with an InvokeAsync method of the proper signature can be an ASP.NET Core middleware. Here is ours. It consumes the Polly policy named middleware, which we need to configure exactly as in the above examples. The only difficulty to overcome is that we need to supply a restartable HttpContext to the downstream handler because an exception could happen in the middle of writing the HTTP response, and retrying the whole operation could cause a duplication of the output that has been written before the exception.

// Noted that keyed service does not seem available for middleware.
public class ResilienceMiddleware( RequestDelegate next, ResiliencePipelineProvider<string> pipelineProvider )
{
    public async Task InvokeAsync( HttpContext httpContext )
    {
        var pipeline = pipelineProvider.GetPipeline( "middleware" );

        var bufferingContext = new RestartableHttpContext( httpContext );
        await bufferingContext.InitializeAsync( httpContext.RequestAborted );

        await pipeline.ExecuteAsync(
            async _ =>
            {
                bufferingContext.Reset();
                await next( bufferingContext );
            },
            httpContext.RequestAborted );

        await bufferingContext.AcceptAsync();
    }
}

To add the middleware in the ASP.NET Core pipeline, use the UseMiddleware method:

var app = builder.Build();
app.UseMiddleware<ResilienceMiddleware>();

Now all the requests served by our microservice are handled by Polly. The microservice then behaves more reliably, even when depending on services that experience transient failures.

Summary

Polly is a useful .NET library that helps make our .NET app resilient using various strategies. Polly can be added using a component-specific API, directly to your code, or using the decorator pattern, either by creating a wrapping type, or by moving the resiliency logic to method decorators (aspect-oriented), to keep your code clean, maintainable, and scalable, without losing the resiliency power of Polly.

This article was first published on a https://blog.postsharp.net under the title 5 Practical Ways to Add Polly to Your C# Application [2024].

Fast and Compact Structured Logging in C# Using String Interpolation

Gael Fraiteur — Mon, 03 Jun 2024 07:00:01 +0000

If you want to easily filter your application logs, structured logging is the way to go. However, using just the standard .NET libraries, structured logging in C# can be both cumbersome and inefficient. In this article, we will explore how to make structured logging efficient and simple.

In this article, you will learn:

What is structured logging
What’s wrong with using the vanilla LogInformation method for structured logging?
Why not use the standard C# string interpolation for logging?
What are custom string interpolation handlers?
How to make structured logging efficient?
How to make structured logging clean and compact?
How to automatically add structured logging to your code?

What is structured logging?

Structured logging involves reporting errors, warnings, and messages in a consistent and organized format, making it straightforward to read, search, and analyze by applications or individuals. A common format for structured logging is JSON because it is both compact and readable. By contrast, unstructured logging reports unstructured text that can only be searched by substrings or regular expressions.

In C#, the most common logging API is the Microsoft.Extensions.Logging namespace. Typically, your structured logging code might look something like this:

_logger.LogInformation("Product {product} sold for {price}.", product, price);

Here, we specify the names of the arguments, product and price, allowing a structured logging backend to recognize them. The server might receive and index a JSON message like this:

{
  "EventId": 0,
  "LogLevel": "Information",
  "Category": "LoggingWithInterpolation.Shop",
  "Message": "Product llama T-shirt sold for 10.0.",
  "State": {
    "Message": "Product llama T-shirt sold for 10.0.",
    "product": "llama T-shirt",
    "productPrice": 10.0,
    "{OriginalFormat}": "Product {product} sold for {productPrice}."
  }
}

As you can see, the message parameters are stored and indexed separately from the formatted message text, making it easier to filter messages.

What’s wrong with using the vanilla LogInformation method for structured logging?

From a server point of view, structured logging looks great. However, there are two problems with the C# code above:

There is some redundancy between the argument names in the formatting string and the argument list, i.e. product and price are duplicated. Not only does it require additional work, but it may also lead to inconsistencies if the formatting string is not synchronized with its argument list.
This code isn’t efficient. It allocates an array for the arguments and boxes value type arguments every time, even when logging for this log level isn’t enabled. The inefficiency escalates if the computation of one of the arguments is expensive.

If you’re familiar with the string.Format method, you know that string interpolation was invented specifically to address these two issues. So, why not use string interpolation here?

Why not use the standard C# string interpolation for logging?

You might think of the following code:

_logger.LogInformation($"Product {product} sold for {price}.");

But you’ll quickly realize that this approach is actually worse!

It doesn’t resolve the performance issue.
It loses argument names for structured logging.

This is why using string interpolation for logging is generally not recommended.

What are custom string interpolation handlers?

Both problems of the default C# string interpolation can be solved thanks to the improvements to string interpolation brought by C# 10. Unfortunately, the necessary supporting classes and extension methods have not been implemented in the system libraries, so we need to implement them ourselves.

We must implement two artifacts ourselves:

a custom string interpolation handler that gathers all the information and then calls the ILogger.Log method, and
a custom extension method of the ILogger extension, called LogInformation, but accepting our custom string interpolation handler instead of the formatting string.

Let’s start with a basic implementation. It will not address any of the two issues above, but at least we can learn the API.

public static class LoggingExtensions
{
    public static void LogInformation(
        this ILogger logger,
        [InterpolatedStringHandlerArgument( nameof(logger) )]
        LogInformationInterpolatedStringHandler handler )
        => handler.Log();
}

[InterpolatedStringHandler]
public ref struct LogInformationInterpolatedStringHandler
{
    private readonly StringBuilder messageBuilder;
    private readonly ILogger logger;

    public LogInformationInterpolatedStringHandler( 
               int literalLength, int formattedCount, ILogger logger )
    {
        this.messageBuilder = new StringBuilder( literalLength + (formattedCount * 10) );
        this.logger = logger;
    }

    public void AppendLiteral( string literal ) 
     => this.messageBuilder.Append( literal );

    public void AppendFormatted<T>( T value ) 
     => this.messageBuilder.Append( value?.ToString() );

    public void Log() 
      => this.logger.Log( 
          LogLevel.Information, this.messageBuilder.ToString() );
}

To get the full source code of the article, please refer to the original article.

What does this do? When the C# compiler encounters a call to a method like LogInformation, where an interpolated string is passed to a parameter whose type is marked with the [InterpolatedStringHandler] attribute, it first calls the constructor of that type. The constructor is passed basic information about the interpolated string, which can be used to guess an initial size of a buffer (here, StringBuilder). The constructor also receives the values of other arguments, as specified by the [InterpolatedStringHandlerArgument] attribute. Next, the compiler calls the AppendLiteral and AppendFormatted methods for each part of the interpolated string. Finally, the method is called with the newly constructed handler.

When applied to the earlier logging code, this process results in the following code:

var handler = new LogInformationInterpolatedStringHandler(19, 2, _logger);
handler.AppendLiteral("Product ");
handler.AppendFormatted(product);
handler.AppendLiteral(" sold for ");
handler.AppendFormatted(price);
handler.AppendLiteral(".");
_logger.LogInformation(handler);

This works, but, as announced, it neither addresses the issue of parameter names nor of efficiency.

Let’s address the performance problem first.

How to make structured logging efficient?

Let’s first make sure that we don’t do too much work when logging is disabled. This is relatively straightforward: a custom string interpolation handler can have an out bool parameter on its constructor. When this parameter is set to false, it signifies that the handler is not enabled, so no further processing is done. This avoids all performance costs associated with that:

public LogInformationInterpolatedStringHandler( 
   int literalLength, 
   int formattedCount, 
   ILogger logger, 
   out bool isEnabled )
{
    isEnabled = logger.IsEnabled( LogLevel.Information );

    if ( isEnabled )
    {
        this.messageBuilder = new StringBuilder(
                      literalLength + (formattedCount * 10) );
        this.logger = logger;
    }

    this.isEnabled = isEnabled;
}

For this modified handler, the C# compiler generates the following code:

var handler = new LogInformationInterpolatedStringHandler(
                                   19, 2, _logger, out var isEnabled);
if (isEnabled)
{
    handler.AppendLiteral("Product ");
    handler.AppendFormatted(product);
    handler.AppendLiteral(" sold for ");
    handler.AppendFormatted(price);
    handler.AppendLiteral(".");
}
_logger.LogInformation(handler);

This is a significant improvement: most of the code is executed only when logging is enabled, which is exactly what we wanted.

How to make structured logging more compact?

So, what about argument names? For this, we’ll leverage another C# feature: caller argument expression. This feature allows us to obtain the C# code for an argument passed to a method. In the case of the AppendFormatted method, that would be the interpolated value, which is precisely what we’re looking for.

To ensure we don’t lose the ability to have a custom name for a logged argument, we can (ab)use the format string component of an interpolated value, i.e. the substring after the : character:

_logger.LogInformation($"Product {product} sold for {price:productPrice}.");

In the example above, the name of the product argument would be left to its default value (product), while the name of the second argument would be overridden to productPrice. Fortunately, this can be combined with a caller argument expression, provided we name the parameter to AppendFormatted as format.

When we combine these elements, we get:

[InterpolatedStringHandler]
public ref struct LogInformationInterpolatedStringHandler
{
    private readonly StringBuilder messageBuilder;
    private readonly object?[] arguments;
    private readonly ILogger logger;
    private readonly bool isEnabled;

    private int argumentsIndex;

    public LogInformationInterpolatedStringHandler( 
    int literalLength, 
    int formattedCount, 
    ILogger logger, 
    out bool isEnabled )
    {
        isEnabled = logger.IsEnabled( LogLevel.Information );

        if ( isEnabled )
        {
            this.messageBuilder = new StringBuilder( 
              literalLength + (formattedCount * 10) );
            this.arguments = new object?[formattedCount];
            this.logger = logger;
        }

        this.isEnabled = isEnabled;
    }

    public readonly void AppendLiteral( string literal )
        => this.messageBuilder.Append( literal.Replace( "{", "{{", StringComparison.Ordinal ).Replace( "}", "}}", StringComparison.Ordinal ) );

    public void AppendFormatted<T>( T value, 
     [CallerArgumentExpression( nameof(value) )] string format = null! )
    {
        this.messageBuilder.Append( $"{{{format}}}" );
        this.arguments[this.argumentsIndex] = value;
        this.argumentsIndex++;
    }

    public readonly void Log()
    {
        if ( this.isEnabled )
        {
            this.logger.Log( LogLevel.Information, this.messageBuilder.ToString(), this.arguments );
        }
    }
}

This time, the C# compiler generates the following code:

var handler = new LogInformationInterpolatedStringHandler(19, 2, _logger, out var isEnabled);
if (isEnabled)
{
    handler.AppendLiteral("Product ");
    handler.AppendFormatted(product, "product");
    handler.AppendLiteral(" sold for ");
    handler.AppendFormatted(price, "productPrice");
    handler.AppendLiteral(".");
}
_logger.LogInformation(handler);

Note how the format argument for product is set by the caller argument expression to "product" and for price to "productPrice" from the format string.

When logging with the simple console logger, this produces output like:

info: LoggingWithInterpolation.Shop[0]
      Product llama T-shirt sold for 10.0.

And with JSON logger (to see the structured logging in action):

{
  "EventId": 0,
  "LogLevel": "Information",
  "Category": "LoggingWithInterpolation.Shop",
  "Message": "Product llama T-shirt sold for 10.0.",
  "State": {
    "Message": "Product llama T-shirt sold for 10.0.",
    "product": "llama T-shirt",
    "productPrice": 10.0,
    "{OriginalFormat}": "Product {product} sold for {productPrice}."
  }
}

How to automatically add structured logging to your code?

We’ve seen how to achieve compact and efficient logging with clean code. However, there are situations where you want to log all method executions and their parameter values. This can lead to code repetition! Traditional code generators like T4 or source generators don’t help because they can only generate new files - they cannot add code into our hand-written code.

Enter Metalama, a code generation toolkit that can mix generated code with user-written code, all at build time.

Let’s see how we can use it to automate the logging of method calls.

First, we need to create an aspect, which is like a template describing how the code should be transformed.

public class LogAttribute : OverrideMethodAspect
{
    [IntroduceDependency]
    private readonly ILogger _logger;

    public override dynamic? OverrideMethod()
    {
        // LogInformation can't be called as an extension method here,
        // because Metalama uses the dynamic type to represent 
        // run-time values
        // and dynamic is not compatible with extension methods.
        LoggingExtensions.LogInformation( this._logger, 
                                 BuildInterpolatedString().ToValue() );

        return meta.Proceed();
    }

    [CompileTime]
    private static InterpolatedStringBuilder BuildInterpolatedString()
    {
        var stringBuilder = new InterpolatedStringBuilder();

        // Include the type and method name.
        stringBuilder.AddText( 
              $"{meta.Target.Type}.{meta.Target.Method.Name}(" );

        var first = true;

        // Include each parameter.
        foreach ( var parameter in meta.Target.Parameters )
        {
            if ( !first )
            {
                stringBuilder.AddText( ", " );
            }

            stringBuilder.AddText( $"{parameter.Name}: " );

            stringBuilder.AddExpression( parameter.Value );

            first = false;
        }

        stringBuilder.AddText( ") started." );

        return stringBuilder;
    }
}

Most of the above code executes at compile time.

You can add the logging aspect to methods one at a time using the [Log] attribute. This feature is completely free. Another option is to use a Metalama fabric to add logging more widely using a single piece of code, but this is a commercial feature:

internal class Fabric : ProjectFabric
{
    public override void AmendProject( IProjectAmender amender )
    {
        amender.Outbound
            .SelectMany( c => c.Types )
            .Where( t => t.TypeKind is not TypeKind.RecordClass 
                                        or TypeKind.RecordStruct )
            .Where( t => t.Accessibility == Accessibility.Public )
            .SelectMany( c => c.Methods )
            .Where( m => m.Accessibility == Accessibility.Public )
            .AddAspectIfEligible<LogAttribute>();
    }
}

When applied to the void Sell(Product product, decimal price) method (whether using an attribute or using a fabric), the aspect generates the following code at the start of the method:

LoggingExtensions.LogInformation(_logger,
            $"Shop.Sell(product: {product}, price: {price}) started.");

This code produces the following output from the simple console logger:

info: LoggingWithInterpolation.Metalama.Shop[0]
      Shop.Sell(product: llama T-shirt, price: 10.0) started.

And with JSON:

{
  "EventId": 0,
  "LogLevel": "Information",
  "Category": "LoggingWithInterpolation.Metalama.Shop",
  "Message": "Shop.Sell(product: llama T-shirt, price: 10.0) started.",
  "State": {
    "Message": "Shop.Sell(product: llama T-shirt, price: 10.0) started.",
    "product": "llama T-shirt",
    "price": 10.0,
    "{OriginalFormat}": "Shop.Sell(product: {product}, price: {price}) started."
  }
}

Of course, you can also customize what is logged to suit your needs, such as logging exceptions or return values.

Summary

Structured logging can be incredibly useful in production, and with just a couple of additional files in your project, you can use it easily and efficiently with familiar interpolated string builders. As a bonus, to automatically add structured logging to your code, you can create a Metalama aspect.

This article was first published on a https://blog.postsharp.net under the title Fast and Compact Structured Logging in C# Using String Interpolation.

The Decorator Pattern in Modern C#

Gael Fraiteur — Thu, 30 May 2024 08:48:27 +0000

The Decorator design pattern allows software developers to extend the functionalities of a component without altering its code. This article explores the primary techniques for implementing the decorator pattern in modern .NET while adhering to the Single Responsibility Principle (SRP) and avoiding boilerplate code.

When to use a Decorator pattern in C

The Decorator pattern is useful when you want to add behavior to an existing component but either cannot or do not want to modify the source code. This is typically done to adhere to the single responsibility principle (SRP) to keep our code clean, readable, and maintainable.

Some real-world use cases for the decorator design pattern include:

Execution policies, such as exception handling, retrying, or caching, which help improve the performance and reliability of your apps.
Observability, for instance, by adding logging to all calls to an external component.
User Interface, like adding a scrollbar to a large textbox. Another example is the Adorner concept in WPF.
Streams, with features such as buffering, encryption, or compression.

What is the Decorator pattern

A Decorator is essentially a wrapper that implements the same contract as the entity it’s wrapping. We are intentionally using the vague term contract. As we will see in this article, it can mean two things: a C# interface if we implement a type decorator, or a method signature if we implement a method decorator. In both cases, the caller does not need to know that it is talking to a decorator rather than the final implementation. The pattern is recursive: we can add a decorator to a decorator, creating a chain of responsibility.

For instance, instead of just calling an unreliable service, we may want to retry a couple of times upon transient failure, and finally assign a unique ID to each exception, log it, and wrap the exception. We can represent the chain as follows:

In this article, we will explore two kinds of decorators: type decorators and method decorators.

The classic Type Decorator pattern

The classic type decorator pattern is a purely object-oriented variant of the decorator pattern that relies on type interfaces.

To illustrate the idea, let’s say we want to build a simple messaging app. We’d need a component that handles sending and receiving messages. This component implements the IMessenger interface and is implemented in a third-party library.

public interface IMessenger
{
    void Send( Message message );

    public Message Receive();
}

We are using the IMessenger service from a client class:

public class Client( IMessenger messenger )
{
    public void Greet()
    {
        messenger.Send( new Message( "Hello, world" ) );
        Console.WriteLine( "--> " + messenger.Receive().Text );
    }
}

We are instantiating the Client class from Program.cs:

var messenger = new Messenger();
var client = new Client( messenger );
client.Greet();

That all works nicely on our development environment. However, as soon as we move things to production, we realize that the messenger service is unreliable and occasionally causes our app to crash. Since we don’t own the source code of the IMessenger implementation, we cannot simply add the logic we need to each method.

How does the Decorator pattern help us tackle this problem?

Take a look at our design using the type decorator pattern in the class diagram below. Along with the decorators for error handling and retrying, we’ve introduced the MessengerDecorator abstract class that holds the wrapped IMessenger object, making it easier to implement individual decorators.

Here is the implementation of the ExceptionReportingMessenger class:

public class ExceptionReportingMessenger : MessengerDecorator
{
    private readonly IExceptionReportingService _reportingService;

    public ExceptionReportingMessenger( 
      IMessenger underlying, 
      IExceptionReportingService reportingService ) :
      base( underlying )
    {
        this._reportingService = reportingService;
    }

    public override void Send( Message message )
    {
        try
        {
            this.Underlying.Send( message );
        }
        catch ( Exception e )
        {
            this._reportingService.ReportException( 
                                           "Failed to send message", e );

            throw;
        }
    }

    public override Message Receive()
    {
        try
        {
            return this.Underlying.Receive();
        }
        catch ( Exception e )
        {
            this._reportingService.ReportException( 
"Failed to receive message", e );

            throw;
        }
    }

}

The RetryingMessenger messenger is very similar.

Now, instead of passing the original Messenger component to the Client class, we are wrapping the Messenger into RetryingMessenger, then into an ExceptionReportingMessenger. This is finally the ExceptionReportingMessenger that we pass to the Client

var originalMessenger = new Messenger();

var retryingMessenger = new ExceptionReportingMessenger(
    new RetryingMessenger( originalMessenger ),
    new ExceptionReportingService() );

var clientUsingDecorator = new Client( retryingMessenger );
clientUsingDecorator.Greet();

When the program calls Client.Greet, the control flow is the following:

Using type decorators with dependency injection

Obviously, in any modern C# application, you would not instantiate the components manually as in the examples above, but you would let dependency injection do the job.

If you are using .NET Core’s IServiceCollection, there is a nice library called Scrutor that can easily wrap a service with a decorator.

For example, this is how to apply the decorators by using Scrutor. Note the calls to the Decorate methods: they are defined by Scrutor.

var services = new ServiceCollection()
    .AddSingleton<IExceptionReportingService, ExceptionReportingService>()
    .AddSingleton<IMessenger, Messenger>()
    .AddSingleton<Client>()
    .Decorate<IMessenger, RetryingMessenger>()
    .Decorate<IMessenger, ExceptionReportingMessenger>()
    .BuildServiceProvider();

var client = services.GetRequiredService<Client>();
client.Greet();

Many dependency injection frameworks have built-in support for decorators. See for instance how Autofac handles this problem,

The Abstract Type Decorator pattern

At first glance, our solution design seems perfect. But when we dig into the implementation of the decorators, we notice that the exception handling would be duplicated across other methods. Here, we’re violating the Don’t Repeat Yourself principle. The code is now harder to maintain than before because any change to the error handling must be made in each method of ExceptionReportingMessenger and any decorator of another type decorator.

We will now see how to improve the Type Decorator pattern to make the decorator logic more reusable.

Let’s use the word policy to designate the logic that we wrap a method call with. Policies can be abstracted away and encapsulated in a reusable way. In the following diagram, we have represented policies as an interface.

Here is the exception-reporting policy:

public class ReportExceptionPolicy( 
   IExceptionReportingService reportingService ) : IPolicy
{
    public T Invoke<T>( Func<T> func )
    {
        try
        {
            return func();
        }
        catch ( Exception e )
        {
            reportingService.ReportException( 
                                "Failed to send message", e );

            throw;
        }
    }
}

We then define an AbstractDecorator, an abstract class that can be used as a base for any decorator:

public abstract class AbstractDecorator( IPolicy policy )
{
    protected T Invoke<T>( Func<T> func ) => policy.Invoke( func );

    protected void Invoke( Action action )
        => policy.Invoke<object?>(
            () =>
            {
                action();

                return null!;
            } );
}

In practice, you’ll also need to implement async versions of the Invoke methods in both IPolicy and AbstractDecorators.

With this setup, all the MessengerDecorator has to do is wrap method implementations with calls to the Invoke methods of AbstractDecorator:

public class MessengerDecorator( IMessenger underlying, IPolicy policy ) :
    AbstractDecorator( policy ), IMessenger
{
    public void Send( Message message ) =>
 this.Invoke( () => underlying.Send( message ) );

    public Message Receive() => this.Invoke( underlying.Receive );
}

Note that this decorator is now abstracted from any policy. The only repetitive code is now in calling the Invoke method.

Finally, we wire the service collection using Scrutor’s Decorate method by supplying one of the Policies to the MessengerDecorator class:

var services = new ServiceCollection()
    .AddSingleton<IExceptionReportingService, ExceptionReportingService>()
    .AddSingleton<IMessenger, Messenger>()
    .Decorate<IMessenger>(
        ( inner, _ ) => new MessengerDecorator(
            inner,
            new RetryPolicy() ) )
    .Decorate<IMessenger>(
        ( inner, serviceProvider ) => new MessengerDecorator(
            inner,
            new ReportExceptionPolicy( serviceProvider.GetRequiredService<IExceptionReportingService>() ) ) )
    .BuildServiceProvider();

var client = services.GetRequiredService<Client>();
client.Greet();

We’ve now consolidated error-handling logic in one place.

The control flow now becomes:

Generating Type Decorators automatically

There’s still repetitive code in the MessengerDecorator class. Arguably, MessengerDecorator is purely boilerplate and should ideally be removed from your codebase. There are two ways to generate this class:

At run time, using an approach known as dynamic proxies, or
At compile time, using source generators.

In this article, we will only explore the first solution.

The principle behind dynamic proxies is to generate the decorator class at run time, when the application initializes. Among the few libraries that implement this feature, the most popular is Castle DynamicProxy. The concept of policy developed earlier maps to Castle’s IInterceptor interface. Here is the implementation of the retry policy as a Castle interceptor. Notice the similarity to the RetryPolicy class in the example above.

internal class RetryInterceptor( 
                     int retryAttempts = 3, double retryDelay = 1000 ) 
 : IInterceptor
{
    public void Intercept( IInvocation invocation )
    {
        for ( var i = 0;; i++ )
        {
            try
            {
                invocation.Proceed();
            }
            catch ( Exception ) when ( i < retryAttempts )
            {
                var delay = retryDelay * Math.Pow( 2, i );

                Console.WriteLine(
                    "Failed to receive message. " +
                    $"Retrying in {delay / 1000} seconds... " +
                    $"({i + 1}/{retryAttempts})" );

                Thread.Sleep( (int) delay );
            }
        }
    }
}

As promised, there’s no longer a need for any decorator code since Castle has implemented it.

We can now proceed to the startup sequence of our application. We will need a ProxyGenerator:

var proxyGenerator = new ProxyGenerator();

We can now use the proxyGenerator.CreateInterfaceProxyWithTarget method to create the proxy class, and supply the two interceptors implementing our policies:

var services = new ServiceCollection()
    .AddSingleton<IExceptionReportingService, ExceptionReportingService>()
    .AddSingleton<IMessenger, Messenger>()
    .Decorate<IMessenger>(
        ( inner, _ ) => new MessengerDecorator(
            inner,
            new RetryPolicy() ) )
    .Decorate<IMessenger>(
        ( inner, serviceProvider ) => new MessengerDecorator(
            inner,
            new ReportExceptionPolicy( serviceProvider.GetRequiredService<IExceptionReportingService>() ) ) )
    .BuildServiceProvider();

var client = services.GetRequiredService<Client>();
client.Greet();

The Method Decorator Pattern

So far, we’ve discussed techniques that help replace a type with another type implementing the same interface, but providing additional services. The principal benefits of this approach are:

It’s purely object-oriented,
It works with code you don’t own,
It’s composable at run time.

However, there’s a significant drawback: it only works if you can inject yourself into the communication between the caller and the service – typically through an interface, although the same could be achieved using abstract or virtual methods. If you appreciate the benefits of being able to decorate a method with any behavior, it’s a pity to have to limit yourself so much. Even worse: you may be tempted to split your application into more smaller components to benefit from decorators. This is a case of framework dictatorship and it should be avoided.

An alternative to the type decorator pattern is the method decorator. As its name suggests, method decorators target a single method and not the entire type. Method decorators are commonly used in dynamic languages such as Python. They aren’t directly supported by C#, but some toolkits like Metalama make it possible.

The idea of C# method decorators is to move the logic of the policy to a special kind of custom attribute called an aspect, which could be compared to code templates. Unlike other custom attributes, aspects are applied to the code during compilation. Since this approach is compile-time, we’re not limited to the limitations of the .NET runtime, namely we’re not limited to virtual or interface methods, but we can intercept anything (including static private fields, if you ask).

Here is the Metalama version of the retry policy:

internal class RetryAttribute : OverrideMethodAspect
{
    public int Attempts { get; set; } = 3;

    public double Delay { get; set; } = 1000;

    public override dynamic? OverrideMethod()
    {
        for ( var i = 0;; i++ )
        {
            try
            {
                return meta.Proceed();
            }
            catch ( Exception e ) when ( i < this.Attempts )
            {
                var delay = this.Delay * Math.Pow( 2, i + 1 );

                Console.WriteLine(
$"Method {meta.Target.Method.DeclaringType.Name}.{meta.Target.Method} " +
$"has failed on {e.GetType().Name}. Retrying in {delay / 1000} seconds. " + ({i + 1}/{this.Attempts})" );

                Thread.Sleep( (int) delay );
            }
        }
    }

    // TODO: Implement OverrideMethodAsync and call Task.Delay instead of Thread.Sleep.
}

To add the policy to a method, apply it as a custom attribute:

public partial class Messenger
{
    private int _receiveCount;
    private int _sendCount;

    [Retry]
    [ReportExceptions]
    public void Send( Message message )
    {
        Console.WriteLine( "Sending message..." );

        // Simulate unreliable message sending
        if ( ++this._sendCount % 3 == 0 )
        {
            Console.WriteLine( "Message sent successfully." );
        }
        else
        {
            throw new IOException( "Failed to send message." );
        }
    }

    [Retry]
    [ReportExceptions]
    public Message Receive()
    {
        Console.WriteLine( "Receiving message..." );

        // Simulate unreliable message receiving
        if ( ++this._receiveCount % 3 == 0 )
        {
            Console.WriteLine( "Message received successfully." );

            return new Message( "Hi!" );
        }

        throw new IOException( "Failed to receive message." );
    }
}

To further improve maintainability, toolkits like Metalama facilitate the bulk application of aspects, eliminating the need for developers to manually specify where each aspect should be used. For instance, we can stipulate that all public methods within a specific namespace should have exception reporting. Consequently, when new methods are added to this namespace, the exception-reporting aspect is automatically applied. This approach not only enhances the readability and maintainability of the codebase but also simplifies scalability. In Metalama, this is achieved using fabrics. The following example demonstrates how to add exception reporting to all public methods in a project:

internal class AddExceptionReportingToPublicMethodsFabric : ProjectFabric
{
    public override void AmendProject( IProjectAmender amender )
    {
        amender.Outbound.SelectMany( t => t.AllTypes )
            .SelectMany( t => t.Methods )
            .Where( m => m.Accessibility == Accessibility.Public )
            .AddAspectIfEligible<ReportExceptionsAttribute>();
    }
}

Summary

Decorators are an effective way to maintain clean code and uphold the single responsibility principle. Opt for type decorators when you don’t own the code that you wish to enhance with new behaviors, or when you need to dynamically add behaviors at runtime. Use method decorators (aspect-oriented) when you own the source code and aim to adhere to the Single Responsibility Principle.

Discover Metalama, the leading code generation and validation toolkit for C#

Write and maintain less code by eliminating boilerplate, generating it dynamically during compilation, typically reducing code lines and bugs by 15%.
Validate your codebase against your own rules in real-time to enforce adherence to your architecture, patterns, and conventions. No need to wait for code reviews.
Excel with large, complex, or old codebases. Metalama does not require you to change your architecture. Beyond getting started, it's at scale that it really shines.

Dealing with NullReferenceException

Gael Fraiteur — Wed, 22 May 2024 16:23:56 +0000

As a C# programmer, you've likely come across this exception: System.NullReferenceException: Object reference not set to an instance of an object. So, what does it mean? How can you deal with it? And better yet, how can you prevent it from happening?

What is a NullReferenceException?

With reference types (classes, interfaces, delegates, or arrays, but not structs), variables (i.e. local variables, fields, or parameters) do not contain the object itself but a reference to the object. This allows two different variables to reference the same object. A reference can either point to an object or be null, which means it doesn't point to any object.

NullReferenceException, as the name suggests, is thrown when you attempt to get the object referenced by a null reference. Specifically, it occurs when you try to access a non-static member (a property, method, field, or event) of a null reference.

Confusingly, C# also has reference parameters, variables, and return values, which are denoted by the ref keyword. This allows representing a reference to value types or... references to references! But since ref locations can't be null, you can't get a NullReferenceException from accessing them, and we won't cover them further in this article.

Another related concept is nullable value types, which are value types that can be null. These are denoted as T? (or Nullable<T>), where T is a value type (a struct or enum). You can't get a NullReferenceException by accessing a nullable value type because they're not references. Instead, accessing Value on a null nullable value type throws an InvalidOperationException.

How to identify the cause of a NullReferenceException?

Before you can fix a NullReferenceException, you first need to understand where it happened.

The simplest approach is to use the debugger of your IDE (like Visual Studio or Rider) to run your code. Consider this simple example:

class Shop(Cart cart)
{
     public void AddToCartWithDiscount(Product product, int discount)
     {
         product.Discount = discount;
         cart.Add(product);
     }
}

When running code that calls the AddToCartWithDiscount method in the Visual Studio debugger, you might see something like this:

Here, you can see where the exception occurred, the values of local variables, the call stack, and even which variable caused the exception by being null. If the null reference was passed as a parameter, as in this case, you can double-click a method in the call stack to see its state, getting closer to the root cause of the exception.

But what if you can't use a debugger? For instance, if you're working on a production issue and can't reproduce it locally. In that scenario, you should have access to the exception and its stack trace, which might look like this:

System.NullReferenceException: Object reference not set to an instance of an object.
   at Shop.AddToCartWithDiscount(Product product, Int32 discount) in Shop.cs:line 5
   at Program.Main() in Program.cs:line 5

There's less information here, but you can still see where the exception occurred and what the call stack is, which should suffice to figure out which value is null. However, keep in mind that the line numbers might not be accurate, especially if you're using a release build.

You could also add logging to your code to get more information about the state of your application when the exception happens.

If debugging information wasn't available when the exception occurred (it's usually contained in a .pdb file, but can also be embedded directly in your .NET executable), you won't even get line numbers. In that case, unraveling what's null will require more detective work, making logging even more valuable.

When tracing the source of the null value, remember that it might not be visible as null in the code. In C#, null is the default value for reference types, so it could also come from a field that was never assigned or an array element that was never set.

By convention, in well-written applications using well-written libraries, all instances of NullReferenceException must be blamed on the author of the code that throws this exception. However, in codebases that do not follow defensive programming practices, finding the root cause can be more challenging.

Before showing how you can implement defensive programming and make these exceptions easier to diagnose, we will see how to cope with null values when they are legitimate and expected.

How to fix a NullReferenceException?

Once you've identified what variable is null, you can address the issue.

There are essentially two different kinds of situations, each with a different solution:

When null is a legitimate value for the variable or parameter because it is optional, and you need to cope with it. For instance, if in a class representing a tree node, the Parent property can legitimately contain a null reference for the root node.
When null is not an expected value, but some external code sent it to you by mistake, you must defend your code against it. This technique is called defensive programming.

Let's look at the first approach first.

How to cope with legitimate null references?

When a reference can legitimately be null (likely because the value is optional), you must cope with it. In that case, you must check if the reference is null before accessing the member. The C# language provides various ways to do this, from a simple != null check in an if statement, through pattern matching with is and switch expressions, to null-related operators ?. and ??. Which one to use depends on the context and your (or your team's) preference.

Here are some examples:

// Simple null check.
if ( product != null )
{
    cart.Add( product );
}

// Pattern matching with an empty pattern checks for null 
// and assigns to a variable.
if ( shop.SpecialOffer is { } specialOffer )
{
    cart.SetDiscount( specialOffer );
}

// Pattern matching using negated null pattern.
if ( shop is { SpecialOffer: not null } )
{
    cart.SetDiscount( shop.SpecialOffer );
}

// Null-conditional and null-coalescing operators combined 
// to set a default value when the reference is null.
var actualDiscount = discount?.Percent ?? 0;

How do I get rid of a NullReferenceException altogether?

Okay. You fixed a NullReferenceException. Now, you want to avoid them from happening in the future. How to do that? This is not a trivial question. Indeed, this problem has been called the billion-dollar mistake.

We suggest a four-step approach:

Enable nullability analysis in your project and fix all warnings
Adopt a contract-based mindset
Check preconditions at the surface of your component
Reduce boilerplate code and human errors

Step 1. Enable nullability analysis in your project and fix all warnings

You might wonder how you can know if a variable, parameter, or field can or cannot contain a null reference so that you can decide if you need to write the code to handle the null case.

To resolve this problem, the C# language now includes a feature called nullable reference types. It allows you, and the libraries you use, to specify which references can be null and which can't. This feature is optional but enabled by default in new projects targeting modern .NET. We suggest you enable this feature in all projects.

You can enable it by adding the following code to your csproj file or, better, to Directory.Build.props:

<PropertyGroup>
  <Nullable>enable</Nullable>
  <LangVersion>latest</LangVersion>
</PropertyGroup>

To enable the latest C# language features with legacy target frameworks (such as .NET Framework 4), you can use PolySharp.

Now that nullable reference types are enabled, you mark a reference type as nullable by adding a ? to it. Conversely, if you don't mark a reference as nullable, it is considered to be non-nullable. Based on these annotations, the compiler or the IDE will report warnings in two cases:

when you try to access a member of a nullable reference without checking if it's null, and
when you try to assign null or a nullable reference to a non-nullable variable, field, or parameter.

For instance, the C# compiler reports a warning in the following code:

public decimal ComputePrice( Product product, 
                             decimal quantity, Discount? discount )
{
    // C# reports a warning on `discount.Percent` 
    // because `discount` can be null.
    return product.Price * quantity * (1 - discount.Percent / 100m);
}

But the following code is correct:

public decimal ComputePrice( Product product, 
                             decimal quantity, Discount? discount )
{
    return product.Price * quantity * (1 - (discount?.Percent ?? 0) / 100m);
}

Step 2. Adopt a contract-based mindset

A key mindset in programming is that of a contract. When you write a method, you are implicitly entering a contract with the caller of the method. This contract is "if you call me and you fulfill conditions A, B, and C, I will perform actions P, Q, R, and return Z".

Nullability annotations on a method can be seen as a contract between your method and its caller. If you mark a parameter as nullable, it means that you are willingly accepting null values. However, if you don't, it means that it is the caller's responsibility to provide you with a non-null value.

The same applies to the return value: unless you mark the return type as nullable, you are committed to returning a non-null value to the caller. If you return null, then your method, and not the caller, is to blame for the NullReferenceException it will cause.

So, who is to blame for the NullReferenceException in the following snippet? The author of GetLastOrder or RepeatLastOrder? Of course, RepeatLastOrder, because GetLastOrder clearly stated it could return null!

// This method requires a Customer but does not promise to return an Order.
public Order? GetLastOrder( Customer customer ) 
    => customer.Orders.OrderByDescending( o => o.Date ).FirstOrDefault();

public void RepeatLastOrder( Customer customer )
{
    var lastOrder = this.GetLastOrder( customer );

    // This will throw NullReferenceException 
    // if the customer never ordered.
    var newOrder = lastOrder.Clone( DateTime.Today );
    this.PostOrder( newOrder );
}

For more complex nullability contracts, Microsoft also provides several nullability contract attributes. For example, consider the Dictionary<TKey, TValue>.TryGetValue(TKey key, out TValue value) method. For a non-nullable reference type TValue, its value parameter is null if the method returns false, but not-null if it returns true. This can't be expressed just by applying ?, which is why these attributes exist.

Step 3. Check preconditions at the surface of your component

To make it easier to diagnose NullReferenceException in large applications, a good practice is to implement a practice called defensive programming. Instead of accepting that your code could fail because of someone else's mistake (because this person could ignore warnings), defensive programming advocates that you should validate all inputs (specifically all preconditions) and throw a specific kind of exception if any precondition is not fulfilled.

There are three categories of exceptions in .NET:

Precondition failure exceptions are the ones that put the blame on the caller of the component:
- ArgumentException: means that the caller sends an invalid parameter. For instance, throw NullArgumentException if the caller sends you a null that you don't expect.
- InvalidOperationException: means that the operation is not available in the current state of the object. For instance, you are trying to write into a file opened in read-only mode.
- NotSupportedException: means that the operation is never available for this kind of object.
Some must be blamed on the component itself: NullReferenceException, ArithmeticException, InvalidCastException, ...
Finally, other exceptions must be blamed on the user or the environment: OutOfMemoryException, IOException, OperationCanceledException, ...

Because any NullReferenceException in your code will be blamed on nobody else than you, it's a good practice, not only to cover your back but also to ease the process of code troubleshooting, to check that the sender does not send you a null reference by mistake. In this case, you must throw an ArgumentNullException.

Another advantage of ArgumentNullException is consistency: if a method mistakenly receives a null reference, it might throw a NullReferenceException only sometimes. Such buggy code might linger in your codebase for a long time, and the exception might be hard to reproduce. Conversely, if you throw an ArgumentNullException, the caller will immediately see the bug and will hopefully fix it promptly.

The C# language offers you three ways to throw ArgumentNullException:

1. Use an if statement

The most obvious approach is to add the following code at the top of your method:

if ( customer == null )
{
    throw new ArgumentNullException( nameof(customer) );
}

2. Use the new ThrowIfNull

Alternatively, you can use the helper ThrowIfNull method:

ArgumentNullException.ThrowIfNull( customer );

This version is shorter and simpler but provides the same information (the name of the parameter that was null is captured using caller argument expression). Plus, it can even be slightly more efficient because it moves the rarely-executed throw statement to a separately compiled method.

3. Use an inline expression check

A less readable but compact approach is to use add ?? throw the first time you are using a parameter.

public Order? GetLastOrder( Customer customer )
    => (customer ?? throw new ArgumentNullException( nameof(customer) ))
        .Orders
        .OrderByDescending( o => o.Date )
        .FirstOrDefault();

Now, where exactly should you place these checks? Best practice is to do this at the surface area of your components. That means checking parameters of public methods and constructors, and the set value in public property setters. This way, you can focus on one component at a time, ensuring that null reference issues don't arise from interactions between components.

Step 4. Reduce boilerplate code and human errors

If you're thinking that checking parameters of all public methods sounds like tedious work, you're absolutely right. Not only is this work boring and time-consuming, but it's also error-prone. It's easy to forget to check for nullability in an entry point of your API, and the C# compiler won't remind you.

This redundant code can be avoided using a tool like Metalama. With the Metalama.Patterns.Contracts package, you can annotate a parameter with the [NotNull] attribute, and it will automatically generate the null check for you. The same attribute works on fields and properties as well. This is a slight improvement, but nothing too exciting.

public Order? GetLastOrder( [NotNull] Customer customer )
    => customer.Orders.OrderByDescending( o => o.Date ).FirstOrDefault();

The significant improvement comes from the ability to apply [NotNull] to an entire project at once. You can do this using a fabric that calls the VerifyNotNullableDeclarations method:

internal class Fabric : ProjectFabric
{
    public override void AmendProject( IProjectAmender amender )
        => amender.Outbound.VerifyNotNullableDeclarations();
}

This will add null checks to non-nullable reference types in all public members of your project. There are also ways to limit this to certain namespaces or types, or to expand it to include private members. If you need even more customization, all the relevant code is open source, so you can modify it to suit your exact needs.

Summary

NullReferenceException is one of the most common exception types in C#. They can be avoided by adopting a few best practices:

Enable null reference types in your C# project and use nullability annotations.
Aim for zero warnings.
Implement defensive programming, i.e., validate your inputs and proactively throw an ArgumentNullException.

Defensive programming makes it much easier to diagnose the root cause of a NullReferenceException because it logically isolates components from each other, with each component being responsible for itself and applying a least-trust principle to other components. Although it's a best practice, it also requires writing a lot of redundant code. Tools like Metalama can help add precondition checks without boilerplate code, leveraging the benefits of precondition checking without its inconveniences.

If you liked this article, make sure to subscribe to the Timeless .NET Engineer newsletter