DEV Community

Cover image for What every ASP.NET Core Web API project needs - Part 1 - Serilog
Mohsen Esmailpour
Mohsen Esmailpour

Posted on • Updated on

What every ASP.NET Core Web API project needs - Part 1 - Serilog

In a series of articles, I'm going to show the implementation of an architecture that is suitable for a thin Web API project or Web API that fits in a microservices architecture. In the first few articles, I'm going to introduce several useful libraries.

Let's get started with logging. Logging is just essential for debugging, troubleshooting and monitoring. A good logging system makes life much easier.

Why Serilog? It is easy to set up, has a clean API, and is portable between recent .NET platforms. The big difference between Serilog and the other frameworks is that it is designed to do structured logging out of the box. Another thing I really like about Serilog is that it can be configured via the appsetting.json file alongside configuring through code. Changing logging configuration without touching the codebase is really helpful, especially in the production environment.

Let's start with creating a new project.

Step 1 - New project

Create a new ASP.NET Core 5.0 API project.

Setp 2 - Install package

Install Serilog.AspNetCore nuget package.

Step 3 - Add UseSerilog extension method

Open Program.cs file and modify CreateHostBuilder method:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        })
        .UseSerilog((hostingContext, loggerConfiguration) =>
            loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration));
Enter fullscreen mode Exit fullscreen mode

UseSerilog sets Serilog as the logging provider. We are going to config Serilog via the appsettings.json file.

Step 4 - Remove default configuration

Open appsettings.json and appsettings.Development.json file and get rid of the logging section:

"Logging": {
  "LogLevel": {
    "Default": "Information",
    "Microsoft": "Warning",
    "Microsoft.Hosting.Lifetime": "Information"
  }
}
Enter fullscreen mode Exit fullscreen mode

Step 5: Add Serilog configuration

Add Serilog configuration section to appsettings.Development.json file:

"Serilog": {
  "MinimumLevel": {
    "Default": "Debug",
    "Override": {
      "Microsoft": "Information",
      "System": "Information"
    },
    "Using": [ ],
  },
  "WriteTo": [
    { }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Serilog.AspNetCore nuget package has dependency on Serilog.Settings.Configuration nuget package and it is a Serilog settings provider that reads from Microsoft.Extensions.Configuration sources. The above configuration is equivalent to this:

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
    .MinimumLevel.Override("System", LogEventLevel.Information)
    .Enrich.FromLogContext()
    .CreateLogger();
Enter fullscreen mode Exit fullscreen mode

Step 6 - Installing Sinks

Serilog uses sinks to write log events to storage for example database, file, etc. One of the most popular sinks for debugging environment is the Console sink.

  • Install Serilog.Sinks.Consolenuget package
  • Add following configuration:
"Serilog": {
  "MinimumLevel": {
    "Default": "Debug",
    "Override": {
      "Microsoft": "Information",
      "System": "Information"
    }
  },
  "Using": [ "Serilog.Sinks.Console" ],
  "WriteTo": [
    { "Name": "Console" }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Console Sink
After installing the sink

  • In the Using section add the sink's nuget package name "Using": [ "Serilog.Sinks.Console" ]
  • In the WriteTo section add sink name and arguments "WriteTo":[ { "Name": "Console" } ]

Now we want to use SQL Server sink for other environments:

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Error",
      "System": "Error"
    },
    "Using": [ "Serilog.Sinks.MSSqlServer" ]
  },
  "WriteTo": [
    {
      "Name": "MSSqlServer",
      "Args": {
        "connectionString": "ConnectionString",
        "tableName": "Logs",
        "autoCreateSqlTable": true
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Step 7 - How to configure a sink

Well, configuring a sink via appsettings.json could be harder than configuring through the code, and for each sink, you might not be able to find a JSON configuration sample. Normally each sink accepts several parameters to configure the sink. For instance, the Console sink accepts the below parameters:
Console Sink Configuration
Each one of these parameters can be configured through JSON:

"WriteTo": [
  {
    "Name": "Console",
    "Args": {
      "restrictedToMinimumLevel": "Verbose",
      "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} <s:{SourceContext}>{NewLine}{Exception}",
      "theme": "Serilog.Sinks.SystemConsole.Themes.AnsiConsoleTheme::Code, Serilog.Sinks.Console"
    }
  }
]
Enter fullscreen mode Exit fullscreen mode

Colored Console Sink
To see complete SQL Server sink JSON configuration check out this.

Step 8 - Enrichers

Log events can be enriched with properties in various ways. You can add additional data by enrichers. For instance, in the production environment, we want to add the IP of the client to the log events.

  • Install Serilog.Enrichers.ClientInfo package
  • Add enriched package name to Using section
  • Add Enrich section with WithClientIp value (enriched name normally starts with With word)
"Using": [ "Serilog.Sinks.MSSqlServer", "Serilog.Enrichers.ClientInfo" ],
"Enrich": [ "WithClientIp" ]
Enter fullscreen mode Exit fullscreen mode

All events written through log will carry a property ClientIp with the IP of the client. Check out the list of available enrichers here.

Step 8 - Filters

By using filters you can exclude or include some log events.

  • Install Serilog.Expressions nuget package
  • Add the "Filter" section to Serilog settings
"Filter": [
  {
    "Name": "ByExcluding",
    "Args": {
      "expression": "RequestPath like '%swagger%'"
    }
  }
]
Enter fullscreen mode Exit fullscreen mode

All log events that contain swagger will be excluded.
Serilog exclude
To see all possible configurations check out Serilog.Settings.Configuration Github repository.

Step 9 - HTTP requests loging

Moreover, you can log the HTTP requests.

  • In Startup.cs file, add the middleware with UseSerilogRequestLogging():
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    ...
    app.UseSerilogRequestLogging();
Enter fullscreen mode Exit fullscreen mode
  • In MinimumLevel.Override section add "Microsoft.AspNetCore": "Warning":
"MinimumLevel": {
  "Default": "Information",
  "Override": {
    "Microsoft": "Error",
    "Microsoft.AspNetCore": "Warning",
    "System": "Error"
  },
Enter fullscreen mode Exit fullscreen mode

Step 10 - Overridng configuration in docker

Last but not least here I want to mention is that you can override the Serilog setting by Docker environment variable. Consider the following configuration:

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Error",
      "System": "Error"
    },
    "Using": [ "Serilog.Sinks.MSSqlServer" ]
  },
  "WriteTo": [
    {
      "Name": "MSSqlServer",
      "Args": {
        "connectionString": "",
        "tableName": "Logs",
        "autoCreateSqlTable": true
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Now in the dcoker-compose file we want pass the actual connection string:

  my-api:
    environment:
      - ASPNETCORE_ENVIRONMENT=Production
      - Serilog__MinimumLevel__Default=Warning
      - Serilog__WriteTo__0__Args__connectionString="Your connection string"
Enter fullscreen mode Exit fullscreen mode

The value of each section can be accessed by __, for instance, Serilog__MinimumLevel__Default is equivalent to:

"Serilog": {
  "MinimumLevel": {
    "Default": "",
Enter fullscreen mode Exit fullscreen mode

In a section to access an item inside the array, use the item index number. "WriteTo" section accepts an array of sinks configuration. If you are using two sinks use Serilog__WriteTo__0__ to access the first sink and Serilog__WriteTo__1__ to access the second sink.

Test

Let's do a simple test. Open a CMD or Powershell at the project directory:

  • Type dotnet add package Serilog.Sinks.File to install File sink Alt Text
  • Open appsettings.josn file and change the logging configuration like this:
"Serilog": {
  "MinimumLevel": {
    "Default": "Debug",
    "Override": {
      "Microsoft": "Information",
      "System": "Information"
    }
  },
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.File" ],
  "WriteTo": [
    { "Name": "Console" },
    {
      "Name": "File",
      "Args": {
        "path": "log.txt",
        "rollingInterval": "Day"
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode
  • Type dotnet build then dotnet run
  • After running the application you should see a log file inside the project directory Alt Text As you can see without touching codes we added another sink to save the log events.

If you're using SQL Server, PostgreSQL or MongoDB sinks, I have developed a small log viewer for small projects. It helps a lot, especially in the production environment and you don't need to query the database to view logs.
Alt Text

Source code for this walkthrough could found on the Github.

Discussion (16)

Collapse
rafalpienkowski profile image
Rafal Pienkowski

Nice article.

There is one thing that you should be aware of when it comes to the Console sink. I’ve seen that you used it with the docker container.
Serilog’s Console the sink is synchronous and has no buffer. That means during the high load it slows down your application. The tread waits until the log is written. As a solution, there is an async sink nuget package. It introduces asynchronously logging and configurable buffers.
As a side note, other sinks like SQL, Application Insights are asynchronous by design.

Collapse
moesmp profile image
Mohsen Esmailpour Author

Yes, you right @rafal
I think using console sink with docker container it's not a good idea and doesn't make sense. I just wanted to make an example of how to override configuration. I'm going to change it with a proper example.

Collapse
rafalpienkowski profile image
Rafal Pienkowski

According to 12-factor-app it does have sense. I see your point of view. I just want to warn everyone:)

Collapse
jasonsbarr profile image
Jason Barr

This is great, but I can't find anything on how to set up and configure Serilog in ASP.NET 6, which completely revamps the Program class (and does away with having a separate Startup class). Do you know anything about how to do that? I really need logging in an API I'm setting up for work and I can't find any information at all on configuring Serilog with ASP.NET 6.

Collapse
moesmp profile image
Mohsen Esmailpour Author

I haven't tried .NET 6 and minimal APIs yet but this weekend I will try and let you know If I had any luck.

Collapse
jasonsbarr profile image
Jason Barr • Edited on

I would appreciate that! Logging is the one significant piece my app needs that I can't figure out how to set up. It's a work project.

I know, I know, building a work project on a pre-release version of .NET probably wasn't the brightest idea, but since it's an LTS it made sense to me since it will be supported for a longer time than .NET 5.

The sticking point is I can't figure out where to put UseSerilog, or what else to use if that method isn't going to work.

Thread Thread
moesmp profile image
Mohsen Esmailpour Author • Edited on

I did same thing with .NET Core RC1 :) It was very risky, but we did the project successfully.
This code works for me and try it

using Serilog;

IHostBuilder host = builder.Host
    .ConfigureDefaults(args)
    .UseSerilog((hostingContext, loggerConfiguration) =>
                    loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration)
                        .Enrich.FromLogContext());
Enter fullscreen mode Exit fullscreen mode

Code

dev-to-uploads.s3.amazonaws.com/up...

Thread Thread
jasonsbarr profile image
Jason Barr

Well, it doesn't generate an error or crash, so that's good. When I run dotnet run it hangs, but when I run dotnet build it runs without a hitch.

Sorry for being such a n00b but this is literally my first .NET project so everything is new and uncertain.

Thread Thread
jasonsbarr profile image
Jason Barr

I figured it out - I forgot to tell it in appsettings.Development.json where to output the log info. Silly beginner mistake.

Thread Thread
jasonsbarr profile image
Jason Barr

Now how do I actually access the logger and use it? Do I need to create an instance somewhere or set it up for dependency injection?

Thread Thread
moesmp profile image
Mohsen Esmailpour Author

No, just inject I ILogger into classes you want to log data.

Thread Thread
jasonsbarr profile image
Jason Barr

Got it working! Thanks for taking the time to help me out.

Collapse
alexisincogito profile image
Alexis Incogito • Edited on

That's exactly what I was looking for, thank you for putting this project together.
I already understand all of the pieces, but being unfamiliar with ASP.NET I was missing a boilerplate template that save me the time of piecing everything together, one blog/documentation at a time.

Great idea!

Collapse
marcosbelorio profile image
Marcos Belorio

Nice post, thanks for sharing, it's simple and full of content

Collapse
rmaurodev profile image
Ricardo

Me too. Very nice article. Thanks for sharing.

Collapse
uthmanrahimi profile image
Uthman Rahimi

Looking forward to reading the rest of this series.