Modern Logging and Telemetry in .Net

Introduction

Logging and telemetry are essential pillars of operational excellence in modern software systems. They provide real-time visibility into application behavior, system performance, and user interactions, enabling DevOps teams to detect and troubleshoot issues quickly before they impact users. Beyond just technical insights, this data informs product management decisions and enhances customer engagement by enabling proactive improvements and a deeper understanding of how users interact with the product.

🎯 Article Objective

The objective of this article is to introduce you to the fundamentals of OpenTelemetry and its integration with Azure Application Insights in a .NET application. It aims to highlight the importance of logging and telemetry in achieving operational excellence, particularly in modern cloud-native systems. By the end of this article, readers will understand how to:

• Set up OpenTelemetry for distributed tracing and metrics collection in a .NET Web API.
• Enable real-time visibility into application performance using Application Insights.
• Enhance troubleshooting, DevOps workflows, and customer experience through effective telemetry.
• Implement correlation IDs for traceability across services.
• Collect critical performance metrics such as CPU usage, memory availability, and request duration.

This hands-on guide empowers developers to build more reliable, observable, and user-centric applications.

---

🔍 What is OpenTelemetry?

OpenTelemetry is an open-source standard for collecting telemetry data — logs, metrics, and traces — from your application.

It helps answer:

• How long does a request take?
• What services were involved in processing it?
• Where are the bottlenecks?

---

📘 Why Not Just Use Logging?

Traditional logging is useful, but lacks context. Here’s how OpenTelemetry compares:

Feature	Logging	OpenTelemetry
Simple log messages	✅	✅
Tracing across services	❌	✅
Built-in metrics (CPU, memory)	❌	✅
Visualization in tools like Application Insights	❌	✅
Correlation across services	❌	✅

---

🎯 Why Use OpenTelemetry with Application Insights?

• Application Insights is a powerful observability platform in Azure.
• OpenTelemetry allows your app to send standardized, portable telemetry data.
• Gain end-to-end visibility into distributed systems, including APIs, microservices, and external dependencies.

---

🛠️ Step-by-Step: Use OpenTelemetry in .NET 7

We’ll build a simple Web API that uses OpenTelemetry to trace requests and collect system metrics.

---

📦 Step 1: Add Required Packages

Run this in your terminal:

dotnet add package Microsoft.ApplicationInsights.AspNetCore
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http
dotnet add package OpenTelemetry.Exporter.ApplicationInsights

🧠 Step 2: Configure Program.cs

using Microsoft.ApplicationInsights.Extensibility;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Enable Application Insights
builder.Services.AddApplicationInsightsTelemetry();

// OpenTelemetry setup
builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService("MyWebApi"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSource("MyWebApi")
        .SetSampler(new AlwaysOnSampler())
        .AddApplicationInsightsExporter(options =>
        {
            options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
        }))
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddRuntimeInstrumentation()
        .AddProcessInstrumentation()
        .AddApplicationInsightsMetricsExporter(options =>
        {
            options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
        }));

var app = builder.Build();

// Middleware to add correlation ID to response
app.Use(async (context, next) =>
{
    var traceId = System.Diagnostics.Activity.Current?.TraceId.ToString();
    if (!string.IsNullOrEmpty(traceId))
    {
        context.Response.Headers.Add("X-Correlation-ID", traceId);
    }
    await next();
});

app.MapControllers();
app.Run();

📂 Step 3: Create a Controller (TraceDemoController.cs)

using Microsoft.AspNetCore.Mvc;
using System.Diagnostics;

namespace MyWebApi.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class TraceDemoController : ControllerBase
    {
        private static readonly ActivitySource ActivitySource = new("MyWebApi");

        [HttpGet("hello")]
        public IActionResult Get()
        {
            using var activity = ActivitySource.StartActivity("SayHello");

            var traceId = Activity.Current?.TraceId.ToString();
            return Ok(new
            {
                Message = "Hello, world!",
                TraceId = traceId
            });
        }
    }
}

⚙️ Step 4: Add Configuration (appsettings.json)

Add your Application Insights connection string:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information"
    }
  },
  "APPLICATIONINSIGHTS_CONNECTION_STRING": "InstrumentationKey=YOUR-KEY-HERE"
}

📊 Metrics You Get Automatically

OpenTelemetry + Application Insights gives you:

Metric	Source
Request duration	AddAspNetCoreInstrumentation
CPU usage	AddRuntimeInstrumentation
Memory available	AddProcessInstrumentation
Request count	AddAspNetCoreInstrumentation

All these show up in Azure Application Insights under Metrics.

🧪 Testing the API

Run the app:

dotnet run

Call the endpoint:

GET http://localhost:5000/TraceDemo/hello

You’ll see the X-Correlation-ID in the response headers — that’s your trace ID!

📈 Viewing in Application Insights

Go to Azure Portal > Application Insights.

Use:

• Application Map for service relationships.
• Metrics to view CPU, memory, and requests.
• Traces to search with X-Correlation-ID.
• Performance to analyze slow endpoints.

✅ Summary

With OpenTelemetry + Application Insights, you now have:

✔ Distributed tracing

✔ Built-in performance metrics

✔ Context-rich diagnostics

✔ Cloud-native observability

This is a big step up from traditional logging and a must-have for any production-ready .NET app.

🧭 What’s Next?

Connect logs (ILogger) to trace context.

Export telemetry to other tools (Jaeger, Prometheus, etc.).

Set up alerts based on CPU, memory, or request duration.

🧠 Got Questions?

Feel free to drop issues or suggestions!