(I am not AI and I too can make mistakes, but I did not write this with AI [only some technical parts]. I do not want to perpetuate the overuse of AI in the arts, or writing or creative world so enjoy some human-generated content)
The Plot - Why We Need Effective Tracing
If you've ever found yourself in a production situation where you "lost" a request and had no insight, no tangible logs, no links to what happened, you'll know the pain and suffering of trying to develop without proper logging and tracing. Having lived and worked in multiple styles of logging configurations I now know which types are my favorite, but there's more to it than just picking your preferred sink(s). Given the importance of traceability as a developer, I felt it would be good to dig deeper, learn more and write about distributed tracing with OpenTelemetry and .NET.
I'm not a huge fan of explaining what something is in a nutshell when I write about topics, since there's already a plethora of information available at your fingertips. But if you are in need of a refresher here's a link with an overview of what OpenTelemetry is and a link explaining the basics of distributed tracing in .NET.
Distributed Systems, Distributed Headaches
One of the added complexities of working with distributed systems is tracing or correlating a request with another request. This could be an HTTP request from your Single Page Application (SPA) front-end that goes through a Web Application Firewall (WAF) which gets proxied to your backend API, which then that API likely makes calls into a database and repeats all of the steps above by making calls to other APIs. All of these events occur on their own, and without a Correlation ID, Trace ID, or some form of identifier there's no way to track a single request all the way through the pipeline. This makes debugging a needle-in-a-haystack form of cruel punishment for anyone trying to find out what happened and often a wasted effort.
There's a subtle nuance to some of the terminology used in tracing. A Correlation ID and a Trace ID are essentially the same thing and will likely be used interchangeably. But technically speaking (I won't fault anyone if they do use them to mean the same thing, I do so myself) a Correlation ID is an older term where you did this manually using an identifier such as a GUID and set an
X-Correlation-IDheader yourself on every request. A Trace ID is the more modern term which is native to the OpenTelemetry specification and by design uses a header name oftraceparent.
A Brief History of the Activity Class in .NET
There's a little excerpt from the Microsoft team right on the .NET distributed tracing concepts page that gives us a hint about why you don't find a direct 1:1 implementation of the OpenTelemetry (OTel) standard as part of the .NET Base Class Libraries:
Another common industry name for units of work in a distributed trace are 'Spans'. .NET adopted the term 'Activity' many years ago, before the name 'Span' was well established for this concept.
Hat tip to the developers at Microsoft as they naturally ran into the problem of needing some sort of standardization to solve for tracing in such a vast and complex system years before OpenTelemetry was a thing. So, what they came up with was their System.Diagnostics.Activity class. Interestingly enough had they tried to use the term Span they'd have clashed with an already existing class called System.Span so instead of trying to sort that out they just left their original implementation of the Activity class and made it work with OTel.
This, at least for me, is where some confusion sets in. I find some of the utility and system classes in .NET difficult to remember simply because they can get so low-level and require a good bit of mental power to keep straight. They are also often named rather vague or ambiguously, which I get why, but this doesn't help with abstract thought turning into applied logic. That's why if something can simply be mirrored 1:1 to a broader standard and shared terminology I prefer that option. So, to help commit this to permanent memory reading a lot about it and really understanding the nuances was my only option.
How to Use Activity and OpenTelemetry in .NET
Client and Backend Configured to Use OpenTelemetry Libraries
Each time a new request is received by an application, it can be associated with a trace. In application components written in .NET, units of work in a trace are represented by instances of System.Diagnostics.Activity and the trace as a whole forms a tree of these Activities, potentially spanning across many distinct processes. The first Activity created for a new request forms the root of the trace tree and it tracks the overall duration and success/failure handling the request. - Traces and Activities, Microsoft Docs
So, let's do this using a very simple front-end web page that will follow the OpenTelemetry standard, send a request to a .NET backend, capture the Activity tree, create a nested Activity and send back the trace logs to the client. I'll provide links to a repo that I stored all this code in, so all I'll do in the following sections is highlight the main parts that wire up OTel to work properly.
Without getting too into the weeds here I'll just focus on the important bits. Luckily, there's no need to reinvent the wheel when it comes to our client-side implementation of OpenTelemetry as there are libraries available to help out. Some extra Googling or AI use will help you get what you need, but for purposes of this example, which I'm just using VanillaJS and no frameworks, here's how to get started in your client app to configure it for OTel. In this example I'm not even using any libraries, this is just manually configured.
1) Configure Client Functions to Generate a Trace ID
Get the Full Code Sample in My Repo Here
Note: This is not recommended for large or production-grade apps. I'm just showing this as the fastest way to get started. I'd recommend using npm with webpack or something similar to use this in a production app and relying on proper OpenTelemetry packages in various frameworks. Some Googling will give you the most up-to-date versions for your framework.
Add JavaScript to generate your Trace IDs
Here's a snippet of the code needed to help generate our custom Trace IDs that follow the OpenTelemetry standard formatting:
// Generate a random trace ID (128-bit hex)
function generateTraceId() {
const bytes = new Uint8Array(16); // 16 bytes
crypto.getRandomValues(bytes); // Random values
return Array.from(bytes, byte => byte.toString(16).padStart(2, '0')).join('');
// Each byte becomes 2 hex chars → 16 bytes × 2 = 32 characters
}
// Generate a random span ID (64-bit hex)
function generateSpanId() {
const bytes = new Uint8Array(8);
crypto.getRandomValues(bytes);
return Array.from(bytes, byte => byte.toString(16).padStart(2, '0')).join('');
}
// Create W3C traceparent header
// Format: version-traceId-spanId-flags
function createTraceParent() {
const version = '00';
const traceId = generateTraceId();
const spanId = generateSpanId();
const flags = '01'; // sampled
return {
header: `${version}-${traceId}-${spanId}-${flags}`,
traceId: traceId,
spanId: spanId
};
}
The generateTraceId() Function
The generateTraceId() function creates a 32-character hex string, representing a 128-bit (16 bytes) random ID:
function generateTraceId() {
const bytes = new Uint8Array(16); // 16 bytes
crypto.getRandomValues(bytes); // Random values
return Array.from(bytes, byte => byte.toString(16).padStart(2, '0')).join('');
// Each byte becomes 2 hex chars → 16 bytes × 2 = 32 characters
}
The generateTraceParent() Function
The '00' prefix is added separately in the createTraceParent() function as the version field:
function createTraceParent() {
const version = '00'; // ← W3C version prefix
const traceId = generateTraceId(); // ← 32 hex chars
const spanId = generateSpanId(); // ← 16 hex chars
const flags = '01'; // ← sampling flags
return {
header: `${version}-${traceId}-${spanId}-${flags}`,
// Final format: 00-{32 chars}-{16 chars}-01
traceId: traceId,
spanId: spanId
};
}
So the final structure of the Trace ID when these functions are all put together looks like this when sent to the server using the traceparent header:
00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01
│ └────── 32 chars ──────┘ └─ 16 chars ─┘ │
version (W3C v1.0) span ID sampled flag
When you finally make the call from the client to the server you will see this header added to every network request with a Trace ID:
2) Wire up the final call to your .NET API backend
The code to call your backend creates a traceparent ID, uses the fetch API to make the call and adds the traceparent header to the request and then displays the output on the page:
async function callApi() {
const button = document.getElementById('callApiBtn');
const output = document.getElementById('outputContent');
// Disable button during request
button.disabled = true;
// Show loading state
output.innerHTML = '<div class="loading">Sending request to backend...</div>';
try {
// Generate trace context
const trace = createTraceParent();
console.log('Sending request with traceparent:', trace.header);
// Make the fetch request with traceparent header
const response = await fetch(API_URL, {
method: 'GET',
headers: {
'traceparent': trace.header,
'Content-Type': 'application/json'
}
});
if (!response.ok) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
}
const data = await response.json();
// Display the results
displayResults(data, trace);
} catch (error) {
console.error('Error calling API:', error);
output.innerHTML = `
<div class="error">
<strong>Error:</strong> ${error.message}
<br><br>
Make sure your .NET backend is running on ${API_URL}
</div>
`;
} finally {
// Re-enable button
button.disabled = false;
}
}
3) All Client Code Coming Together
The full code for this process will be available in my GitHub repo, for purposes of keeping this readable and somewhat short I'll skip some code here. Please visit my repo to copy out exactly what I used in my example. Here's a look at the final request and response from my client app to my backend:
Configuring Our .NET Backend
I know at this point this is getting rather verbose, I'm trying my best to make it make sense and flow. But, honestly this sort of highlights exactly why I felt the need to go deeper in learning about distributed tracing and furthermore why I wanted to write about it. I find writing and explaining solidifies something much more effectively in my mind that simply reading about it. At this point, after several hours of research and typing, I can tell I already have a much better grasp on all the moving parts. So, I'll keep trying to keep this as clear and concise as possible. Stick with me!
1) Update your Program.cs file in .NET to accept headers
I'm making a few assumptions here, but the requirements for this next part are as simple or as complex as you want them to be. All I did was create a brand new ASP.NET Web API project in Visual Studio 2022 and made these minors modifications. The first step is to update the Program.cs file so that it accepts our traceparent header:
// Program.cs file - add this
builder.Services.AddCors(options => {
options.AddDefaultPolicy(policy => {
policy.WithOrigins("http://localhost:5500") // Or your client port
.AllowAnyMethod()
.WithHeaders("traceparent") // Necessary!
.WithExposedHeaders("traceparent");
});
});
2) Configure .NET Routing to Capture, Create and Return Activity
Now that all the plumbing is in place we can finally start to work with the incoming Trace, or more specifically the Traceparent header that our client now sends in every request. In our .NET app we'll create a controller endpoint to test this out. Stub in the following code as follows into a new TraceController.cs file:
// TraceController.cs file
using Microsoft.AspNetCore.Mvc;
using System.Diagnostics;
namespace OpenTelemetry.Controllers
{
[ApiController]
[Route("api/[controller]")]
public class TraceController : ControllerBase
{
// ActivitySource for creating custom activities
private static readonly ActivitySource ActivitySource = new("OpenTelemetry.Demo", "1.0.0");
private readonly ILogger<TraceController> _logger;
public TraceController(ILogger<TraceController> logger)
{
_logger = logger;
}
/// <summary>
/// Demonstrates OpenTelemetry distributed tracing with automatic W3C Trace Context propagation
/// </summary>
[HttpGet]
public IActionResult GetTraceInfo()
{
// Create a custom activity that will inherit the trace context from the incoming traceparent header
using var activity = ActivitySource.StartActivity("ProcessDemoRequest");
// Add some custom tags/attributes to the activity
activity?.SetTag("demo.type", "blog-example");
activity?.SetTag("demo.purpose", "opentelemetry-demonstration");
activity?.SetTag("demo.timestamp", DateTime.UtcNow.ToString("o"));
// Get the current activity (which has the trace context from the incoming request)
var currentActivity = Activity.Current;
// Create a list of log messages to return
var logs = new List<string>
{
"Processing demo request",
"Custom activity created successfully",
$"TraceId: {currentActivity?.TraceId}",
$"SpanId: {currentActivity?.SpanId}",
$"Current time: {DateTime.UtcNow:yyyy-MM-dd HH:mm:ss} UTC"
};
// Log these messages using structured logging (in production, these would go to your logging provider)
_logger.LogInformation("Processing demo request with TraceId: {TraceId}", currentActivity?.TraceId);
_logger.LogInformation("Custom activity '{ActivityName}' created", currentActivity?.DisplayName);
// Additional demonstration: simulate some work
Thread.Sleep(10); // Simulate processing time
logs.Add("Simulated processing completed");
// Build the response object
var response = new TraceInfoResponse
{
TraceId = currentActivity?.TraceId.ToString() ?? "N/A",
SpanId = currentActivity?.SpanId.ToString() ?? "N/A",
ParentSpanId = currentActivity?.ParentSpanId.ToString() ?? "N/A",
ActivityName = currentActivity?.DisplayName ?? currentActivity?.OperationName ?? "N/A",
Logs = logs,
AdditionalData = new Dictionary<string, object>
{
{ "timestamp", DateTime.UtcNow },
{ "serverTime", DateTime.Now },
{ "activityDuration", activity?.Duration.TotalMilliseconds ?? 0 },
{ "customMessage", "This demonstrates distributed tracing!" },
{ "tags", activity?.Tags.ToDictionary(t => t.Key, t => t.Value) ?? new Dictionary<string, string?>() }
}
};
_logger.LogInformation("Returning trace information. TraceId matched: {Matched}",
currentActivity?.TraceId != null);
return Ok(response);
}
}
/// <summary>
/// Response model for trace information
/// </summary>
public class TraceInfoResponse
{
public string TraceId { get; set; } = string.Empty;
public string SpanId { get; set; } = string.Empty;
public string ParentSpanId { get; set; } = string.Empty;
public string ActivityName { get; set; } = string.Empty;
public List<string> Logs { get; set; } = new();
public Dictionary<string, object>? AdditionalData { get; set; }
}
}
The Aha! Moment Behind .NET's Implementation
This, for me, was a fun revelation. If you look at our class, and our Program.cs file you'll notice we never had to import any new libraries to work with our client OpenTelemetry implementation. This is exactly the genius behind why Activity still exists and why the developers at Microsoft chose to do it this way. They already had the necessary plumbing in place, that serendipitously was nearly identical in contract to OTel, but years before it became a thing. So, when in Rome, I suppose!
And because of this we just need to be aware of the way things map between the OTel definitions and the Activity definitions. Full transparency, because I'm not in the mood to spend hours putting this together, I did use Copilot to help generate a cheat sheet for how the two designs map to each other. So, here's a breakdown of how the .NET Activity API maps to the OpenTelemetry spec:
Core Properties & Methods of How Activity Maps to OTel
.NET Activity API |
OpenTelemetry Concept |
Description |
Usage Example |
|---|---|---|---|
Activity.Current |
Current Span |
Gets the current activity in the execution context (automatically set by ASP.NET Core) |
|
TraceId |
Trace ID |
128-bit unique identifier for the entire distributed trace |
|
SpanId |
Span ID |
64-bit unique identifier for this specific span/activity |
|
ParentSpanId |
Parent Span ID |
Span ID of the parent activity (from |
|
ActivitySource |
Tracer |
Creates and manages activities/spans for a logical component |
|
StartActivity() |
Start Span |
Creates a new child activity/span |
|
SetTag(key, value) |
Set Attribute |
Adds metadata key-value pairs to the span |
|
SetStatus() |
Set Status |
Sets the span status (Ok, Error, Unset) |
|
ActivityKind |
Span Kind |
Type of span: Server, Client, Internal, Producer, Consumer |
|
Duration |
Span Duration |
How long the activity took (auto-calculated) |
|
DisplayName |
Span Name |
Human-readable name for the operation |
|
AddEvent() |
Add Event |
Adds a timestamped event to the span |
|
Context |
Span Context |
Propagation context (TraceId, SpanId, TraceFlags) |
Used internally for propagation |
Baggage |
Baggage |
Key-value pairs propagated across service boundaries |
|
W3C Trace Context Mapping
Component |
Format |
.NET Activity Property |
Notes |
|---|---|---|---|
traceparent header |
|
Auto-extracted by ASP.NET Core |
Sets |
Version |
|
N/A (handled internally) |
W3C Trace Context v1.0 |
Trace ID |
32 hex chars (128-bit) |
|
Immutable across entire trace |
Span ID |
16 hex chars (64-bit) |
|
Unique to this activity |
Trace Flags |
2 hex chars (8-bit) |
|
|
tracestate header |
Vendor-specific data |
|
Optional, rarely used |
Essential Methods You Should Know
Method |
Purpose |
When to Use |
|---|---|---|
|
Create a new span |
Tracing a specific operation or method |
|
Add metadata |
Enriching spans with context (user ID, request params, etc.) |
|
Mark success/failure |
Error handling, operation result |
|
Log timestamped event |
Significant moments during span lifecycle |
|
End the span |
Always use |
Common Patterns You'll Use as a .NET Developer with Activity
Pattern 1: Access Current Trace Context
Important things to note are that Activity.Current is a static property that returns the current activity instance for the current execution context. I.e. it's available anywhere during a request lifecycle. Some more fun facts around it are that Activity.Current uses AsyncLocal<T> under the hood, which means that it's thread-safe (each thread has its own current activity), it's async-safe (flows through async/await automatically) and request scoped (each HTTP request has its own activity context. So, what does this all mean? You can access it anywhere in your request like this:
// In your controller
public IActionResult MyAction()
{
var traceId = Activity.Current?.TraceId; // Gets current activity
DoSomeWork();
return Ok();
}
// In a service class (same request context)
public void DoSomeWork()
{
var traceId = Activity.Current?.TraceId; // Same trace ID
await DoAsync();
}
// Even after async calls
public async Task DoAsync()
{
await Task.Delay(100);
var traceId = Activity.Current?.TraceId; // Still the same trace ID
}
// Other ways to get the current activity
var currentActivity = Activity.Current;
var traceId = currentActivity?.TraceId.ToString();
var spanId = currentActivity?.SpanId.ToString();
var parentSpanId = currentActivity?.ParentSpanId.ToString();
Pattern 2: Create Custom Activity
If you want to create, capture or trace your own custom activity it's as simple as wiring things up like the example below. This can be useful for things like isolating specific tasks, like database calls, API calls or some long running bit of business logic. You could capture diagnostic information about how long something took and view that in a meaningful way in your logs. To create a custom activity, you set it up like the following:
private static readonly ActivitySource Source = new("MyService", "1.0.0");
using var activity = Source.StartActivity("ProcessOrder");
activity?.SetTag("order.id", orderId);
activity?.SetTag("customer.id", customerId);
// Work happens here
// Activity automatically ends when disposed
Pattern 3: Error Handling
And to really round things out here is how you can handle errors when using the Activity class and tracing. This helps to enrich your logs with very human-readable information and to be able to pinpoint exactly where errors are happening with context included. Which when you're pouring over thousands of rows of log data this becomes a powerful tool to have:
try
{
// Work
activity?.SetStatus(ActivityStatusCode.Ok);
}
catch (Exception ex)
{
activity?.SetStatus(ActivityStatusCode.Error, ex.Message);
activity?.RecordException(ex); // .NET 7+
throw;
}
(End of AI assisted info generation. I'm trying not to use AI to actually write anything for me, but this one made sense as it's a lot of code-heavy technical samples)
Wrapping Things Up
Honestly, I'd hoped writing this wasn't going to be very involved, but that was a huge oversight on my part. I should have known that a topic so in-depth and also so powerful wouldn't just be a simple summary in the end. But, I learned a lot myself, I now feel like I could certainly configure, use and customize the .NET Activity class and use in a lot of ways.
Most of my professional life now is in the land of Azure App Insights, which captures troves of data across hundreds of distributed systems, so the motivation here was really to fully grasp what's going on and how to implement from scratch the modern OpenTelemetry spec, but to also use the native BCL Activity components that exist in the .NET framework. Libraries specifically for OTel do exist but being able to tap into tools and features that are already there, reduce yet another dependency, and are just as robust seems like a win-win.
If you found this post helpful, or even if you found it not super helpful, I'd love to hear from you. I also would love to hear how in-depth, or how often other devs find themselves taking a deep dive into this or similar topics to help them debug and maintain their sanity.
Otherwise, thanks for reading, and happy coding!
~ Charles
P.S. if you enjoyed reading this, you'll surely enjoy reading my ramblings about how AI Didn't Kill Software Development, and Here's Why It Never Will
Top comments (0)