DEV Community: Adrián López

OpenTelemetry in .NET Microservices: A Practical Guide to Observable Distributed Systems

Adrián López — Wed, 29 Apr 2026 17:40:19 +0000

You've deployed your fifth microservice. Requests flow through an API gateway, hit an order service, fan out to inventory and payment, and eventually land in a notification pipeline. Everything works — until it doesn't. A customer reports that checkout takes 12 seconds. Your logs show nothing unusual. Your metrics say average latency is fine. But somewhere in that chain of services, something is dragging, and you have no idea where.

This is the observability gap that kills microservice architectures. Monoliths gave us stack traces. Microservices gave us distributed amnesia.

OpenTelemetry exists to close that gap — and in the .NET ecosystem, it's matured from an experimental curiosity into the de facto standard for instrumentation. This post covers everything you need to go from zero to production-grade observability in your .NET microservices, with concrete C# examples, real debugging scenarios, and deployment patterns for Azure.

Why Observability Is Hard in Microservices

If you've operated distributed systems, you already know the pain points:

Distributed tracing is non-trivial. A single user request might touch 8 services, 3 databases, and 2 message queues. Correlating that flow requires propagating context across HTTP calls, gRPC channels, and async messaging — consistently.
Logs are scattered. Each service writes its own logs to its own sink. Reconstructing a timeline means grepping across dozens of streams and mentally stitching timestamps together.
Metrics lack context. Knowing that P99 latency spiked on the payment service tells you where to look, but not why. Was it a slow database query? A downstream timeout? A garbage collection pause?
Vendor lock-in is real. You instrumented everything for Datadog, and now leadership wants to move to Azure Monitor. Ripping out and replacing telemetry code across 20 services is not how anyone wants to spend a quarter.

OpenTelemetry addresses all of these problems with a single, vendor-neutral instrumentation framework.

What Is OpenTelemetry?

OpenTelemetry (OTel) is a CNCF project that provides a unified set of APIs, SDKs, and conventions for generating, collecting, and exporting telemetry data — specifically traces, metrics, and logs.

The key value proposition is decoupling instrumentation from backends. You instrument your code once with OTel APIs, and then choose where to send the data: Azure Monitor, Jaeger, Grafana, Coralogix, Datadog, Honeycomb, or any OTLP-compatible backend. Swap backends without touching application code.

OpenTelemetry in the .NET Ecosystem

The .NET OTel story is particularly strong because it builds on primitives that already exist in the runtime:

System.Diagnostics.Activity is the native .NET distributed tracing API, and OTel's .NET SDK wraps it directly — no parallel abstraction layer.
System.Diagnostics.Metrics (introduced in .NET 6) provides the metrics API that OTel hooks into.
ILogger<T> integration means your existing structured logging flows naturally into OTel's log pipeline.

This isn't a foreign framework bolted onto .NET. It's a thin orchestration layer over APIs you're probably already using.

Core Concepts

Traces, Spans, and Context Propagation

A trace represents the entire lifecycle of a request as it flows through your system. Each trace is composed of spans — individual units of work. A span records an operation's name, start/end timestamps, status, and key-value attributes.

Spans form a tree: a root span (e.g., the incoming HTTP request) has child spans (e.g., a database query, an outbound HTTP call), which may themselves have children. The parent-child relationship is established via context propagation — passing a trace ID and span ID across service boundaries.

In .NET, context propagation over HTTP is handled automatically when you use HttpClient with the OTel SDK configured. The SDK injects W3C traceparent headers on outbound calls and extracts them on inbound requests. For messaging scenarios (RabbitMQ, Azure Service Bus), you'll need to propagate context manually through message headers.

[API Gateway] ──► [Order Service] ──► [Payment Service]
  span: gateway       span: order         span: payment
  traceId: abc123     traceId: abc123     traceId: abc123
  spanId: 001         spanId: 002         spanId: 003
                      parentId: 001       parentId: 002

Metrics and Logs

Metrics capture aggregated, numerical measurements over time: request counts, histogram of response times, gauge of active connections. Unlike traces (which capture individual requests), metrics are cheap at scale because they're pre-aggregated.

Logs provide the richest contextual detail — exception stack traces, business-level events, debug output — but are the most expensive signal to store and query.

The three signals complement each other:

Metrics alert you that something is wrong (latency spike).
Traces show you where in the call chain the problem is.
Logs tell you why (the specific error, the slow query text, the unexpected null).

OpenTelemetry correlates all three by attaching trace and span IDs to log records and metric exemplars, so you can jump from a metric anomaly to the specific traces that caused it.

Instrumentation: Automatic vs. Manual

Automatic instrumentation (also called library instrumentation) is provided by community packages that hook into common libraries. For .NET, this includes:

OpenTelemetry.Instrumentation.AspNetCore — traces for incoming HTTP requests
OpenTelemetry.Instrumentation.HttpClient — traces for outgoing HTTP calls
OpenTelemetry.Instrumentation.SqlClient — traces for SQL queries
OpenTelemetry.Instrumentation.EntityFrameworkCore — traces for EF Core operations
OpenTelemetry.Instrumentation.GrpcNetClient — traces for gRPC calls

These packages generate spans automatically with no code changes beyond registration.

Manual instrumentation is what you add for your own business logic — things automatic instrumentation can't know about. Processing a payment? Validating inventory? Calling a third-party API with retry logic? You create custom spans and record custom metrics for these.

Exporters and Backends

An exporter is the component that sends telemetry data to a backend. Common exporters in .NET:

OTLP (OpenTelemetry Protocol): The standard wire format. Works with the OpenTelemetry Collector, Grafana Tempo, Jaeger, and most modern backends.
Azure Monitor: Sends data to Application Insights / Azure Monitor via the Azure.Monitor.OpenTelemetry.Exporter package.
Console: Writes telemetry to stdout — useful for local development and debugging.
Zipkin / Jaeger (direct): Native exporters for these backends, though OTLP is generally preferred now.

Hands-On: Setting Up OpenTelemetry in ASP.NET Core

Step 1: Install the Packages

dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.HttpClient
dotnet add package OpenTelemetry.Instrumentation.SqlClient
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
dotnet add package OpenTelemetry.Exporter.Console
dotnet add package Azure.Monitor.OpenTelemetry.Exporter

Step 2: Configure the SDK in `Program.cs`

using OpenTelemetry;
using OpenTelemetry.Metrics;
using OpenTelemetry.Trace;
using OpenTelemetry.Logs;
using OpenTelemetry.Resources;

var builder = WebApplication.CreateBuilder(args);

// Define a shared resource describing this service
var serviceName = "OrderService";
var serviceVersion = "1.0.0";

var resourceBuilder = ResourceBuilder.CreateDefault()
    .AddService(serviceName: serviceName, serviceVersion: serviceVersion)
    .AddAttributes(new Dictionary<string, object>
    {
        ["deployment.environment"] = builder.Environment.EnvironmentName,
        ["host.name"] = Environment.MachineName
    });

// Configure Tracing
builder.Services.AddOpenTelemetry()
    .ConfigureResource(r => r.AddService(serviceName, serviceVersion: serviceVersion))
    .WithTracing(tracing =>
    {
        tracing
            .SetResourceBuilder(resourceBuilder)
            .AddAspNetCoreInstrumentation(opts =>
            {
                // Filter out health check noise
                opts.Filter = ctx => !ctx.Request.Path.StartsWithSegments("/health");
            })
            .AddHttpClientInstrumentation()
            .AddSqlClientInstrumentation(opts =>
            {
                opts.SetDbStatementForText = true; // Capture SQL text (careful in prod)
                opts.RecordException = true;
            })
            .AddSource("OrderService.Activities") // Custom ActivitySource
            .AddOtlpExporter(opts =>
            {
                opts.Endpoint = new Uri("http://otel-collector:4317");
            });

        // Add Azure Monitor in deployed environments
        if (!builder.Environment.IsDevelopment())
        {
            tracing.AddAzureMonitorTraceExporter(opts =>
            {
                opts.ConnectionString = builder.Configuration
                    .GetValue<string>("AzureMonitor:ConnectionString");
            });
        }
        else
        {
            tracing.AddConsoleExporter();
        }
    })
    .WithMetrics(metrics =>
    {
        metrics
            .SetResourceBuilder(resourceBuilder)
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            .AddMeter("OrderService.Metrics") // Custom Meter
            .AddOtlpExporter()
            .AddPrometheusExporter(); // /metrics endpoint for scraping
    });

// Configure Logging
builder.Logging.AddOpenTelemetry(logging =>
{
    logging
        .SetResourceBuilder(resourceBuilder)
        .AddOtlpExporter()
        .IncludeFormattedMessage = true;
});

var app = builder.Build();
app.MapPrometheusScrapingEndpoint(); // Expose /metrics
// ... rest of pipeline
app.Run();

Step 3: Add Custom Instrumentation

using System.Diagnostics;
using System.Diagnostics.Metrics;

public class OrderProcessingService
{
    // ActivitySource is the .NET equivalent of a Tracer
    private static readonly ActivitySource ActivitySource =
        new("OrderService.Activities", "1.0.0");

    // Custom metrics
    private static readonly Meter Meter = new("OrderService.Metrics", "1.0.0");
    private static readonly Counter<long> OrdersProcessed =
        Meter.CreateCounter<long>("orders.processed", "orders", "Total orders processed");
    private static readonly Histogram<double> OrderProcessingDuration =
        Meter.CreateHistogram<double>("orders.processing_duration", "ms",
            "Time to process an order");

    private readonly ILogger<OrderProcessingService> _logger;
    private readonly PaymentClient _paymentClient;
    private readonly InventoryClient _inventoryClient;

    public OrderProcessingService(
        ILogger<OrderProcessingService> logger,
        PaymentClient paymentClient,
        InventoryClient inventoryClient)
    {
        _logger = logger;
        _paymentClient = paymentClient;
        _inventoryClient = inventoryClient;
    }

    public async Task<OrderResult> ProcessOrderAsync(Order order)
    {
        // Start a custom span
        using var activity = ActivitySource.StartActivity(
            "ProcessOrder",
            ActivityKind.Internal);

        // Add structured attributes to the span
        activity?.SetTag("order.id", order.Id);
        activity?.SetTag("order.item_count", order.Items.Count);
        activity?.SetTag("order.customer_id", order.CustomerId);

        var stopwatch = Stopwatch.StartNew();

        try
        {
            // Validate inventory — this creates a child span automatically
            // because HttpClient instrumentation is active
            var inventoryResult = await _inventoryClient
                .CheckAvailabilityAsync(order.Items);

            if (!inventoryResult.AllAvailable)
            {
                activity?.SetStatus(ActivityStatusCode.Error, "Insufficient inventory");
                activity?.AddEvent(new ActivityEvent("inventory.insufficient",
                    tags: new ActivityTagsCollection
                    {
                        { "unavailable_skus", string.Join(",",
                            inventoryResult.UnavailableSkus) }
                    }));

                return OrderResult.InsufficientInventory(inventoryResult.UnavailableSkus);
            }

            // Process payment
            var paymentResult = await _paymentClient
                .ChargeAsync(order.CustomerId, order.Total);

            activity?.SetTag("payment.transaction_id", paymentResult.TransactionId);

            _logger.LogInformation(
                "Order {OrderId} processed successfully. Transaction: {TransactionId}",
                order.Id, paymentResult.TransactionId);

            activity?.SetStatus(ActivityStatusCode.Ok);
            OrdersProcessed.Add(1,
                new KeyValuePair<string, object?>("order.status", "completed"));

            return OrderResult.Success(paymentResult.TransactionId);
        }
        catch (Exception ex)
        {
            activity?.SetStatus(ActivityStatusCode.Error, ex.Message);
            activity?.RecordException(ex);
            OrdersProcessed.Add(1,
                new KeyValuePair<string, object?>("order.status", "failed"));

            _logger.LogError(ex, "Failed to process order {OrderId}", order.Id);
            throw;
        }
        finally
        {
            stopwatch.Stop();
            OrderProcessingDuration.Record(stopwatch.Elapsed.TotalMilliseconds,
                new KeyValuePair<string, object?>("order.status",
                    activity?.Status == ActivityStatusCode.Ok ? "completed" : "failed"));
        }
    }
}

Step 4: Correlating Requests Across Services

For HTTP-based communication, correlation is automatic. The HttpClient instrumentation injects traceparent headers, and the receiving service's AspNetCore instrumentation picks them up.

For message-based communication (e.g., Azure Service Bus, RabbitMQ), you need to propagate context manually:

// Producer: inject trace context into message headers
public async Task PublishOrderEventAsync(OrderEvent orderEvent)
{
    using var activity = ActivitySource.StartActivity(
        "PublishOrderEvent", ActivityKind.Producer);

    activity?.SetTag("messaging.system", "servicebus");
    activity?.SetTag("messaging.destination", "order-events");

    var message = new ServiceBusMessage(JsonSerializer.Serialize(orderEvent));

    // Inject current context into message properties
    if (activity?.Context is ActivityContext context)
    {
        message.ApplicationProperties["traceparent"] =
            $"00-{context.TraceId}-{context.SpanId}-{(context.TraceFlags.HasFlag(ActivityTraceFlags.Recorded) ? "01" : "00")}";
    }

    await _sender.SendMessageAsync(message);
}

// Consumer: extract trace context from message headers
public async Task HandleMessageAsync(ServiceBusReceivedMessage message)
{
    ActivityContext parentContext = default;

    if (message.ApplicationProperties.TryGetValue("traceparent", out var traceparent)
        && traceparent is string tp)
    {
        // Parse W3C traceparent header
        var parts = tp.Split('-');
        if (parts.Length == 4)
        {
            var traceId = ActivityTraceId.CreateFromString(parts[1]);
            var spanId = ActivitySpanId.CreateFromString(parts[2]);
            var flags = parts[3] == "01"
                ? ActivityTraceFlags.Recorded : ActivityTraceFlags.None;
            parentContext = new ActivityContext(traceId, spanId, flags, isRemote: true);
        }
    }

    using var activity = ActivitySource.StartActivity(
        "ProcessOrderEvent",
        ActivityKind.Consumer,
        parentContext);

    activity?.SetTag("messaging.system", "servicebus");
    activity?.SetTag("messaging.operation", "process");

    // Process the message...
}

Real-World Scenarios

Scenario 1: Debugging Latency Across Services

A customer complains that placing an order takes too long. With OpenTelemetry configured, you query your tracing backend for traces where order.processing_duration > 5000ms.

You pull up a trace and see:

[OrderController.Post]          0ms ─────────────────────── 6200ms
  [ProcessOrder]                5ms ──────────────────────  6195ms
    [HTTP GET /inventory]       10ms ─── 120ms
    [HTTP POST /payments]       125ms ────────────────────  6100ms
      [SQL: INSERT payment]     130ms ── 180ms
      [HTTP POST /fraud-check]  200ms ────────────────────  6050ms  ← BOTTLENECK

The fraud-check service is taking nearly 6 seconds. Without distributed tracing, you'd be staring at order service logs wondering why ProcessOrderAsync is slow, with no visibility into what the payment service is doing downstream.

Scenario 2: Tracking End-to-End Request Flow

A webhook delivery to a partner is failing intermittently. You search for traces by webhook.partner_id = "acme" and find that successful deliveries take 200ms, but failures show a pattern: the notification service's outbound HTTP call to the partner endpoint returns 503 after exactly 30 seconds — a timeout.

The trace reveals the full chain: API → Order → Notification → Partner webhook. The partner's endpoint is intermittently overloaded. You add a retry policy with exponential backoff and add a span event to track each retry attempt.

Scenario 3: Identifying Production Bottlenecks

Your orders.processing_duration histogram shows P95 degradation over two weeks. You correlate with exemplars — specific trace IDs attached to the slow metric data points — and find they all share a common pattern: the SQL span for inventory reservation is growing linearly.

Digging into the span attributes, you see db.statement contains a query that's missing an index on the sku column. As the product catalog grew, that query went from 5ms to 500ms. The trace data told you exactly which query, in which service, against which table.

Best Practices

Naming Conventions

Use semantic conventions from the OpenTelemetry specification. This ensures consistency across services and teams:

Span names should be low-cardinality: ProcessOrder, HTTP GET /api/orders/{id} (not /api/orders/12345). Include the verb and resource, not dynamic IDs.
Attribute keys follow dot.notation: order.id, payment.method, customer.tier. Avoid inventing your own when a semantic convention exists (http.method, db.system, rpc.service).
Meter and instrument names use reverse-domain style: orderservice.orders.processed, orderservice.processing.duration.

Reducing Telemetry Noise and Cost

Not every span is worth sending to your backend:

Filter health checks and readiness probes — as shown in the setup code, exclude /health and /ready endpoints from trace instrumentation.
Use the Filter callback on instrumentation options to drop low-value spans programmatically.
Limit db.statement capture in production. Full SQL text is invaluable for debugging but expensive to store at scale. Consider enabling it only in non-production environments or for sampled traces.
Control attribute cardinality. Adding user.email to every span creates millions of unique attribute values and explodes storage costs. Prefer IDs over free-text values.

Sampling Strategies

Sampling controls which traces get recorded and exported:

AlwaysOn: Record every trace. Fine for low-traffic services or development.
AlwaysOff: Record nothing. Useful for suppressing telemetry in batch jobs that would generate excessive noise.
TraceIdRatioBased: Record a fixed percentage of traces (e.g., 10%). Simple and predictable, but you might miss the rare-but-critical slow request.
ParentBased: Respect the sampling decision of the upstream service. If the parent was sampled, sample the child too. This is critical for keeping traces complete.

tracing.SetSampler(new ParentBasedSampler(
    new TraceIdRatioBasedSampler(0.1))); // 10% base rate, but always honor parent

In practice, tail-based sampling (deciding after the trace is complete, based on whether it was slow or errored) is far more useful than head-based sampling. This requires the OpenTelemetry Collector with a tail-sampling processor — the SDK doesn't support it natively.

Securing Telemetry Data

Telemetry data can contain sensitive information: query parameters, user IDs, database statements, error messages with PII.

Scrub sensitive attributes before export. Use the SDK's Enrich and Filter callbacks to redact or remove PII.
Use TLS for OTLP exports. The OTLP exporter supports gRPC and HTTP/protobuf — ensure endpoints use https:// or encrypted gRPC channels.
Restrict access to your telemetry backend. Traces contain internal architecture details. Treat your observability platform with the same access control rigor as production databases.
Never log secrets. This sounds obvious, but automatic instrumentation on HttpClient can capture request headers. Explicitly exclude Authorization and other sensitive headers.

.AddHttpClientInstrumentation(opts =>
{
    opts.FilterHttpRequestMessage = req =>
    {
        // Don't trace calls to the token endpoint
        return !req.RequestUri!.AbsolutePath.Contains("/oauth/token");
    };
    opts.EnrichWithHttpRequestMessage = (activity, request) =>
    {
        // Remove sensitive query parameters
        activity?.SetTag("http.url",
            request.RequestUri?.GetLeftPart(UriPartial.Path));
    };
})

Deploying and Monitoring in Azure

Integration with Application Insights / Azure Monitor

The Azure.Monitor.OpenTelemetry.Exporter package sends traces, metrics, and logs directly to Application Insights without requiring the OpenTelemetry Collector as an intermediary. For many teams on Azure, this is the fastest path to production observability.

Alternatively, the newer Azure Monitor OpenTelemetry Distro (Azure.Monitor.OpenTelemetry.AspNetCore) bundles common instrumentation libraries with the exporter in a single package:

builder.Services.AddOpenTelemetry()
    .UseAzureMonitor(opts =>
    {
        opts.ConnectionString = builder.Configuration["AzureMonitor:ConnectionString"];
    });

This auto-configures ASP.NET Core, HttpClient, and SQL Client instrumentation with the Azure Monitor exporter. It's opinionated but gets you to 80% coverage with minimal code.

In Application Insights, you get:

Transaction search: Find traces by operation ID, custom attributes, or time range.
Application Map: Visualize service dependencies and identify failure hotspots.
Performance blade: Drill from P95 latency to individual slow traces.
Failures blade: Aggregate exceptions by type and trace them back to the originating request.

Observability in Containerized Environments (Kubernetes)

When running .NET microservices on AKS or any Kubernetes cluster, the recommended architecture is:

Deploy the OpenTelemetry Collector as a DaemonSet or sidecar. Services export telemetry via OTLP to the Collector on localhost or the node's IP — fast, low-latency, no cross-network traffic.
The Collector handles routing, batching, and export. It can fan out to multiple backends (Azure Monitor + Grafana Tempo), apply tail-based sampling, enrich spans with Kubernetes metadata (k8s.pod.name, k8s.namespace.name), and buffer during backend outages.
Use the Kubernetes Attributes Processor in the Collector to automatically attach pod, node, and deployment metadata to every span — invaluable for correlating application behavior with infrastructure events.

A minimal Collector config for Azure:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s
    send_batch_size: 1024
  k8sattributes:
    extract:
      metadata:
        - k8s.pod.name
        - k8s.namespace.name
        - k8s.deployment.name
  tail_sampling:
    decision_wait: 10s
    policies:
      - name: errors
        type: status_code
        status_code: { status_codes: [ERROR] }
      - name: slow-traces
        type: latency
        latency: { threshold_ms: 2000 }
      - name: probabilistic
        type: probabilistic
        probabilistic: { sampling_percentage: 10 }

exporters:
  azuremonitor:
    connection_string: ${APPLICATIONINSIGHTS_CONNECTION_STRING}
  otlp/tempo:
    endpoint: tempo.monitoring.svc.cluster.local:4317
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [k8sattributes, tail_sampling, batch]
      exporters: [azuremonitor, otlp/tempo]
    metrics:
      receivers: [otlp]
      processors: [k8sattributes, batch]
      exporters: [azuremonitor]
    logs:
      receivers: [otlp]
      processors: [k8sattributes, batch]
      exporters: [azuremonitor]

This gives you tail-based sampling (always capture errors and slow requests, sample 10% of everything else), Kubernetes metadata enrichment, and dual export to both Azure Monitor and Grafana Tempo.

Key Takeaways

OpenTelemetry is the industry standard for vendor-neutral observability. In .NET, it builds on native APIs (Activity, Meter, ILogger), making adoption low-friction.
The three signals — traces, metrics, and logs — are complementary. Metrics detect anomalies, traces locate the problem, logs explain the cause. Correlation via trace IDs ties them together.
Automatic instrumentation covers the infrastructure layer (HTTP, SQL, gRPC). Manual instrumentation is where you capture business-domain semantics (order processing, payment flow, inventory checks).
Sampling and filtering are essential at scale. Tail-based sampling in the OTel Collector is the most effective strategy for capturing interesting traces without drowning in data.
The OTel Collector is your telemetry control plane. Use it to decouple services from backends, apply processing rules, and route data to multiple destinations.

Implementation Checklist

[ ] Install OTel SDK and instrumentation packages for ASP.NET Core, HttpClient, and your data access layer.
[ ] Configure ResourceBuilder with service name, version, and environment attributes.
[ ] Register automatic instrumentation for all infrastructure libraries in use.
[ ] Create an ActivitySource and Meter for each service's custom business telemetry.
[ ] Add meaningful span attributes and events to manual instrumentation — follow semantic conventions.
[ ] Implement context propagation for asynchronous messaging (Service Bus, RabbitMQ, Kafka).
[ ] Configure exporters: OTLP to a Collector for production, Console for local development.
[ ] Set up sampling: ParentBasedSampler with TraceIdRatioBasedSampler as a starting point; evaluate tail-based sampling in the Collector.
[ ] Filter out noise: health checks, readiness probes, internal polling.
[ ] Scrub PII from span attributes and suppress sensitive headers.
[ ] Deploy the OpenTelemetry Collector in your Kubernetes cluster with batching, K8s metadata enrichment, and your chosen exporters.
[ ] Validate end-to-end trace correlation across at least two services before going to production.

Next Steps

Once you have baseline observability in place, consider exploring:

Distributed context baggage for passing business-context values (tenant ID, feature flags) across service boundaries without adding them to every call explicitly.
Exemplars — linking specific metric data points to the traces that produced them — for bridging from metric alerts to root-cause traces.
Custom span processors for advanced enrichment: attaching git commit SHAs, feature flag states, or canary deployment markers to every span.
SLO-driven alerting built on OTel metrics: define error budgets and burn rates using the histogram data you're already collecting.
OpenTelemetry instrumentation for Azure SDK calls (Azure.Core tracing) to get visibility into blob storage, Key Vault, and other Azure service interactions.
Performance tuning the telemetry pipeline itself: batch sizes, export intervals, memory limits on the Collector — telemetry infrastructure at scale needs its own observability.

Observability isn't a feature you ship once. It's a practice you refine as your system grows. OpenTelemetry gives you the foundation — a single, portable instrumentation layer that evolves with your architecture. Start with automatic instrumentation, add manual spans for the operations that matter most to your business, and iterate from there.

Azure Service Bus for Event-Driven Systems: A Practical Deep Dive

Adrián López — Mon, 20 Apr 2026 09:30:53 +0000

Introduction: Why Event-Driven Architecture Matters Now More Than Ever

If you've been building distributed systems on Azure for any meaningful amount of time, you've hit the wall. The wall where synchronous HTTP calls between services start cascading failures. Where tight coupling between your ordering service and your inventory service means a deployment to one brings down the other. Where your system can't absorb a spike in traffic without everything grinding to a halt.

Event-driven architecture (EDA) isn't a silver bullet, but it solves a category of problems that request-response patterns fundamentally cannot. By decoupling producers from consumers, introducing temporal buffers, and enabling reactive processing pipelines, EDA gives distributed systems the elasticity and fault tolerance they need to operate at scale.

At the heart of Azure's messaging ecosystem sits Azure Service Bus — a fully managed enterprise message broker that handles the heavy lifting of reliable, ordered, transactional message delivery. This post is a practitioner's guide: we'll go deep on the concepts that matter, look at real production scenarios, write actual code, and cover the operational concerns that separate a working system from a production-grade one.

What Is Azure Service Bus, and When Should You Reach for It?

Azure Service Bus is a cloud-native message broker supporting both message queuing and publish-subscribe patterns. It operates at the PaaS level — you don't manage infrastructure, brokers, or clusters. It provides:

Guaranteed message delivery with at-least-once semantics
FIFO ordering via sessions
Transactions across multiple operations
Dead-lettering and deferred message handling
Built-in duplicate detection
Message scheduling and delayed delivery

Service Bus vs. Event Grid vs. Event Hubs: Choosing the Right Tool

This is the question that comes up in every architecture review, so let's settle it with a decision framework.

Azure Service Bus is your choice when you need reliable command/message delivery between services. Think: "process this order," "send this notification," "update this record." It excels at transactional workloads where every message matters and must be processed exactly as intended.

Azure Event Grid is built for reactive event routing. It's ideal for lightweight, high-fanout notifications — "a blob was uploaded," "a resource was created." It's push-based, operates on a per-event pricing model, and is optimized for low-latency event distribution rather than queuing.

Azure Event Hubs is a high-throughput event streaming platform. If you're ingesting telemetry, logs, or clickstream data at millions of events per second and need to replay or process streams in order, Event Hubs (or its Kafka-compatible interface) is the right fit.

The decision heuristic: if losing a message is unacceptable and consumers need guaranteed processing → Service Bus. If you're distributing notifications reactively → Event Grid. If you're streaming high-volume data for analytics → Event Hubs.

In practice, production systems often combine all three. An order placed in Service Bus might trigger an Event Grid notification to update a dashboard, while telemetry from the process flows into Event Hubs for analytics.

Core Concepts in Depth

Queues vs. Topics vs. Subscriptions

Queues implement a point-to-point messaging pattern. A message sent to a queue is received by exactly one consumer. If multiple consumers are listening, they compete for messages — this is the competing consumers pattern, and it's how you scale processing horizontally.

Producer → [Queue] → Consumer A
                   → Consumer B  (competing; each message goes to one)

Topics and Subscriptions implement publish-subscribe. A message published to a topic is delivered to every subscription on that topic. Each subscription acts like a virtual queue with its own independent cursor. Subscriptions can have filters (SQL-like expressions or correlation filters) that determine which messages they receive.

Producer → [Topic] → Subscription A (filter: OrderType = 'Premium') → Consumer A
                   → Subscription B (filter: Region = 'EU')         → Consumer B
                   → Subscription C (no filter — gets everything)   → Consumer C

This distinction matters for your architecture: queues for work distribution, topics for event broadcasting with selective consumption.

Messages, Sessions, and Ordering

A Service Bus message consists of a binary body (up to 256 KB on Standard, 100 MB on Premium) and a set of broker-managed and user-defined properties. Properties are key-value pairs that ride alongside the payload without requiring deserialization — this is what makes subscription filters possible.

Sessions solve the ordering problem. Standard queues and subscriptions offer best-effort FIFO within a single partition, but no strict guarantees. When you need guaranteed ordering for a group of related messages, you assign them a common SessionId. All messages with the same session ID are delivered in order to a single consumer that holds an exclusive lock on that session.

A practical example: if you're processing events for a specific customer — account created, address updated, order placed — you set SessionId = customerId. This ensures those events are processed sequentially, even with multiple competing consumers handling different customers in parallel.

Dead-Letter Queues

Every queue and subscription has a companion dead-letter queue (DLQ) — a sidecar that captures messages that cannot be processed. Messages land in the DLQ when:

They exceed the maximum delivery count (too many processing failures)
Their TTL expires before being consumed
A subscription filter evaluation fails
The receiver explicitly dead-letters them (e.g., a poison message that fails validation)

The DLQ is not a trash can — it's an operations signal. Production systems need monitoring on DLQ depth and automated or semi-automated processes to inspect, remediate, and resubmit dead-lettered messages. Ignoring the DLQ is one of the most common operational mistakes in Service Bus deployments.

Message Delivery Guarantees

Service Bus provides at-least-once delivery by default. When a consumer receives a message in PeekLock mode, the message becomes invisible to other consumers but isn't removed from the queue. The consumer must explicitly complete the message after successful processing. If the lock expires or the consumer crashes, the message becomes visible again and is redelivered.

The alternative is ReceiveAndDelete mode — the message is removed from the queue immediately upon delivery. This gives you at-most-once semantics with lower latency, but no safety net. Use it only when losing occasional messages is acceptable (e.g., non-critical telemetry).

Duplicate detection is a broker-side feature that prevents the same message from being enqueued twice within a configurable time window. It works by tracking the MessageId property. This is invaluable when producers might retry sends after ambiguous failures (network timeouts, for instance), but it only deduplicates at the ingestion side — it doesn't prevent a consumer from processing the same message twice after redelivery.

Scheduling and Delayed Delivery

Service Bus supports scheduled enqueue time — you can send a message now but have it become visible to consumers at a future point in time. This is implemented broker-side, which means your producer doesn't need to maintain timers or polling loops.

Use cases include: delaying a retry after a transient failure, scheduling a reminder notification, implementing a timeout pattern ("if the order isn't confirmed within 30 minutes, cancel it"), or staging messages for batch processing at a specific time window.

// Schedule a message for 30 minutes from now
var sequenceNumber = await sender.ScheduleMessageAsync(
    message,
    DateTimeOffset.UtcNow.AddMinutes(30));

// Cancel it if needed before it fires
await sender.CancelScheduledMessageAsync(sequenceNumber);

Decoupling and Scalability in Microservices

The real value of Service Bus in a microservices architecture goes beyond "services don't call each other directly." Here's what decoupling actually gives you in practice:

Temporal decoupling: the producer and consumer don't need to be running at the same time. Your API can accept and enqueue an order even if the fulfillment service is down for deployment. The queue absorbs the gap.

Load leveling: during a flash sale, your web tier might enqueue thousands of orders per second. Your processing tier can consume them at a sustainable rate without being overwhelmed. The queue acts as a shock absorber.

Independent scaling: queue consumers can be scaled out horizontally. With competing consumers, you simply add more instances. Each instance pulls messages independently. Azure Container Apps, Azure Functions, or KEDA-scaled Kubernetes pods can auto-scale consumer count based on queue depth.

Independent deployment: because services communicate through messages (contracts) rather than direct API calls, you can deploy, version, and scale them independently. A schema change on the producer side doesn't require a synchronized deployment on the consumer side — as long as the message contract is honored.

Real-World Scenarios

Scenario 1: Order Processing Pipeline

An e-commerce platform decomposes order processing into discrete stages: validation, payment, inventory reservation, and fulfillment. Each stage is a separate service. The order flows through a series of queues:

API Gateway → [orders-validation] → Validation Service
                                         ↓
                              [orders-payment] → Payment Service
                                                      ↓
                                           [orders-fulfillment] → Fulfillment Service

Each service reads from its input queue, performs its work, and publishes to the next queue (or to a topic if multiple downstream services need to react). Failures at any stage result in retries via the lock mechanism or dead-lettering for manual review. The entire pipeline is resilient to individual service outages.

Scenario 2: Cross-Service Integration Events

A SaaS platform publishes domain events (e.g., UserRegistered, SubscriptionUpgraded) to a Service Bus topic. Multiple downstream services subscribe selectively:

The email service subscribes to UserRegistered to send welcome emails
The billing service subscribes to SubscriptionUpgraded to adjust invoicing
The analytics service subscribes to all events for audit logging

Each subscription has its own filter and processes at its own pace. Adding a new consumer means adding a new subscription — no changes to the producer.

Scenario 3: Background Job Offloading

A web API needs to generate PDF reports, a CPU-intensive operation. Instead of blocking the HTTP request, it enqueues a GenerateReport message and returns 202 Accepted with a job ID. A background worker pool processes the queue, generates the PDF, uploads it to blob storage, and publishes a completion event. The client polls or subscribes for the result.

C# Examples with Azure.Messaging.ServiceBus SDK

All examples use the Azure.Messaging.ServiceBus NuGet package (current stable: 7.x). The ServiceBusClient is designed to be a singleton — create one instance and reuse it across your application lifetime.

Setting Up the Client

using Azure.Messaging.ServiceBus;
using Azure.Identity;

// Preferred: Managed Identity (no secrets in config)
var client = new ServiceBusClient(
    "your-namespace.servicebus.windows.net",
    new DefaultAzureCredential());

// Alternative: connection string (dev/test only)
// var client = new ServiceBusClient(connectionString);

Sending Messages

public class OrderPublisher : IAsyncDisposable
{
    private readonly ServiceBusSender _sender;

    public OrderPublisher(ServiceBusClient client)
    {
        _sender = client.CreateSender("orders");
    }

    public async Task PublishOrderAsync(Order order, CancellationToken ct)
    {
        var message = new ServiceBusMessage(
            BinaryData.FromObjectAsJson(order))
        {
            // MessageId enables duplicate detection at the broker
            MessageId = order.OrderId.ToString(),
            // SessionId guarantees ordering per customer
            SessionId = order.CustomerId.ToString(),
            // Correlation for end-to-end tracing
            CorrelationId = Activity.Current?.Id,
            ContentType = "application/json",
            Subject = "OrderPlaced",
            // Custom properties for filtering
            ApplicationProperties =
            {
                ["OrderType"] = order.Type.ToString(),
                ["Region"] = order.Region
            }
        };

        await _sender.SendMessageAsync(message, ct);
    }

    // Batch sending for throughput
    public async Task PublishOrderBatchAsync(
        IEnumerable<Order> orders, CancellationToken ct)
    {
        using ServiceBusMessageBatch batch =
            await _sender.CreateMessageBatchAsync(ct);

        foreach (var order in orders)
        {
            var message = new ServiceBusMessage(
                BinaryData.FromObjectAsJson(order))
            {
                MessageId = order.OrderId.ToString(),
                SessionId = order.CustomerId.ToString()
            };

            if (!batch.TryAddMessage(message))
            {
                // Batch is full — send what we have, start a new one
                await _sender.SendMessagesAsync(batch, ct);
                batch.Dispose();

                using var newBatch =
                    await _sender.CreateMessageBatchAsync(ct);
                if (!newBatch.TryAddMessage(message))
                    throw new InvalidOperationException(
                        "Message too large for an empty batch.");

                // Continue filling newBatch...
            }
        }

        if (batch.Count > 0)
            await _sender.SendMessagesAsync(batch, ct);
    }

    public async ValueTask DisposeAsync()
    {
        await _sender.DisposeAsync();
    }
}

Receiving and Processing Messages

public class OrderProcessor : BackgroundService
{
    private readonly ServiceBusClient _client;
    private readonly IOrderService _orderService;
    private readonly ILogger<OrderProcessor> _logger;

    public OrderProcessor(
        ServiceBusClient client,
        IOrderService orderService,
        ILogger<OrderProcessor> logger)
    {
        _client = client;
        _orderService = orderService;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken ct)
    {
        var processor = _client.CreateProcessor("orders",
            new ServiceBusProcessorOptions
            {
                // Number of concurrent message handlers
                MaxConcurrentCalls = 10,
                // PeekLock is the default and recommended mode
                ReceiveMode = ServiceBusReceiveMode.PeekLock,
                // Auto-complete is off — we complete manually
                // after successful processing
                AutoCompleteMessages = false,
                // How long before the lock expires
                MaxAutoLockRenewalDuration = TimeSpan.FromMinutes(10),
                // Prefetch for throughput (see best practices)
                PrefetchCount = 20
            });

        processor.ProcessMessageAsync += HandleMessageAsync;
        processor.ProcessErrorAsync += HandleErrorAsync;

        await processor.StartProcessingAsync(ct);

        // Keep running until cancellation
        await Task.Delay(Timeout.Infinite, ct);

        await processor.StopProcessingAsync();
        await processor.DisposeAsync();
    }

    private async Task HandleMessageAsync(
        ProcessMessageEventArgs args)
    {
        var order = args.Message.Body
            .ToObjectFromJson<Order>();

        _logger.LogInformation(
            "Processing order {OrderId} for customer {CustomerId}",
            order.OrderId, order.CustomerId);

        try
        {
            await _orderService.ProcessAsync(order, args.CancellationToken);

            // Explicitly complete — removes message from queue
            await args.CompleteMessageAsync(args.Message);
        }
        catch (InvalidOrderException ex)
        {
            // Poison message — dead-letter it with a reason
            _logger.LogWarning(ex,
                "Order {OrderId} is invalid, dead-lettering", order.OrderId);

            await args.DeadLetterMessageAsync(args.Message,
                deadLetterReason: "InvalidOrder",
                deadLetterErrorDescription: ex.Message);
        }
        catch (TransientException ex)
        {
            // Transient failure — abandon so it's retried
            _logger.LogWarning(ex,
                "Transient failure for order {OrderId}, abandoning",
                order.OrderId);

            await args.AbandonMessageAsync(args.Message);
        }
    }

    private Task HandleErrorAsync(ProcessErrorEventArgs args)
    {
        _logger.LogError(args.Exception,
            "Service Bus error. Source: {Source}, Entity: {Entity}",
            args.ErrorSource, args.EntityPath);

        return Task.CompletedTask;
    }
}

Handling Failures and Retries

The SDK handles transient Service Bus errors (throttling, connectivity) internally with built-in retry policies. You can configure them:

var client = new ServiceBusClient(
    "your-namespace.servicebus.windows.net",
    new DefaultAzureCredential(),
    new ServiceBusClientOptions
    {
        RetryOptions = new ServiceBusRetryOptions
        {
            Mode = ServiceBusRetryMode.Exponential,
            MaxRetries = 5,
            Delay = TimeSpan.FromSeconds(1),
            MaxDelay = TimeSpan.FromSeconds(30),
            TryTimeout = TimeSpan.FromSeconds(60)
        }
    });

For application-level retries (your processing logic fails), the pattern is:

On transient failure: call AbandonMessageAsync(). The message becomes visible again after the lock expires. The broker tracks the delivery count.
Once DeliveryCount exceeds MaxDeliveryCount (configured on the queue, default 10), the broker automatically dead-letters the message.
On permanent/poison failures: call DeadLetterMessageAsync() immediately to skip retries.

This gives you a natural retry loop without any custom retry framework — the broker manages it.

Best Practices

Idempotency and Message Handling

At-least-once delivery means your handlers will receive duplicates — after crashes, lock expirations, or network hiccups. Your processing logic must be idempotent.

Strategies for achieving idempotency:

Natural idempotency: some operations are inherently idempotent. Setting a value (e.g., status = 'shipped') is safe to repeat. Incrementing a counter is not.
Idempotency keys: store the MessageId or a business-level idempotency key in your database within the same transaction as your state change. Before processing, check if the key exists. This is the most reliable approach.
Conditional writes: use optimistic concurrency (ETags, row versions) so that duplicate processing attempts fail gracefully on the second write.

// Idempotency via deduplication table
public async Task ProcessAsync(Order order, CancellationToken ct)
{
    await using var transaction = await _db.Database
        .BeginTransactionAsync(ct);

    // Check if already processed
    var exists = await _db.ProcessedMessages
        .AnyAsync(m => m.MessageId == order.OrderId.ToString(), ct);

    if (exists)
    {
        _logger.LogInformation(
            "Order {OrderId} already processed, skipping", order.OrderId);
        return;
    }

    // Process the order
    await _db.Orders.AddAsync(MapToEntity(order), ct);

    // Record the message ID
    await _db.ProcessedMessages.AddAsync(
        new ProcessedMessage { MessageId = order.OrderId.ToString() }, ct);

    await _db.SaveChangesAsync(ct);
    await transaction.CommitAsync(ct);
}

Error Handling Strategies

Classify errors upfront: transient (network, throttling, temporary unavailability) vs. permanent (validation failure, deserialization error, business rule violation). Transient errors get retried via abandon; permanent errors get dead-lettered immediately.
Set MaxDeliveryCount thoughtfully: too low and you dead-letter messages that would have succeeded on the next attempt. Too high and a poison message clogs your consumer with repeated failures. A value between 5 and 10 is a reasonable starting point.
Monitor dead-letter queues actively: set up Azure Monitor alerts on DLQ message count. Build tooling (or use Service Bus Explorer) to inspect, edit, and resubmit dead-lettered messages.
Structured logging with correlation: propagate CorrelationId across services so you can trace a message's journey end-to-end through Application Insights or your observability stack.

Throughput and Scaling Considerations

Use batching: SendMessagesAsync(batch) amortizes the cost of a single AMQP operation across many messages. On the consumer side, PrefetchCount pulls multiple messages in a single round trip.
Scale consumers horizontally: with competing consumers, throughput scales linearly with consumer count — up to the number of partitions (16 on Standard/Premium).
Premium tier for performance-sensitive workloads: Premium gives you dedicated resources (Messaging Units), predictable latency, and support for messages up to 100 MB. Standard tier shares resources and is subject to throttling under load.
AMQP over HTTP: the SDK uses AMQP by default. Don't switch to HTTP unless you have a specific constraint (e.g., firewall rules) — AMQP maintains persistent connections and is significantly more efficient.

Security and Authentication

Use Managed Identity in production: DefaultAzureCredential or ManagedIdentityCredential eliminates connection strings entirely. Assign the Azure Service Bus Data Sender and Azure Service Bus Data Receiver roles at the namespace or entity level.
Avoid connection strings in production: if you must use them (legacy systems), store them in Azure Key Vault with automatic rotation. Never commit them to source control.
Network isolation: Premium tier supports Private Endpoints and Virtual Network service endpoints. Combine with IP firewall rules to lock down the namespace.
Shared Access Policies: scope them to the narrowest entity (queue or topic) with the minimum required permissions (Send, Listen, or Manage).

Performance and Cost Optimization

Cost Drivers

On the Standard tier, you pay per operation (messaging operation = send, receive, or management call) plus a base hourly rate. On Premium, you pay per Messaging Unit (MU) per hour — a fixed cost model that's more predictable but higher baseline.

Key optimization levers:

Batching reduces operation count: a batch send of 100 messages counts as a single operation. This can cut costs dramatically at scale.
Prefetching reduces receive round trips: setting PrefetchCount on the processor fetches multiple messages per AMQP call.
Short-lived idle consumers are expensive: Azure Functions with Service Bus triggers spin up on-demand and scale to zero — ideal for intermittent workloads where running a dedicated consumer pool would waste Messaging Units or compute.
Right-size your Premium tier: each MU provides a defined throughput ceiling. Start with 1 MU and scale up based on actual metrics. Use auto-scale rules based on CPU and throttling metrics.
TTL and auto-delete: set reasonable DefaultMessageTimeToLive values. Configure AutoDeleteOnIdle for temporary queues/subscriptions to clean up unused entities.
Avoid unnecessary forwarding chains: each forward is an additional operation. Design your topology to minimize hops.

Performance Benchmarks to Keep in Mind

Standard tier: expect ~1,000–3,000 operations/sec depending on message size and concurrency.
Premium (1 MU): ~1,000 messages/sec for 1 KB messages, scaling linearly with additional MUs.
P99 latency on Premium: typically under 10 ms for send/receive operations in the same region.

Architecture Patterns

Publish-Subscribe with Filtered Subscriptions

OrderService → [order-events topic]
    → Subscription: "billing" (filter: Subject = 'OrderPlaced')      → BillingService
    → Subscription: "shipping" (filter: Amount > 100)                 → ShippingService  
    → Subscription: "analytics" (no filter)                           → AnalyticsService

Each downstream service gets exactly the events it cares about. Adding a new consumer is a subscription configuration change — no code changes to the publisher.

Competing Consumers for Horizontal Scaling

[orders-queue] → Consumer Instance 1  (auto-scaled by KEDA / Azure Functions)
               → Consumer Instance 2
               → Consumer Instance 3
               → ...

All instances read from the same queue. The broker ensures each message is delivered to exactly one instance. Scale the instance count based on queue depth using KEDA (Kubernetes), Azure Functions auto-scale, or Azure Container Apps scaling rules.

Saga/Choreography with Service Bus

For distributed transactions across services (e.g., order → payment → inventory), each service publishes domain events after completing its step. Compensating actions handle failures:

OrderService: publishes OrderPlaced
    → PaymentService: processes, publishes PaymentConfirmed OR PaymentFailed
        → InventoryService: reserves stock, publishes StockReserved OR StockUnavailable
            → If failure at any stage → compensating events roll back prior steps

Sessions ensure ordering per saga instance. Dead-letter queues capture stuck sagas for manual intervention.

Request-Reply Over Service Bus

When you need asynchronous request-reply (the caller expects a response, but not synchronously), use the ReplyTo and ReplyToSessionId properties:

// Sender sets up a temporary reply queue
var request = new ServiceBusMessage(payload)
{
    ReplyTo = "reply-queue",
    ReplyToSessionId = Guid.NewGuid().ToString(),
    MessageId = correlationId
};
await sender.SendMessageAsync(request);

// Receiver processes and replies
var reply = new ServiceBusMessage(responsePayload)
{
    SessionId = args.Message.ReplyToSessionId,
    CorrelationId = args.Message.MessageId
};
await replySender.SendMessageAsync(reply);

Summary

Azure Service Bus is the backbone of reliable, asynchronous communication in Azure-based distributed systems. Its strength lies in the combination of guaranteed delivery, flexible routing (queues and topics), session-based ordering, and enterprise-grade features like dead-lettering, duplicate detection, and scheduling — all without infrastructure management overhead.

The key decision points are: use queues for point-to-point work distribution, topics for event broadcasting with selective consumption, sessions when ordering matters, and Premium tier when you need predictable performance and network isolation.

Best Practices Checklist

[ ] Use Managed Identity (not connection strings) for authentication in all deployed environments
[ ] Make all message handlers idempotent — track processed message IDs
[ ] Set AutoCompleteMessages = false and complete messages explicitly after successful processing
[ ] Classify errors as transient (abandon) or permanent (dead-letter) — don't retry poison messages
[ ] Monitor dead-letter queue depth with Azure Monitor alerts
[ ] Use batching (send and receive) for throughput-sensitive workloads
[ ] Enable duplicate detection on queues/topics where producers might retry
[ ] Set SessionId on messages that require strict ordering per entity
[ ] Configure MaxDeliveryCount between 5–10 based on your failure profile
[ ] Use PrefetchCount to reduce AMQP round trips (start with 20, tune from there)
[ ] Set DefaultMessageTimeToLive to prevent unbounded message accumulation
[ ] Propagate CorrelationId for distributed tracing across services
[ ] Scope shared access policies to minimum required permissions
[ ] Right-size your tier: Standard for moderate workloads, Premium for latency-sensitive or high-throughput
[ ] Build tooling to inspect and resubmit dead-lettered messages

Further Exploration

Advanced patterns: look into the Claim Check pattern for large payloads (store in Blob Storage, send a reference via Service Bus), Priority Queues using multiple queues with weighted consumers, and Sequential Convoy using sessions for complex workflows.
Azure Functions Service Bus bindings: for serverless consumption with auto-scaling based on queue depth, Azure Functions offer the lowest-friction integration path.
Dapr and Service Bus: if you're building polyglot microservices, Dapr's pub/sub component abstracts Service Bus behind a portable API.
MassTransit / NServiceBus: these frameworks add saga support, outbox patterns, and higher-level abstractions over the raw SDK. Evaluate them for complex workflows where the raw SDK would require significant boilerplate.
Azure Service Bus emulator: for local development, the Service Bus emulator (currently in preview) provides a local instance that mimics the cloud service behavior.
Monitoring deep dive: explore Application Insights integration, custom metrics via ServiceBusProcessor events, and Azure Monitor workbooks for operational dashboards.

ASP.NET Core Performance Optimization Techniques

Adrián López — Tue, 07 Apr 2026 10:13:00 +0000

Introduction: Why Performance Is a Cloud-Native Imperative

Every millisecond matters. In a cloud-based world where you pay per compute-second, where users abandon pages after 3 seconds of loading, and where a single slow endpoint can cascade into a system-wide bottleneck, performance optimization isn't a luxury — it's a business requirement.

ASP.NET Core is already one of the fastest mainstream web frameworks available. But "fast by default" doesn't mean "fast in practice." Poorly written LINQ queries, careless allocations, misconfigured middleware, and naive caching strategies can erode that advantage quickly — and in Azure, those inefficiencies translate directly into higher bills and lower reliability.

This post is a practitioner's guide. We won't rehash what async/await means or how dependency injection works. Instead, we'll walk through concrete, battle-tested techniques to squeeze meaningful performance out of your ASP.NET Core applications — with code, tooling, and the reasoning behind each decision.

Let's get into it.

1. Middleware Optimization and Request Pipeline Tuning

The ASP.NET Core middleware pipeline processes every single HTTP request. The order, quantity, and weight of your middleware components have a direct impact on latency.

Order Matters — A Lot

Middleware executes in registration order. Place short-circuiting middleware (like health checks, static files, and CORS) as early as possible to avoid unnecessary work downstream.

var app = builder.Build();

// ✅ Short-circuit early — these requests don't need auth, MVC, etc.
app.UseHealthChecks("/health");
app.UseStaticFiles();

// CORS and auth come next
app.UseCors();
app.UseAuthentication();
app.UseAuthorization();

// MVC/endpoint routing last — heaviest processing
app.MapControllers();

Strip Out What You Don't Need

Every middleware you register adds overhead. Audit your pipeline ruthlessly. If you're building an API that never serves static files, remove UseStaticFiles(). If you don't use sessions, don't register session middleware.

Write Lean Custom Middleware

When writing custom middleware, avoid blocking calls and heavy allocations in the hot path.

// ❌ Anti-pattern: Allocating and doing I/O on every request
public class HeavyMiddleware
{
    private readonly RequestDelegate _next;

    public HeavyMiddleware(RequestDelegate next) => _next = next;

    public async Task InvokeAsync(HttpContext context)
    {
        // Allocates a new list and hits the database on EVERY request
        var bannedIps = await LoadBannedIpsFromDatabase();

        if (bannedIps.Contains(context.Connection.RemoteIpAddress?.ToString()))
        {
            context.Response.StatusCode = 403;
            return;
        }

        await _next(context);
    }
}

// ✅ Better: Cache the result, minimize per-request work
public class LeanMiddleware
{
    private readonly RequestDelegate _next;
    private readonly IMemoryCache _cache;

    public LeanMiddleware(RequestDelegate next, IMemoryCache cache)
    {
        _next = next;
        _cache = cache;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var bannedIps = await _cache.GetOrCreateAsync("banned-ips", async entry =>
        {
            entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5);
            return await LoadBannedIpsFromDatabase();
        });

        if (bannedIps!.Contains(context.Connection.RemoteIpAddress?.ToString()))
        {
            context.Response.StatusCode = 403;
            return;
        }

        await _next(context);
    }
}

2. Efficient Use of Async/Await

You already know async/await. The question is whether you're using it in ways that actually help — or in ways that add overhead for no benefit.

Avoid Async for Trivial Operations

If a method is purely CPU-bound and completes in microseconds, making it async adds state machine overhead with no benefit.

// ❌ Unnecessary async — no I/O, no real awaiting
public async Task<string> GetGreetingAsync(string name)
{
    return await Task.FromResult($"Hello, {name}");
}

// ✅ Just return synchronously
public string GetGreeting(string name)
{
    return $"Hello, {name}";
}

Use ConfigureAwait(false) in Library Code

In library or service-layer code that doesn't need the synchronization context, use ConfigureAwait(false) to avoid unnecessary context captures.

public async Task<Product?> GetProductAsync(int id)
{
    return await _dbContext.Products
        .AsNoTracking()
        .FirstOrDefaultAsync(p => p.Id == id)
        .ConfigureAwait(false);
}

Parallelize Independent I/O Calls

When you have multiple independent async operations, use Task.WhenAll instead of awaiting them sequentially.

// ❌ Sequential — total time = sum of all calls
var user = await _userService.GetUserAsync(userId);
var orders = await _orderService.GetOrdersAsync(userId);
var recommendations = await _recommendationService.GetAsync(userId);

// ✅ Parallel — total time = longest call
var userTask = _userService.GetUserAsync(userId);
var ordersTask = _orderService.GetOrdersAsync(userId);
var recommendationsTask = _recommendationService.GetAsync(userId);

await Task.WhenAll(userTask, ordersTask, recommendationsTask);

var user = userTask.Result;
var orders = ordersTask.Result;
var recommendations = recommendationsTask.Result;

Never Block on Async Code

This is well-known, but still common in practice. Never call .Result or .Wait() on a task in synchronous code — it risks deadlocks and thread pool starvation.

// ❌ Thread pool starvation risk
public IActionResult GetData()
{
    var data = _service.GetDataAsync().Result; // Blocks a thread
    return Ok(data);
}

// ✅ Stay async all the way
public async Task<IActionResult> GetData()
{
    var data = await _service.GetDataAsync();
    return Ok(data);
}

3. Caching Strategies

Caching is one of the highest-leverage optimizations available. But choosing the right layer and invalidation strategy makes the difference between a fast app and a stale one.

In-Memory Caching

Best for: single-instance deployments, reference data that changes infrequently.

builder.Services.AddMemoryCache();

// In your service:
public class ProductService
{
    private readonly IMemoryCache _cache;
    private readonly AppDbContext _db;

    public ProductService(IMemoryCache cache, AppDbContext db)
    {
        _cache = cache;
        _db = db;
    }

    public async Task<List<Category>> GetCategoriesAsync()
    {
        return (await _cache.GetOrCreateAsync("categories", async entry =>
        {
            entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(30);
            entry.SlidingExpiration = TimeSpan.FromMinutes(10);

            return await _db.Categories
                .AsNoTracking()
                .OrderBy(c => c.Name)
                .ToListAsync();
        }))!;
    }
}

Distributed Caching with Redis

Essential for: multi-instance deployments, App Service scale-out, Kubernetes pods.

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = builder.Configuration
        .GetConnectionString("Redis");
    options.InstanceName = "myapp:";
});

// Usage in a service
public class CatalogService
{
    private readonly IDistributedCache _cache;
    private readonly AppDbContext _db;

    private static readonly DistributedCacheEntryOptions CacheOptions = new()
    {
        AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(15),
        SlidingExpiration = TimeSpan.FromMinutes(5)
    };

    public async Task<ProductDetail?> GetProductAsync(int id)
    {
        var cacheKey = $"product:{id}";
        var cached = await _cache.GetStringAsync(cacheKey);

        if (cached is not null)
            return JsonSerializer.Deserialize<ProductDetail>(cached);

        var product = await _db.Products
            .AsNoTracking()
            .Where(p => p.Id == id)
            .Select(p => new ProductDetail
            {
                Id = p.Id,
                Name = p.Name,
                Price = p.Price,
                Category = p.Category.Name
            })
            .FirstOrDefaultAsync();

        if (product is not null)
        {
            await _cache.SetStringAsync(
                cacheKey,
                JsonSerializer.Serialize(product),
                CacheOptions);
        }

        return product;
    }
}

Response Caching

For GET endpoints returning data that doesn't change per-user, use response caching to skip controller execution entirely.

builder.Services.AddResponseCaching();

// In pipeline
app.UseResponseCaching();

// On your controller action
[HttpGet("categories")]
[ResponseCache(Duration = 300, VaryByHeader = "Accept-Encoding")]
public async Task<IActionResult> GetCategories()
{
    var categories = await _categoryService.GetAllAsync();
    return Ok(categories);
}

Caveat: Response caching doesn't work with authenticated endpoints or responses that include Set-Cookie. For those scenarios, use output caching (available in .NET 7+) or application-level caching.

4. Database Performance: EF Core Optimization

The database is almost always the primary bottleneck. EF Core is powerful, but it's easy to write queries that generate terrible SQL without realizing it.

Use AsNoTracking for Read-Only Queries

Change tracking is expensive. If you're just reading data for display, turn it off.

// ❌ Tracking enabled by default — unnecessary overhead
var products = await _db.Products
    .Where(p => p.IsActive)
    .ToListAsync();

// ✅ No tracking — faster, less memory
var products = await _db.Products
    .AsNoTracking()
    .Where(p => p.IsActive)
    .ToListAsync();

For read-heavy services, consider setting it globally:

builder.Services.AddDbContext<AppDbContext>(options =>
    options.UseSqlServer(connectionString)
           .UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking));

Project Only What You Need

Select only the columns you need. Returning full entity graphs when you need three fields is wasteful.

// ❌ Fetches entire entity with all columns and nav properties
var orders = await _db.Orders
    .Include(o => o.Customer)
    .Include(o => o.Items)
    .ToListAsync();

// ✅ Project to a DTO — smaller query, less data over the wire
var orders = await _db.Orders
    .AsNoTracking()
    .Select(o => new OrderSummaryDto
    {
        OrderId = o.Id,
        CustomerName = o.Customer.Name,
        TotalAmount = o.Items.Sum(i => i.Price * i.Quantity),
        ItemCount = o.Items.Count,
        OrderDate = o.OrderDate
    })
    .ToListAsync();

Avoid the N+1 Query Problem

This is the single most common EF Core performance issue in production codebases.

// ❌ N+1: one query for orders, then one query PER order for items
var orders = await _db.Orders.ToListAsync();
foreach (var order in orders)
{
    // This triggers a lazy-load query for EACH order
    var total = order.Items.Sum(i => i.Price);
}

// ✅ Eager load with Include, or better yet, project in a single query
var orderTotals = await _db.Orders
    .AsNoTracking()
    .Select(o => new
    {
        o.Id,
        Total = o.Items.Sum(i => i.Price * i.Quantity)
    })
    .ToListAsync();

Compiled Queries for Hot Paths

If a query runs thousands of times per minute, compile it to eliminate repeated expression tree translation.

public class ProductRepository
{
    // Compiled once, reused every call
    private static readonly Func<AppDbContext, int, Task<Product?>> GetByIdQuery =
        EF.CompileAsyncQuery((AppDbContext db, int id) =>
            db.Products
              .AsNoTracking()
              .FirstOrDefault(p => p.Id == id));

    private readonly AppDbContext _db;

    public ProductRepository(AppDbContext db) => _db = db;

    public Task<Product?> GetByIdAsync(int id) => GetByIdQuery(_db, id);
}

Connection Pooling

Make sure connection pooling is configured correctly. The defaults are usually fine, but under heavy load you may need tuning.

// In your connection string:
"Server=myserver;Database=mydb;Min Pool Size=5;Max Pool Size=100;
 Connection Timeout=30;Connection Lifetime=300;"

Tip: Monitor active-db-connections with dotnet-counters to detect connection leaks or pool exhaustion. If you see the count climbing without dropping, you likely have a DbContext or SqlConnection that isn't being disposed properly.

5. Minimizing Allocations and Reducing GC Pressure

The garbage collector in .NET is excellent, but it's not free. Frequent Gen 0/1/2 collections cause latency spikes. The goal is to allocate less, not to outsmart the GC.

Use Span, ReadOnlySpan, and stackalloc

For parsing and transformation operations on hot paths, Span<T> avoids heap allocations entirely.

// ❌ String allocations on every call
public string ExtractDomain(string email)
{
    var parts = email.Split('@'); // Allocates a string[]
    return parts[1];             // Allocates a new string
}

// ✅ Zero-allocation with Span
public ReadOnlySpan<char> ExtractDomain(ReadOnlySpan<char> email)
{
    int atIndex = email.IndexOf('@');
    return email[(atIndex + 1)..];  // Slice — no allocation
}

Use ArrayPool for Temporary Buffers

When you need temporary arrays, rent them instead of allocating.

// ❌ Allocates a new byte[] on every call
public byte[] ProcessData(Stream input)
{
    var buffer = new byte[8192];
    int bytesRead = input.Read(buffer, 0, buffer.Length);
    // ... process
    return buffer;
}

// ✅ Rent from the pool — returned for reuse
public void ProcessData(Stream input)
{
    var buffer = ArrayPool<byte>.Shared.Rent(8192);
    try
    {
        int bytesRead = input.Read(buffer, 0, buffer.Length);
        // ... process
    }
    finally
    {
        ArrayPool<byte>.Shared.Return(buffer);
    }
}

StringBuilder for String Concatenation in Loops

This is well-known but still frequently violated:

// ❌ Creates a new string object on every iteration
string result = "";
foreach (var item in items)
{
    result += item.ToString() + ", ";
}

// ✅ Single allocation, much less GC pressure
var sb = new StringBuilder(items.Count * 20); // Estimate capacity
foreach (var item in items)
{
    if (sb.Length > 0) sb.Append(", ");
    sb.Append(item);
}
var result = sb.ToString();

Object Pooling with ObjectPool

For expensive-to-create objects used frequently, use Microsoft.Extensions.ObjectPool.

builder.Services.AddSingleton<ObjectPoolProvider, DefaultObjectPoolProvider>();
builder.Services.AddSingleton(sp =>
{
    var provider = sp.GetRequiredService<ObjectPoolProvider>();
    return provider.Create(new DefaultPooledObjectPolicy<StringBuilder>());
});

// In your service:
public class ReportGenerator
{
    private readonly ObjectPool<StringBuilder> _sbPool;

    public ReportGenerator(ObjectPool<StringBuilder> sbPool) => _sbPool = sbPool;

    public string GenerateReport(IEnumerable<ReportLine> lines)
    {
        var sb = _sbPool.Get();
        try
        {
            foreach (var line in lines)
            {
                sb.AppendLine($"{line.Label}: {line.Value}");
            }
            return sb.ToString();
        }
        finally
        {
            sb.Clear();
            _sbPool.Return(sb);
        }
    }
}

6. Logging and Diagnostics Impact on Performance

Logging is essential, but aggressive logging is a silent performance killer — especially in hot paths.

Use Source Generators for High-Performance Logging

The LoggerMessage source generator avoids boxing and string interpolation overhead.

// ❌ Interpolation allocates even when log level is disabled
_logger.LogInformation($"Processing order {orderId} for customer {customerId}");

// ❌ Structured logging is better but still has boxing overhead
_logger.LogInformation("Processing order {OrderId} for customer {CustomerId}",
    orderId, customerId);

// ✅ Source-generated: zero allocation when level is disabled
public static partial class LogMessages
{
    [LoggerMessage(
        EventId = 1001,
        Level = LogLevel.Information,
        Message = "Processing order {OrderId} for customer {CustomerId}")]
    public static partial void ProcessingOrder(
        ILogger logger, int orderId, int customerId);
}

// Usage:
LogMessages.ProcessingOrder(_logger, orderId, customerId);

Guard Expensive Log Operations

If you're logging something that requires computation, check the level first.

// ❌ Serialization happens even if Debug logging is off
_logger.LogDebug("Request payload: {Payload}",
    JsonSerializer.Serialize(request));

// ✅ Only serialize when Debug is actually enabled
if (_logger.IsEnabled(LogLevel.Debug))
{
    _logger.LogDebug("Request payload: {Payload}",
        JsonSerializer.Serialize(request));
}

Right-Size Your Log Levels in Production

In production, your minimum level should typically be Warning or Information — never Debug or Trace. Configure this per-category in appsettings.Production.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning",
      "Microsoft.AspNetCore": "Warning",
      "MyApp.Services": "Information",
      "Microsoft.EntityFrameworkCore.Database.Command": "Warning"
    }
  }
}

7. API Response Optimization

Enable Response Compression

Compress responses at the application level — especially useful when you're not behind a reverse proxy that handles compression.

builder.Services.AddResponseCompression(options =>
{
    options.EnableForHttps = true;
    options.Providers.Add<BrotliCompressionProvider>();
    options.Providers.Add<GzipCompressionProvider>();
    options.MimeTypes = ResponseCompressionDefaults
        .MimeTypes
        .Concat(["application/json", "application/xml"]);
});

builder.Services.Configure<BrotliCompressionProviderOptions>(options =>
{
    options.Level = CompressionLevel.Optimal;
});

// In the pipeline — place BEFORE response caching
app.UseResponseCompression();

Implement Pagination Properly

Never return unbounded collections. Cursor-based pagination is more performant than offset-based for large datasets.

// Offset-based (simpler, worse for deep pages)
[HttpGet("products")]
public async Task<IActionResult> GetProducts(
    [FromQuery] int page = 1,
    [FromQuery] int pageSize = 25)
{
    pageSize = Math.Clamp(pageSize, 1, 100); // Enforce maximum

    var query = _db.Products.AsNoTracking().Where(p => p.IsActive);

    var totalCount = await query.CountAsync();
    var items = await query
        .OrderBy(p => p.Id)
        .Skip((page - 1) * pageSize)
        .Take(pageSize)
        .Select(p => new ProductDto(p.Id, p.Name, p.Price))
        .ToListAsync();

    return Ok(new PagedResult<ProductDto>
    {
        Items = items,
        TotalCount = totalCount,
        Page = page,
        PageSize = pageSize,
        TotalPages = (int)Math.Ceiling(totalCount / (double)pageSize)
    });
}

// Cursor-based (better for large datasets — avoids OFFSET)
[HttpGet("products")]
public async Task<IActionResult> GetProducts(
    [FromQuery] int? afterId = null,
    [FromQuery] int pageSize = 25)
{
    pageSize = Math.Clamp(pageSize, 1, 100);

    var query = _db.Products
        .AsNoTracking()
        .Where(p => p.IsActive);

    if (afterId.HasValue)
        query = query.Where(p => p.Id > afterId.Value);

    var items = await query
        .OrderBy(p => p.Id)
        .Take(pageSize + 1) // Fetch one extra to check if more exist
        .Select(p => new ProductDto(p.Id, p.Name, p.Price))
        .ToListAsync();

    var hasMore = items.Count > pageSize;
    if (hasMore) items.RemoveAt(items.Count - 1);

    return Ok(new CursorResult<ProductDto>
    {
        Items = items,
        HasMore = hasMore,
        NextCursor = items.LastOrDefault()?.Id
    });
}

Use Minimal APIs for Lightweight Endpoints

For simple endpoints, Minimal APIs have less overhead than full MVC controllers.

app.MapGet("/api/health", () => Results.Ok(new { Status = "Healthy" }));

app.MapGet("/api/products/{id:int}", async (int id, AppDbContext db) =>
    await db.Products.FindAsync(id) is { } product
        ? Results.Ok(product)
        : Results.NotFound());

8. Measuring Performance: Tools and Techniques

You can't optimize what you can't measure. Here are the tools you should be using.

BenchmarkDotNet — Micro-Benchmarks

Use BenchmarkDotNet to compare implementations with statistical rigor.

[MemoryDiagnoser]
[SimpleJob(RuntimeMoniker.Net80)]
public class SerializationBenchmarks
{
    private readonly List<Product> _products = Enumerable
        .Range(1, 1000)
        .Select(i => new Product { Id = i, Name = $"Product {i}", Price = i * 1.5m })
        .ToList();

    [Benchmark(Baseline = true)]
    public string NewtonsoftJson()
    {
        return Newtonsoft.Json.JsonConvert.SerializeObject(_products);
    }

    [Benchmark]
    public string SystemTextJson()
    {
        return System.Text.Json.JsonSerializer.Serialize(_products);
    }

    [Benchmark]
    public byte[] SystemTextJsonUtf8()
    {
        return System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(_products);
    }
}

// Run with: dotnet run -c Release

The [MemoryDiagnoser] attribute is critical — it shows you allocations per operation, which is often more important than raw speed.

dotnet-counters — Runtime Diagnostics

Monitor your app's runtime behavior in real time:

# Install
dotnet tool install --global dotnet-counters

# Monitor your app (find PID with dotnet-counters ps)
dotnet-counters monitor --process-id <PID> \
    --counters System.Runtime,Microsoft.AspNetCore.Hosting,Microsoft.EntityFrameworkCore

# Key metrics to watch:
# - gen-0-gc-count, gen-1-gc-count, gen-2-gc-count
# - threadpool-thread-count
# - threadpool-queue-length (should stay near 0)
# - active-db-connections
# - requests-per-second
# - current-requests

Application Insights — Production Monitoring

Enable dependency tracking and custom metrics to find bottlenecks in production.

builder.Services.AddApplicationInsightsTelemetry();

// Custom metrics for business-critical operations
public class OrderService
{
    private readonly TelemetryClient _telemetry;

    public OrderService(TelemetryClient telemetry) => _telemetry = telemetry;

    public async Task<Order> PlaceOrderAsync(OrderRequest request)
    {
        var stopwatch = Stopwatch.StartNew();

        try
        {
            var order = await ProcessOrderInternal(request);

            _telemetry.TrackMetric("OrderProcessing.Duration",
                stopwatch.Elapsed.TotalMilliseconds);
            _telemetry.TrackMetric("OrderProcessing.ItemCount",
                request.Items.Count);

            return order;
        }
        catch (Exception ex)
        {
            _telemetry.TrackException(ex);
            throw;
        }
    }
}

9. Azure Deployment Performance Considerations

Choose the Right App Service Tier

Not all tiers are created equal. The Basic tier shares CPU. The Standard and Premium tiers give you dedicated compute. For performance-sensitive workloads, Premium v3 (P1v3+) offers better CPU, faster storage I/O, and support for more instances.

Enable Auto-Scaling Based on the Right Metric

CPU percentage is the default trigger, but it's not always the best signal. Consider scaling on HTTP queue length or custom metrics from Application Insights.

Use Deployment Slots to Avoid Cold Starts

Deployment slots let you warm up a new version before swapping it into production. Configure warm-up in your web.config:

<system.webServer>
  <applicationInitialization doAppInitAfterRestart="true">
    <add initializationPage="/health" />
    <add initializationPage="/api/warmup" />
  </applicationInitialization>
</system.webServer>

Use Azure Redis Cache Instead of In-Memory Cache

When running multiple instances (and you should be, for reliability), in-memory cache causes inconsistencies. Use Azure Cache for Redis as your IDistributedCache implementation.

Connection Strings and Service Configuration

// Enable connection resiliency for Azure SQL
builder.Services.AddDbContext<AppDbContext>(options =>
    options.UseSqlServer(connectionString, sqlOptions =>
    {
        sqlOptions.EnableRetryOnFailure(
            maxRetryCount: 5,
            maxRetryDelay: TimeSpan.FromSeconds(30),
            errorNumbersToAdd: null);
        sqlOptions.CommandTimeout(30);
    }));

Enable HTTP/2 and Keep-Alive

builder.WebHost.ConfigureKestrel(options =>
{
    options.AddServerHeader = false; // Minor — removes the Server header
    options.Limits.Http2.MaxStreamsPerConnection = 100;
    options.Limits.KeepAliveTimeout = TimeSpan.FromMinutes(2);
    options.Limits.RequestHeadersTimeout = TimeSpan.FromSeconds(30);
});

10. Common Performance Pitfalls and Anti-Patterns

Here are the mistakes we see most often in production ASP.NET Core applications:

Anti-Pattern	Why It Hurts	Fix
Calling `.Result` or `.Wait()` on tasks	Thread pool starvation, deadlocks	Use `async/await` throughout
Using `Include()` instead of `Select()`	Over-fetches data, large memory footprint	Project to DTOs
No pagination on list endpoints	Unbounded queries, timeouts, OOM	Always paginate; enforce max page size
Logging at `Debug`/`Trace` in production	I/O overhead on every request	Set minimum level to `Warning`
Creating `HttpClient` with `new`	Socket exhaustion	Use `IHttpClientFactory`
Allocating large strings in loops	GC pressure, Gen 2 collections	Use `StringBuilder` or `Span<T>`
Not using `AsNoTracking()`	Unnecessary change tracking overhead	Default to no-tracking for reads
Catching and re-throwing exceptions	Stack trace reset, perf cost of exceptions	Use `throw;` not `throw ex;`
Synchronous I/O in middleware	Blocks thread pool threads	Use async overloads exclusively
No caching on stable data	Repeated DB/API calls for the same data	Cache with appropriate TTL

Performance Optimization Checklist

Apply these immediately to any ASP.NET Core project:

Request Pipeline

[ ] Middleware is ordered correctly (short-circuit first)
[ ] Unused middleware is removed
[ ] Response compression is enabled

Async/Await

[ ] No .Result or .Wait() calls anywhere
[ ] Independent I/O operations are parallelized with Task.WhenAll
[ ] ConfigureAwait(false) is used in library/service code

Caching

[ ] Frequently read, rarely changed data is cached
[ ] Distributed cache is configured for multi-instance deployments
[ ] Cache entries have appropriate expiration policies

Database

[ ] AsNoTracking() is used on all read-only queries
[ ] Queries project to DTOs instead of loading full entities
[ ] No N+1 query patterns exist (check with EF Core logging)
[ ] Hot-path queries are compiled with EF.CompileAsyncQuery
[ ] Connection strings include pooling configuration

Allocations

[ ] StringBuilder is used instead of string concatenation in loops
[ ] ArrayPool<T> is used for temporary buffers
[ ] Span<T> is used for parsing operations on hot paths
[ ] Large object allocations are minimized (avoid byte[] > 85KB)

Logging

[ ] LoggerMessage source generators are used in hot paths
[ ] Production log level is Warning or higher by default
[ ] Expensive log payloads are guarded with IsEnabled()

API Design

[ ] All list endpoints are paginated with an enforced maximum
[ ] HttpClient is created via IHttpClientFactory
[ ] Response compression (Brotli/Gzip) is enabled

Monitoring

[ ] Application Insights (or equivalent) is configured
[ ] Key operations have custom performance metrics
[ ] dotnet-counters has been run at least once to baseline GC and thread pool behavior

Key Takeaways

Performance optimization in ASP.NET Core isn't about finding a single silver bullet — it's about systematically identifying and eliminating waste across every layer of your application.

The highest-impact areas, in order of typical ROI:

Database queries — this is almost always where the biggest wins are. Fix N+1 queries, project to DTOs, add AsNoTracking().
Caching — cache what you can at the right layer. Even a 5-minute cache on a hot endpoint can reduce database load by 99%.
Async discipline — don't block, don't allocate unnecessarily, parallelize where possible.
Pipeline hygiene — strip unused middleware, order correctly, compress responses.
Measure everything — use dotnet-counters, BenchmarkDotNet, and Application Insights. Don't guess, profile.

The best performance work is boring. It's not about clever tricks — it's about consistent application of known best practices, combined with measurement to validate that your changes actually matter.

Start with your slowest endpoint. Profile it. Fix the biggest bottleneck. Repeat.

Happy optimizing.

Modular Monolith Architecture in .NET: The Pragmatic Middle Ground

Adrián López — Sat, 28 Mar 2026 10:47:03 +0000

The Problem No One Talks About in the Microservices Hype

It's 2019. Your team has just finished migrating a perfectly functional ASP.NET MVC app to microservices. You now have 14 services, a Kubernetes cluster, a service mesh, distributed tracing, a message broker, and three engineers whose full-time job is keeping it all running. Feature velocity? Cut in half. Onboarding a new developer? Two weeks just to understand the infrastructure.

Sound familiar?

The architecture community spent a decade evangelizing microservices as the solution to every scalability and maintainability problem. And while microservices genuinely solve hard problems at scale, they come with an enormous operational and cognitive tax — one that most product teams can't afford, and frankly don't need to pay.

Enter the Modular Monolith: not a retreat to the big ball of mud, and not the complexity of distributed systems. It's an architectural style that enforces clear internal boundaries while keeping the simplicity of a single deployable unit. For many .NET teams, it's the most pragmatic choice they never considered.

What Is a Modular Monolith?

A modular monolith is a single deployable application composed of well-defined, loosely coupled internal modules, each owning its own domain logic, data, and public API surface.

The key distinction from a traditional monolith is intentional structure. A classic monolith tends to evolve into spaghetti — shared database tables, cross-cutting service dependencies, and no clear ownership boundaries. A modular monolith fights this entropy from day one by treating each module as if it were a microservice candidate, without actually making it one yet.

The Three-Way Comparison:

Dimension	Traditional Monolith	Modular Monolith	Microservices
Deployment	Single unit	Single unit	Multiple units
Module boundaries	Weak / absent	Strong, enforced	Strong, enforced
Data isolation	Shared DB	Logical isolation	Physical isolation
Communication	Direct calls	In-process events	Network (HTTP/gRPC/MQ)
Operational complexity	Low	Low	High
Refactoring cost	High (tightly coupled)	Medium	High (distributed)
Team autonomy	Low	Medium	High
Latency overhead	None	None	Network latency

Think of it this way: a modular monolith is a microservices architecture collapsed into a single process. The boundaries are real — they're just not physical.

Why Not Just Use Microservices?

Microservices are genuinely powerful when you need:

Independent scaling of specific workloads
Polyglot persistence and technology choices per service
Complete team autonomy with separate CI/CD pipelines
Fault isolation at the process boundary level

But the prerequisites are steep. You need mature DevOps culture, a platform team, distributed tracing, eventual consistency patterns, saga orchestration, and more. The distributed fallacies (network is reliable, latency is zero, bandwidth is infinite…) become your daily enemies.

The hidden costs of premature microservices decomposition:

Distributed transactions replaced simple database transactions — now you're managing sagas and compensating actions
A cross-service feature requires coordinating 3 teams and 3 PRs
Local development requires Docker Compose with 8 containers just to run the app
Debugging requires correlating traces across 5 services

A modular monolith defers these costs until they're actually justified. And for many applications — even those at serious scale — that day may never come.

Core Principles

1. Module Boundaries

A module is a cohesive grouping of functionality around a bounded context (to borrow DDD terminology). In an e-commerce system, natural modules might be Orders, Catalog, Users, Payments, and Notifications.

The boundary rule is simple: modules do not reference each other's internals. If Orders needs a product name, it doesn't call Catalog's internal ProductService — it goes through a defined public contract.

2. High Cohesion / Low Coupling

Everything related to the Catalog domain lives inside Catalog: its entities, repositories, domain services, use cases, and event handlers. Nothing leaks out. Other modules don't know Catalog has an EfCore repository — they only know the public interface it exposes.

3. Independent Data Per Module

This is the most important and most violated principle. Each module should own its own data. In a modular monolith, you typically use a single database with schema-level separation (e.g., catalog.Products, orders.Orders), or separate DbContext instances that map to different schemas.

No module queries another module's tables directly. Ever. This is the line between a modular monolith and a well-organized big ball of mud.

4. Communication Patterns

Modules communicate in two ways:

Synchronous (in-process): One module exposes an interface (e.g., ICatalogModule) and another calls it through the DI container. This is appropriate for read queries where you need an immediate result.

Asynchronous (domain events): When Orders places an order, it publishes an OrderPlacedEvent. Inventory and Notifications handle it independently. This is done in-process using a mediator or event dispatcher — no message broker required.

Implementing in .NET

Project Structure

src/
├── ECommerce.Api/                  # Host project (entry point)
│   ├── Program.cs
│   └── appsettings.json
│
├── ECommerce.Shared/               # Cross-cutting contracts
│   ├── Events/                     # Domain event interfaces
│   └── Modules/                    # IModule interface
│
├── ECommerce.Modules.Catalog/      # Catalog module (Class Library)
│   ├── CatalogModule.cs            # Module registration
│   ├── Domain/
│   │   └── Product.cs
│   ├── Application/
│   │   ├── GetProductQuery.cs
│   │   └── GetProductQueryHandler.cs
│   ├── Infrastructure/
│   │   ├── CatalogDbContext.cs
│   │   └── ProductRepository.cs
│   └── Api/
│       └── CatalogEndpoints.cs
│
├── ECommerce.Modules.Orders/       # Orders module (Class Library)
│   ├── OrdersModule.cs
│   ├── Domain/
│   │   └── Order.cs
│   ├── Application/
│   │   ├── PlaceOrderCommand.cs
│   │   └── PlaceOrderCommandHandler.cs
│   ├── Infrastructure/
│   │   └── OrdersDbContext.cs
│   └── Api/
│       └── OrdersEndpoints.cs
│
└── ECommerce.Modules.Users/        # Users module (Class Library)

Each module is a separate class library project. The Api host project references all modules, but modules reference only ECommerce.Shared — never each other directly.

The IModule Contract

Define a shared registration interface that every module implements:

// ECommerce.Shared/Modules/IModule.cs
public interface IModule
{
    string Name { get; }
    IServiceCollection RegisterServices(
        IServiceCollection services,
        IConfiguration configuration);
    WebApplication MapEndpoints(WebApplication app);
}

Module Registration in Program.cs

// ECommerce.Api/Program.cs
var builder = WebApplication.CreateBuilder(args);

// Each module self-registers
var modules = new IModule[]
{
    new CatalogModule(),
    new OrdersModule(),
    new UsersModule(),
};

foreach (var module in modules)
    module.RegisterServices(builder.Services, builder.Configuration);

builder.Services.AddMediatR(cfg =>
    cfg.RegisterServicesFromAssemblies(
        typeof(CatalogModule).Assembly,
        typeof(OrdersModule).Assembly,
        typeof(UsersModule).Assembly));

var app = builder.Build();

foreach (var module in modules)
    module.MapEndpoints(app);

app.Run();

DI Per Module

Each module's RegisterServices method wires up its own dependencies in isolation:

// ECommerce.Modules.Catalog/CatalogModule.cs
public class CatalogModule : IModule
{
    public string Name => "Catalog";

    public IServiceCollection RegisterServices(
        IServiceCollection services,
        IConfiguration configuration)
    {
        services.AddDbContext<CatalogDbContext>(opts =>
            opts.UseSqlServer(
                configuration.GetConnectionString("Catalog"),
                sql => sql.MigrationsHistoryTable("__EFMigrationsHistory", "catalog")));

        services.AddScoped<IProductRepository, ProductRepository>();
        services.AddScoped<ICatalogModule, CatalogModuleFacade>();

        return services;
    }

    public WebApplication MapEndpoints(WebApplication app)
    {
        app.MapGroup("/catalog").MapCatalogEndpoints();
        return app;
    }
}

Real-World Example: E-Commerce Order Placement

Let's trace an order placement flow across modules.

The Cross-Module Contract

Orders needs to verify product prices from Catalog. It does this through a public facade interface — not by importing Catalog's internal types:

// ECommerce.Shared/Modules/ICatalogModule.cs
public interface ICatalogModule
{
    Task<ProductDto?> GetProductAsync(Guid productId);
}

public record ProductDto(Guid Id, string Name, decimal Price, bool IsAvailable);

Placing an Order

// ECommerce.Modules.Orders/Application/PlaceOrderCommandHandler.cs
public class PlaceOrderCommandHandler
    : IRequestHandler<PlaceOrderCommand, Guid>
{
    private readonly IOrderRepository _orders;
    private readonly ICatalogModule _catalog;       // cross-module contract
    private readonly IPublisher _publisher;          // MediatR publisher

    public PlaceOrderCommandHandler(
        IOrderRepository orders,
        ICatalogModule catalog,
        IPublisher publisher)
    {
        _orders = orders;
        _catalog = catalog;
        _publisher = publisher;
    }

    public async Task<Guid> Handle(
        PlaceOrderCommand command,
        CancellationToken cancellationToken)
    {
        var product = await _catalog.GetProductAsync(command.ProductId);

        if (product is null || !product.IsAvailable)
            throw new DomainException("Product not available.");

        var order = Order.Create(command.CustomerId, product.Id, product.Price);
        await _orders.AddAsync(order, cancellationToken);

        // Publish domain event — handled by other modules in-process
        await _publisher.Publish(
            new OrderPlacedEvent(order.Id, product.Id, command.CustomerId),
            cancellationToken);

        return order.Id;
    }
}

Handling the Event in Another Module

// ECommerce.Modules.Notifications/Application/SendOrderConfirmationHandler.cs
public class SendOrderConfirmationHandler
    : INotificationHandler<OrderPlacedEvent>
{
    private readonly IEmailService _email;
    private readonly IUsersModule _users;

    public async Task Handle(
        OrderPlacedEvent notification,
        CancellationToken cancellationToken)
    {
        var user = await _users.GetUserAsync(notification.CustomerId);
        await _email.SendOrderConfirmationAsync(user.Email, notification.OrderId);
    }
}

Orders publishes an event. Notifications handles it. Neither knows the other exists. This is the power of in-process messaging with MediatR — you get the decoupling benefits of a message bus without the operational overhead.

Common Pitfalls

1. The Shared Database Anti-Pattern

The most common failure mode: all modules share a single DbContext and freely query each other's tables. You end up with JOINs across module boundaries, and suddenly changing a column in catalog.Products breaks 4 other modules.

The fix: One DbContext per module, separate DB schemas, zero cross-schema queries. If module B needs data from module A, it calls A's public API — not its tables.

2. Tight Coupling Through Shared Domain Models

Referencing another module's domain entities directly creates invisible coupling. If Orders imports Catalog.Domain.Product, you've created a compile-time dependency that defeats the entire architecture.

The fix: Shared contracts live in ECommerce.Shared as DTOs or interfaces. Domain models are private to their module.

3. Overengineering Small Applications

If your application is a CRUD API with 5 entities and no meaningful domain complexity, a modular monolith is overkill. You're adding project structure, DI ceremony, and module facades for no real benefit.

The fix: Apply this architecture when you have genuine domain complexity, multiple teams, or a realistic expectation of growth. Don't architect for hypothetical futures.

When to Choose Modular Monolith vs. Microservices

Choose a Modular Monolith when:

You're building a new product and the domain isn't fully understood yet
Team size is under ~15 engineers with 2–4 feature teams
Operational maturity for Kubernetes/service mesh isn't there yet
Your workloads are relatively uniform in compute/memory needs
You value developer experience and fast local development

Choose Microservices when:

Specific services have vastly different scaling requirements (e.g., a search service vs. a checkout service)
You have large, autonomous teams that need independent deployment pipelines
You need polyglot persistence (different databases per service for legitimate reasons)
You've already extracted a clear, stable bounded context that rarely changes

The honest truth: Most applications should start as a modular monolith. If they grow to need microservices, the modular structure makes extraction far less painful than it would be from a traditional monolith.

Scaling and Evolution Strategy

One of the strongest arguments for this architecture is its evolutionary path. Because your modules already have clean boundaries, migrating a single module to a standalone service is a surgical operation, not a rewrite.

The extraction checklist for a module:

✅ The module already has its own DbContext and schema — point it at a separate database
✅ All cross-module communication already goes through interfaces — swap the in-process implementation for an HTTP or message-based adapter
✅ MediatR domain events switch from in-process IPublisher to publishing to a message broker (e.g., MassTransit + RabbitMQ)
✅ The module's endpoints already exist — lift them into a new host project

In practice, extraction of a well-bounded module can take days, not months. Compare this to extracting from a traditional monolith where untangling shared state, circular dependencies, and cross-table queries can take quarters.

A practical evolution path looks like this:

Phase 1: Modular Monolith (single deployment, all modules)
    ↓  (Catalog service gets high read traffic)
Phase 2: Extract Catalog to a standalone service
         → Orders calls Catalog via HTTP
         → Events go through a message broker
    ↓  (Payments needs PCI compliance isolation)
Phase 3: Extract Payments to a standalone service
         → Now you have 3 deployable units

You scale surgically, not speculatively.

Key Takeaways

A modular monolith is not a traditional monolith — it's a disciplined architecture with enforced module boundaries, data isolation, and clean communication contracts
The single biggest rule: no module accesses another module's data store directly. Ever.
In .NET, implement this with separate class libraries per module, module-scoped DbContext instances, public facade interfaces in a shared contracts project, and MediatR for in-process event-driven communication
Avoid the trap of premature microservices — the operational complexity is real and often unjustified at early product stages
A well-built modular monolith is the best migration path to microservices if and when you genuinely need them
Start here by default. Migrate to microservices only when you have a specific, measurable reason to do so — not because it's fashionable

The architecture world likes clean, binary choices. Microservices or monolith. Distributed or single process. But the most pragmatic choice is often in the middle: a system that's well-structured enough to evolve, and simple enough to actually ship.

Have you migrated to or from a modular monolith? The edge cases are always where the real lessons live — reach out or drop a comment.

DEV Community: Adrián López

OpenTelemetry in .NET Microservices: A Practical Guide to Observable Distributed Systems

Why Observability Is Hard in Microservices

What Is OpenTelemetry?

OpenTelemetry in the .NET Ecosystem

Core Concepts

Traces, Spans, and Context Propagation

Metrics and Logs

Instrumentation: Automatic vs. Manual

Exporters and Backends

Hands-On: Setting Up OpenTelemetry in ASP.NET Core

Step 1: Install the Packages

Step 2: Configure the SDK in Program.cs

Step 3: Add Custom Instrumentation

Step 4: Correlating Requests Across Services

Real-World Scenarios

Scenario 1: Debugging Latency Across Services

Scenario 2: Tracking End-to-End Request Flow

Scenario 3: Identifying Production Bottlenecks

Best Practices

Naming Conventions

Reducing Telemetry Noise and Cost

Sampling Strategies

Securing Telemetry Data

Deploying and Monitoring in Azure

Integration with Application Insights / Azure Monitor

Observability in Containerized Environments (Kubernetes)

Key Takeaways

Implementation Checklist

Next Steps

Azure Service Bus for Event-Driven Systems: A Practical Deep Dive

Introduction: Why Event-Driven Architecture Matters Now More Than Ever

What Is Azure Service Bus, and When Should You Reach for It?

Service Bus vs. Event Grid vs. Event Hubs: Choosing the Right Tool

Core Concepts in Depth

Queues vs. Topics vs. Subscriptions

Messages, Sessions, and Ordering

Dead-Letter Queues

Message Delivery Guarantees

Scheduling and Delayed Delivery

Decoupling and Scalability in Microservices

Real-World Scenarios

Scenario 1: Order Processing Pipeline

Scenario 2: Cross-Service Integration Events

Scenario 3: Background Job Offloading

C# Examples with Azure.Messaging.ServiceBus SDK

Setting Up the Client

Sending Messages

Receiving and Processing Messages

Handling Failures and Retries

Best Practices

Idempotency and Message Handling

Error Handling Strategies

Throughput and Scaling Considerations

Security and Authentication

Performance and Cost Optimization

Cost Drivers

Performance Benchmarks to Keep in Mind

Architecture Patterns

Publish-Subscribe with Filtered Subscriptions

Competing Consumers for Horizontal Scaling

Saga/Choreography with Service Bus

Request-Reply Over Service Bus

Summary

Best Practices Checklist

Further Exploration

ASP.NET Core Performance Optimization Techniques

Introduction: Why Performance Is a Cloud-Native Imperative

1. Middleware Optimization and Request Pipeline Tuning

Order Matters — A Lot

Strip Out What You Don't Need

Write Lean Custom Middleware

2. Efficient Use of Async/Await

Avoid Async for Trivial Operations

Use ConfigureAwait(false) in Library Code

Parallelize Independent I/O Calls

Never Block on Async Code

3. Caching Strategies

In-Memory Caching

Distributed Caching with Redis

Step 2: Configure the SDK in `Program.cs`