DEV Community: Nick

LINQ Performance: The Tricks That Make Queries Scream

Nick — Thu, 01 Oct 2026 20:36:00 +0000

LINQ Performance: The Tricks That Make Queries Scream (And The Pitfalls That Make Them Crawl)

You've learned LINQ. Your queries work. But working isn't the same as fast. The difference between a 50ms query and a 5-second timeout is usually a handful of patterns the documentation doesn't emphasize.

Let's fix that.

The Materialization Cost

Every ToList(), ToArray(), ToDictionary() allocates memory and copies data. Chain multiple? You're copying multiple times:

// Bad — three allocations
var result = items
    .Where(x => x.IsActive)
    .ToList()               // Allocation 1
    .OrderBy(x => x.Name)
    .ToList()               // Allocation 2
    .Take(10)
    .ToList();              // Allocation 3

// Good — one allocation
var result = items
    .Where(x => x.IsActive)
    .OrderBy(x => x.Name)
    .Take(10)
    .ToList();              // Single allocation at the end

Materialize once, at the end, after all transformations.

Any() vs Count() > 0

Need to check if something exists? Any() stops at the first match:

// Bad — counts everything
if (products.Count() > 0) { }

// Good — stops at first item
if (products.Any()) { }

// With condition
if (products.Any(p => p.Price > 100)) { }  // Stops at first matching item

In SQL, Any() becomes EXISTS (stops early). Count() scans the entire table.

Fun fact: The performance difference scales with data size. On a table with 10 million rows where 1 exists, Any() returns immediately while Count() still scans all 10 million. I've seen this single change turn 30-second queries into 10ms responses.

First() vs Single() vs FirstOrDefault()

First() — returns first, throws if empty
FirstOrDefault() — returns first or default, no throw
Single() — returns one, throws if empty OR if multiple exist
SingleOrDefault() — returns one or default, throws if multiple exist

The crucial difference: Single() must verify no second element exists. That means fetching (or at least checking for) two rows.

// When you expect exactly one and want validation
var user = dbContext.Users.Single(u => u.Email == email);  // Throws if 0 or 2+

// When you want the first and don't care about duplicates
var user = dbContext.Users.First(u => u.Email == email);  // Faster, takes first

For database queries, First() generates TOP 1. Single() needs to verify uniqueness.

The Index Awareness Gap

LINQ doesn't know about your indexes. But your query shapes determine whether indexes are used:

// If Email is indexed, this is fast
var user = dbContext.Users.FirstOrDefault(u => u.Email == email);

// If Email is NOT indexed, this scans the table
var user = dbContext.Users.FirstOrDefault(u => u.Email == email);

Same LINQ, different performance based on database schema. Profile your actual queries. Check execution plans.

Computed Columns in WHERE

Wrapping a column in a function kills index usage:

// Bad — YEAR(OrderDate) prevents index use
var orders = dbContext.Orders
    .Where(o => o.OrderDate.Year == 2024)
    .ToList();

// Good — range query uses index
var start = new DateTime(2024, 1, 1);
var end = new DateTime(2025, 1, 1);
var orders = dbContext.Orders
    .Where(o => o.OrderDate >= start && o.OrderDate < end)
    .ToList();

The rewrite generates OrderDate >= '2024-01-01' AND OrderDate < '2025-01-01', which can use an index on OrderDate.

The Tracking Overhead

By default, EF tracks every entity for change detection. For read-only queries, disable it:

// Default — entities are tracked
var products = dbContext.Products.ToList();

// Faster for read-only — no tracking overhead
var products = dbContext.Products.AsNoTracking().ToList();

AsNoTracking() skips the change tracker, reduces memory, and speeds up materialization. Use it whenever you're just displaying data.

For an entire context that's read-only:

dbContext.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;

Batch vs Individual Operations

// Terrible — N database calls
foreach (var product in products)
{
    product.Price *= 1.1m;
    await dbContext.SaveChangesAsync();  // Save inside loop!
}

// Better — one database call
foreach (var product in products)
{
    product.Price *= 1.1m;
}
await dbContext.SaveChangesAsync();  // Save after loop

// Best for bulk — raw SQL, no entity loading
await dbContext.Database.ExecuteSqlRawAsync(
    "UPDATE Products SET Price = Price * 1.1 WHERE CategoryId = {0}", categoryId);

Each SaveChanges is a database round-trip. Batch operations when possible. For massive updates, raw SQL avoids loading entities entirely.

The Pagination Pattern

// Bad — loads all, then slices
var allProducts = dbContext.Products.ToList();
var page = allProducts.Skip(100).Take(20);

// Good — SQL handles pagination
var page = dbContext.Products
    .OrderBy(p => p.Id)  // Required for consistent paging
    .Skip(100)
    .Take(20)
    .ToList();

Skip and Take on IQueryable translate to OFFSET/FETCH or LIMIT. On IEnumerable (after ToList), they filter in memory.

Always page in the database.

String Operations: The Hidden Cost

String methods have different database translations:

// These usually translate well
.Where(p => p.Name.Contains("phone"))     // LIKE '%phone%'
.Where(p => p.Name.StartsWith("Smart"))   // LIKE 'Smart%' — can use index
.Where(p => p.Name.EndsWith("Pro"))       // LIKE '%Pro' — no index

// This might not translate
.Where(p => p.Name.ToLower() == searchTerm.ToLower())  // May cause issues

StartsWith can use indexes. Contains and EndsWith generally can't. And string manipulation functions vary by database provider.

Compiled Queries for Hot Paths

If a query runs thousands of times, compile it once:

private static readonly Func<AppDbContext, int, Product?> GetProductById =
    EF.CompileQuery((AppDbContext db, int id) =>
        db.Products.FirstOrDefault(p => p.Id == id));

// Usage — no expression tree compilation overhead
var product = GetProductById(dbContext, 42);

Compiled queries skip the expression tree parsing on each call. For cold paths (run rarely), the overhead is negligible. For hot paths (called per-request), it adds up.

The Checklist

Materialize once — ToList() at the end, not mid-chain
Any() over Count() > 0 — stop at first match
First() over Single() — unless you need the uniqueness check
AsNoTracking() — for read-only queries
Paginate in database — Skip/Take on IQueryable
Avoid function wrappers in WHERE — kills index usage
Profile actual SQL — LINQ is an abstraction; the database decides performance

That wraps our LINQ series! We've covered deferred execution, IQueryable vs IEnumerable, projections, grouping, joins, the N+1 problem, async patterns, aggregates, raw SQL, and performance. Each of these concepts builds on the others to make you dangerous with LINQ.

The real learning happens when you apply these patterns to your own queries, profile the results, and see the database doing what you intended. Good luck out there!

When LINQ Isn't Enough: Raw SQL in Entity Framework Core

Nick — Wed, 30 Sep 2026 17:35:00 +0000

When LINQ Isn't Enough: Raw SQL in Entity Framework Core

Sometimes LINQ can't express what you need. A complex stored procedure. A database-specific function. A performance-critical query the ORM mangles. That's when you reach for raw SQL — but EF Core still wants to help.

Let's explore the escape hatches.

FromSqlRaw: Queries That Return Entities

When you need raw SQL but still want entity tracking and LINQ composition:

var products = dbContext.Products
    .FromSqlRaw("SELECT * FROM Products WHERE Price > 100")
    .ToList();

The results are tracked entities. You can modify and save them.

Even better — you can chain LINQ operators:

var products = dbContext.Products
    .FromSqlRaw("SELECT * FROM Products WHERE Price > 100")
    .OrderBy(p => p.Name)   // Added by EF
    .Take(10)                // Added by EF
    .ToList();

EF wraps your SQL as a subquery and adds the LINQ parts. The generated SQL looks like:

SELECT * FROM (
    SELECT * FROM Products WHERE Price > 100
) AS p
ORDER BY p.Name
LIMIT 10

FromSqlInterpolated: Safe Parameterization

Never concatenate user input into SQL. Use interpolation:

var category = "Electronics";
var minPrice = 50m;

var products = dbContext.Products
    .FromSqlInterpolated($"SELECT * FROM Products WHERE Category = {category} AND Price > {minPrice}")
    .ToList();

EF extracts the interpolated values as parameters. The actual SQL becomes:

SELECT * FROM Products WHERE Category = @p0 AND Price > @p1

SQL injection? Prevented. The interpolation syntax looks like string building, but EF converts it to parameters.

Fun fact: FromSqlInterpolated uses C#'s FormattableString feature under the hood. The compiler treats $"..." differently when the method expects FormattableString instead of string. This trick lets EF intercept the interpolation values before they become a string, extracting them as parameters.

ExecuteSqlRaw: Commands That Don't Return Entities

For UPDATE, DELETE, INSERT, or anything that doesn't return rows you want as entities:

// Simple execution
int rowsAffected = dbContext.Database.ExecuteSqlRaw(
    "UPDATE Products SET Price = Price * 1.1 WHERE CategoryId = 5");

// With parameters
int rowsAffected = dbContext.Database.ExecuteSqlInterpolated(
    $"UPDATE Products SET Price = Price * {multiplier} WHERE CategoryId = {categoryId}");

Returns the number of affected rows. No entity tracking involved.

Stored Procedures

Procedures That Return Entities

var products = dbContext.Products
    .FromSqlRaw("EXEC GetProductsByCategory @p0", categoryId)
    .ToList();

// Or with named parameters
var products = dbContext.Products
    .FromSqlRaw("EXEC GetProductsByCategory @CategoryId = {0}", categoryId)
    .ToList();

The procedure must return columns matching the entity shape.

Procedures That Return Scalars or Non-Entities

// Using ADO.NET through EF
var connection = dbContext.Database.GetDbConnection();
await connection.OpenAsync();

using var command = connection.CreateCommand();
command.CommandText = "EXEC GetProductCount @CategoryId";
command.Parameters.Add(new SqlParameter("@CategoryId", categoryId));

var count = (int)await command.ExecuteScalarAsync();

For complex scenarios, you drop down to raw ADO.NET. EF manages the connection; you manage the command.

Keyless Entity Types for Custom Queries

When your SQL returns a shape that isn't a table:

// Define a keyless type
[Keyless]
public class ProductSummary
{
    public string Category { get; set; }
    public int Count { get; set; }
    public decimal TotalValue { get; set; }
}

// Register it
modelBuilder.Entity<ProductSummary>().HasNoKey();

// Query it
var summaries = dbContext.Set<ProductSummary>()
    .FromSqlRaw(@"
        SELECT Category, COUNT(*) as Count, SUM(Price) as TotalValue 
        FROM Products 
        GROUP BY Category")
    .ToList();

Keyless tells EF this type isn't tracked and has no primary key. Perfect for aggregated results, views, or stored procedure results.

Database Functions

EF Core can call database-specific functions:

// SQL Server specific
var products = dbContext.Products
    .Where(p => EF.Functions.Like(p.Name, "%phone%"))
    .ToList();

// Date functions
var recent = dbContext.Orders
    .Where(o => EF.Functions.DateDiffDay(o.OrderDate, DateTime.Now) < 30)
    .ToList();

EF.Functions provides database-specific operations that translate to native SQL functions.

When to Use Raw SQL

Complex stored procedures — especially legacy ones returning non-entity shapes
Database-specific features — window functions, CTEs, recursive queries
Performance-critical paths — when EF's generated SQL is inefficient
Bulk operations — UPDATE/DELETE affecting many rows without loading entities
Administrative queries — database maintenance, statistics, metadata

When NOT to Use Raw SQL

Simple CRUD — LINQ handles this better with compile-time checking
Dynamic filters — LINQ composes; SQL concatenation leads to injection
When you haven't measured — EF's SQL is usually fine. Profile first.

The Hybrid Approach

Combine raw SQL for the complex part, LINQ for the rest:

var baseQuery = dbContext.Products
    .FromSqlRaw(@"
        SELECT p.* 
        FROM Products p
        INNER JOIN ProductMetrics pm ON p.Id = pm.ProductId
        WHERE pm.ViewCount > 1000");

// Add LINQ filtering
if (minPrice.HasValue)
    baseQuery = baseQuery.Where(p => p.Price >= minPrice.Value);

// Add LINQ projection
var results = baseQuery
    .Select(p => new { p.Id, p.Name, p.Price })
    .ToList();

Raw SQL gets the complex join. LINQ handles dynamic filtering and projection safely.

The Rule

Returns entities? FromSqlRaw / FromSqlInterpolated + optional LINQ chain
Modifying data? ExecuteSqlRaw / ExecuteSqlInterpolated
Non-entity results? Keyless entity type or raw ADO.NET
User input? Always use Interpolated or parameters — never concatenate
Try LINQ first — raw SQL is an escape hatch, not the first tool

Next time, we'll wrap up with LINQ performance tips — the tricks and pitfalls that separate a query that screams from one that crawls. Hope to see you!

LINQ Aggregate Operators: Beyond Sum and Count

Nick — Tue, 29 Sep 2026 20:35:00 +0000

LINQ Aggregate Operators: Beyond Sum and Count

Everyone knows Count() and Sum(). But LINQ's aggregate family goes deeper — and the general-purpose Aggregate() operator can implement any of them, plus custom reductions you haven't imagined yet.

Let's explore what's beyond the basics.

The Usual Suspects

var products = dbContext.Products;

var count = products.Count();                      // Number of items
var conditionalCount = products.Count(p => p.Price > 100);  // Filtered count

var sum = products.Sum(p => p.Price);              // Total price
var average = products.Average(p => p.Price);     // Mean price
var min = products.Min(p => p.Price);             // Lowest price
var max = products.Max(p => p.Price);             // Highest price

These translate to SQL aggregates — executed in the database, not in memory. Efficient.

The Null Trap

What happens when the sequence is empty?

var emptyProducts = Enumerable.Empty<Product>();

emptyProducts.Count();    // 0 — safe
emptyProducts.Sum(p => p.Price);     // 0 — safe (by design)
emptyProducts.Average(p => p.Price); // InvalidOperationException!
emptyProducts.Min(p => p.Price);     // InvalidOperationException!
emptyProducts.Max(p => p.Price);     // InvalidOperationException!

Sum and Count handle emptiness gracefully. The others throw.

The fix: use nullable overloads:

decimal? avg = emptyProducts.Average(p => (decimal?)p.Price);  // null instead of exception
decimal? min = emptyProducts.Min(p => (decimal?)p.Price);      // null
decimal? max = emptyProducts.Max(p => (decimal?)p.Price);      // null

// Or with DefaultIfEmpty
decimal avg = emptyProducts
    .Select(p => p.Price)
    .DefaultIfEmpty()  // Provides 0 if empty
    .Average();        // Returns 0

Fun fact: This null behavior matches SQL. SELECT AVG(Price) FROM Products WHERE 1=0 returns NULL, not zero. LINQ chose to throw exceptions because returning a misleading default (like 0 for average) could hide bugs. The nullable overloads give you SQL semantics when you want them.

MinBy and MaxBy: The Element, Not the Value

New in .NET 6, these return the element with the min/max value, not the value itself:

// Old way — two passes
var minPrice = products.Min(p => p.Price);
var cheapest = products.First(p => p.Price == minPrice);

// New way — one pass
var cheapest = products.MinBy(p => p.Price);  // Returns the Product, not the price
var mostExpensive = products.MaxBy(p => p.Price);

MinBy/MaxBy are cleaner and more efficient. In-memory, it's one iteration instead of two.

Note: these are LINQ to Objects only — EF Core doesn't translate them to SQL. Use OrderBy().First() for database:

// EF Core equivalent
var cheapest = await dbContext.Products
    .OrderBy(p => p.Price)
    .FirstOrDefaultAsync();

The General-Purpose Aggregate

Aggregate() is the Swiss Army knife. It takes:

A seed value
A function that combines accumulator + element
(Optionally) a result selector

Build any aggregation you want:

// Sum, implemented manually
var sum = numbers.Aggregate(0m, (acc, n) => acc + n);

// Product (multiply all values)
var product = numbers.Aggregate(1, (acc, n) => acc * n);

// String concatenation
var csv = names.Aggregate("", (acc, name) => acc + "," + name).TrimStart(',');

// Better string concat with StringBuilder
var csv = names.Aggregate(
    new StringBuilder(),
    (sb, name) => sb.Append(name).Append(","),
    sb => sb.ToString().TrimEnd(',')
);

Real-World Aggregate Patterns

Running Total

var runningTotals = transactions
    .Aggregate(
        new List<(string Name, decimal Running)>(),
        (list, t) => 
        {
            var lastTotal = list.LastOrDefault().Running;
            list.Add((t.Name, lastTotal + t.Amount));
            return list;
        }
    );

Nested Object Building

var menuTree = menuItems
    .Where(m => m.ParentId == null)
    .Aggregate(
        new List<MenuNode>(),
        (list, item) => 
        {
            list.Add(BuildNode(item, menuItems));
            return list;
        }
    );

Counting by Condition

var counts = products.Aggregate(
    (Active: 0, Inactive: 0),
    (counts, p) => p.IsActive 
        ? (counts.Active + 1, counts.Inactive) 
        : (counts.Active, counts.Inactive + 1)
);

Console.WriteLine($"Active: {counts.Active}, Inactive: {counts.Inactive}");

Aggregate vs GroupBy for Summaries

Need multiple aggregates by group? GroupBy with Select:

var categoryStats = products
    .GroupBy(p => p.CategoryId)
    .Select(g => new 
    {
        Category = g.Key,
        Count = g.Count(),
        Total = g.Sum(p => p.Price),
        Average = g.Average(p => p.Price),
        Min = g.Min(p => p.Price),
        Max = g.Max(p => p.Price)
    })
    .ToList();

This translates to efficient SQL with GROUP BY and aggregate functions.

LongCount for Big Data

Count() returns int. If you have more than 2.1 billion items (rare, but possible):

long count = dbContext.HugeTable.LongCount();

Database vs Memory Aggregation

When aggregating on IQueryable, EF executes in the database:

// Executed in SQL — efficient
var dbSum = dbContext.Products.Sum(p => p.Price);

If you materialize first, aggregation happens in memory:

// Executed in C# — pulls all data first
var memSum = dbContext.Products.ToList().Sum(p => p.Price);

For large datasets, database aggregation is dramatically faster. Always aggregate before .ToList() when possible.

The Rule

Aggregate in database — call Sum/Count on IQueryable, not after ToList()
Watch for nulls — use nullable overloads for Min/Max/Average on possibly-empty sequences
Need the element? Use MinBy/MaxBy (.NET 6+) or OrderBy().First()
Custom reduction? Aggregate() can implement anything
Per-group stats? GroupBy + aggregate functions

Next time, we'll tackle EF Core's raw SQL escape hatch — FromSqlRaw, ExecuteSqlRaw, and when to break out of LINQ entirely. Hope to see you!

Async LINQ: Why ToListAsync Exists (And Why WhereAsync Doesn't)

Nick — Mon, 28 Sep 2026 18:34:00 +0000

Async LINQ: Why ToListAsync Exists (And Why WhereAsync Doesn't)

You write await dbContext.Products.ToListAsync(). Good. Then you wonder: where's WhereAsync? Why isn't await dbContext.Products.WhereAsync(p => p.Price > 100)?

The answer reveals something fundamental about how LINQ actually works.

Where the Async Boundary Lives

LINQ operators like Where, Select, OrderBy don't execute anything. They build a query. An expression tree. A recipe.

The execution happens at the terminal operator: ToList, First, Count, Any. That's when bytes flow over the network. That's when you want async.

// Building the query — instant, no I/O
var query = dbContext.Products
    .Where(p => p.Price > 100)      // No execution
    .OrderBy(p => p.Name)           // Still no execution
    .Select(p => new { p.Name });   // Still building

// Actually executing — this is the I/O
var results = await query.ToListAsync();  // NOW it hits the database

There's no WhereAsync because Where doesn't touch the database. Making it async would be meaningless.

The Async Terminal Operators

EF Core provides async versions of all execution-triggering operators:

// Materialization
await query.ToListAsync();
await query.ToArrayAsync();
await query.ToDictionaryAsync(p => p.Id);

// Aggregation
await query.CountAsync();
await query.SumAsync(p => p.Price);
await query.AverageAsync(p => p.Price);
await query.MaxAsync(p => p.Price);

// Element access
await query.FirstAsync();
await query.FirstOrDefaultAsync();
await query.SingleAsync();
await query.SingleOrDefaultAsync();

// Existence
await query.AnyAsync();
await query.AnyAsync(p => p.Stock > 0);
await query.AllAsync(p => p.Price > 0);

// Finding
await query.FindAsync(id);  // DbSet only — uses primary key

Each one triggers actual database communication. Each one should be awaited.

The Streaming Async Pattern

ToListAsync waits for all rows before returning. For large result sets, you might want to process as you receive:

await foreach (var product in dbContext.Products.AsAsyncEnumerable())
{
    await ProcessProductAsync(product);
}

AsAsyncEnumerable() returns an IAsyncEnumerable<T> — each iteration asynchronously fetches the next item. You don't hold the entire result set in memory.

Fun fact: IAsyncEnumerable<T> was added in C# 8 (2019), but the concept of async iteration goes back to Rx (Reactive Extensions) from 2009. Rx's IObservable<T> pushed data to you; IAsyncEnumerable<T> lets you pull. Ten years from push to pull.

The ConfigureAwait Question

In library code, you often see:

var results = await dbContext.Products
    .ToListAsync()
    .ConfigureAwait(false);

ConfigureAwait(false) tells the await not to capture the synchronization context. In ASP.NET Core, this doesn't matter (there's no sync context). In desktop/UI apps or older ASP.NET, it prevents deadlocks and improves performance.

Rule of thumb:

Application code? Skip ConfigureAwait
Library code? Add ConfigureAwait(false) everywhere

Async Pitfalls

Mixing Sync and Async

// Bad — blocks the thread
var products = dbContext.Products.ToList();  // Sync
await SomeOtherAsyncMethod();

// Good — stays async
var products = await dbContext.Products.ToListAsync();
await SomeOtherAsyncMethod();

One sync call in an async method can negate the benefits. Under load, thread pool starvation follows.

The False Async

// This is NOT async — the query executes in the Task.Run
var results = await Task.Run(() => dbContext.Products.ToList());

Task.Run offloads to a thread pool thread. You're still blocking a thread, just a different one. True async (ToListAsync) releases the thread during I/O.

Parallel Execution Trap

// Dangerous — DbContext is not thread-safe!
var task1 = dbContext.Products.CountAsync();
var task2 = dbContext.Orders.CountAsync();
await Task.WhenAll(task1, task2);  // Concurrent access to same context!

DbContext isn't thread-safe. For parallel queries, use separate contexts:

await using var context1 = new AppDbContext();
await using var context2 = new AppDbContext();

var task1 = context1.Products.CountAsync();
var task2 = context2.Orders.CountAsync();
await Task.WhenAll(task1, task2);  // Safe — separate contexts

Cancellation Tokens

All async operators accept cancellation:

public async Task<List<Product>> GetProductsAsync(CancellationToken ct)
{
    return await dbContext.Products
        .Where(p => p.IsActive)
        .ToListAsync(ct);  // Cancellable
}

When the token cancels, EF aborts the database command. Always wire cancellation through from your ASP.NET controller:

[HttpGet]
public async Task<IActionResult> GetProducts(CancellationToken ct)
{
    var products = await _productService.GetProductsAsync(ct);
    return Ok(products);
}

The Async LINQ to Objects Gap

Standard LINQ to Objects (in-memory) doesn't have async operators. If you're filtering an in-memory list, it's just:

var filtered = products.Where(p => p.Price > 100).ToList();

No await needed — no I/O involved. The async variants are for I/O-bound operations (database, file, network), not CPU-bound in-memory work.

The Rule

Query building (Where, Select) — no async needed
Query execution (ToList, First, Count) — use async versions
Large results? AsAsyncEnumerable() for streaming
Parallel queries? Separate DbContext per task
Always pass CancellationToken to async EF operations

Next time, we'll explore LINQ's aggregate operators — Sum, Average, Max, and the lesser-known Aggregate that lets you define custom aggregations. Hope to see you!

The N+1 Query Problem: Why Your LINQ Query Makes 1001 Database Calls

Nick — Fri, 25 Sep 2026 05:42:00 +0000

The N+1 Query Problem: Why Your LINQ Query Makes 1001 Database Calls

Your query returns 1000 orders. Your database log shows 1001 queries. One to fetch the orders, then one for each order's customer. This is the N+1 problem — and it's probably happening in your codebase right now.

The Innocent-Looking Code

var orders = dbContext.Orders.ToList();

foreach (var order in orders)
{
    Console.WriteLine($"Order {order.Id} by {order.Customer.Name}");
}

What you expect: one query fetching orders with customer names.

What happens:

SELECT * FROM Orders — fetches 1000 orders
SELECT * FROM Customers WHERE Id = 1 — for order 1
SELECT * FROM Customers WHERE Id = 2 — for order 2
... 998 more queries ...

If each query takes 5ms, you've turned a 10ms operation into a 5-second disaster.

Why It Happens

Entity Framework uses lazy loading by default (or used to — EF Core made it opt-in). When you access order.Customer, EF sees the navigation property isn't loaded and fires a query.

The insidious part: it works perfectly in development with 10 records. In production with 10,000? Your API times out.

The Solution: Eager Loading with Include

var orders = dbContext.Orders
    .Include(o => o.Customer)  // Load customers with orders
    .ToList();

foreach (var order in orders)
{
    Console.WriteLine($"Order {order.Id} by {order.Customer.Name}");
}

Now EF generates one query with a JOIN. Customer data comes with the order in a single round-trip.

Nested Includes

For deeper relationships:

var orders = dbContext.Orders
    .Include(o => o.Customer)
    .Include(o => o.OrderItems)
        .ThenInclude(oi => oi.Product)  // Nested: OrderItem's Product
    .ToList();

ThenInclude continues from the last included collection. You can chain them for complex graphs.

The Projection Alternative

Include loads entire entities. Often, you don't need all columns:

// Instead of Include...
var orders = dbContext.Orders
    .Include(o => o.Customer)
    .Include(o => o.OrderItems)
    .ToList();

// ...project to what you actually need
var orders = dbContext.Orders
    .Select(o => new 
    {
        o.Id,
        CustomerName = o.Customer.Name,
        ItemCount = o.OrderItems.Count,
        Total = o.OrderItems.Sum(oi => oi.Price)
    })
    .ToList();

The projection approach generates one query that fetches exactly the columns needed. No N+1, minimal data transfer.

Fun fact: The term "N+1" comes from the mathematical pattern: 1 query for the parent collection, plus N queries for each child. It was popularized by the Ruby on Rails community around 2006, but the problem existed anywhere ORMs did automatic lazy loading. The ActiveRecord pattern (from Martin Fowler's 2002 book) made it endemic.

AsSplitQuery: When Include Gets Too Wide

Wide includes (many navigation properties or deep nesting) produce massive JOINs. EF Core 5+ offers split queries:

var orders = dbContext.Orders
    .Include(o => o.OrderItems)
    .Include(o => o.ShippingAddress)
    .Include(o => o.BillingAddress)
    .AsSplitQuery()  // Separate queries instead of mega-JOIN
    .ToList();

AsSplitQuery fires multiple simple queries instead of one giant JOIN. For complex includes with many columns, split queries can actually be faster.

Trade-off: multiple round-trips vs one huge result set. Profile to decide.

Detecting N+1 in the Wild

Option 1: EF Core Logging

optionsBuilder.LogTo(Console.WriteLine, LogLevel.Information);

Watch the output. Multiple SELECT statements for navigation properties = N+1.

Option 2: Simple Query Count

int queryCount = 0;
optionsBuilder.LogTo(msg => 
{
    if (msg.Contains("Executing DbCommand")) queryCount++;
}, LogLevel.Information);

If queryCount >> 1 for what should be one query, investigate.

Option 3: Third-Party Tools

MiniProfiler, Glimpse, or similar tools visualize queries per request. N+1 shows as a fan of identical query patterns.

Common N+1 Patterns

The Loop Print

// Bad
foreach (var order in dbContext.Orders)
    Console.WriteLine(order.Customer.Name);  // N queries

// Good
foreach (var order in dbContext.Orders.Include(o => o.Customer))
    Console.WriteLine(order.Customer.Name);  // 1 query

The Innocent Serialize

// Bad — serializer triggers lazy loading
return Json(dbContext.Orders.ToList());  // Triggers N+1 during serialization

// Good
return Json(dbContext.Orders
    .Select(o => new { o.Id, CustomerName = o.Customer.Name })
    .ToList());

The Collection Count

// Bad
var orders = dbContext.Orders.ToList();
var withItems = orders.Where(o => o.OrderItems.Any());  // N queries

// Good
var orders = dbContext.Orders.Include(o => o.OrderItems).ToList();
var withItems = orders.Where(o => o.OrderItems.Any());  // 0 additional queries

The Rule

Accessing navigation in loops? Include before iterating
Don't need full entities? Project to DTOs (avoids N+1 and fetches less data)
Wide includes? Consider AsSplitQuery
Serialize entities? Project to anonymous/DTO first
Always profile — EF logging reveals the real query count

Next time, we'll look at async LINQ — when to use ToListAsync, why you can't make LINQ operators truly async, and the Task patterns that work with database queries. Hope to see you!

LINQ Joins: The Operator You Probably Don't Need

Nick — Thu, 24 Sep 2026 06:42:00 +0000

LINQ Joins: The Operator You Probably Don't Need

In SQL, you write JOINs constantly. In LINQ with Entity Framework, explicit joins are often a code smell.

Let me explain why — and when you actually need them.

Navigation Properties: The EF Way

If you have proper relationships configured:

public class Order
{
    public int Id { get; set; }
    public int CustomerId { get; set; }
    public Customer Customer { get; set; }  // Navigation property
}

public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }
    public List<Order> Orders { get; set; }  // Collection navigation
}

You don't need Join. Just navigate:

// EF automatically generates the JOIN
var orderDetails = dbContext.Orders
    .Where(o => o.Customer.Name.StartsWith("A"))  // Navigate through relationship
    .Select(o => new 
    {
        o.Id,
        CustomerName = o.Customer.Name  // EF handles the JOIN
    })
    .ToList();

SQL generated:

SELECT o.Id, c.Name as CustomerName
FROM Orders o
INNER JOIN Customers c ON o.CustomerId = c.Id
WHERE c.Name LIKE 'A%'

Clean. No explicit join syntax.

When You Actually Need Join

Explicit Join makes sense when:

No navigation property — raw tables without relationships
In-memory collections — joining two lists
Non-key relationships — joining on arbitrary columns

Scenario 1: No Relationship Configured

var orderWithCustomer = dbContext.Orders
    .Join(
        dbContext.Customers,
        order => order.CustomerId,      // Outer key
        customer => customer.Id,        // Inner key
        (order, customer) => new        // Result selector
        {
            OrderId = order.Id,
            CustomerName = customer.Name
        }
    )
    .ToList();

The syntax is verbose — four parameters. This is why navigation properties are preferred.

Scenario 2: In-Memory Collections

var products = GetProducts();
var prices = GetPriceUpdates();

var updated = products
    .Join(
        prices,
        p => p.Sku,
        u => u.Sku,
        (product, update) => new 
        {
            product.Name,
            OldPrice = product.Price,
            NewPrice = update.Price
        }
    )
    .ToList();

For in-memory joins, there's no EF to generate SQL. The Join operator uses hash-based matching — efficient even on large collections.

Fun fact: LINQ's Join uses a hash join algorithm internally. It builds a lookup table from the inner sequence, then probes it for each outer element. Time complexity: O(n + m) instead of O(n × m) for nested loops. The same optimization that makes database joins fast.

Left Join: The GroupJoin Pattern

LINQ's Join is an inner join — no match, no result. For left joins, you need GroupJoin:

var customersWithOrders = dbContext.Customers
    .GroupJoin(
        dbContext.Orders,
        customer => customer.Id,
        order => order.CustomerId,
        (customer, orders) => new 
        {
            customer.Name,
            OrderCount = orders.Count(),
            Orders = orders.ToList()
        }
    )
    .ToList();

GroupJoin gives you each customer with a collection of their orders (possibly empty).

For the classic LEFT JOIN behavior (one row per result, null if no match):

var leftJoin = dbContext.Customers
    .GroupJoin(
        dbContext.Orders,
        c => c.Id,
        o => o.CustomerId,
        (customer, orders) => new { customer, orders }
    )
    .SelectMany(
        x => x.orders.DefaultIfEmpty(),  // Flatten, keeping nulls
        (x, order) => new 
        {
            CustomerName = x.customer.Name,
            OrderId = order != null ? order.Id : (int?)null
        }
    )
    .ToList();

Ugly, right? This is why navigation properties with .Include() are the EF way.

The Query Syntax Alternative

Join looks cleaner in query syntax:

var result = 
    from order in dbContext.Orders
    join customer in dbContext.Customers 
        on order.CustomerId equals customer.Id
    select new { order.Id, customer.Name };

// Left join in query syntax
var leftJoin = 
    from customer in dbContext.Customers
    join order in dbContext.Orders 
        on customer.Id equals order.CustomerId into orders
    from order in orders.DefaultIfEmpty()
    select new { customer.Name, OrderId = order.Id };

The into keyword creates a grouped join. The second from with DefaultIfEmpty() flattens it into a left join.

Multiple Joins

Chaining joins gets messy fast:

var result = dbContext.Orders
    .Join(dbContext.Customers, o => o.CustomerId, c => c.Id, (o, c) => new { o, c })
    .Join(dbContext.Products, oc => oc.o.ProductId, p => p.Id, (oc, p) => new 
    {
        OrderId = oc.o.Id,
        CustomerName = oc.c.Name,
        ProductName = p.Name
    })
    .ToList();

Versus navigation properties:

var result = dbContext.Orders
    .Select(o => new 
    {
        o.Id,
        CustomerName = o.Customer.Name,
        ProductName = o.Product.Name
    })
    .ToList();

Night and day.

The Rule

Have navigation properties? Just navigate — EF handles joins
In-memory collections? Join is the right tool
Need left join in EF? Use GroupJoin + SelectMany, or include nullable navigation
Complex multi-table queries? Query syntax is more readable than chained Join calls

Next time, we'll explore the dangerous world of N+1 queries — the silent performance killer that makes your 1-second query take 30 seconds. Hope to see you!

LINQ GroupBy: The Operator Everyone Uses Wrong

Nick — Wed, 23 Sep 2026 07:41:00 +0000

LINQ GroupBy: The Operator Everyone Uses Wrong

GroupBy in LINQ looks like SQL GROUP BY. It isn't. At least, not in the way you'd expect.

When you grasp the difference, you stop fighting the operator and start wielding it.

The SQL Mental Model (That Misleads You)

In SQL, GROUP BY collapses rows into aggregates:

SELECT CategoryId, COUNT(*) as Count, AVG(Price) as AvgPrice
FROM Products
GROUP BY CategoryId

Result: one row per category with aggregate values. Clean.

So you write this in LINQ:

var groups = dbContext.Products
    .GroupBy(p => p.CategoryId)
    .ToList();

And you expect... what? Aggregates? No. You get a list of IGrouping<int, Product> objects. Each grouping has a Key (the CategoryId) and contains all the products in that category.

foreach (var group in groups)
{
    Console.WriteLine($"Category {group.Key}:");
    foreach (var product in group)  // Iterate individual products
    {
        Console.WriteLine($"  - {product.Name}");
    }
}

LINQ's GroupBy is more powerful than SQL's — it doesn't force you to aggregate. But that power confuses SQL-trained developers.

The Aggregation Pattern

To get SQL-like grouped aggregates, you combine GroupBy with Select:

var categoryStats = dbContext.Products
    .GroupBy(p => p.CategoryId)
    .Select(g => new 
    {
        CategoryId = g.Key,
        Count = g.Count(),
        AvgPrice = g.Average(p => p.Price),
        MaxPrice = g.Max(p => p.Price)
    })
    .ToList();

Now you get one row per category with computed values. This is what translates cleanly to SQL.

The Database vs In-Memory Split

Here's where it gets tricky. Not all GroupBy operations translate to SQL:

// This works in SQL
var simple = dbContext.Products
    .GroupBy(p => p.CategoryId)
    .Select(g => new { g.Key, Count = g.Count() })
    .ToList();

// This might NOT work in SQL (depending on EF version)
var complex = dbContext.Products
    .GroupBy(p => p.CategoryId)
    .Select(g => new 
    { 
        g.Key, 
        Products = g.ToList()  // Trying to materialize full objects in group
    })
    .ToList();

EF Core has historically struggled with "group then expand" patterns. The workaround is to group in memory:

var products = await dbContext.Products.ToListAsync();
var grouped = products
    .GroupBy(p => p.CategoryId)
    .Select(g => new { CategoryId = g.Key, Products = g.ToList() })
    .ToList();

Fun fact: SQL's GROUP BY was standardized in SQL-86 (yes, 1986). But the concept of grouping as collections — where each group is a container you can enumerate — comes from functional programming. LINQ merged both worlds, which is why it feels different from either pure SQL or pure functional code.

Grouping by Multiple Keys

Group by a composite key using anonymous types:

var grouped = dbContext.Products
    .GroupBy(p => new { p.CategoryId, p.SupplierId })
    .Select(g => new 
    {
        g.Key.CategoryId,
        g.Key.SupplierId,
        Count = g.Count()
    })
    .ToList();

The anonymous type becomes your grouping key. All properties must match for items to land in the same group.

GroupBy with Element Selector

You can transform elements during grouping:

// Group products, but only keep their names
var productNamesByCategory = products
    .GroupBy(
        p => p.CategoryId,           // Key selector
        p => p.Name                   // Element selector
    );

foreach (var group in productNamesByCategory)
{
    Console.WriteLine($"Category {group.Key}: {string.Join(", ", group)}");
}

The Lookup Alternative

When you'll access groups multiple times by key, ToLookup is more efficient:

var lookup = products.ToLookup(p => p.CategoryId);

// Direct key access — O(1) instead of iterating
var electronics = lookup[5];
var clothing = lookup[7];

ToLookup is like GroupBy but immediately materialized into a dictionary-like structure. Unlike Dictionary, it returns an empty collection for missing keys instead of throwing.

The "I Want This But Grouped" Pattern

Common scenario: you have a flat list, need it grouped for display:

var ordersByMonth = orders
    .GroupBy(o => new { o.OrderDate.Year, o.OrderDate.Month })
    .OrderByDescending(g => g.Key.Year)
    .ThenByDescending(g => g.Key.Month)
    .Select(g => new 
    {
        Period = $"{g.Key.Year}-{g.Key.Month:D2}",
        Orders = g.OrderBy(o => o.OrderDate).ToList(),
        Total = g.Sum(o => o.Amount)
    })
    .ToList();

The Rule

For aggregates only? GroupBy + Select with aggregation functions — translates to SQL
Need full objects per group? Materialize first, then GroupBy in memory
Multiple key access? Use ToLookup instead of GroupBy
Composite keys? Anonymous types with all grouping columns

Next time, we'll dive into Join operations — the LINQ equivalent of SQL JOINs, and why they're often unnecessary when you have proper navigation properties. Hope to see you!

Select Like a Surgeon: LINQ Projection That Doesn't Bleed Data

Nick — Tue, 22 Sep 2026 06:21:00 +0000

Select Like a Surgeon: LINQ Projection That Doesn't Bleed Data

Your entity has 47 columns. Your UI needs 3. You're loading all 47 every single time.

This is the silent performance killer in most .NET applications — and Select is the scalpel that fixes it.

The Problem With Loading Everything

var products = dbContext.Products
    .Where(p => p.CategoryId == 5)
    .ToList();

// In the view, you only use:
foreach (var p in products)
{
    Console.WriteLine($"{p.Name}: {p.Price}");
}

Entity Framework dutifully loads every column: Id, Name, Description, Price, Cost, Stock, SupplierId, CreatedAt, ModifiedAt, ImageBlob, Specifications... the full 47-column monstrosity.

You used two.

The Select Fix

var products = dbContext.Products
    .Where(p => p.CategoryId == 5)
    .Select(p => new { p.Name, p.Price })  // Only these two columns
    .ToList();

Generated SQL:

SELECT Name, Price FROM Products WHERE CategoryId = 5

Less data transferred. Faster query. Lower memory. Same result.

Anonymous Types vs DTOs

For quick projections, anonymous types work great:

var results = query.Select(p => new { p.Name, p.Price, p.Stock });

But you can't return anonymous types from methods. For that, create a DTO:

public record ProductSummary(string Name, decimal Price);

public List<ProductSummary> GetSummaries()
{
    return dbContext.Products
        .Select(p => new ProductSummary(p.Name, p.Price))
        .ToList();
}

Fun fact: The record keyword in C# 9+ generates Equals, GetHashCode, and ToString automatically — perfect for DTOs. Before records, we wrote 30 lines of boilerplate for what's now a single line.

Projection With Computed Values

Select isn't just for cherry-picking columns — you can compute:

var analysis = dbContext.Products
    .Select(p => new 
    {
        p.Name,
        Profit = p.Price - p.Cost,
        InStock = p.Stock > 0,
        DisplayPrice = "$" + p.Price.ToString("F2")  // Careful!
    })
    .ToList();

Wait — that DisplayPrice line. Will it run in SQL or C#?

The answer: EF Core will try to translate it. Simple string operations often work. Complex formatting might fail. When in doubt, project the raw value and format after materialization.

Nested Projections

Here's where it gets powerful. Related entities without loading full objects:

var orders = dbContext.Orders
    .Select(o => new 
    {
        o.OrderNumber,
        CustomerName = o.Customer.Name,  // Traverses relationship
        Items = o.OrderItems.Select(i => new 
        {
            i.Product.Name,
            i.Quantity,
            Total = i.Quantity * i.UnitPrice
        }).ToList()
    })
    .ToList();

This generates a single efficient query with JOINs — no N+1 problem, no loading full Customer or Product entities.

The Conditional Projection

Sometimes you need different shapes based on conditions:

var products = dbContext.Products
    .Select(p => new 
    {
        p.Name,
        p.Price,
        Status = p.Stock > 10 ? "Available" 
               : p.Stock > 0 ? "Low Stock" 
               : "Out of Stock"
    })
    .ToList();

This ternary logic translates to SQL CASE WHEN statements.

SelectMany: Flattening Nested Collections

When each source element has a collection, and you want a flat result:

// Each order has multiple items. Get all items across all orders:
var allItems = dbContext.Orders
    .SelectMany(o => o.OrderItems)  // Flatten
    .Select(i => new { i.ProductId, i.Quantity })
    .ToList();

SelectMany is the LINQ equivalent of SQL's implicit join that expands rows.

The Rule

Always project when you don't need all columns
Project early in the query chain — before ToList()
Use DTOs when returning from methods or crossing boundaries
Nest projections to avoid N+1 and reduce transferred data
Test complex expressions — not everything translates to SQL

Next time, we'll tackle the GroupBy operator — the one LINQ feature that's simultaneously powerful and confusing. We'll see how it differs between in-memory and database execution, and why your grouping query might not behave as expected. Hope to see you!

IQueryable vs IEnumerable: The Line Between C# and SQL

Nick — Mon, 21 Sep 2026 06:19:00 +0000

IQueryable vs IEnumerable: The Line Between C# and SQL

You're filtering a million records. With one interface, the database does the work. With the other, your application drowns in memory. Same LINQ syntax. Wildly different execution.

Let's demystify where C# ends and SQL begins.

The Two Worlds

IEnumerable<Product> inMemory = products.Where(p => p.Price > 100);
IQueryable<Product> inDatabase = dbContext.Products.Where(p => p.Price > 100);

Same Where. Same lambda. Completely different animals.

IEnumerable — evaluates in your app's memory using delegates
IQueryable — builds an expression tree, translated to SQL by the database provider

The Million-Row Mistake

Watch this common anti-pattern:

public IEnumerable<Product> GetExpensive()
{
    return dbContext.Products;  // Oops — cast to IEnumerable
}

// Somewhere else
var results = GetExpensive()
    .Where(p => p.Price > 100)  // This runs IN MEMORY
    .Take(10)
    .ToList();

The Where looks innocent. But because we returned IEnumerable, the database returns all products first, then C# filters them. On a million rows, your server weeps.

The fix:

public IQueryable<Product> GetExpensive()
{
    return dbContext.Products;  // Keep IQueryable
}

// Now filtering happens at the database
var results = GetExpensive()
    .Where(p => p.Price > 100)  // SQL: WHERE Price > 100
    .Take(10)                    // SQL: TOP 10
    .ToList();

Expression Trees: The Magic Behind IQueryable

When you write .Where(p => p.Price > 100) on an IQueryable, the lambda isn't compiled to IL code. Instead, it becomes a data structure — an expression tree:

GreaterThan
├── MemberAccess: p.Price  
└── Constant: 100

Entity Framework walks this tree and generates:

SELECT * FROM Products WHERE Price > 100

Fun fact: the LINQ expression tree system was so powerful that other languages adopted it. F# got quotations, and JavaScript transpilers like TypeScript use similar AST concepts. Your LINQ query is code describing code.

The Boundary: Where IQueryable Fails

Not everything translates to SQL:

// This breaks at runtime
var results = dbContext.Products
    .Where(p => MyCustomMethod(p.Name))  // Can't translate to SQL!
    .ToList();

IQueryable doesn't know what MyCustomMethod does — it can only translate standard LINQ operators and expressions the provider understands.

Two solutions:

// Option 1: Filter in the database first, then apply custom logic
var results = dbContext.Products
    .Where(p => p.Price > 100)        // Runs in SQL
    .AsEnumerable()                    // Switch to in-memory
    .Where(p => MyCustomMethod(p.Name)) // Runs in C#
    .ToList();

// Option 2: Project to anonymous type first
var results = dbContext.Products
    .Where(p => p.Price > 100)
    .Select(p => new { p.Name, p.Price })  // Only fetch needed columns
    .AsEnumerable()
    .Where(x => MyCustomMethod(x.Name))
    .ToList();

The Rule of Thumb

Keep IQueryable as long as possible — push filtering to the database
Use AsEnumerable() or ToList() only when you need C#-specific logic
Return IQueryable from repository methods when callers need to add conditions
Return IEnumerable (or materialized List) when results are final

Quick Test: Where Does This Run?

dbContext.Products
    .Where(p => p.Price > 100)        // ?
    .OrderBy(p => p.Name)             // ?
    .ToList()                          // ?
    .Where(p => p.Stock > 0);         // ?

First three: SQL. Last one: C# (after ToList() materializes).

Next time we'll explore the dark art of projection — turning database rows into exactly the shape you need while minimizing data transfer. Hope to see you!

The LINQ Query That Never Runs (Until You Force It)

Nick — Fri, 18 Sep 2026 09:19:22 +0000

The LINQ Query That Never Runs (Until You Force It)

You write a LINQ query. It compiles. It looks correct. But something is off — it executes at the weirdest times, or worse, executes multiple times when you expected once.

Welcome to deferred execution — LINQ's most misunderstood feature.

The Illusion of Immediate

Most developers assume this runs immediately:

var filtered = products.Where(p => p.Price > 100);

It doesn't. Not a single item is touched. The filtered variable holds a recipe, not a meal. The actual filtering happens only when you iterate — foreach, ToList(), Count(), First().

Here's where it gets interesting: LINQ remembers the source, not the snapshot.

var numbers = new List<int> { 1, 2, 3 };
var doubled = numbers.Select(n => n * 2);

numbers.Add(4);  // Modify the source

foreach (var n in doubled)
    Console.WriteLine(n);  // Prints 2, 4, 6, 8 — includes the new element!

The query re-evaluates against the current state of numbers. Every. Single. Time.

The Multiple Execution Trap

This pattern destroys performance in real codebases:

var expensive = GetProducts()
    .Where(p => p.Category == "Electronics")
    .Select(p => new { p.Name, p.Price });

// First enumeration — database hit
var count = expensive.Count();

// Second enumeration — ANOTHER database hit
var first = expensive.First();

// Third enumeration — you get the idea
foreach (var item in expensive) { }

Three database calls. Same query. Because expensive is a recipe, not results.

Force Immediate Execution

The fix is simple — materialize when you need stable data:

var products = GetProducts()
    .Where(p => p.Category == "Electronics")
    .ToList();  // Execute NOW, store results

var count = products.Count;     // Just counts the list
var first = products.First();   // Just reads from memory

ToList(), ToArray(), ToDictionary() — these force execution and store results.

Fun Fact

The term "deferred execution" comes from functional programming's "lazy evaluation" — a concept dating back to 1976 with SASL language. LINQ brought this academic concept into mainstream C# development in 2007, and developers have been confused by it ever since.

When Deferred Execution Shines

It's not a bug — it's a feature. Consider:

IEnumerable<Product> BuildQuery(bool includeInactive)
{
    var query = products.Where(p => p.Price > 0);

    if (!includeInactive)
        query = query.Where(p => p.IsActive);

    return query.OrderBy(p => p.Name);
}

You can compose queries conditionally before execution. The database sees one optimized query, not three separate ones.

The Rule

Building a query? Let it stay deferred — compose freely.
Passing results around? Materialize with ToList().
Multiple enumerations? Materialize first.
Single enumeration? Deferred is fine.

Next time, we'll look at how IQueryable takes deferred execution even further — translating C# into SQL you never wrote. Hope to see you!

DelegatingHandler Magic: Build a Pipeline for Your HTTP Calls

Nick — Thu, 10 Sep 2026 10:14:00 +0000

What's up, .NET devs! 👋

Today I want to show you one of the most underrated features in HttpClient — the DelegatingHandler pipeline. It's like middleware for your HTTP calls!

What's a DelegatingHandler?

Think of DelegatingHandler as a chain of interceptors. Every HTTP request passes through each handler, and every response comes back through the same chain (in reverse).

🔗 Fun Fact: The handler pipeline is modeled after the "Chain of Responsibility" design pattern from the Gang of Four book!

Request:  Client → Handler1 → Handler2 → Handler3 → Network
Response: Client ← Handler1 ← Handler2 ← Handler3 ← Network

Your First Handler: Logging

public class LoggingHandler : DelegatingHandler
{
    private readonly ILogger<LoggingHandler> _logger;

    public LoggingHandler(ILogger<LoggingHandler> logger)
    {
        _logger = logger;
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var stopwatch = Stopwatch.StartNew();

        _logger.LogInformation("→ {Method} {Uri}", request.Method, request.RequestUri);

        var response = await base.SendAsync(request, cancellationToken);

        stopwatch.Stop();

        _logger.LogInformation("← {StatusCode} in {ElapsedMs}ms",
            (int)response.StatusCode, stopwatch.ElapsedMilliseconds);

        return response;
    }
}

Register It

services.AddTransient<LoggingHandler>();

services.AddHttpClient<IMyApiClient, MyApiClient>()
    .AddHttpMessageHandler<LoggingHandler>();

Correlation ID Propagation

In microservices, trace requests across services:

public class CorrelationIdHandler : DelegatingHandler
{
    private readonly IHttpContextAccessor _httpContextAccessor;

    public CorrelationIdHandler(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var correlationId = _httpContextAccessor.HttpContext?
            .Request.Headers["X-Correlation-ID"].FirstOrDefault()
            ?? Guid.NewGuid().ToString();

        request.Headers.Add("X-Correlation-ID", correlationId);
        return base.SendAsync(request, cancellationToken);
    }
}

Auth Token Injection

Never manually add auth headers again:

public class AuthTokenHandler : DelegatingHandler
{
    private readonly ITokenService _tokenService;

    public AuthTokenHandler(ITokenService tokenService)
    {
        _tokenService = tokenService;
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var token = await _tokenService.GetAccessTokenAsync(cancellationToken);
        request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
        return await base.SendAsync(request, cancellationToken);
    }
}

Metrics Collection

public class MetricsHandler : DelegatingHandler
{
    private readonly IMeterFactory _meterFactory;
    private readonly Histogram<double> _requestDuration;

    public MetricsHandler(IMeterFactory meterFactory)
    {
        var meter = meterFactory.Create("HttpClient.Metrics");
        _requestDuration = meter.CreateHistogram<double>("http_request_duration_ms");
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var stopwatch = Stopwatch.StartNew();
        var tags = new TagList
        {
            { "method", request.Method.Method },
            { "host", request.RequestUri?.Host ?? "unknown" }
        };

        try
        {
            var response = await base.SendAsync(request, cancellationToken);
            tags.Add("status_code", ((int)response.StatusCode).ToString());
            return response;
        }
        finally
        {
            stopwatch.Stop();
            _requestDuration.Record(stopwatch.Elapsed.TotalMilliseconds, tags);
        }
    }
}

Chaining Multiple Handlers

services.AddTransient<LoggingHandler>();
services.AddTransient<CorrelationIdHandler>();
services.AddTransient<AuthTokenHandler>();
services.AddTransient<MetricsHandler>();

services.AddHttpClient<IMyApiClient, MyApiClient>()
    .AddHttpMessageHandler<CorrelationIdHandler>()  // First in, last out
    .AddHttpMessageHandler<AuthTokenHandler>()
    .AddHttpMessageHandler<LoggingHandler>()
    .AddHttpMessageHandler<MetricsHandler>();       // Last in, first out

🎯 Pro Tip: Order Matters!

Handlers execute in the order you add them for requests, and reverse order for responses.

Logging should be first (sees raw request) or last (sees final request)
Auth should be after correlation ID
Retry policies (Polly) should wrap most other handlers

Wrapping Up

DelegatingHandler is your Swiss Army knife for HTTP concerns. Instead of polluting your business logic with logging, auth, and metrics — put it in handlers!

Happy pipelining! 🚀

HttpClient Observability: Logs, Metrics, and Traces Done Right

Nick — Wed, 09 Sep 2026 08:13:00 +0000

Hey observability enthusiasts! 👋

Your HttpClient is making calls, but can you see what's happening? Today we'll add proper logging, metrics, and distributed tracing to your HTTP calls!

The Built-in Logging

.NET's IHttpClientFactory includes logging out of the box. Just configure your log levels:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "System.Net.Http.HttpClient": "Information"
    }
  }
}

This gives you:

info: System.Net.Http.HttpClient.MyClient.LogicalHandler[100]
      Start processing HTTP request GET https://api.example.com/users
info: System.Net.Http.HttpClient.MyClient.ClientHandler[100]
      Sending HTTP request GET https://api.example.com/users
info: System.Net.Http.HttpClient.MyClient.ClientHandler[101]
      Received HTTP response headers after 145.2ms - 200

🔍 Fun Fact: HttpClient emits events at two levels: LogicalHandler (before/after your handlers) and ClientHandler (actual network call). This helps you see what your handlers add!

Custom Logging Handler

Want more control? Build a logging handler:

public class HttpLoggingHandler : DelegatingHandler
{
    private readonly ILogger<HttpLoggingHandler> _logger;

    public HttpLoggingHandler(ILogger<HttpLoggingHandler> logger)
    {
        _logger = logger;
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var requestId = Guid.NewGuid().ToString("N")[..8];
        var sw = Stopwatch.StartNew();

        _logger.LogInformation(
            "[{RequestId}] → {Method} {Uri}",
            requestId,
            request.Method,
            request.RequestUri);

        if (request.Content != null && _logger.IsEnabled(LogLevel.Debug))
        {
            var body = await request.Content.ReadAsStringAsync(cancellationToken);
            _logger.LogDebug("[{RequestId}] Request body: {Body}", requestId, body);
        }

        try
        {
            var response = await base.SendAsync(request, cancellationToken);
            sw.Stop();

            _logger.LogInformation(
                "[{RequestId}] ← {StatusCode} in {ElapsedMs}ms",
                requestId,
                (int)response.StatusCode,
                sw.ElapsedMilliseconds);

            return response;
        }
        catch (Exception ex)
        {
            sw.Stop();
            _logger.LogError(
                ex,
                "[{RequestId}] ✗ Failed after {ElapsedMs}ms: {Message}",
                requestId,
                sw.ElapsedMilliseconds,
                ex.Message);
            throw;
        }
    }
}

Metrics with .NET 8

.NET 8 introduced native metrics for HttpClient:

public class HttpMetricsHandler : DelegatingHandler
{
    private readonly IMeterFactory _meterFactory;
    private readonly Histogram<double> _duration;
    private readonly Counter<long> _requests;

    public HttpMetricsHandler(IMeterFactory meterFactory)
    {
        var meter = meterFactory.Create("MyApp.HttpClient");

        _duration = meter.CreateHistogram<double>(
            "http.client.request.duration",
            unit: "ms",
            description: "HTTP request duration");

        _requests = meter.CreateCounter<long>(
            "http.client.request.count",
            description: "Total HTTP requests");
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var sw = Stopwatch.StartNew();
        var tags = new TagList
        {
            { "http.method", request.Method.Method },
            { "server.address", request.RequestUri?.Host ?? "unknown" }
        };

        try
        {
            var response = await base.SendAsync(request, cancellationToken);

            tags.Add("http.status_code", ((int)response.StatusCode).ToString());
            tags.Add("http.response.status_code", ((int)response.StatusCode).ToString());

            return response;
        }
        catch (TaskCanceledException)
        {
            tags.Add("error.type", "timeout");
            throw;
        }
        catch (HttpRequestException ex)
        {
            tags.Add("error.type", ex.GetType().Name);
            throw;
        }
        finally
        {
            sw.Stop();
            _duration.Record(sw.Elapsed.TotalMilliseconds, tags);
            _requests.Add(1, tags);
        }
    }
}

Export to Prometheus/OpenTelemetry

builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics =>
    {
        metrics
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation() // Built-in HttpClient metrics!
            .AddMeter("MyApp.HttpClient")   // Our custom metrics
            .AddPrometheusExporter();
    });

Distributed Tracing

Trace requests across services:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing =>
    {
        tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation(options =>
            {
                options.RecordException = true;
                options.EnrichWithHttpRequestMessage = (activity, request) =>
                {
                    activity.SetTag("http.request.body_size", 
                        request.Content?.Headers.ContentLength ?? 0);
                };
                options.EnrichWithHttpResponseMessage = (activity, response) =>
                {
                    activity.SetTag("http.response.body_size",
                        response.Content.Headers.ContentLength ?? 0);
                };
            })
            .AddJaegerExporter();
    });

💡 Pro Tip: EventSource for Deep Debugging

.NET's HttpClient emits detailed events via EventSource:

public class HttpEventListener : EventListener
{
    protected override void OnEventSourceCreated(EventSource eventSource)
    {
        if (eventSource.Name == "System.Net.Http" ||
            eventSource.Name == "System.Net.Sockets" ||
            eventSource.Name == "System.Net.Security")
        {
            EnableEvents(eventSource, EventLevel.Verbose);
        }
    }

    protected override void OnEventWritten(EventWrittenEventArgs eventData)
    {
        Console.WriteLine($"[{eventData.EventSource.Name}] {eventData.EventName}");

        if (eventData.Payload != null)
        {
            for (int i = 0; i < eventData.Payload.Count; i++)
            {
                Console.WriteLine($"  {eventData.PayloadNames?[i]}: {eventData.Payload[i]}");
            }
        }
    }
}

// Enable in your app
using var listener = new HttpEventListener();

This shows DNS resolution, socket connections, TLS handshakes — everything!

Health Checks

Monitor your downstream dependencies:

services.AddHealthChecks()
    .AddUrlGroup(
        new Uri("https://api.example.com/health"),
        name: "example-api",
        tags: new[] { "ready" })
    .AddUrlGroup(
        new Uri("https://api.payment.com/ping"),
        name: "payment-api",
        failureStatus: HealthStatus.Degraded,
        tags: new[] { "ready" });

Custom Health Check

public class ApiHealthCheck : IHealthCheck
{
    private readonly HttpClient _client;

    public ApiHealthCheck(IHttpClientFactory factory)
    {
        _client = factory.CreateClient("HealthCheck");
    }

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            var sw = Stopwatch.StartNew();
            var response = await _client.GetAsync("/health", cancellationToken);
            sw.Stop();

            var data = new Dictionary<string, object>
            {
                { "responseTime", sw.ElapsedMilliseconds },
                { "statusCode", (int)response.StatusCode }
            };

            if (response.IsSuccessStatusCode)
            {
                return sw.ElapsedMilliseconds > 1000
                    ? HealthCheckResult.Degraded("Slow response", data: data)
                    : HealthCheckResult.Healthy("OK", data);
            }

            return HealthCheckResult.Unhealthy(
                $"Status: {response.StatusCode}", 
                data: data);
        }
        catch (Exception ex)
        {
            return HealthCheckResult.Unhealthy(ex.Message, ex);
        }
    }
}

Complete Setup

// Program.cs
builder.Services.AddTransient<HttpLoggingHandler>();
builder.Services.AddTransient<HttpMetricsHandler>();

builder.Services.AddHttpClient<IMyApiClient, MyApiClient>()
    .AddHttpMessageHandler<HttpLoggingHandler>()
    .AddHttpMessageHandler<HttpMetricsHandler>();

builder.Services.AddOpenTelemetry()
    .WithMetrics(m => m
        .AddHttpClientInstrumentation()
        .AddMeter("MyApp.HttpClient")
        .AddPrometheusExporter())
    .WithTracing(t => t
        .AddHttpClientInstrumentation()
        .AddJaegerExporter());

builder.Services.AddHealthChecks()
    .AddCheck<ApiHealthCheck>("downstream-api");

What to Monitor

Metric	Why
Request duration	Spot slowdowns early
Error rate	Know when things break
Status code distribution	Understand failure patterns
Request count	Capacity planning
Connection pool usage	Detect exhaustion

Wrapping Up

Observability isn't optional — it's how you know your app is healthy. With .NET's built-in instrumentation plus custom handlers, you can:

Debug production issues faster
Track SLOs and SLAs
Correlate requests across services
Catch problems before users do

Start with the built-in AddHttpClientInstrumentation() and add custom metrics as needed.

Happy observing! 👁️🚀