DEV Community: Sukhpinder Singh

Four Million Checks for Two Million Orders

Sukhpinder Singh — Sat, 31 Oct 2026 08:15:54 +0000

A reporting endpoint got slow enough that someone pinged me about it, and the method at the center of it looked completely harmless. It took a sequence of orders, logged how many were billable, then added up their amounts. Three lines of real work. The kind of code you skim past in review because there's nothing to argue with.

public static decimal Summarize(IEnumerable<Order> billable)
{
    int n = billable.Count();
    decimal total = 0m;
    foreach (var o in billable)
        total += o.Amount;

    log.Information("{Count} billable orders", n);
    return total;
}

The part I slid right past was the parameter type. IEnumerable<Order>. And here's what the caller was handing in:

Summarize(orders.Where(IsBillable));

A Where doesn't filter anything when you call it. It hands back a little object that promises to filter later, once somebody walks it. And my method walks it twice: once for Count(), once for the foreach. Each walk re-runs the predicate on every element from scratch. The filtering I assumed happened once was happening every single time I touched that variable.

Counting the calls instead of guessing

I didn't want to argue from a hunch, so I made the predicate tell on itself. One static counter, bumped on every call, and a filter that does a bit of real work so re-running it isn't free:

static bool IsBillable(Order o)
{
    Probe.PredicateCalls++;
    return o.Active && o.Amount >= 50m;
}

Then two versions of the summary. The one I had, and one that walks the query a single time into a list before it reads anything:

// Two walks of the same deferred query.
static decimal SummarizeTwice(IEnumerable<Order> billable)
{
    int n = billable.Count();   // walk #1
    decimal total = 0m;
    foreach (var o in billable) // walk #2
        total += o.Amount;
    _ = n;
    return total;
}

// Walk it once into a list, then read the list.
static decimal SummarizeOnce(IEnumerable<Order> billable)
{
    var materialized = billable as IReadOnlyList<Order> ?? billable.ToList();
    int n = materialized.Count;
    decimal total = 0m;
    foreach (var o in materialized)
        total += o.Amount;
    _ = n;
    return total;
}

Two million orders, a fresh deferred query each round, best-of-seven, JIT warmed first. The data gets built once up front so the list itself doesn't land on the scoreboard.

Orders:              2,000,000
Checksums match:     True (both 330,342,136)

SummarizeTwice  predicate calls: 4,000,000   best time: 55.9 ms
SummarizeOnce   predicate calls: 2,000,000   best time: 45.5 ms

Extra calls from the second walk: 2,000,000
Time ratio (twice / once):        1.23x

Four million predicate calls to process two million orders. Exactly double, which is what you'd expect once you can see it: two full passes, every element filtered twice over. The counter catches each call as it happens, so that's an exact count rather than a profiler's estimate.

The time didn't double, and that's the honest part

Here's where I owe you some fairness about the thing I'm warning you off. The clock only moved about 1.2x, nowhere near 2x. My predicate is cheap, a bool check and a decimal compare, and materializing into a list isn't free either, so SummarizeOnce hands some of its win back in allocation. On this toy, the second walk is annoying, not fatal.

So why care? Because the predicate is almost never this cheap in the code that actually hurts. Put a database round trip behind that Where, or a parse, or a call that touches a file or a service, and "I did it twice" stops being 20% and becomes the entire bill. The in-memory version is where you learn the shape of the bug for cheap. The expensive version is where it pages you at 2am. The call count is the number that travels between the two; the milliseconds are specific to how heavy each element happens to be.

The fix is boring, which is how I like my fixes. Walk it once, then read off the result.

One opinion I'll actually plant a flag on: the real bug isn't the double walk, it's the parameter type. IEnumerable<T> is a quiet promise that you'll enumerate it at most once and won't go asking for a count. The moment a method wants both a count and the items, it should say so in its signature. Take IReadOnlyCollection<T> or IReadOnlyList<T> and let the caller decide where materializing happens. That as IReadOnlyList<T> ?? ToList() dance up above is me papering over a contract I should have written properly to begin with.

When not to reach for this: if you enumerate the sequence exactly once, leave it deferred. Materializing a stream you only walk a single time, just to feel safe, is how you turn a lazy pipeline into a buffer holding the whole thing in memory, which is the opposite of what IEnumerable streaming is good for. The problem was never deferral. It was me deferring and then walking twice anyway.

I went spelunking through the rest of that service afterward and found three more .Count()-then-foreach pairs sitting over LINQ queries. None of them were slow yet. All of them were waiting for the source to get expensive.

Got a favorite spot where a deferred query bites? The .Any() guard right before a foreach is the classic one, but I bet some of you have seen sneakier. Tell me in the comments.

The whole thing, if you want to run it

No external packages, just dotnet run -c Release on the .NET 10 SDK. (The GitHub push for this one got blocked by my sandbox's egress rules, so the full sample lives right here instead of in the usual repo.)

using System.Diagnostics;

const int Count = 2_000_000;
const int Rounds = 7;

var orders = Demo.BuildOrders(Count);

var twice = Measure(Demo.SummarizeTwice);
var once = Measure(Demo.SummarizeOnce);

Console.WriteLine($"Orders:              {Count:N0}");
Console.WriteLine($"Checksums match:     {twice.result == once.result} (both {twice.result:N0})");
Console.WriteLine();
Console.WriteLine($"SummarizeTwice  predicate calls: {twice.calls:N0}   best time: {twice.ms:F1} ms");
Console.WriteLine($"SummarizeOnce   predicate calls: {once.calls:N0}   best time: {once.ms:F1} ms");
Console.WriteLine();
Console.WriteLine($"Extra calls from the second walk: {twice.calls - once.calls:N0}");
Console.WriteLine($"Time ratio (twice / once):        {twice.ms / once.ms:F2}x");

(decimal result, long calls, double ms) Measure(Func<IEnumerable<Order>, decimal> run)
{
    _ = run(orders.Where(Demo.IsBillable)); // warm the JIT

    double best = double.MaxValue;
    decimal result = 0m;
    long calls = 0;

    for (int r = 0; r < Rounds; r++)
    {
        var query = orders.Where(Demo.IsBillable); // fresh deferred query each round
        Probe.PredicateCalls = 0;
        var sw = Stopwatch.StartNew();
        result = run(query);
        sw.Stop();
        if (sw.Elapsed.TotalMilliseconds < best)
            best = sw.Elapsed.TotalMilliseconds;
        calls = Probe.PredicateCalls;
    }

    return (result, calls, best);
}

record Order(int Id, decimal Amount, bool Active);

static class Probe
{
    public static long PredicateCalls;
}

static class Demo
{
    public static List<Order> BuildOrders(int count)
    {
        var rng = new Random(1234);
        var list = new List<Order>(count);
        for (int i = 0; i < count; i++)
        {
            bool active = rng.Next(3) != 0; // roughly two thirds active
            list.Add(new Order(i, rng.Next(1, 500), active));
        }
        return list;
    }

    public static bool IsBillable(Order o)
    {
        Probe.PredicateCalls++;
        return o.Active && o.Amount >= 50m;
    }

    public static decimal SummarizeTwice(IEnumerable<Order> billable)
    {
        int n = billable.Count();
        decimal total = 0m;
        foreach (var o in billable)
            total += o.Amount;
        _ = n;
        return total;
    }

    public static decimal SummarizeOnce(IEnumerable<Order> billable)
    {
        var materialized = billable as IReadOnlyList<Order> ?? billable.ToList();
        int n = materialized.Count;
        decimal total = 0m;
        foreach (var o in materialized)
            total += o.Amount;
        _ = n;
        return total;
    }
}

— still counting things nobody asked me to count

I Only Wanted Six Sums. GroupBy Gave Me 100 MB

Sukhpinder Singh — Sat, 31 Oct 2026 07:16:33 +0000

A reporting endpoint I wrote months ago started turning up in our gen0 collection graphs, and for a while I couldn't work out why. All it does is total an amount column per region and hand back six numbers. Six. Nothing that small should be making the garbage collector sweat, but the allocation counter said otherwise, and that counter has never once flattered me.

The offending line was the one I'd have waved through in review without a second look:

var totals = rows
    .GroupBy(r => r.Region)
    .ToDictionary(g => g.Key, g => g.Sum(r => (long)r.Amount));

It reads clean and does exactly what it says. And to produce those six sums, it first builds a grouping that holds every single row, bucketed by key, so it can walk each bucket and add things up. I only ever look at the totals. I'm paying to materialize two million rows into groups, then throwing the groups away one line later.

Measuring it instead of guessing

I didn't want to argue from a hunch, so I wrote the smallest honest version. Two million rows of (region, amount) across six regions, built once up front so the data itself doesn't land on the scorecard. Then the same total computed a few ways, each wrapped in a best-of-five loop with GC.GetAllocatedBytesForCurrentThread() and a checksum so a wrong answer couldn't slip through.

The GroupBy version printed this:

GroupBy().ToDictionary(Sum)     100,667,344 bytes     97.4 ms

A hundred megabytes, to keep six longs.

What AggregateBy actually does

AggregateBy shipped in .NET 9 and I'd walked straight past it in the release notes without registering what it was for. It's Aggregate, but keyed: you give it a key selector, a seed, and an accumulator, and it keeps one running value per key as it goes. No buckets of rows, no intermediate groupings. One pass, one small dictionary of accumulators that it hands back as key/value pairs.

var totals = rows
    .AggregateBy(r => r.Region, seed: 0L, (acc, r) => acc + r.Amount)
    .ToDictionary();

Same six numbers. Here's what it cost, next to the old line, best-of-five in a cloud container with workstation GC. This isn't a lab, so I care about the ratio, not the absolute milliseconds:

GroupBy().ToDictionary(Sum)     100,667,344 bytes     97.4 ms
AggregateBy(seed, acc)                1,168 bytes     30.1 ms

From a hundred megabytes down to a bit over a kilobyte, and about three times quicker on the clock. The allocation figure came back byte-for-byte identical on every run, which is exactly what you'd expect: GroupBy has to hold all the rows, AggregateBy holds six longs.

The part that keeps me honest

Before anyone files AggregateBy under free speed, I added a third contestant: the dumb dictionary loop I'd have written years ago without thinking.

var totals = new Dictionary<string, long>();
foreach (var r in rows)
{
    totals.TryGetValue(r.Region, out long acc);
    totals[r.Region] = acc + r.Amount;
}

hand-rolled Dictionary loop            504 bytes     33.2 ms

It allocated less than AggregateBy and ran at basically the same speed. That's the honest result, and I'd rather you have it than a flattering one. AggregateBy isn't beating a hand-written loop, because it pretty much is a hand-written loop with a nicer face. What it beats is GroupBy pressed into service as a counting machine. So the lesson here isn't "LINQ got faster." It's "I was using the wrong LINQ method, and the right one happens to read just as well as the wrong one."

My own take: when what you want out of a collection is one value per key, a sum, a max, a running count, AggregateBy (or CountBy, its count-only sibling) should be the first reach, and GroupBy should wait for when you actually need the grouped items. If you need the rows inside each bucket, GroupBy is correct and AggregateBy can't help you, because by then it's already discarded them.

And the usual note about scale: at a few hundred rows none of this shows up. The 100 MB only appears because there are two million. Don't go rewriting a loop that runs over a config list of twelve entries. But a reporting path, an import job, anything that sees real volume, that's where reaching for GroupBy out of habit taxes you.

The whole sample

I couldn't push this one to the usual repo today, so here's the complete file. Drop it in a console project targeting net10.0 (it runs fine on net9.0 too, since that's where AggregateBy landed) and dotnet run -c Release:

using System.Diagnostics;

const int RowCount = 2_000_000;
string[] regions = ["NA", "EU", "APAC", "LATAM", "MEA", "ANZ"];

var rows = new SaleLine[RowCount];
var rng = new Random(1234);
for (int i = 0; i < RowCount; i++)
    rows[i] = new SaleLine(regions[rng.Next(regions.Length)], rng.Next(1, 500));

Console.WriteLine($"Totalling {RowCount:N0} rows into {regions.Length} regions, best-of-5:\n");

Measure("GroupBy().ToDictionary(Sum)", () =>
{
    var totals = rows
        .GroupBy(r => r.Region)
        .ToDictionary(g => g.Key, g => g.Sum(r => (long)r.Amount));
    return Checksum(totals);
});

Measure("AggregateBy(seed, acc)", () =>
{
    var totals = rows
        .AggregateBy(r => r.Region, seed: 0L, (acc, r) => acc + r.Amount)
        .ToDictionary();
    return Checksum(totals);
});

Measure("hand-rolled Dictionary loop", () =>
{
    var totals = new Dictionary<string, long>();
    foreach (var r in rows)
    {
        totals.TryGetValue(r.Region, out long acc);
        totals[r.Region] = acc + r.Amount;
    }
    return Checksum(totals);
});

static void Measure(string label, Func<long> run)
{
    long check = run();
    GC.Collect();
    GC.WaitForPendingFinalizers();
    GC.Collect();

    long bestBytes = long.MaxValue;
    double bestMs = double.MaxValue;

    for (int i = 0; i < 5; i++)
    {
        long before = GC.GetAllocatedBytesForCurrentThread();
        var sw = Stopwatch.StartNew();
        long result = run();
        sw.Stop();
        long after = GC.GetAllocatedBytesForCurrentThread();

        if (result != check)
            throw new Exception($"{label} disagreed: {result} vs {check}");

        bestBytes = Math.Min(bestBytes, after - before);
        bestMs = Math.Min(bestMs, sw.Elapsed.TotalMilliseconds);
    }

    Console.WriteLine($"{label,-32} {bestBytes,14:N0} bytes   {bestMs,7:F1} ms   (checksum {check})");
}

static long Checksum(Dictionary<string, long> totals)
{
    long sum = 0;
    foreach (var kv in totals)
        sum += kv.Value;
    return sum;
}

readonly record struct SaleLine(string Region, int Amount);

Anyone else got a LINQ method you ignored for a year before realizing it was the one you needed the whole time? I'd like to hear which.

— still measuring allocations nobody asked me to measure

The Backing Field a Typo Could Reach

Sukhpinder Singh — Fri, 30 Oct 2026 08:18:15 +0000

A discount came back as 110% in a test last week and I spent a solid ten minutes insisting that was impossible. I had a setter. The setter clamped. Math.Clamp(value, 0, 100), sitting right there, no way past it. Except there was a way past it, and I'd built the door myself.

Here's the property, more or less how it looked:

sealed class DiscountOld
{
    private int _percent;

    public int Percent
    {
        get => _percent;
        set => _percent = Math.Clamp(value, 0, 100);
    }
}

Nothing wrong with that. Assign Percent = 150 and you get 100 back. I'd tested exactly that case and moved on. The setter does its job every single time it runs. That last bit is the part I didn't think hard enough about.

The line I wrote three weeks later

A seasonal promo needed a flat bump, and I added a method to the same class to do it. Past me, in a hurry, reached for the field because it was shorter and it was right there:

public void ApplySeasonalBump()
{
    _percent += 90;   // this never asks the setter for permission
}

So a row that started at a sensible 20 goes to 110, and the clamp never gets a vote, because _percent += 90 touches the backing field and the backing field doesn't clamp anything. The setter was never the guard I thought it was. It was a guard on one specific door, and I'd left a second door open the day I gave the field a name.

Running both side by side makes it plain:

DiscountOld.Percent  = 110

The clamp is still there. It just didn't run on the path that mattered.

The field keyword removes the second door

C# 14 (the field contextual keyword, stable now in .NET 10) lets a property refer to its own backing store without you ever naming it:

sealed class DiscountNew
{
    public int Percent
    {
        get => field;
        set => field = Math.Clamp(value, 0, 100);
    }

    public void ApplySeasonalBump()
    {
        Percent += 90;   // the only door left is the property
    }
}

The interesting word there is nameless. The compiler still generates a backing field — same as before, nothing clever happening underneath. But there's no identifier for it anywhere in your source. So when I go to write the bump method, I can't reach for _percent, because _percent doesn't exist. My hand is forced onto Percent, which runs the setter, which clamps:

DiscountNew.Percent  = 100

And if you don't believe the field is truly unreachable, try it. Drop _percent += 90 into the second class and the build stops cold:

error CS0103: The name '_percent' does not exist in the current context

That error is the whole point. The bug from the top of this post isn't fixed here so much as it's made unspellable.

This isn't a speed thing, and I want to be honest about that

I write a lot of posts where the payoff is a number on a stopwatch. This isn't one of them. The field keyword compiles down to the same backing field you'd have written by hand, so there's no allocation saved, no cycle shaved, nothing to benchmark. If you go looking for a performance win here you'll come back empty, and that's fine. The win is a different currency: it's one fewer category of bug that a tired colleague can introduce at 5pm. I'll take that trade over a microsecond most days.

My honest opinion, for whatever it's worth: the sweet spot is any property that does real work in its setter — clamping, trimming, normalizing, raising a changed event. Those are exactly the properties where skipping the setter is a bug, and field makes skipping it impossible. For a plain get; set; with no logic, it buys you nothing, so I don't bother.

Where it bites

One thing to watch, and it's the reason this wasn't just quietly turned on for everyone. If you already have something named field in scope, the keyword and the name collide. A local called field inside an accessor is now a hard error:

error CS9273: In language version 14.0, 'field' is a keyword within a property
accessor. Rename the variable or use the identifier '@field' instead.

And a member named field gets you a warning that the keyword is quietly binding to the synthesized backing store instead of your member (warning CS9258), with @field or this.field as the escape hatch. Not common names, but if you've got a domain where "field" is a real noun — forms, databases, farming apps — grep for it before you bump your language version. I'd rather know about the collision from a grep than from a warning I scrolled past.

Full runnable sample, both classes and the output above, is the two snippets in this post — drop them in a net10.0 console project with <LangVersion>14.0</LangVersion> and run it.

Have you found a good use for field yet, or hit a collision you didn't expect? Curious which camp people land in.

— still clamping things that were already clamped, apparently

I Clamped Two Million Readings and Nothing Happened

Sukhpinder Singh — Fri, 30 Oct 2026 07:32:03 +0000

A job of mine ingests sensor readings, and one cleanup step clamps anything above 500 back down to 500. Garden-variety stuff. I wrote the loop, the tests I cared about stayed green, and the "reading over threshold" alert kept firing in staging anyway. Values above 500 were sailing straight through. The loop was right there in front of me, clearly clamping every element. Except it clamped none of them.

Here's the shape of it. The reading is a struct, because there are millions of them and I didn't want a couple million tiny objects churning through the heap:

struct Reading
{
    public long Value;
    // ...a pile of metadata fields I rarely touch

    public void Clamp()
    {
        if (Value > 500) Value = 500;
    }
}

And the loop:

foreach (var r in readings) // readings is a List<Reading>
    r.Clamp();

Compiles. Runs. Clamps nothing.

foreach hands you a copy

When you foreach over a List<Reading>, each r is a fresh copy of the element, because Reading is a value type. Clamp() faithfully clamps that copy, and then the copy falls out of scope at the bottom of the loop body and takes the change with it. The list never hears about it. The compiler stays quiet, because calling a method on a local copy is an ordinary thing to do. It only looks wrong once you know the method mutates this.

I proved it to myself with the smallest harness I could manage. Two million readings with values 0 to 1023, sum the field before and after, and watch whether the number moves:

before             : sum 1022942656
after foreach copy : sum 1022942656  (UNCHANGED)

Not off by a little. Identical. Two million method calls, every one of them working on a doomed copy.

The part that stung is that C# will actually stop you if you reach for the obvious mutations directly. Both of these are compile errors, not runtime surprises:

foreach (var r in readings) r.Value = 500; // CS1654: iteration variable is readonly
readings[0].Value = 500;                    // CS1612: indexer return is not a variable

CS1612 is the telling one. List<T>'s indexer returns the element by value, so readings[0] is a throwaway copy, and assigning to a field of a throwaway that's about to vanish means nothing. The compiler says as much. The mutating-method version slips past only because the method call wraps that same copy in a friendlier shape.

CollectionsMarshal.AsSpan gives you the real element

CollectionsMarshal.AsSpan returns a Span<Reading> over the list's actual backing array. A span indexes by reference, so there's no copy sitting between you and the storage:

using System.Runtime.InteropServices;

foreach (ref var r in CollectionsMarshal.AsSpan(readings))
    r.Clamp();

after AsSpan clamp : sum 755330878  (changed)

The number moved. That's the whole fix. ref var r binds to the element where it lives, Clamp() mutates that element, and the list keeps it.

It isn't faster, and I'm fine with that

I half-expected a speed win too. Skipping two million 120-byte struct copies sounds like it ought to show up somewhere. So I measured the read side, just summing the field, which is identical work either way. Best-of-50, on a shared cloud container where I trust the ratio a lot more than the absolute milliseconds:

foreach copy   :  15.3 ms/pass
foreach AsSpan :  14.5 ms/pass
copy/span      :  about 1.1x

It bounced between a dead tie and maybe ten percent across runs. Summing is memory-bound: you're streaming the whole backing array out of RAM no matter how you walk it, and the register-sized copy into a temporary is lost in that noise. So I won't sell you AsSpan as a read-speed trick. It isn't one here. The reason to reach for it is that it edits the real element, not that it shaves microseconds.

When I'd leave it alone

The span from AsSpan is a live window onto the backing array, and it goes stale the instant the list resizes. If you Add, Remove, or Clear while you're still holding the span, you're reading the old array or slicing out of bounds. So take the span, do your pass, and let it go. Don't stash it in a field, and don't change the list's length underneath it.

And honestly, if you find yourself needing any of this, that's worth a pause on its own. A List<struct> you mutate in place is a design that fights the language a bit, and sometimes the cleaner answer is a class, or rebuilding elements instead of editing them. But when you've measured, you genuinely want the value type, and you need to edit in place, this beats the read-copy-write-back-by-index dance that people usually fall into when they hit CS1612.

My rule of thumb now: if a method mutates a struct and I'm calling it inside a foreach over a collection, I stop and ask what exactly I'm mutating. More often than I'd like, the answer is "a copy nobody keeps."

Here's the complete thing if you want to run it yourself. New console project on the .NET 10 SDK, drop this in Program.cs, dotnet run -c Release:

using System.Diagnostics;
using System.Runtime.InteropServices;

struct Reading
{
    public long Value;
    public long F1, F2, F3, F4, F5, F6, F7;
    public long F8, F9, F10, F11, F12, F13, F14; // ~120 bytes total

    public Reading(long value) => Value = value;

    public void Clamp()
    {
        if (Value > 500) Value = 500;
    }
}

static class Program
{
    const int Count = 2_000_000;
    const int Repeats = 50;

    static List<Reading> BuildReadings()
    {
        var list = new List<Reading>(Count);
        for (int i = 0; i < Count; i++)
            list.Add(new Reading(i & 1023));
        return list;
    }

    static long Sum(List<Reading> readings)
    {
        long sum = 0;
        foreach (ref readonly var r in CollectionsMarshal.AsSpan(readings))
            sum += r.Value;
        return sum;
    }

    static void MutationDemo()
    {
        var a = BuildReadings();
        long before = Sum(a);

        foreach (var r in a)   // r is a copy; this clamps nothing
            r.Clamp();

        Console.WriteLine($"before             : sum {before}");
        Console.WriteLine($"after foreach copy : sum {Sum(a)}  ({(Sum(a) == before ? "UNCHANGED" : "changed")})");

        // These two don't even compile, for the record:
        //   foreach (var r in a) r.Value = 500; // CS1654
        //   a[0].Value = 500;                    // CS1612

        var b = BuildReadings();
        long beforeB = Sum(b);

        foreach (ref var r in CollectionsMarshal.AsSpan(b)) // ref into the backing array
            r.Clamp();

        Console.WriteLine($"after AsSpan clamp : sum {Sum(b)}  ({(Sum(b) == beforeB ? "UNCHANGED" : "changed")})");
    }

    static long SumForeachCopy(List<Reading> readings)
    {
        long sum = 0;
        foreach (var r in readings)
            sum += r.Value;
        return sum;
    }

    static (double ms, long bytes) Measure(Func<List<Reading>, long> f, List<Reading> data, long expect)
    {
        for (int i = 0; i < 5; i++) _ = f(data); // warm the JIT

        long bestTicks = long.MaxValue, check = 0;
        long allocBefore = GC.GetAllocatedBytesForCurrentThread();
        var sw = new Stopwatch();
        for (int i = 0; i < Repeats; i++)
        {
            sw.Restart();
            check = f(data);
            sw.Stop();
            if (sw.ElapsedTicks < bestTicks) bestTicks = sw.ElapsedTicks;
        }
        long allocAfter = GC.GetAllocatedBytesForCurrentThread();
        if (check != expect) throw new Exception("checksum mismatch - benchmark is lying");
        return (bestTicks * 1000.0 / Stopwatch.Frequency, (allocAfter - allocBefore) / Repeats);
    }

    static void Main()
    {
        Console.WriteLine($"{RuntimeInformation.FrameworkDescription}, struct {Marshal.SizeOf<Reading>()} bytes, {Count:N0} elements, best-of-{Repeats}.\n");
        MutationDemo();

        var data = BuildReadings();
        long expect = Sum(data);
        var copy = Measure(SumForeachCopy, data, expect);
        var span = Measure(Sum, data, expect);
        Console.WriteLine($"\nforeach copy   : {copy.ms,7:F3} ms/pass");
        Console.WriteLine($"foreach AsSpan : {span.ms,7:F3} ms/pass");
        Console.WriteLine($"copy/span      : {copy.ms / span.ms:F2}x");
    }
}

Ever shipped a loop that was technically running and practically doing nothing? I'd like to hear what tipped you off.

— still benchmarking the things nobody asked me to clamp

GetOrAdd Ran My Factory Three Times

Sukhpinder Singh — Thu, 29 Oct 2026 08:20:47 +0000

I had a client object that was expensive to build. It opens a pooled connection, reads a bit of config, and I only ever want one per name for the life of the process. So I did the obvious thing and cached it in a ConcurrentDictionary, keyed by name, built on first use with GetOrAdd. Thread-safe dictionary, lazy construction, done.

Then I noticed the startup logs. For a single key, "opening connection" showed up three times. Only one connection was actually in the dictionary afterward, and every caller was using it, so nothing looked broken downstream. But two connections had been opened and then quietly abandoned. On a cold start with a few of these firing at once, that adds up to real work nobody asked for.

The race is hiding in the one line I trusted most.

Where the extra calls come from

GetOrAdd(key, valueFactory) is not a lock around your factory. When several threads hit a missing key at the same moment, more than one of them can run the factory. The dictionary only keeps one of the results and hands that same instance to everyone, but the losing factories already executed. They opened their connection, did their work, and got thrown away.

I didn't want to take that on faith, so I built the smallest thing that shows it. Sixty-four threads, one key, a factory that stamps a serial number onto each object it builds and spins a little to mimic real setup cost:

static int _factoryCalls;

static Connection BuildConnection()
{
    int serial = Interlocked.Increment(ref _factoryCalls);
    Thread.SpinWait(200_000); // stand in for a handshake / DNS / file read
    return new Connection(serial);
}

Parallel.For(0, 64, _ =>
{
    var conn = cache.GetOrAdd("db", _ => BuildConnection());
    handedOut.Add(conn.SerialNumber);
});

Here's what that run printed:

== GetOrAdd(key, factory) ==
  factory ran:            3 time(s)
  connections built:      3 (and 2 thrown away)
  serial stored in cache: #2
  distinct serials callers actually got: 1

Three factory calls. All 64 callers walked away with the same connection, so the "distinct serials" count is one, which is exactly why this is so easy to miss. The cache is doing its job. It just paid three times to fill one slot. Notice too that the instance it kept was serial #2, not #1. The winner is whichever thread finishes the insert first, and that's not the one that started first.

That SpinWait widens the window so the race is obvious, but it doesn't create it. Take the spin out and you still get duplicate calls now and then, you just need more threads or worse luck to see them. The count isn't stable either. I got three every time on this little container because it doesn't have many cores. On a 16-core box under a real stampede I've seen it climb a lot higher.

One Lazy and it stops

The fix I reach for is to stop storing the connection and start storing a Lazy<T> of it. Building a Lazy is cheap, so it doesn't matter if GetOrAdd makes a few of those. Only the winner's .Value ever gets touched, and Lazy guarantees the real factory behind it runs exactly once:

var cache = new ConcurrentDictionary<string, Lazy<Connection>>();

var conn = cache.GetOrAdd("db",
    _ => new Lazy<Connection>(BuildConnection, LazyThreadSafetyMode.ExecutionAndPublication)).Value;

Same 64 threads, same key:

== GetOrAdd(key, new Lazy<T>(factory)).Value ==
  factory ran:            1 time(s)
  distinct serials callers actually got: 1

One call. That's the whole trick. The dictionary is still allowed to race on the wrapper, but the expensive part is behind a gate that only opens once. ExecutionAndPublication is the default mode and it's the one you want here: it both runs the factory once and makes sure everyone sees the same finished object.

When I don't bother

I don't wrap everything in Lazy out of reflex, and I don't think you should either. If the factory is cheap and has no side effects, building a throwaway int or a small record twice costs you nothing you'll ever measure, and the wrapper just adds an allocation and a layer of .Value noise to read past. The plain GetOrAdd is fine there, and honestly it's the more readable code.

The Lazy version earns its keep when the factory is expensive, or when running it twice is a bug and not just waste: it opens a socket, writes a file, increments something, registers a handler. That's the real tell. Not "is this slow" but "what happens if this runs more than once." If the answer is "nothing good," reach for the wrapper.

One more thing worth saying out loud: this was never a thread-safety hole. Nobody got a half-built object and the dictionary never got corrupted. It's a "ran more than I expected" problem, which is a quieter kind of bug and the sort that sits in your startup path for months looking like a slightly slow boot.

Here's the whole thing in one file so you can run it yourself. It's a plain net10.0 console app, nothing else in the project:

using System.Collections.Concurrent;
using System.Diagnostics;

sealed class Connection
{
    public int SerialNumber { get; }
    public Connection(int serial) => SerialNumber = serial;
}

static class Program
{
    static int _factoryCalls;

    static Connection BuildConnection()
    {
        int serial = Interlocked.Increment(ref _factoryCalls);
        Thread.SpinWait(200_000); // stand in for a handshake / DNS / file read
        return new Connection(serial);
    }

    static void Main()
    {
        const int callers = 64;
        Console.WriteLine($"Spinning up {callers} threads, all asking for the same key at once.\n");
        RunPlain(callers);
        Console.WriteLine();
        RunLazy(callers);
    }

    static void RunPlain(int callers)
    {
        _factoryCalls = 0;
        var cache = new ConcurrentDictionary<string, Connection>();
        var handedOut = new ConcurrentBag<int>();

        var sw = Stopwatch.StartNew();
        Parallel.For(0, callers, _ =>
        {
            var conn = cache.GetOrAdd("db", _ => BuildConnection());
            handedOut.Add(conn.SerialNumber);
        });
        sw.Stop();

        Console.WriteLine("== GetOrAdd(key, factory) ==");
        Console.WriteLine($"  factory ran:            {_factoryCalls} time(s)");
        Console.WriteLine($"  connections built:      {_factoryCalls} (and {_factoryCalls - 1} thrown away)");
        Console.WriteLine($"  serial stored in cache: #{cache["db"].SerialNumber}");
        Console.WriteLine($"  distinct serials callers got: {handedOut.Distinct().Count()}");
        Console.WriteLine($"  elapsed:                {sw.Elapsed.TotalMilliseconds:F1} ms");
    }

    static void RunLazy(int callers)
    {
        _factoryCalls = 0;
        var cache = new ConcurrentDictionary<string, Lazy<Connection>>();
        var handedOut = new ConcurrentBag<int>();

        var sw = Stopwatch.StartNew();
        Parallel.For(0, callers, _ =>
        {
            var conn = cache.GetOrAdd("db",
                _ => new Lazy<Connection>(BuildConnection, LazyThreadSafetyMode.ExecutionAndPublication)).Value;
            handedOut.Add(conn.SerialNumber);
        });
        sw.Stop();

        Console.WriteLine("== GetOrAdd(key, new Lazy<T>(factory)).Value ==");
        Console.WriteLine($"  factory ran:            {_factoryCalls} time(s)");
        Console.WriteLine($"  distinct serials callers got: {handedOut.Distinct().Count()}");
        Console.WriteLine($"  elapsed:                {sw.Elapsed.TotalMilliseconds:F1} ms");
    }
}

Have you been bitten by this one? I'm curious whether people hit it through an expensive factory like I did or through a factory with a side effect that broke something louder.

— still counting things that were only ever supposed to run once.

OrderBy().First() Wasn't Sorting Anything

Sukhpinder Singh — Thu, 29 Oct 2026 07:17:53 +0000

I left a comment on a pull request last week that I now regret. The code found the cheapest offer for a product like this:

var cheapest = offers.OrderBy(o => o.Price).First();

And I did the thing. "This sorts the whole list just to grab one item. Use MinBy, it's a single pass." I even added a little explainer about O(n log n) versus O(n), feeling helpful. Then I went to prove it with a benchmark so the author couldn't argue, and the benchmark made me look silly instead.

What I expected

My mental model was simple and, I thought, obviously correct. OrderBy sorts everything. Sorting a hundred thousand items means roughly 1.7 million comparisons and a buffer big enough to hold them all. MinBy walks the list once, keeps the smallest it's seen, and allocates nothing. So swapping one for the other should be a clean win on both time and garbage. That's the kind of change you make without thinking.

Here's the setup I ran it against. A hundred thousand offers, each a small record, cheapest price sitting somewhere in the middle so neither approach gets to cheat by position:

record Offer(int Id, string Seller, decimal Price, int StockDays);

static Offer CheapestViaOrderBy(List<Offer> offers)
    => offers.OrderBy(o => o.Price).First();

static Offer CheapestViaMinBy(List<Offer> offers)
    => offers.MinBy(o => o.Price)!;

I measured best-of-N per call, warmed the JIT first, and tracked allocations with GC.GetAllocatedBytesForCurrentThread(). This is a shared cloud container, not a quiet lab machine, so I care about the ratios between these numbers, not the absolute milliseconds.

What actually happened

OrderBy().First():    0.675 ms/call      128 bytes/call
MinBy():              0.654 ms/call       40 bytes/call

A tie. Same speed, within noise. MinBy allocated less, but we're talking 40 bytes against 128, not the megabytes I was picturing. There was no sort to avoid, because OrderBy().First() wasn't sorting anything.

That stopped me. I'd been so sure. So I went and read what LINQ actually does, and it turns out the ordering operators are lazier than I gave them credit for. OrderBy on its own doesn't sort. It builds an IOrderedEnumerable that describes a sort and defers the work until someone enumerates it. And First() doesn't enumerate the sorted result. It has a special path that asks the ordered sequence for just its first element, and that path computes the minimum in a single comparison pass. No buffer, no full sort. The same is true for Last(), and ElementAt has its own partial-selection trick too.

So the code I told someone to "fix" was already doing the O(n) single pass I was about to recommend. MinBy expresses the intent better, and it shaves a few bytes, but it is not rescuing the performance of that line. It can't. There was never a fire.

Where the sort is real

The sort only happens when you make it happen, which is the moment you ask for the whole ordered sequence instead of one end of it. Materialize it and the bill comes due:

static Offer CheapestViaFullSort(List<Offer> offers)
    => offers.OrderBy(o => o.Price).ToList()[0];

OrderBy().ToList()[0]:   26.466 ms/call   3,602,961 bytes/call

There's the sort I kept imagining. Against OrderBy().First() that's about 39 times slower and roughly twenty-eight thousand times the allocation per call. That gap is the actual lesson. The danger was never OrderBy().First(). It's the .ToList(), the .ToArray(), the foreach over the ordered result, or taking the top three with .OrderBy(...).Skip(2).First(), which genuinely does have to order things. If you only want one end of a sequence, say so with First, Last, Min, or Max, and LINQ keeps it cheap. The expensive version is the one that keeps the sorted copy around.

So is MinBy pointless?

No, and this is where I landed after being wrong. I still reach for MinBy in that pull request, just not for the reason I gave. It says what the code means in one word. Nobody reading MinBy(o => o.Price) has to pause and remember that OrderBy().First() happens to be optimized. And it's a little quieter on allocation. Those are fine reasons. "It avoids a sort" is not one, because there wasn't a sort.

One honest caveat before you swap everything on sight: the two don't behave the same on an empty sequence, and I got bitten by exactly this once.

OrderBy().First() -> throws InvalidOperationException
MinBy()           -> returns null   (for a reference type)

First() throws when there's nothing there. MinBy over a sequence of reference types hands you back null instead, which is easy to miss if you're not expecting it and leads to a NullReferenceException three lines later that points at the wrong place. Over value types like List<int>, MinBy throws just like First does. So the swap can quietly turn a loud exception into a silent null. Worth a glance at the call site before you trust it.

My takeaway, stated as the opinion it is: micro-advice about LINQ performance is worth almost nothing without a measurement next to it, and I'm including my own advice in that. I was confidently, specifically wrong about a line I've "optimized" a dozen times. The benchmark took ten minutes and saved me from being wrong in writing.

Here's the whole thing if you want to run it yourself. I'd meant to drop a GitHub link here, but the repo push was blocked in my build environment this run, so the full sample is inline instead:

using System.Diagnostics;

record Offer(int Id, string Seller, decimal Price, int StockDays);

static class Bench
{
    const int Count = 100_000;
    const int Iterations = 200;

    static List<Offer> BuildOffers(int n)
    {
        var rng = new Random(1234);
        var list = new List<Offer>(n);
        for (int i = 0; i < n; i++)
            list.Add(new Offer(i, "seller-" + (i % 500),
                rng.Next(100, 100_000) / 100m, rng.Next(0, 30)));
        return list;
    }

    static Offer CheapestViaOrderBy(List<Offer> o) => o.OrderBy(x => x.Price).First();
    static Offer CheapestViaMinBy(List<Offer> o)   => o.MinBy(x => x.Price)!;
    static Offer CheapestViaFullSort(List<Offer> o) => o.OrderBy(x => x.Price).ToList()[0];

    static (double ms, long bytes) Measure(Func<List<Offer>, Offer> find, List<Offer> offers)
    {
        find(offers);
        for (int i = 0; i < 20; i++) find(offers);

        double best = double.MaxValue;
        long bytesBest = long.MaxValue;
        var sw = new Stopwatch();
        for (int rep = 0; rep < 5; rep++)
        {
            long before = GC.GetAllocatedBytesForCurrentThread();
            sw.Restart();
            for (int i = 0; i < Iterations; i++) find(offers);
            sw.Stop();
            long after = GC.GetAllocatedBytesForCurrentThread();
            best = Math.Min(best, sw.Elapsed.TotalMilliseconds / Iterations);
            bytesBest = Math.Min(bytesBest, (after - before) / Iterations);
        }
        return (best, bytesBest);
    }

    static void Main()
    {
        var offers = BuildOffers(Count);
        var (a, ba) = Measure(CheapestViaOrderBy, offers);
        var (b, bb) = Measure(CheapestViaMinBy, offers);
        var (c, bc) = Measure(CheapestViaFullSort, offers);

        Console.WriteLine($"OrderBy().First():     {a,8:F3} ms   {ba,12:N0} bytes/call");
        Console.WriteLine($"MinBy():               {b,8:F3} ms   {bb,12:N0} bytes/call");
        Console.WriteLine($"OrderBy().ToList()[0]: {c,8:F3} ms   {bc,12:N0} bytes/call");
    }
}

Did you also carry around the "OrderBy().First() sorts everything" belief, or was that just me? Curious whether anyone's hit the empty-sequence null in production.

Still benchmarking lines of code that nobody filed a bug against.

440 MB of Closures on Pure Cache Hits

Sukhpinder Singh — Wed, 28 Oct 2026 08:18:35 +0000

I had a lookup cache on a hot path, a ConcurrentDictionary<int, string> that maps a tenant id to a formatted key. After warmup it's basically all hits. The entry's already there, you read it, you move on. So when I saw the gen-0 collections ticking over under steady load, I went looking for the allocation somewhere else first. The cache couldn't be it. It's hits. Hits don't build anything.

Except mine did. The factory I handed to GetOrAdd was quietly allocating an object on every single call, including the millions of calls where it never ran.

Two overloads that read the same and aren't

GetOrAdd has more than one factory shape. The one everybody reaches for is GetOrAdd(key, k => Build(k)), taking a Func<TKey, TValue>. There's another that takes Func<TKey, TArg, TValue> plus a state argument: GetOrAdd(key, (k, arg) => Build(k, arg), arg). On the surface they do the same thing, and most comparisons of them talk about the factory invocation.

But the invocation isn't where my cost was. Mine was at the call site, before GetOrAdd decided anything. My factory closed over a value that changed every time through the loop:

int salt = i;                       // different each iteration
string v = dict.GetOrAdd(key, k => $"tenant-42:user:{k}:{salt}");

Because salt is a fresh local each iteration, the compiler can't cache that lambda. It builds a new display-class instance to hold salt, and a new delegate pointing at it, every time the line runs. The dictionary then looks up key, finds it, hands back the existing string, and never touches my shiny new closure. I allocated it for nothing.

The fix is the state-arg overload, with a static lambda so there's physically nothing to capture:

int salt = i;
string v = dict.GetOrAdd(key, static (k, s) => $"tenant-42:user:{k}:{s}", salt);

The value I need rides in as the argument. The lambda captures nothing, so the compiler caches a single delegate for the whole program and the call allocates zero.

I counted it

I don't trust my gut on allocation, so I wrote the smallest thing that shows it. A dictionary pre-filled with a thousand keys, then five million GetOrAdd calls cycling over those keys, so after the warmup pass essentially everything is a hit:

var dict = new ConcurrentDictionary<int, string>();
for (int k = 0; k < 1_000; k++)
    dict[k] = "tenant-42:user:" + k;

Both hammer loops are the ones above, wrapped in a method. The harness is nothing fancy, a stopwatch, a warmup call per path so the JIT is done, best of seven, allocations read around each pass:

static (long bestMs, long bytes, long result) Measure(Func<long> f)
{
    long bestMs = long.MaxValue, bytes = 0, result = 0;
    var sw = new Stopwatch();
    for (int run = 0; run < 7; run++)
    {
        long before = GC.GetAllocatedBytesForCurrentThread();
        sw.Restart();
        result = f();
        sw.Stop();
        long used = GC.GetAllocatedBytesForCurrentThread() - before;
        if (sw.ElapsedMilliseconds < bestMs) { bestMs = sw.ElapsedMilliseconds; bytes = used; }
    }
    return (bestMs, bytes, result);
}

Single-file console app, .NET 10, nothing else in it. Here's what came back:

GetOrAdd calls          : 5,000,000  (hits on 1,000 keys)
capturing lambda time   :    112 ms
state-arg    time       :     24 ms
capturing lambda alloc  :    440,000,000 bytes
state-arg    alloc      :              0 bytes
bytes per capturing call: 88.0

88 bytes a call, on pure cache hits. The display class plus the delegate, every time, swept up a moment later having done no work. Across five million calls that's 440 MB of churn for a loop whose entire job was to read strings that already existed. The state-arg version did the identical reads at zero bytes, and came out roughly four and a half times faster because it wasn't feeding the collector.

This is a container, not a lab bench, and I trust the ratio more than the millisecond count. But 440 MB against zero holds up anywhere.

When this is a non-issue

Here's the part that bit me and might save you a wild goose chase. This only costs you when the factory captures something that varies per call. If your lambda captures nothing, or captures a value that's fixed for the whole method, the compiler already caches the closure and allocates it once. I proved that to myself first with a factory that closed over a method-level constant and got a grand total of 96 bytes for the entire run. So don't go rewriting every GetOrAdd you own. Go find the ones whose factory closes over a loop variable, a method argument that changes, a per-request value.

And yes, the static (k, s) => ... form reads a little worse than the plain lambda. My honest opinion: on a hot path that's a trade I take without thinking, and everywhere else I leave the readable version alone.

Since the usual repo push got blocked on this run, the whole sample is up above. Two hammer methods, the dictionary setup, the harness, drop them in a dotnet new console on .NET 10 and it runs.

What's the sneakiest allocation you've found hiding behind a line you were sure was free? I collect these.

Right, back to it. Still putting counters on code that was never on trial.

I Split Two Million Rows and the GC Noticed

Sukhpinder Singh — Wed, 28 Oct 2026 07:18:43 +0000

I had a nightly job that reads a partner CSV and sums the amount column, just to sanity-check a total before the real import runs. Boring code. row.Split(','), grab field one, parse it, add it up. It worked the day I wrote it and I forgot about it.

Then the job's memory graph grew a sawtooth I didn't like, and I opened a profiler expecting to find the database client or the file reader. Nope. The garbage collector was working full time on nothing but comma splitting. For a loop that reads one column out of four.

string.Split is generous with memory

Here's what I'd stopped seeing. string.Split(',') doesn't just find the commas. It allocates a string[] to hold the pieces, and then a brand-new string for every field it carved out. Four columns means five allocations per row before I've parsed a single number. And I was throwing away three of those strings immediately, because I only wanted field one.

Multiply that by a couple million rows and the GC has plenty to do.

The newer Span<char>.Split overloads take a different deal. You hand them a Span<Range> to fill, they write the boundaries of each field into it, and nothing gets copied. You slice the original string yourself when you actually need a piece. double.Parse happily takes a ReadOnlySpan<char>, so I never have to materialize the substring at all.

static double SumWithSpanSplit(string[] rows, CultureInfo culture)
{
    double total = 0;
    Span<Range> fields = stackalloc Range[8]; // 4 columns, room to spare
    foreach (string row in rows)
    {
        ReadOnlySpan<char> line = row;
        int count = line.Split(fields, ',');
        total += double.Parse(line[fields[1]], culture);
    }
    return total;
}

The stackalloc Range[8] lives on the stack and gets reused on every iteration, so the per-row cost is zero. Compare that to the version I actually shipped first:

static double SumWithStringSplit(string[] rows, CultureInfo culture)
{
    double total = 0;
    foreach (string row in rows)
    {
        string[] fields = row.Split(',');
        total += double.Parse(fields[1], culture);
    }
    return total;
}

I counted it

I don't trust my gut on allocations, so I built two million CSV-ish rows in memory (id,amount,region,active), then summed the amount column both ways. A stopwatch, a warmup call per path, best time out of seven, and the allocation counter read around each pass. This is a container, not a lab bench, so I only care about the ratio.

Rows: 2,000,000   Runtime: .NET 10.0.14

string.Split       231.8 ms      383,039,160 bytes   sum=9960642900.00
span Split         122.5 ms               40 bytes   sum=9960642900.00

time:  span is 1.9x faster
alloc: span allocated 40 bytes
sums match: True

365 megabytes versus 40 bytes. The 40 bytes is the harness breathing, not the loop. And the loop got about 1.8x faster across repeated runs, which is the part I didn't expect, I went in thinking I'd just save memory. Turns out all that allocating and collecting isn't free on the clock either.

The sums matching is the line I care about most. A faster parser that quietly reads the wrong column is worse than a slow correct one.

When I'd leave string.Split alone

Most of the time, honestly. If I'm reading a 30-line config file once at startup, string.Split is clearer and the allocations vanish into the noise. I'm not going to stack-allocate a range buffer to save microseconds on something that runs once. The span version is also fussier: you cap the number of fields at the size of your Span<Range>, and if a row has more commas than you planned for, the extra columns fold into the last range instead of throwing. For my fixed-shape partner feed that's fine. For arbitrary user input I'd check the returned count before trusting fields[1].

My rule of thumb: reach for span Split when the same parse runs in a hot loop over a lot of rows, and leave the friendly version everywhere else. The readability cost is real, so I want the allocation win to be real too.

The rest of the harness, if you want to run it

I usually link these out to a repo, but the push to my samples repo was blocked on this run, so here's the whole thing in one place. Drop it in a net10.0 console project (<TargetFramework>net10.0</TargetFramework>) and dotnet run -c Release. The two Sum... methods above go right below this.

using System.Diagnostics;
using System.Globalization;

const int RowCount = 2_000_000;
var culture = CultureInfo.InvariantCulture;

// Build the rows once: id,amount,region,active  e.g. "1048576,37.52,emea,1"
string[] rows = new string[RowCount];
for (int i = 0; i < RowCount; i++)
{
    double amount = (i % 9973) + (i % 100) / 100.0;
    string region = (i % 4) switch { 0 => "emea", 1 => "apac", 2 => "amer", _ => "latam" };
    rows[i] = string.Create(culture, $"{i},{amount:0.00},{region},{i & 1}");
}

static (double ms, long bytes, double result) Measure(Func<double> pass)
{
    double result = pass();           // warm up the JIT and the code paths
    double best = double.MaxValue;
    long bytesBest = 0;
    for (int run = 0; run < 7; run++)
    {
        long before = GC.GetTotalAllocatedBytes(precise: true);
        var sw = Stopwatch.StartNew();
        result = pass();
        sw.Stop();
        long after = GC.GetTotalAllocatedBytes(precise: true);
        if (sw.Elapsed.TotalMilliseconds < best)
        {
            best = sw.Elapsed.TotalMilliseconds;
            bytesBest = after - before;
        }
    }
    return (best, bytesBest, result);
}

var a = Measure(() => SumWithStringSplit(rows, culture));
var b = Measure(() => SumWithSpanSplit(rows, culture));
Console.WriteLine($"string.Split  {a.ms,8:0.0} ms  {a.bytes,14:N0} bytes  sum={a.result:0.00}");
Console.WriteLine($"span Split    {b.ms,8:0.0} ms  {b.bytes,14:N0} bytes  sum={b.result:0.00}");

If you've got a parsing loop that shows up on your allocation graph, this is a cheap swap to try. What's the worst allocation hot spot you've found hiding behind a one-line convenience method?

Catch you in the next one — still putting stopwatches on code nobody asked me to time.

Every Comma Was Costing Me an Array

Sukhpinder Singh — Tue, 27 Oct 2026 08:16:23 +0000

A nightly job that chews through a CSV export got moved onto a smaller box, and it started spending more time in gen0 collections than I was comfortable with. The work is boring: read a few hundred thousand lines, pull one numeric column, add it up. Nothing in that loop should be stressing the garbage collector. The allocation counter disagreed, and that counter has never once lied to make me feel better.

The culprit turned out to be the least suspicious line in the whole thing: line.Split(',').

Here's the part I keep under-weighting. string.Split doesn't just find the commas. It allocates a string[] for the pieces and then a fresh string for every field it carves out, including the ones I never look at. I wanted field number three. I was paying for all of them, a million times over, then handing each little string straight to the GC a microsecond later.

Measuring it instead of trusting the vibe

I didn't want to argue from a profiler screenshot, so I wrote the smallest honest version of the job. A million CSV-shaped lines of id,sku,amount, built once up front so the data itself doesn't land on either scorecard. Then two functions that sum the amount column and differ only in how they split a line.

static long SumWithStringSplit(string[] lines)
{
    long total = 0;
    foreach (string line in lines)
    {
        string[] fields = line.Split(',');
        total += int.Parse(fields[2], CultureInfo.InvariantCulture);
    }
    return total;
}

I wrapped a best-of-five loop with GC.GetAllocatedBytesForCurrentThread() and kept a checksum so I'd notice if either version quietly computed the wrong total.

Summing the amount column over 1,000,000 lines, best-of-5:

string.Split(',')     157,759,960 bytes/run      57 ms   (checksum 25500000)

Just over 150 MB for one pass, to read one column. That's an array plus three substrings per line, a million times, all of it garbage before the next iteration even starts.

Keeping the commas on the stack

ReadOnlySpan<char> has had a Split overload for a few versions now that I'd somehow never reached for on a real problem. Instead of handing back new strings, it writes the slice boundaries as Range values into a buffer you pass in. Stack-allocate that buffer and the whole operation touches the heap zero times.

static long SumWithSpanSplit(string[] lines)
{
    long total = 0;
    Span<Range> fields = stackalloc Range[4];
    foreach (string line in lines)
    {
        ReadOnlySpan<char> span = line;
        span.Split(fields, ',');
        total += int.Parse(span[fields[2]], CultureInfo.InvariantCulture);
    }
    return total;
}

span[fields[2]] slices the third range back out of the original line without copying it, and int.Parse has a ReadOnlySpan<char> overload, so the number comes straight off that slice.

Span<Range> split              40 bytes/run      21 ms   (checksum 25500000)

Same checksum, so same answer. The 40 bytes is measurement noise rather than per-line cost; the per-line allocation is flat zero. About 150 MB of garbage gone, and it finished in roughly a third of the time as a bonus. I came for the allocations and the speed tagged along.

Where I'd stop short

This isn't a "go replace every Split you own" post. If you split a config value once at startup, the array version is clearer and its allocation is a rounding error, so reach for the readable thing. The span version earns its keep on the hot paths where the same shape of line streams past millions of times.

Two sharp edges are worth knowing. The Span<Range> overload won't grow your buffer for you. If a line has more fields than the destination holds, the last Range gets the entire unsplit remainder instead of throwing. For fixed-shape data that's fine, since you size the buffer to the columns you expect, but ragged input will surprise you quietly. The other edge: the moment you actually need those fields as string values to keep past the loop, you're allocating them anyway and the span version buys you nothing. Its whole advantage is reading through data you're about to throw away.

My honest take is that the Span/Range APIs still read a little awkwardly. Slicing with span[fields[2]] isn't as obvious as fields[2], and that friction is probably why I skipped over them for so long. On a cold path that awkwardness isn't worth it. On the one loop lighting up your gen0 graph, 150 MB of avoidable garbage buys a slightly uglier line of code every time.

Here's the whole thing, runnable as-is on the .NET 10 SDK with dotnet run -c Release:

using System.Diagnostics;
using System.Globalization;

const int LineCount = 1_000_000;
const int Runs = 5;

string[] lines = new string[LineCount];
for (int i = 0; i < LineCount; i++)
    lines[i] = $"{i},widget-{i % 97},{(i % 50) + 1}";

// Warm up the JIT for both paths before measuring.
_ = SumWithStringSplit(lines);
_ = SumWithSpanSplit(lines);

Console.WriteLine($"Summing the amount column over {LineCount:n0} lines, best-of-{Runs}:");
Console.WriteLine();
Measure("string.Split(',')  ", () => SumWithStringSplit(lines));
Measure("Span<Range> split  ", () => SumWithSpanSplit(lines));

static long SumWithStringSplit(string[] lines)
{
    long total = 0;
    foreach (string line in lines)
    {
        string[] fields = line.Split(',');
        total += int.Parse(fields[2], CultureInfo.InvariantCulture);
    }
    return total;
}

static long SumWithSpanSplit(string[] lines)
{
    long total = 0;
    Span<Range> fields = stackalloc Range[4];
    foreach (string line in lines)
    {
        ReadOnlySpan<char> span = line;
        span.Split(fields, ',');
        total += int.Parse(span[fields[2]], CultureInfo.InvariantCulture);
    }
    return total;
}

static void Measure(string label, Func<long> work)
{
    GC.Collect();
    GC.WaitForPendingFinalizers();
    GC.Collect();

    long before = GC.GetAllocatedBytesForCurrentThread();
    long best = long.MaxValue;
    long checksum = 0;
    for (int r = 0; r < Runs; r++)
    {
        var sw = Stopwatch.StartNew();
        checksum = work();
        sw.Stop();
        best = Math.Min(best, sw.ElapsedMilliseconds);
    }
    long after = GC.GetAllocatedBytesForCurrentThread();

    long perRun = (after - before) / Runs;
    Console.WriteLine($"{label}{perRun,14:n0} bytes/run   {best,5} ms   (checksum {checksum})");
}

Measured in a container, best-of-five, nothing fancy. I care about the ratio, not the exact milliseconds your hardware would print.

Ever found a hot Split hiding inside a loop you'd written off as simple? Tell me where yours turned up.

— still benchmarking the boring loops nobody asked me to

My Logging Middleware Ate the Request Body

Sukhpinder Singh — Tue, 27 Oct 2026 07:16:42 +0000

I added request-body logging to an API one afternoon, the kind where you dump the raw payload so you can see exactly what a client sent. Harmless, right? The next morning a teammate pinged me because one of the POST endpoints had started rejecting perfectly good orders with validation errors. The payload looked fine in the logs. The handler swore the body was empty. Both were telling the truth.

The request body is a forward-only stream. Once something reads it to the end, the next reader gets nothing. My logging middleware read it to log it, and by the time model binding looked, there was nothing left to bind.

I wanted to see the byte counts instead of just nodding along to that explanation, so I built the smallest thing that reproduces it. One middleware that reads the body, one handler that reads it again, and a client that POSTs a real JSON payload over loopback. This is .NET 10, Release build, in a small Linux container. Nothing fancy.

Here's the shape of it. A middleware stands in for the "log every request body" code, and the handler does what your real endpoint would do when it reads the body itself:

app.Use(async (context, next) =>
{
    var read = await ReadBodyBytes(context.Request.Body);
    Console.WriteLine($"  [middleware] read {read} bytes from the body");
    await next(context);
});

app.MapPost("/orders", async (HttpContext ctx) =>
{
    var read = await ReadBodyBytes(ctx.Request.Body);
    Console.WriteLine($"  [handler]    read {read} bytes from the body");
    return Results.Ok(new { bytesHandlerSaw = read });
});

ReadBodyBytes just copies the stream into a MemoryStream and returns the length. The client POSTs a 61-byte JSON body. Here's what came back:

Client is POSTing a 61-byte JSON body.

== WITHOUT EnableBuffering ==
  [middleware] read 61 bytes from the body
  [handler]    read 0 bytes from the body
  [client]     server answered 200: {"bytesHandlerSaw":0}

There it is. The middleware got all 61 bytes. The handler got zero, and never knew anything was wrong. If that handler had been doing model binding instead, you'd get a null model or a "required field missing" error, and you'd spend a while staring at a request that looks completely valid in your logs.

The one line that fixes it

The fix is HttpRequest.EnableBuffering(). It swaps the raw request stream for one you can seek, so after you read to the end you can set Position = 0 and read it again. Call it before the first read, and rewind after:

app.Use(async (context, next) =>
{
    context.Request.EnableBuffering();

    var read = await ReadBodyBytes(context.Request.Body);
    Console.WriteLine($"  [middleware] read {read} bytes from the body");

    context.Request.Body.Position = 0; // rewind for whoever reads next
    await next(context);
});

Same run, buffering on:

== WITH EnableBuffering ==
  [middleware] read 61 bytes from the body
  [handler]    read 61 bytes from the body
  [client]     server answered 200: {"bytesHandlerSaw":61}

Both readers see 61 bytes now. The rewind is the part people forget, by the way. EnableBuffering alone makes the stream seekable, but it doesn't move the position back for you. If you read to the end and don't reset, the next reader still starts at the end and still gets zero. The seekable stream just means the zero is now your fault instead of the framework's.

What EnableBuffering actually costs

This isn't free, and the name undersells what it does. Behind the scenes it wraps the body in a FileBufferingReadStream. Small bodies stay in memory, but once a request crosses a threshold (around 30 KB by default) the buffer spills to a temporary file on disk. So if you turn this on globally and someone uploads a 200 MB file, you've just written 200 MB to disk that you may not have needed to.

My opinion, for what it's worth: reading the request body in middleware at all is the thing to be suspicious of, not just the buffering. If you can get what you need from the already-parsed model in the handler, or log a hash or a size instead of the full payload, do that and never touch the raw stream. Reach for EnableBuffering when you genuinely need two readers of the same bytes, cap it to the routes that need it, and set a reasonable buffer limit rather than flipping it on for the whole app.

When you do need it, here's the whole runnable sample. It builds the same tiny app twice, once without buffering and once with, and POSTs to each one so you can watch the counts yourself:

using System.Text;

var payload = """{"orderId":4821,"sku":"KB-87","qty":2,"note":"leave at door"}""";
Console.WriteLine($"Client is POSTing a {Encoding.UTF8.GetByteCount(payload)}-byte JSON body.\n");

await RunScenario("WITHOUT EnableBuffering", enableBuffering: false, 5080, payload);
Console.WriteLine();
await RunScenario("WITH EnableBuffering", enableBuffering: true, 5081, payload);

static async Task RunScenario(string label, bool enableBuffering, int port, string payload)
{
    Console.WriteLine($"== {label} ==");

    var builder = WebApplication.CreateBuilder();
    builder.Logging.ClearProviders();
    builder.WebHost.UseUrls($"http://127.0.0.1:{port}");
    var app = builder.Build();

    app.Use(async (context, next) =>
    {
        if (enableBuffering)
            context.Request.EnableBuffering();

        var read = await ReadBodyBytes(context.Request.Body);
        Console.WriteLine($"  [middleware] read {read} bytes from the body");

        if (enableBuffering)
            context.Request.Body.Position = 0;

        await next(context);
    });

    app.MapPost("/orders", async (HttpContext ctx) =>
    {
        var read = await ReadBodyBytes(ctx.Request.Body);
        Console.WriteLine($"  [handler]    read {read} bytes from the body");
        return Results.Ok(new { bytesHandlerSaw = read });
    });

    await app.StartAsync();

    using var http = new HttpClient(new SocketsHttpHandler { UseProxy = false });
    var content = new StringContent(payload, Encoding.UTF8, "application/json");
    var resp = await http.PostAsync($"http://127.0.0.1:{port}/orders", content);
    Console.WriteLine($"  [client]     server answered {(int)resp.StatusCode}: {await resp.Content.ReadAsStringAsync()}");

    await app.StopAsync();
}

static async Task<int> ReadBodyBytes(Stream body)
{
    using var ms = new MemoryStream();
    await body.CopyToAsync(ms);
    return (int)ms.Length;
}

Drop that into a Microsoft.NET.Sdk.Web project and dotnet run -c Release. The byte counts are the whole point: 61, then 0, then 61 again.

Have you been bitten by the empty-body-on-second-read thing? I'm curious whether it got you in middleware like me or somewhere sneakier, like an exception filter. Tell me where it bit you.

— still measuring bytes nobody asked me to count

StartsWith Was Asking ICU Every Time

Sukhpinder Singh — Mon, 26 Oct 2026 08:27:43 +0000

I had an endpoint that sorts log lines as they come off a queue: route the ERROR ones one way, everything else another. The hot loop is one check, does this line start with "ERROR", and I'd written line.StartsWith("ERROR") without a second thought. It's the most obvious thing you can type.

Then I had the service under load with a profiler open, and a surprising slice of time was parked inside CompareInfo and the globalization stack underneath it. For a prefix check on ASCII log lines. That's when it clicked that StartsWith(string) doesn't do what I thought it did.

The default overload is linguistic, not literal

Here's the thing I'd forgotten. string.StartsWith(string) and string.IndexOf(string), the overloads that take only a string, don't compare characters. They compare text, using the current culture's rules, which on most boxes means routing through ICU for a full linguistic comparison. That's the machinery that knows "ﬀ" and "ff" are the same word, that a soft hyphen shouldn't count, that casing folds differently in Turkish. Handy when you're sorting names a person will read. Total overkill when you're asking whether a byte string begins with five known ASCII letters.

The fix is to say what you actually meant:

line.StartsWith("ERROR", StringComparison.Ordinal)

Ordinal means compare the char values, in order, and stop. No culture, no collation table, no ICU.

I counted it

I don't trust my gut on this stuff, so I wrote the smallest thing that would show the gap. Two million log-ish lines in an array, five prefixes mixed in so the comparison has to actually look at characters:

string[] prefixes = { "INFO ", "WARN ", "DEBUG ", "ERROR ", "TRACE " };
var lines = new string[2_000_000];
for (int i = 0; i < lines.Length; i++)
    lines[i] = prefixes[i % prefixes.Length] + "request " + i + " finished in " + (i % 400) + "ms";

Then count how many start with "ERROR", the polite way and the ordinal way:

static long CountCulture(string[] lines)
{
    long n = 0;
    foreach (string s in lines)
        if (s.StartsWith("ERROR")) n++;                       // CurrentCulture
    return n;
}

static long CountOrdinal(string[] lines)
{
    long n = 0;
    foreach (string s in lines)
        if (s.StartsWith("ERROR", StringComparison.Ordinal)) n++;
    return n;
}

The harness is nothing fancy. A stopwatch, a warmup call for each path so the JIT and the ICU tables are hot, then the best time out of seven, with the allocation counter read around each pass:

static long Measure(Action a, out long bestMs)
{
    bestMs = long.MaxValue;
    long bytes = 0;
    var sw = new Stopwatch();
    for (int run = 0; run < 7; run++)
    {
        long before = GC.GetAllocatedBytesForCurrentThread();
        sw.Restart();
        a();
        sw.Stop();
        bytes = GC.GetAllocatedBytesForCurrentThread() - before;
        if (sw.ElapsedMilliseconds < bestMs) bestMs = sw.ElapsedMilliseconds;
    }
    return bytes;
}

It's a single-file console app on .NET 10, top-level statements, nothing else in it. Here's what came back:

StartsWith (culture) :    27 ms
StartsWith (ordinal) :    18 ms
ratio                : 1.5x
allocated (culture)  : 0 bytes
allocated (ordinal)  : 0 bytes

That was a bit of a letdown, if I'm honest. I'd built this up in my head as the villain and it came in at 1.5x. Real, repeatable, but not the kind of number that changes your afternoon. And neither path allocates a byte, so this is pure CPU, no garbage to chase.

Then I ran the same question through IndexOf, because plenty of my "does this contain" checks are written the same lazy way:

IndexOf (culture)    :   109 ms
IndexOf (ordinal)    :    30 ms
ratio                : 3.6x

There's the one that was actually hurting. A substring search under a linguistic comparison is a lot more work than a prefix check. It can't just glance at the front, it has to weigh matches at every position with full collation rules. Roughly 3.6x on the same data. That's the call I should have been worried about from the start.

These are ratios off a Linux container, not a lab bench with the clocks pinned, so the absolute milliseconds will land somewhere else on your hardware. The shape holds up though: ordinal is cheaper, and substring search is cheaper by a lot more than a prefix check.

Ordinal isn't a free find-and-replace

Here's the caveat, and it's a real one. Ordinal and culture don't always give the same answer. That's the entire reason they're separate methods, and it's why you can't just sweep the codebase swapping one for the other.

string shy = "coop.log";                       // a soft hyphen sits between "co" and "op"
shy.StartsWith("coop");                               // True  - culture treats the hyphen as invisible
shy.StartsWith("coop", StringComparison.Ordinal);     // False - ordinal sees the hidden char

Culture-aware comparison treats that soft hyphen as nothing, so it matches "coop". Ordinal sees nine characters and says no. For log routing, ordinal is exactly what I want: the literal content, invisible characters and all. For anything a human typed and expects to match "naturally", though, like search boxes, usernames, display sorting, the linguistic version is the correct one, and the extra cost is the price of being correct.

My rule of thumb now: if the strings are protocol rather than prose, so headers, enum names, file extensions, log levels, anything machine-generated, reach for Ordinal or OrdinalIgnoreCase by default. If a person wrote it and a person will read it, leave the culture-aware comparison where it is. The analyzer CA1310 flags exactly these call sites if you switch it on, which beats grepping for them by hand.

The quiet irony is that ordinal is both faster and usually what you meant for this kind of data. It's just not the default, and the default looks so harmless sitting there.

So what's the laziest string comparison hiding in your hot path? Check the Contains calls first, those are IndexOf underneath.

— still benchmarking corners of the standard library nobody asked me to

The GUID That Sorts Itself (Until It Doesn't)

Sukhpinder Singh — Mon, 26 Oct 2026 07:28:38 +0000

I was looking at an index on a table whose primary key is a uniqueidentifier, and the fragmentation number was ugly. Every insert dropped a fresh Guid.NewGuid() into a random spot in the key range, so the B-tree was constantly splitting pages to make room in the middle. Classic random-GUID problem. I'd read about Version 7 GUIDs fixing this, and .NET has Guid.CreateVersion7() now, so I figured I'd swap it in and move on.

Then I got curious about how far I could actually trust the sort. So I did the thing I always do: wrote the smallest program that would tell me, and ran it instead of guessing.

What v4 keys look like when you line them up

Here's the baseline. Make six ordinary GUIDs a couple milliseconds apart, print them in the order I made them, then sort them and ask if the sorted order matches:

var v4Keys = new List<Guid>();
for (int i = 0; i < 6; i++)
{
    v4Keys.Add(Guid.NewGuid());
    Thread.Sleep(2);
}

var v4Sorted = v4Keys.OrderBy(g => g).ToList();
Console.WriteLine($"Sorted == creation order? {v4Sorted.SequenceEqual(v4Keys)}");

a23b61f1-613d-464a-a2e2-7ac3ccf522e0
ebd6f78a-afbc-42de-9450-cdee1416c9e9
3001f306-6a01-4556-b338-b3649735a280
dcf56d17-4dd3-4087-8409-9958e0e17f61
a06dcf69-ccc2-4642-b610-b21bfd05b99e
b4da4432-b310-46fc-877d-3bf6aa89de0d
Sorted == creation order? False

No surprise. A v4 GUID is 122 random bits. The value you get has nothing to do with when you asked for it, which is exactly why it scatters across the index.

v7 keys carry the time in their front teeth

A Version 7 GUID puts a 48-bit Unix-millisecond timestamp in the high bits and fills the rest with randomness. Same loop, same two-millisecond gaps, swap the factory:

var v7Keys = new List<Guid>();
for (int i = 0; i < 6; i++)
{
    v7Keys.Add(Guid.CreateVersion7());
    Thread.Sleep(2);
}

01a17719-db51-7dc3-b700-0ff42fff55da  @ 07:25:45.681
01a17719-db54-713b-adec-96911e0ab1fe  @ 07:25:45.684
01a17719-db56-7379-9292-eedae31b7e54  @ 07:25:45.686
01a17719-db58-764f-b15b-faa65bb7ff23  @ 07:25:45.688
01a17719-db5a-775e-a3b6-e12cee035baf  @ 07:25:45.690
01a17719-db5c-7722-8037-9f652b6883f6  @ 07:25:45.692
Sorted == creation order? True

Look at the left edge of each one. The prefix climbs in lockstep with the clock, so sorting the bytes sorts by time. New keys land at the end of the range instead of the middle, and that's the whole point for a database: inserts append, pages stop splitting.

And that timestamp isn't a party trick, you can read it straight back out of the key. The byte layout is big-endian for the RFC, so I ask for the bytes in that order and pull the first six:

static DateTimeOffset ReadTimestamp(Guid guid)
{
    Span<byte> bytes = stackalloc byte[16];
    guid.TryWriteBytes(bytes, bigEndian: true, out _);
    Span<byte> ms = stackalloc byte[8];
    bytes[..6].CopyTo(ms[2..]);   // 48-bit value, right-aligned in a long
    long unixMs = BinaryPrimitives.ReadInt64BigEndian(ms);
    return DateTimeOffset.FromUnixTimeMilliseconds(unixMs);
}

That bigEndian: true matters. Guid stores its first three fields in little-endian internally, so the default ToByteArray() gives you a jumbled prefix and the timestamp comes out as nonsense. The overload that takes the flag is the one you want.

The part the blog posts skip

Here's where I stopped trusting the word "sortable" and made it earn it. Those sleeps above are doing a lot of work, each key got its own millisecond. Real inserts don't wait politely. So I generated a hundred thousand of them as fast as the loop would go and asked the same question:

const int burst = 100_000;
var sameMs = new Guid[burst];
for (int i = 0; i < burst; i++)
    sameMs[i] = Guid.CreateVersion7();

var byValue = sameMs.OrderBy(g => g).ToArray();
bool matches = byValue.AsSpan().SequenceEqual(sameMs);

Burst of 100,000 v7 GUIDs took 71.8 ms
  spanned 55 distinct millisecond bucket(s)
  sorting recovers exact creation order? False

False. A hundred thousand keys fell into 55 millisecond buckets, so roughly eighteen hundred of them share each bucket, and inside a bucket the ordering bits are random. Sorting gets you to the right millisecond and then shrugs. I printed a pair to see it plainly:

Same millisecond, created a then b, but b < a byte-wise:
  a (first)  01a17719-db60-79bc-84dd-1b62234a3abb
  b (second) 01a17719-db60-753f-90e6-9be4edb208b7

Created a first, b second, same millisecond, and b sorts ahead of a. That's not a bug. .NET 10's implementation doesn't add a monotonic counter in the low bits, so two keys born in the same millisecond can come back in either order. (The spec leaves room for a counter; this runtime doesn't use one.)

This is a container, not a lab bench, and the 71.8 ms will be different on your box. The number I care about is 55 buckets, because it says the thing that matters: at insert speed, "sorted" means sorted to the millisecond, nothing finer.

So where I actually landed

For database keys, v7 is a straight upgrade over v4 and I'd reach for it by default now. The index locality is real, and getting a creation timestamp baked into the key for free is a nice bonus when you're squinting at rows in production. My one caution: don't let "time-ordered" slide into "ordered." If you need to know which of two rows was inserted first, and they might land in the same millisecond, you still need a real sequence or an identity column. The GUID won't settle that argument, and it'll look like it might, which is worse.

Want to see it on your own hardware? Here's the whole thing in one file, Program.cs, targeting net10.0 (it runs fine on .NET 9 too). No packages.

using System.Buffers.Binary;
using System.Diagnostics;

Console.WriteLine($"Running on .NET {Environment.Version}\n");

var v4Keys = new List<Guid>();
for (int i = 0; i < 6; i++) { v4Keys.Add(Guid.NewGuid()); Thread.Sleep(2); }
Console.WriteLine("v4 in creation order:");
foreach (var g in v4Keys) Console.WriteLine($"  {g}");
var v4Sorted = v4Keys.OrderBy(g => g).ToList();
Console.WriteLine($"  Sorted == creation order? {v4Sorted.SequenceEqual(v4Keys)}\n");

var v7Keys = new List<Guid>();
for (int i = 0; i < 6; i++) { v7Keys.Add(Guid.CreateVersion7()); Thread.Sleep(2); }
Console.WriteLine("v7 in creation order:");
foreach (var g in v7Keys) Console.WriteLine($"  {g}  @ {ReadTimestamp(g):HH:mm:ss.fff}");
var v7Sorted = v7Keys.OrderBy(g => g).ToList();
Console.WriteLine($"  Sorted == creation order? {v7Sorted.SequenceEqual(v7Keys)}\n");

const int burst = 100_000;
var sameMs = new Guid[burst];
var sw = Stopwatch.StartNew();
for (int i = 0; i < burst; i++) sameMs[i] = Guid.CreateVersion7();
sw.Stop();
int buckets = sameMs.Select(ReadTimestamp).Distinct().Count();
var byValue = sameMs.OrderBy(g => g).ToArray();
bool matches = byValue.AsSpan().SequenceEqual(sameMs);
Console.WriteLine($"Burst of {burst:N0} v7 GUIDs took {sw.Elapsed.TotalMilliseconds:F1} ms");
Console.WriteLine($"  spanned {buckets} distinct millisecond bucket(s)");
Console.WriteLine($"  sorting recovers exact creation order? {matches}\n");

for (int i = 1; i < burst; i++)
{
    if (ReadTimestamp(sameMs[i]) == ReadTimestamp(sameMs[i - 1]) &&
        sameMs[i].CompareTo(sameMs[i - 1]) < 0)
    {
        Console.WriteLine("Same millisecond, created a then b, but b < a byte-wise:");
        Console.WriteLine($"  a (first)  {sameMs[i - 1]}");
        Console.WriteLine($"  b (second) {sameMs[i]}");
        break;
    }
}

static DateTimeOffset ReadTimestamp(Guid guid)
{
    Span<byte> bytes = stackalloc byte[16];
    guid.TryWriteBytes(bytes, bigEndian: true, out _);
    Span<byte> ms = stackalloc byte[8];
    bytes[..6].CopyTo(ms[2..]);
    long unixMs = BinaryPrimitives.ReadInt64BigEndian(ms);
    return DateTimeOffset.FromUnixTimeMilliseconds(unixMs);
}

Have you moved any tables over to v7 keys yet, or are you still sitting on sequential integers? Curious what the fragmentation looked like before and after if you have.

— still benchmarking things nobody asked me to measure