AI Wrote a Thread-Safe Counter. The CPU Made It 5x Slower.

Meet the cache line.

I asked an AI assistant for a simple thing: per-thread counters. Four threads, each incrementing its own slot in an array. No shared variables. No locks needed. The code it wrote was clean and correct:

long[] counters = new long[threadCount];
// thread t does: counters[t]++

Every thread writes only to its own element. There is no race here. Any code review would pass it. Every test would pass too.

Then I measured it against a version that does exactly the same work, and the correct code lost by 5x.

The numbers

This is from my machine (Apple Silicon, .NET 10, 4 threads, 200 million increments per thread):

Round 1:  adjacent (one cache line):  464 ms   padded (line per thread):  85 ms   ratio: 5.4x
Round 2:  adjacent (one cache line):  397 ms   padded (line per thread):  85 ms   ratio: 4.7x

Same loop. Same number of increments. Same "each thread touches only its own counter". The only difference between the two versions is where the counters live in memory.

What the hardware actually does

A CPU never reads one byte from memory. It moves data in fixed blocks called cache lines — 64 bytes on x86, 128 bytes on Apple Silicon. Ask for one long and the whole block it lives in travels into the core's cache. Think of a cook whose ingredients are in a basement fridge: going downstairs is expensive, so you never carry one carrot — you carry the whole crate.

(Don't take my word for the 128: run sysctl hw.cachelinesize on an M-series Mac.)

Usually this works for you. Array elements sit side by side, so scanning an array is fast: you touch one element and the next fifteen arrive in the same crate for free.

But with multiple cores there is a rule: to write into a cache line, a core must own it exclusively. The moment core 1 writes, every other core's copy of that line is declared stale. And the unit of ownership is not your variable. It is the whole line.

Now look at my four counters. Four long values, 8 bytes each, side by side — 32 bytes. They all fit in one cache line. Thread 1 increments its counter and takes ownership of the line. A nanosecond later thread 2 increments its own, different counter — and has to rip the same line back. The line ping-pongs between cores on every single write. Four threads that share nothing in the code are fighting over one crate in the hardware.

This is called false sharing. False, because no data is actually shared. The fight is real anyway.

Four counters in one cache line vs. one counter per line — same code, 5x difference.

The demo fix is one attribute

Give every thread its own cache line. Pad each counter so the next one starts in a different line:

[StructLayout(LayoutKind.Explicit, Size = 128)]
struct PaddedCounter
{
    [FieldOffset(64)] public long Value;
}

That is the entire difference between 464 ms and 85 ms. We pay a little memory — 128 bytes per counter instead of 8 — and get back the parallelism we thought we already had. (I use 128, not 64, for two reasons: Apple Silicon lines are 128 bytes, and on x86 the adjacent-line prefetcher likes to drag neighboring lines along.)

Run it yourself

The full demo is about 70 lines, no project file needed — with the .NET 10 SDK you can run a single file directly:

dotnet run -c Release FalseSharingDemo.cs

// FalseSharingDemo.cs — .NET 10 file-based app
using System.Diagnostics;
using System.Runtime.InteropServices;

const long Iterations = 200_000_000;
int threadCount = Math.Min(Environment.ProcessorCount, 4);

for (int round = 0; round < 2; round++)
{
    var slow = MeasureAdjacent(threadCount);
    var fast = MeasurePadded(threadCount);
    Console.WriteLine($"adjacent: {slow.TotalMilliseconds:F0} ms   padded: {fast.TotalMilliseconds:F0} ms   ratio: {slow / fast:F1}x");
}

static TimeSpan MeasureAdjacent(int threadCount)
{
    long[] counters = new long[threadCount];
    return RunThreads(threadCount, t =>
    {
        for (long i = 0; i < Iterations; i++)
            counters[t]++;
        return counters[t];
    });
}

static TimeSpan MeasurePadded(int threadCount)
{
    var counters = new PaddedCounter[threadCount];
    return RunThreads(threadCount, t =>
    {
        for (long i = 0; i < Iterations; i++)
            counters[t].Value++;
        return counters[t].Value;
    });
}

static TimeSpan RunThreads(int threadCount, Func<int, long> body)
{
    long sink = 0;
    var threads = new Thread[threadCount];
    var sw = Stopwatch.StartNew();
    for (int t = 0; t < threadCount; t++)
    {
        int id = t;
        threads[t] = new Thread(() => Interlocked.Add(ref sink, body(id)));
        threads[t].Start();
    }
    foreach (var th in threads) th.Join();
    sw.Stop();
    GC.KeepAlive(sink);
    return sw.Elapsed;
}

[StructLayout(LayoutKind.Explicit, Size = 128)]
struct PaddedCounter
{
    [FieldOffset(64)] public long Value;
}

Where this hides in real code

You will not write four counters in a loop at work. But you will write, or an AI will write for you:

Cache statistics. Almost every in-memory cache keeps hits, misses, evictions. The natural implementation is fields next to each other, or a long[] with one slot per shard, updated with Interlocked.Increment from every thread. Fields next to each other means one cache line. This is literally the demo above, running in your production service.

Sharded counters. The cruel version: you sharded a counter specifically to make it parallel, put the shards in one array — and they still share lines. You did the architecture work and the hardware quietly undid it.

LRU metadata. A compact long[] lastAccessTicks per cache slot means every cache read becomes a write into a hot shared array. A read-heavy cache that is slow because of writes is a fun thing to debug.

And sometimes the problem ships inside the library. ConcurrentDictionary — the base of most homemade .NET caches — internally keeps a counter per lock stripe in a plain array (_countPerLock in the source). Under very hot multi-threaded writes those neighbors can end up sharing lines. I have not benchmarked the real-world impact — but the layout is right there in the source, and now you know what to look for.

When you should NOT care

Honesty section. False sharing hurts when the writes are hot — millions of updates per second from several threads. If your cache updates its stats a thousand times per second, you will never notice, and padding everything "just in case" is cargo cult. The rule is the same as always: measure first. The demo above is the measurement; adapt it to your data layout and see if your ratio is 1.0x or 5x.

The actual point

The AI-generated code was not wrong. It compiled, it was race-free, it passed every test I could write for its correctness. An entire code review process could bless it. The 5x was invisible at every layer we normally check.

That is what changed with AI-assisted coding, and it is why hardware fundamentals became more valuable, not less. The model will happily generate a thread-safe counter, a sharded cache, an LRU eviction policy — and none of its correctness guarantees say anything about cache lines. Correctness and mechanical sympathy are different layers. Tests catch the first. Only understanding catches the second.

You do not need to memorize cache sizes. You need to know the crate exists. Keep data that is used together close. Keep data that is written by different threads apart. That one rule, read in both directions, is most of "cache-aware" programming.

The tools write the code now. Knowing why it is slow is still our job.

---

Originally published on vasyl.blog.

Comments

[Alex Shev]

This is exactly why AI-generated concurrency code needs benchmark and profiling feedback, not just correctness tests. A counter can be technically safe and still be the wrong implementation for the CPU path.

[Vasyl]

Yes, exactly. All unit tests were green, the code was correct. The 5x slowdown showed up only when I ran it under BenchmarkDotNet with contention. So for concurrency code, now I treat benchmark as part of the loop, not as afterthought.

[Alex Shev]

That is the exact lesson: correctness tests and performance tests answer different questions. For concurrency, green unit tests only prove the logic survived; benchmarks under contention prove the design survived.

[Vasyl]

Exactly, "green tests prove the logic survived, benchmarks under contention prove the design survived" is the cleaner way to say it. Stealing that line.

[Alex Shev]

Exactly. The line works because it separates two proofs that often get blurred together. A concurrency change can be logically correct and still operationally bad once contention, cache behavior, or scheduling shows up.

[Vinicius Pereira]

Your padding choice is more right than the article lets on, and for two reasons worth naming. First, the GC only gives managed objects pointer alignment, 8 bytes on x64, and there is no supported way to request cache-line alignment for a heap allocation. So a struct sized exactly 64 bytes inside an array can still straddle two lines depending on where the array happens to land. Sizing to 128 with the value in the middle makes the layout safe at any start alignment, not just the lucky ones. Second, Intel's optimization manual documents a spatial prefetcher that completes every fetched line to its 128-byte pair in L2, which is why libraries like Folly pad to 128 even on 64-byte-line x86.

The nicest receipt: the BCL does exactly what you did. In dotnet/runtime, ConcurrentQueueSegment has a PaddedHeadAndTail struct with LayoutKind.Explicit, Size = 3 * CACHE_LINE_SIZE, Head at offset 1x and Tail at 2x, padding before, between and after. And CACHE_LINE_SIZE there is 128 on ARM64, matching your Apple Silicon numbers. The AI wrote the naive version; the runtime team wrote yours.

On finding this in production: false sharing involves no lock, so lock-contention profilers report a clean bill while throughput craters, and that disagreement is itself the diagnostic. It surfaces as memory-bound stalls. On Linux, perf c2c was built for exactly this and ranks cache lines by cross-core hits on modified lines, with the offsets within the line. Correct under test, slow under load is a bug class only measurement can see. The counter compiled, the tests passed, and the only witness was the wall clock.

[Vasyl]

Nice, the ConcurrentQueue padding is the perfect proof. And perf c2c is exactly the right tool for this. Thanks.