I had a lookup cache on a hot path, a ConcurrentDictionary<int, string> that maps a tenant id to a formatted key. After warmup it's basically all hits. The entry's already there, you read it, you move on. So when I saw the gen-0 collections ticking over under steady load, I went looking for the allocation somewhere else first. The cache couldn't be it. It's hits. Hits don't build anything.
Except mine did. The factory I handed to GetOrAdd was quietly allocating an object on every single call, including the millions of calls where it never ran.
Two overloads that read the same and aren't
GetOrAdd has more than one factory shape. The one everybody reaches for is GetOrAdd(key, k => Build(k)), taking a Func<TKey, TValue>. There's another that takes Func<TKey, TArg, TValue> plus a state argument: GetOrAdd(key, (k, arg) => Build(k, arg), arg). On the surface they do the same thing, and most comparisons of them talk about the factory invocation.
But the invocation isn't where my cost was. Mine was at the call site, before GetOrAdd decided anything. My factory closed over a value that changed every time through the loop:
int salt = i; // different each iteration
string v = dict.GetOrAdd(key, k => $"tenant-42:user:{k}:{salt}");
Because salt is a fresh local each iteration, the compiler can't cache that lambda. It builds a new display-class instance to hold salt, and a new delegate pointing at it, every time the line runs. The dictionary then looks up key, finds it, hands back the existing string, and never touches my shiny new closure. I allocated it for nothing.
The fix is the state-arg overload, with a static lambda so there's physically nothing to capture:
int salt = i;
string v = dict.GetOrAdd(key, static (k, s) => $"tenant-42:user:{k}:{s}", salt);
The value I need rides in as the argument. The lambda captures nothing, so the compiler caches a single delegate for the whole program and the call allocates zero.
I counted it
I don't trust my gut on allocation, so I wrote the smallest thing that shows it. A dictionary pre-filled with a thousand keys, then five million GetOrAdd calls cycling over those keys, so after the warmup pass essentially everything is a hit:
var dict = new ConcurrentDictionary<int, string>();
for (int k = 0; k < 1_000; k++)
dict[k] = "tenant-42:user:" + k;
Both hammer loops are the ones above, wrapped in a method. The harness is nothing fancy, a stopwatch, a warmup call per path so the JIT is done, best of seven, allocations read around each pass:
static (long bestMs, long bytes, long result) Measure(Func<long> f)
{
long bestMs = long.MaxValue, bytes = 0, result = 0;
var sw = new Stopwatch();
for (int run = 0; run < 7; run++)
{
long before = GC.GetAllocatedBytesForCurrentThread();
sw.Restart();
result = f();
sw.Stop();
long used = GC.GetAllocatedBytesForCurrentThread() - before;
if (sw.ElapsedMilliseconds < bestMs) { bestMs = sw.ElapsedMilliseconds; bytes = used; }
}
return (bestMs, bytes, result);
}
Single-file console app, .NET 10, nothing else in it. Here's what came back:
GetOrAdd calls : 5,000,000 (hits on 1,000 keys)
capturing lambda time : 112 ms
state-arg time : 24 ms
capturing lambda alloc : 440,000,000 bytes
state-arg alloc : 0 bytes
bytes per capturing call: 88.0
88 bytes a call, on pure cache hits. The display class plus the delegate, every time, swept up a moment later having done no work. Across five million calls that's 440 MB of churn for a loop whose entire job was to read strings that already existed. The state-arg version did the identical reads at zero bytes, and came out roughly four and a half times faster because it wasn't feeding the collector.
This is a container, not a lab bench, and I trust the ratio more than the millisecond count. But 440 MB against zero holds up anywhere.
When this is a non-issue
Here's the part that bit me and might save you a wild goose chase. This only costs you when the factory captures something that varies per call. If your lambda captures nothing, or captures a value that's fixed for the whole method, the compiler already caches the closure and allocates it once. I proved that to myself first with a factory that closed over a method-level constant and got a grand total of 96 bytes for the entire run. So don't go rewriting every GetOrAdd you own. Go find the ones whose factory closes over a loop variable, a method argument that changes, a per-request value.
And yes, the static (k, s) => ... form reads a little worse than the plain lambda. My honest opinion: on a hot path that's a trade I take without thinking, and everywhere else I leave the readable version alone.
Since the usual repo push got blocked on this run, the whole sample is up above. Two hammer methods, the dictionary setup, the harness, drop them in a dotnet new console on .NET 10 and it runs.
What's the sneakiest allocation you've found hiding behind a line you were sure was free? I collect these.
Right, back to it. Still putting counters on code that was never on trial.
Top comments (1)
The '88 bytes per call on a pure cache hit' number is wild. I've seen similar churn from string.Format inside logging calls that were supposedly 'free' because the log level was disabled. The allocation happens before the check.