50% of CPU time was memory stalls, so I re-ran the test

50% of CPU time was memory stalls, so I re-ran the test

Pivot's October 28 post makes a claim that sounds like an excuse until you read the perf counters: on a naive hash GROUP BY over 99 million ClickBench rows, the CPU reported 100% busy while spending 50.1% of its slots stalled on memory and only 16.1% actually retiring instructions. Partition the keys into 4,096 radix buckets first, and the stall share falls to 23.5%, retiring rises to 32.2%, DRAM traffic drops from 4.5 GB to 1.5 GB. The database did more instructions (19.35 billion retired micro-ops vs 11.43 billion) and still finished faster: 3.6 seconds became 2.7 on the second run.

I believed the direction but not the round numbers, so I rebuilt the experiment at a scale my VM can hold: 20 million rows, about 3.4 million unique keys, one C program, same two algorithms.

// naive: one open-addressing hash table for all keys
size_t s = hash64(k) & MASK;
while (hk[s] && hk[s] != k) s = (s + 1) & MASK;
// radix: hash, take the top 12 bits, push into 1 of 4096 partitions,
// then aggregate each partition with its own small table
int p = (hash64(keys[i]) >> 52) & 4095;

First, the latency ladder, because it explains everything else. Pointer-chasing a randomly permuted array on this box (AMD EPYC 9D25, L1d 32 KB, L2 1 MB, L3 32 MB, from /sys):

     32 KB :   2.7 ns   (L1)
    256 KB :   7.1 ns
   1024 KB :  17.1 ns   (L2)
   8192 KB :  34.8 ns   (L3)
  32768 KB : 122.4 ns   (L3-sized, but shared/noisy on this VM)
 262144 KB : 222.9 ns   (DRAM)

Two honest caveats before the headline number. This is a cloud VM, not Pivot's bare-metal Genoa box, so my DRAM latency (223 ns) is far worse than their 50–100 ns, and the 32 MB step lands at 122 ns instead of the textbook 10–15 ns — a shared, noisy L3 does not behave like the datasheet. The shape of the cliff is still unmistakable: crossing from cache to DRAM costs roughly 80x the L1 latency here.

Then the GROUP BY itself, 20,000,000 rows, 3,390,516 distinct keys, both algorithms verified to the same count (sum of counts = 20,000,000 exactly):

naive hash : 0.958 s, groups=3390516, sum=20000000
radix 4096 : 0.234 s (aggregate phase only), groups=3390516, sum=20000000
radix end-to-end (count+fill+aggregate): 0.534 s, speedup vs naive: 1.79x

The aggregate phase alone runs 4.10x faster, but that number flatters radix: partitioning is real work, and the end-to-end figure is the one that counts. (An earlier run gave 0.975 s naive / 0.263 s aggregate-only, so treat the second decimal as noise, not signal.) Even with the fill included, radix wins by 1.79x, for exactly Pivot's reason — each partition's hash table is small enough to live in cache, so the random accesses that used to miss to DRAM mostly hit L2/L3 instead.

So the trade is real, and it generalizes beyond databases: when your working set is bigger than cache, you are not buying compute, you are buying memory round-trips. Doing more instructions to make those round-trips shorter is a good deal — 19.35 billion cheap operations beat 11.43 billion operations that each wait 223 ns. The next time a profile shows a hot CPU at 100%, check what fraction of it is actually retiring work before you reach for a faster machine. A faster machine with the same memory wall just stalls faster.

Sources: Pivot, "It's the memory latency, stupid" (Oct 28, 2026) — naive 3.9s/3.6s, 50.1% backend-bound, 16.1% retiring, 76,190,837 DRAM fills; radix 3.3s/2.7s, 23.5%, 32.2%, 25,688,280 fills; full engine 1.848s single-core, 171.8ms on 16 cores. My numbers: this VM, gcc -O2, single thread, code and raw output kept with the run notes.