DEV Community

Cover image for malloc did 1,000 allocations with 1 syscall
frank chu
frank chu

Posted on

malloc did 1,000 allocations with 1 syscall

malloc did 1,000 allocations with 1 syscall

One brk syscall grew my heap by 132 KiB and served all 1,000 malloc(16) calls. The same program, compiled the way I first wrote it, made zero heap syscalls at all — because GCC deleted every allocation.

The re-run

Ben Andrew's "What is malloc doing, anyway?" hit Hacker News today (213 points), and it is a beautiful piece of measurement: strace the syscalls, read the chunk headers, time the calls, build a toy allocator stage by stage. I re-ran his numbers on my own machine — Ubuntu, glibc 2.39, 4 vCPUs — to see which of them are universal and which are his machine. Most of them held exactly. One of them did not, and the reason is the most interesting part.

1,000 mallocs, 1 syscall

His program: malloc(16) a thousand times into a static array. His strace showed two brk calls — one to ask where the break is, one to grow the heap by 132 KiB. Mine showed the same shape:

brk(NULL)                 = 0x59362db06000
brk(0x59362db27000)       = 0x59362db27000
Enter fullscreen mode Exit fullscreen mode

The growth is 0x21000 bytes — 135,168, exactly 132 KiB. Same number, different machine, different glibc. The other 999 allocations never touch the kernel; they carve slices out of that one block. That is the whole point of a userspace allocator, and it is reassuring that the number is stable across machines: glibc's main arena grabs 132 KiB on first use and deals from it.

The chunk headers match to the byte

Reading the size word stored just before each returned pointer, glibc's chunk layout is identical on my build:

  request    chunk
        1       32
       24       32
       25       48
     1000     1008
Enter fullscreen mode Exit fullscreen mode

Request plus 8 bytes of header, rounded up to a multiple of 16, minimum 32. His table is byte-for-byte the same. This is the part of malloc that never changes: the rounding rule is load-bearing for alignment, and every glibc you meet will print this table.

Where my numbers diverge: the syscall cost

Timed over a million calls each:

syscall(getppid)    496.0 ns    (his: 575.2 ns)
sbrk(16)            517.6 ns    (his: 610.1 ns)
malloc(16)           58.9 ns    (his:  26.9 ns)
sbrk/malloc           8.8x       (his:  22.7x)
Enter fullscreen mode Exit fullscreen mode

The syscalls are cheaper on my VM than on his machine; malloc is more than twice as slow — 58.9 ns against his 26.9. So the ratio collapses from 22.7x to 8.8x. The lesson is not that one of us is wrong: syscall latency and allocator latency move independently with CPU, virtualization, and spectre mitigations. The ratio is a property of a machine, not of malloc. If you quote "malloc is 20x cheaper than a syscall," quote the machine it was measured on.

The mmap threshold is exactly 128 KiB

Allocations above glibc's mmap threshold bypass the heap and get their own mapping. I probed either side of the default:

120 KiB malloc -> no mmap; served from the brk heap
132 KiB malloc -> mmap(NULL, 139264, ...)   (132 KiB + overhead)
  1 MiB malloc -> mmap(NULL, 1052672, ...)  (1 MiB + overhead)
Enter fullscreen mode Exit fullscreen mode

120 KiB stays on the heap; 132 KiB gets its own anonymous mapping, which is also why freeing a large block returns memory to the kernel immediately while freeing a small one does not. The default threshold is 128 KiB and it is a live, observable boundary, not folklore.

The compiler deleted my benchmark

Here is the part I did not expect. My first build of the 1,000-malloc program showed one brk call — a query, no growth — and the 1 MB program showed no large mmap at all. objdump explained it:

$ objdump -d m1m | grep -c "call.*malloc"
0
Enter fullscreen mode Exit fullscreen mode

GCC at -O2 had removed every malloc call. The pointers went into a static array that nothing ever read, so the compiler proved the allocations unobservable and deleted them — malloc is annotated in a way that permits this. The fix was to make the pointers escape: store them in a global array, touch a byte in each block, print a checksum. Only then did the syscalls appear. If you have ever benchmarked an allocator and gotten suspiciously fast numbers, check whether the compiler ran your benchmark at all.

One thing he didn't measure: thread contention

The post is single-threaded throughout. glibc's answer to multithreaded malloc is per-thread arenas, so I measured 2M malloc(16)/free pairs per thread:

threads   aggregate throughput
1         60.2 Mops/s  (16.6 ns/op)
2        113.3 Mops/s   (8.8 ns/op)
3        169.0 Mops/s   (5.9 ns/op)
4         91.6 Mops/s  (10.9 ns/op)
Enter fullscreen mode Exit fullscreen mode

Scaling is nearly linear to 3 threads — the arenas are doing their job, and the per-op cost here is the tcache fast path, which is why single-threaded numbers look absurdly small. At 4 threads throughput falls off a cliff, from 169 to 92 Mops/s. That is 4 workers plus the main thread on 4 vCPUs: the moment threads outnumber cores, arena isolation stops helping and scheduling takes over. The allocator scales until the machine doesn't.

The scoreboard

claim his machine my machine
heap growth for 1,000 x malloc(16) 132 KiB, 2 brk calls 132 KiB, 2 brk calls
chunk for request 1 / 24 / 25 / 1000 32 / 32 / 48 / 1008 32 / 32 / 48 / 1008
empty syscall 575.2 ns 496.0 ns
sbrk(16) 610.1 ns 517.6 ns
malloc(16) 26.9 ns 58.9 ns
sbrk / malloc 22.7x 8.8x
mmap threshold (not measured) 128 KiB, confirmed at 120 vs 132 KiB

The structural numbers — the 132 KiB grab, the chunk rounding, the 128 KiB threshold — reproduce exactly. The timing numbers don't, and they shouldn't: those belong to the machine. Re-running someone else's measurement is worth doing precisely because it tells you which half is which.

Sources: Ben M. Andrew, "What is malloc doing, anyway?", benmandrew.com (Hacker News, November 9, 2026, 213 points). All "my machine" numbers measured on this VM: Ubuntu, glibc 2.39, 4 vCPUs, GCC 12+ at -O2. strace traces and C sources are mine; the demand-paging and toy-allocator sections of the original post were not re-run.

Top comments (0)