Stack vs Heap: What Actually Happens in Physical Memory and CPU Registers

Every computer science student and bootcamp graduate is taught the standard memory mantra: primitives live on the stack, objects live on the heap. The stack is fast; the heap is slow.

If you ask your instructor or senior developer why the stack is fast and how the heap is structured, you usually get an abstract metaphor about stacks of cafeteria plates and messy piles of laundry.

That metaphor hides how computers actually work.

The stack and the heap are not two different hardware chips on your motherboard. There is no "fast SRAM stack chip" sitting next to a "slow DRAM heap chip". Every byte of your process memory, whether stack or heap, resides in the exact same physical DDR4 or DDR5 RAM modules, routed across the same memory controller and translated by the same Memory Management Unit (MMU).

So why does allocating on the stack take less than a single CPU clock cycle, while allocating on the heap can trigger OS locks, search algorithms, and kernel context switches?

Let's look at what the CPU, the Linux kernel, and runtime allocators actually do when you allocate memory.

---

1. The Virtual Address Space Map

When your operating system executes an x86_64 binary, it sets up a virtual address space (historically 48-bit, providing 256 TB of addressable space).

Your process does not see physical RAM addresses. It sees a flat virtual address map.

+------------------------------------+ 0x7FFFFFFFFFFF (Top of User Space)
| Stack (Grows Downward v)           |
|   [RSP register points here]       |
+------------------------------------+
| Guard Page (PROT_NONE)             |
+ - - - - - - - - - - - - - - - - - -+
|                                    |
|         Unmapped Space             |
|                                    |
+ - - - - - - - - - - - - - - - - - -+
| Memory Mapping Segment (mmap)      |
|   Shared libs (.so), Large chunks  |
+------------------------------------+
| Heap (Grows Upward ^)              |
|   [Program break / brk points here]|
+------------------------------------+
| .bss (Uninitialized globals)       |
+------------------------------------+
| .data (Initialized globals)        |
+------------------------------------+
| .text (Compiled machine code)      |
+------------------------------------+ 0x0000000000400000

Notice the architecture:

• The Stack starts near the ceiling of user space (0x7fff...) and grows downward toward lower addresses.
• The Heap starts just above the .bss segment and grows upward toward higher addresses.

---

2. What Actually Happens on the Stack

The stack is managed directly by CPU hardware architecture. The CPU dedicates a physical register specifically for this purpose: RSP (the Stack Pointer register).

The 1-Cycle Allocation

Look at what happens when you enter a function in C or Rust with 32 bytes of local variables:

void compute(void) {
    int a = 10;
    int b = 20;
    char buffer[24];
    // total: 32 bytes
}

The compiler translates this into a single assembly instruction:

compute:
    sub rsp, 32         ; Allocate 32 bytes instantly
    mov DWORD PTR [rsp+28], 10
    mov DWORD PTR [rsp+24], 20
    ...
    add rsp, 32         ; Deallocate 32 bytes instantly
    ret

That is the entire allocation mechanism: subtracting an integer from a CPU register (sub rsp, 32).

There is no loop, no searching for free slots, no lock contention, and no metadata tracking. It executes in 1 CPU clock cycle (roughly 0.2 to 0.3 nanoseconds on a modern 4GHz processor).

Function Frames & The Calling Convention

When function main() calls compute(), the hardware executes a sequence of micro-operations:

1. call compute: Pushes the return address (the next instruction in main) onto the stack (rsp drops by 8 bytes) and jumps to the target address.
2. Prologue: The function optionally saves the caller's frame pointer (push rbp; mov rbp, rsp).
3. Allocation: sub rsp, N claims contiguous memory for all local variables at once.
4. Epilogue & Return: mov rsp, rbp; pop rbp; ret restores the previous stack frame and pops the return address back into the instruction pointer (RIP).

High Memory
| ... caller frame ...            |
| Return Address (RIP for main)   | <-- pushed by `call`
| Saved Frame Pointer (RBP)       | <-- pushed by prologue
| Local: int a (4 bytes)          |
| Local: int b (4 bytes)          |
| Local: char buffer (24 bytes)   | <-- [RSP points here]
Low Memory

Why Stack Memory Is Hot in L1 Cache

Stack memory enjoys near-perfect spatial and temporal cache locality.

Because your program constantly reuses the same tiny window of memory at the top of the stack as functions push and pop frames, those memory pages are almost permanently resident in the CPU's L1 Data Cache (32KB to 64KB per core, ~1ns access latency).

The Catch: Stack Overflows and Guard Pages

The stack has a strict size limit (typically 8MB on Linux, configurable via ulimit -s).

What happens if you run an infinite recursion?

The OS kernel places an unmapped Guard Page with PROT_NONE permissions immediately below the stack limit. When RSP decrements past the boundary and writes to this page, the CPU Memory Management Unit detects a permission violation and triggers a Page Fault. The kernel traps it and terminates the process with SIGSEGV (Segmentation Fault: Stack Overflow).

---

3. What Actually Happens on the Heap

The heap is not a hardware construct. The CPU has no "heap pointer register". The heap is an abstraction managed entirely in software by a user-space memory allocator (such as glibc's ptmalloc, Google's tcmalloc, or jemalloc).

When you call malloc(64) in C or new User() in higher-level runtimes, a complex multi-tier engine springs into action.

char *ptr = (char *)malloc(64);

Here is what happens behind that call:

Step 1: Thread Caching (Fast Path)

To avoid locking overhead across multiple CPU threads, modern allocators maintain thread-local caches (such as glibc's tcache introduced in glibc 2.26).

• The allocator rounds your 64-byte request up to the nearest bin size (e.g., 80 bytes, accounting for an 8-byte or 16-byte internal chunk header).
• It checks the thread's local singly-linked tcache bin.
• If a free chunk exists, it pops the head pointer and returns it immediately without acquiring any mutex lock.

Step 2: Bin Searching and Arena Locks (Slow Path)

If the thread cache is empty:

• The allocator falls back to the central heap Arena.
• It acquires a mutex lock to protect shared allocator structures.
• It searches through categorized doubly-linked lists:
  • Fastbins: Fixed small chunk sizes.
  • Smallbins: Exact size matches (< 1024 bytes).
  • Unsorted bins / Large bins: Variable-sized chunks that require splitting larger blocks into smaller pieces and inserting remaining fragments back into the free list.

Every heap chunk carries hidden metadata directly preceding the returned pointer:

+------------------------+------------------------+
| Previous Chunk Size    | Chunk Size | A | M | P | <-- 16-byte Header
+------------------------+------------------------+
| User Data Payload (e.g. 64 bytes)              |
| ...                                            | <-- Pointer returned to user
+------------------------------------------------+

(Flags: A = Allocated from non-main arena, M = Allocated via mmap, P = Previous chunk is in use).

Step 3: Requesting Pages from the Kernel (brk vs mmap)

If no existing free chunks can satisfy the request, the allocator must ask the Linux kernel for more memory. It uses one of two system calls:

1. brk() / sbrk() for Small-to-Medium Chunks

The allocator asks the kernel to increment the process "program break" pointer, extending the top of the heap segment.

2. mmap() for Large Allocations

If the requested size exceeds MMAP_THRESHOLD (128 KB by default in glibc), glibc skips the heap entirely. It issues an anonymous mmap syscall:

mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

This creates a dedicated virtual memory mapping outside the traditional heap segment. When you call free(), memory allocated via mmap is immediately unmapped and returned to the OS kernel via munmap().

---

4. The Virtual vs Physical Illusion: Demand Paging

Here is something that catches many developers off guard:

When malloc(), brk(), or mmap() succeeds, the operating system has NOT allocated physical RAM to your process.

Linux uses optimistic demand paging. The kernel only updates its internal page table structures to mark that range of virtual addresses as valid.

Physical RAM is only allocated when your code performs its first read or write to that memory address:

1. CPU executes: mov [rax], 42
2. MMU checks page table for virtual address in RAX -> Page Table Entry (PTE) is empty!
3. Hardware fires Page Fault Exception (Ring 3 -> Ring 0 trap)
4. Linux kernel fault handler (handle_mm_fault) intervenes:
   - Verifies address is within valid VMA (Virtual Memory Area)
   - Allocates an empty 4KB physical page frame from RAM free pool
   - Zeroes the page (security: prevents leaking data from other processes)
   - Writes physical address into Page Table Entry (PTE)
5. Kernel resumes process; CPU re-executes `mov [rax], 42` successfully.

A minor page fault costs roughly 1,000 to 2,000 CPU clock cycles (~300 to 600 nanoseconds). If you allocate 100MB and touch every page, your CPU spends hundreds of thousands of cycles fielding kernel traps before your application logic even runs.

---

5. The Hardware Reality: Cache Lines & Pointer Chasing

The biggest performance difference between stack and heap in real-world software is not the allocation overhead. It is how your CPU reads that memory later.

Modern CPUs do not fetch single bytes or integers from RAM. They fetch memory in 64-byte Cache Lines.

Sequential Access (Stack / Flat Arrays)

int array[1024]; // 4096 bytes contiguous
for (int i = 0; i < 1024; i++) {
    sum += array[i];
}

When the CPU reads array[0], it pulls the entire 64-byte line into L1 cache (16 integers). The CPU's hardware stream prefetcher detects the sequential access pattern and automatically loads upcoming cache lines into L2/L3 before your loop even asks for them.

Cache miss rate: Near 0%.

Pointer Chasing (Heap Objects / Linked Structures)

struct Node {
    int value;
    struct Node *next; // Heap pointer
};

When nodes are allocated individually on the heap across different times, malloc places them at disparate virtual addresses scattered across pages.

Node 1 (0x55a0f120) ------> Node 2 (0x55a08900) ------> Node 3 (0x55a12340)
   [DRAM Page 1]               [DRAM Page 42]              [DRAM Page 118]

When you traverse current = current->next, the CPU cannot predict where the next pointer points. The hardware prefetcher is useless.

On every hop:

• The CPU misses L1, misses L2, misses L3.
• The instruction pipeline stalls for 150 to 250 clock cycles while waiting for DRAM.

This memory stall is why iterating an array of contiguous structs is often 10x to 50x faster than traversing a linked list of identical data.

---

6. Stack vs Heap: Complete Architectural Comparison

Dimension	Stack	Heap
Physical Hardware	Main DRAM (hot in L1/L2 Cache)	Main DRAM (often scattered)
Allocation Mechanism	Pointer arithmetic (sub rsp, N)	Allocator bins, arenas, search algorithms
Allocation Cost	~1 CPU cycle (~0.3 ns)	~20–50 cycles (cached) to ~2,000 cycles (page fault)
Deallocation Cost	~1 CPU cycle (add rsp, N / ret)	Freelist merging, coalescing, lock management
Management	CPU hardware & compiler	User-space runtime library (ptmalloc, jemalloc, GC)
Kernel Syscalls	None during execution	brk(), mmap(), munmap()
Lifetime	Tied strictly to function scope	Manual (free) or managed by Garbage Collector
Size Limit	Small (typically 8MB per thread)	Limited only by Virtual Memory / RAM + Swap
Failure Mode	Stack Overflow (SIGSEGV via Guard Page)	malloc returns NULL or Linux OOM Killer
Cache Behavior	Contiguous, high spatial locality	Non-contiguous, prone to pointer-chasing stalls

---

7. What High-Level Runtimes Do: Escape Analysis

If you code in Go, Java, or JavaScript (Node.js/V8), you don't call malloc directly. Does that mean everything you write goes to the heap?

Not necessarily. Modern optimizing compilers use Escape Analysis.

func createPoint() *Point {
    p := Point{X: 10, Y: 20} // Escapes function scope!
    return &p
}

func calculate() int {
    p := Point{X: 10, Y: 20} // Does NOT escape!
    return p.X + p.Y
}

• In calculate(), the compiler proves that p never outlives the function. It allocates p directly on the stack, avoiding GC pressure entirely.
• In createPoint(), the pointer escapes the stack frame. The Go compiler automatically promotes p to the heap at compile time.

Understanding whether your data structures escape to the heap is one of the most effective ways to optimize high-throughput Go, Java, and C# services.

---

Key Takeaways

1. Stack and heap share the same physical memory. The speed difference comes from the allocation mechanism (a single register subtraction vs bin searching) and CPU cache locality.
2. Stack allocation is O(1) hardware arithmetic. sub rsp, N claims memory in 1 CPU cycle.
3. Heap allocation is an operating system and runtime negotiation. It involves thread caches, free lists, arena mutexes, and virtual page table updates via brk and mmap.
4. Memory access patterns matter more than allocation cost. Sequential contiguous layout on stack or flat heap buffers allows CPU hardware prefetchers to saturate memory bandwidth, while heap pointer chasing stalls CPU execution units.
5. Watch your escape paths. Minimizing heap allocations in high-level languages reduces both allocator lock contention and garbage collection pauses.