Every program you have ever written has been lied to. Not maliciously, and not in a way that causes you harm. But from the moment your program starts running to the moment it exits, the operating system is maintaining an elaborate fiction about the nature of memory, and your program has no idea.
The fiction is this: your program believes it has access to a large, flat, contiguous address space that belongs entirely to it. On a 64-bit system, that address space is theoretically 16 exabytes. Your program can read from address 0x1000, write to address 0x7fffffffe000, and jump to code at address 0x400000, and it does so with the complete confidence that those addresses are real, that the memory at those addresses is yours, and that nothing else is touching it.
None of that is true in the way your program thinks it is. The addresses are not real physical memory addresses. The memory is not necessarily yours in the sense of being physically reserved. Other programs are running in what appears to be the same address space but is not. The OS is intercepting every single memory access and translating it, in hardware, before your program sees the result.
Understanding how this works is not just an academic exercise. It explains behavior that is otherwise confusing: why processes are isolated from each other, how memory mapped files work, what a page fault actually is, why malloc does not immediately consume RAM, how copy-on-write makes fork fast, what a segmentation fault really means at the hardware level, and why certain access patterns are dramatically slower than they should be.
The problem virtual memory solves
To understand why virtual memory exists, you need to understand the world before it.
In the early days of computing, programs ran directly on physical memory. A program was loaded at some physical address, and it used physical addresses to access its data. This created several serious problems.
The first was relocation. If your program was written to run at physical address 0x1000, and something else was already loaded there, your program could not run. Every program had to be written to run at a specific location, or it had to be compiled with relocation in mind and then adjusted at load time, which was complex.
The second was isolation. If two programs were running simultaneously and sharing physical memory, there was nothing preventing one program from reading or writing the memory of the other. A bug in one program could corrupt another. A malicious program could read passwords out of another program's memory. The only security was trusting that programs were well-behaved.
The third was fragmentation. As programs were loaded and unloaded, physical memory developed holes. You might have 100 megabytes of free physical memory but no contiguous 50-megabyte region to load a program that needed 50 megabytes. The memory existed but was not usable.
The fourth was the mismatch between program needs and available RAM. A program might need more memory than the machine physically had. Without some mechanism to handle this, programs were simply limited to what physically existed.
Virtual memory was designed to solve all of these problems simultaneously. It does so by inserting an indirection layer between the addresses a program uses and the physical memory those addresses correspond to.
Pages and page tables
The fundamental unit of virtual memory is the page. A page is a fixed-size block of memory, typically 4096 bytes (4 kilobytes) on most systems, though some architectures support larger page sizes. All memory management in a virtual memory system happens at page granularity. You never map a single byte. You map a page, which means 4096 bytes all mapped together.
The data structure that records the mapping from virtual pages to physical pages is the page table. Every process has its own page table. The page table is a structure in physical memory, maintained by the operating system, that the hardware uses to translate virtual addresses to physical addresses.
When your program accesses a virtual address, the hardware does not use that address directly. Instead, it looks up the virtual page number (derived from the address) in the page table, finds the corresponding physical page number, combines that with the offset within the page, and produces the physical address that actually goes onto the memory bus.
A virtual address on a 64-bit system with 4KB pages breaks down like this: the bottom 12 bits are the page offset (since 2^12 = 4096, these bits index into the page). The remaining bits above that identify the virtual page number. The translation hardware looks up the virtual page number in the page table, retrieves the physical page number, and appends the original offset bits to form the physical address.
This translation happens for every single memory access. Every instruction fetch, every data load, every data store. The hardware does this transparently, in a few nanoseconds, without any involvement from your program. Your program issues a virtual address, and the physical address is what actually happens.
The translation lookaside buffer
If every memory access required looking up the page table, and the page table itself is in memory, every memory access would require at least one additional memory access just for the translation. That would roughly halve memory throughput. For most programs it would be far worse than halving, because page tables are hierarchical structures that require multiple lookups.
The solution is the Translation Lookaside Buffer, or TLB. The TLB is a small, fast cache built into the CPU that stores recent virtual-to-physical page mappings. When the hardware needs to translate a virtual address, it checks the TLB first. If the mapping is there (a TLB hit), the translation is done in a single cycle or two. If the mapping is not there (a TLB miss), the hardware has to walk the page table in memory, which takes many cycles, and then cache the result in the TLB for future use.
TLBs are small. A typical L1 TLB might hold 64 entries. An L2 TLB might hold a few hundred to a few thousand. Each entry covers one page, which is 4KB. A 64-entry TLB covers 256KB of address space with zero-cost translations. Code and data that fits within 256KB will have excellent TLB behavior. Code and data that is scattered across many pages will cause TLB misses and pay the page table walk cost repeatedly.
This is why TLB pressure is a real concern for performance-sensitive code. A program with a large working set, data scattered across thousands of pages accessed in no particular order, will spend significant time on TLB misses. Large pages (2MB or 1GB on x86-64, configured through huge pages on Linux) are one mitigation: each TLB entry covers far more memory, so the same TLB can cover a much larger working set without misses.
Hierarchical page tables
A flat page table would be straightforward but impractical. On a 64-bit system with 4KB pages, the virtual address space has 2^52 pages (ignoring the bits that are not used in practice). A flat array with one entry per page would itself be enormous, larger than the physical memory of most machines. And each process would need its own copy.
The solution is a hierarchical page table, also called a multi-level page table. Instead of a single flat array, the page table is a tree structure. The virtual page number is split into multiple parts, and each part indexes into a different level of the tree. Only the parts of the tree that are actually needed are allocated.
On x86-64, the current standard uses a four-level page table structure, with a fifth level available on newer processors for even larger address spaces. A virtual address is split into four 9-bit indices and a 12-bit page offset. The four indices index into four levels of page table directories. Each level of directory contains 512 entries (2^9). An entry at each level either points to the next level's directory or marks itself as absent.
The tree is sparse. For a typical process, most of the address space is unmapped. Only the regions that have been mapped (code, data, stack, heap, memory-mapped files) have entries in the tree. The directories covering unmapped regions do not need to exist. The kernel only allocates page table memory for the address space regions that actually exist.
When a process terminates, the kernel walks its page table and frees all the allocated directories. When a process forks, the kernel can share page table entries between parent and child (with modifications to support copy-on-write, which we will get to). The hierarchical structure makes both of these operations practical.
What happens when you access unmapped memory
Your program's address space contains many virtual addresses that are not mapped to any physical page. The stack has a bottom. The heap has a current limit. There are gaps between the code segment, the data segment, and the stack. Most of the theoretical 128 terabytes of usable virtual address space (on current x86-64 implementations) is unmapped.
What happens when your program attempts to access an unmapped address?
The hardware attempts the page table walk and finds either a missing entry in one of the page table levels, or a page table entry marked as not present. The hardware raises an exception: a page fault. Control transfers from your program to the operating system's page fault handler.
The page fault handler examines the faulting address and the context of the access. It has several possible responses.
If the address is simply invalid, not in any region that the process has legitimately mapped, the kernel sends a signal to the process. On Unix-like systems, this is SIGSEGV, the segmentation fault signal. By default this terminates the process and optionally produces a core dump. The error message "Segmentation fault (core dumped)" that every C programmer has seen is the result of this sequence: invalid memory access, page fault, kernel sends SIGSEGV, process terminates.
If the address is in a legitimate region but the page is not currently in physical memory (because it was never loaded, or because it was swapped out), the kernel fetches the page from wherever it lives (disk, a file, a swap partition), allocates a physical page to hold it, updates the page table entry to point to that physical page and mark it present, and resumes the process from the faulting instruction. The process never knew anything happened. From its perspective, it issued a memory access and got back a result. The fact that the OS intervened and fetched data from disk is invisible.
If the address is in a legitimate region and the page is present but the access violated the permissions on the page (for example, writing to a read-only page), the kernel again sends a signal. This is the basis for detecting certain bugs and for implementing copy-on-write, which we will discuss shortly.
Page faults are the mechanism through which the operating system maintains control over memory. Every unmapped or missing or protected page gives the kernel an opportunity to intervene, manage resources, enforce isolation, and implement features transparently.
Demand paging: memory that does not exist until you use it
When you call malloc and allocate a hundred megabytes, the C library might ask the kernel for that memory via the brk or mmap system call. The kernel grants the request and marks a hundred megabytes of virtual address space as belonging to your process. But it does not immediately allocate a hundred megabytes of physical RAM.
The pages in that region are marked as reserved but not present. Physical memory is not assigned until your program actually touches a page.
When your program writes to the first byte of that region, a page fault occurs. The kernel sees that the fault is in a valid region, allocates one physical page (4KB), zeros it out, maps the virtual page to the physical page, and resumes your program. That one page is now in RAM. The other 99,999 pages in the hundred-megabyte allocation are still not in RAM, because you have not touched them yet.
This is demand paging. Memory is paged in on demand, as it is accessed. It is the reason that allocating a large buffer does not immediately consume RAM. It is the reason that a program can start up fast even if it has a large initialized data segment: the initialization happens lazily as pages are first touched.
Demand paging also means that the amount of physical RAM a process actually uses can be much less than the virtual address space it has mapped. A program that allocates a hundred-megabyte buffer but only uses ten megabytes of it consumes roughly ten megabytes of RAM, not a hundred. The kernel only has physical pages for what was actually touched.
The trade-off is that the first access to each new page incurs a page fault, which involves a kernel mode switch and some amount of work. For programs that allocate and then sequentially use large buffers, this is usually fine: the faults are amortized over a lot of useful work, and the kernel may even prefault pages when it can predict access patterns. For programs that have performance requirements and cannot tolerate unpredictable latency spikes, demand paging can be a problem, which is why there are system calls like mlock that force pages to be faulted in and pinned in physical memory.
Copy-on-write: how fork is not as expensive as it sounds
On Unix-like systems, the primary way to create a new process is fork. Fork creates a copy of the current process: same code, same data, same heap, same stack, same file descriptors. On a process with a gigabyte of resident memory, you might expect fork to take a while and consume another gigabyte of RAM.
In practice, fork is fast and does not immediately consume the memory you expect.
The mechanism is copy-on-write. When fork is called, the kernel creates a new process with its own page table, but instead of copying all the physical pages from the parent, it makes the child's page table point to the same physical pages as the parent. Both parent and child share the same physical memory.
To prevent one process from modifying the other's data, all those shared pages are marked read-only in both page tables. If either process reads from a shared page, no fault occurs and no copy is made. If either process writes to a shared page, a page fault occurs. The fault handler sees that this is a copy-on-write fault on a shared page, allocates a new physical page, copies the content of the original page into the new page, updates the faulting process's page table to point to the new page with write permission, and resumes the write. The other process continues to use the original page.
The result is that fork is cheap when the parent and child don't write to much memory after forking. On a typical Unix server that uses fork to handle requests and then immediately calls exec to replace the child with a new program, almost no copying happens: the exec replaces the entire address space, so the copy-on-write pages are never actually copied.
Copy-on-write is also why modifying a file that has been memory-mapped as a private mapping does not affect the underlying file. The pages start shared with the file's backing store. When you write to them, the copy-on-write mechanism creates private copies. The file on disk is untouched.
Memory-mapped files: the unified interface
Memory-mapped files are one of the more useful features built on top of the virtual memory mechanism. The mmap system call allows you to map a file (or a portion of a file) directly into your process's address space. Once mapped, reading from the mapped region reads from the file. Writing to the mapped region (if it's a writable mapping) writes to the file.
From your program's perspective, you just read and write memory. There are no read or write system calls. No buffers to manage. No seeking. You access the file by its virtual address.
Under the hood, the kernel sets up virtual-to-physical mappings that point to pages backed by the file. Pages are not loaded until they are accessed: demand paging applies here too. When you access a page of the mapped file that is not in memory, a page fault occurs, the kernel fetches the page from the file on disk, maps it into physical memory, updates your page table, and resumes your program. When memory is tight, the kernel can evict file-backed pages from RAM without needing to write them to swap, because the file itself is the backing store and can be read again later.
This unified model (files and anonymous memory managed the same way through the same page fault mechanism) is one of the cleaner aspects of the Unix memory model. Everything that uses memory goes through the same machinery.
Shared memory between processes works through the same mechanism. Two processes can both mmap the same file (or the same anonymous region created with special flags). Their page tables point to the same physical pages. Writes by one process are visible to the other without any copying. This is the most efficient form of interprocess communication: no data moves, no kernel involvement after the initial setup, just two processes looking at the same physical memory through different virtual addresses.
Swap: extending RAM onto disk
When physical RAM is full and a process needs more, the kernel has to make room. It does this by evicting pages: taking physical pages that are currently in RAM and moving them elsewhere.
For pages backed by files (code segments, memory-mapped files), eviction is free. The data is already on disk. The kernel can simply remove the page table mapping, mark the page as not present, and the physical page is now available for something else. If the evicted page is accessed again, a page fault occurs and the kernel reloads it from the file.
For anonymous pages (heap data, stack data, anything not backed by a file), eviction requires writing the data somewhere before reclaiming the physical page. That somewhere is swap space: a dedicated area on disk (or a swap file) where the kernel stores evicted anonymous pages.
When an anonymous page is evicted to swap, the kernel records the swap location in the page table entry, marks the page as not present, and reclaims the physical page. If the process accesses that virtual address later, a page fault occurs, the kernel reads the page back from swap into a physical page, updates the page table, and resumes the process. This is completely transparent to the program.
Swap allows the total memory demand of all running processes to exceed the amount of physical RAM. A system with 16GB of RAM and 32GB of swap can run programs that collectively need up to 48GB of memory, as long as not all of it is actively needed at the same time.
The cost is performance. RAM access takes nanoseconds. Disk access takes microseconds to milliseconds. When a process accesses pages that have been swapped out, it waits for disk I/O, which can be orders of magnitude slower than RAM. A system that is swapping heavily (sometimes called thrashing) can become almost unusably slow: programs that should run in milliseconds take seconds because every memory access misses and requires a disk read.
Modern SSDs have made swap significantly more usable than it was with spinning disks. An NVMe SSD with microsecond access times makes swap a reasonable overflow buffer. On spinning disks, heavy swap usage was often catastrophic.
What the virtual address space actually looks like
For a running process on a Linux 64-bit system, the virtual address space is organized into several distinct regions, each with a specific purpose.
At the low end of the address space, the first few pages are typically unmapped. This is intentional: it ensures that null pointer dereferences (accessing address zero or nearby addresses) cause a page fault and a segfault rather than silently succeeding. A null pointer is not a valid memory address in the virtual address space of any process.
Above the null guard region sits the text segment: the compiled code of your program, loaded from the executable file. This region is mapped from the executable file, readable and executable but typically not writable (to prevent the program from accidentally modifying its own code at runtime).
After the text segment comes the data segment: global and static variables that have explicit initial values. These are loaded from the executable file and are readable and writable.
The BSS segment follows: global and static variables that are zero-initialized. The kernel does not actually store all these zeros in the executable file; instead, the BSS region is marked as zero-initialized anonymous memory, and the demand paging mechanism zeroes physical pages as they are first accessed.
The heap grows upward from above the BSS. The C library's malloc implementation manages this region, requesting more space from the kernel (via brk or mmap) as needed and subdividing it for individual allocations. The current top of the heap is the program break.
In the middle of the address space are memory-mapped regions: shared libraries, memory-mapped files, and large anonymous allocations. When you dynamically link against libc or any other shared library, the library's code and data are mapped into this region. All processes that use the same shared library map the same physical pages for the library code (read-only), sharing it without duplication, while having their own private copies of the library's writable data.
The stack lives at the high end of the address space and grows downward toward lower addresses. It starts at a high address and expands toward the heap as function calls are made and local variables are allocated. The operating system maps the stack region with a guard page at the bottom: an unmapped page that catches stack overflows by causing a page fault when the stack tries to grow past it.
Above the stack, the kernel reserves the very top of the address space for itself. On Linux, the upper half of the 64-bit address space is always mapped to kernel memory. When a system call occurs, the kernel code runs in this region, which is present in every process's page table but only accessible in kernel mode. This is why system calls are fast compared to switching to a separate kernel process: the kernel is already mapped into your process's address space, just inaccessible from user mode.
Why two processes can have the same address
One of the things that confuses people new to operating systems is running the same program twice and noticing that both instances report the same addresses for their variables. Run a simple C program that prints the address of a local variable in two terminals and you will see the same hexadecimal value in both.
This makes no sense if you think of virtual addresses as physical locations. Two different things cannot be at the same location.
It makes complete sense once you understand that each process has its own page table. The virtual address 0x7ffe1234abcd in process A maps to some physical page via A's page table. The same virtual address in process B maps to a completely different physical page via B's page table. Same virtual address, same program layout, different physical memory. The isolation is enforced in hardware by the MMU using each process's page table when translating addresses for that process.
This is also why process isolation is robust. Process A cannot access process B's memory by guessing virtual addresses, because A's page table simply does not have mappings for B's physical pages. A's page table is authoritative for A's address space. B's memory does not exist in A's page table. Any attempt by A to access an address that is not mapped in its own page table results in a page fault and, if the address is not in any valid region, a segfault.
The kernel's role in all of this
The kernel is the entity that maintains page tables and handles page faults. Every time the hardware raises a page fault, execution transfers to the kernel's fault handler. The kernel decides what to do: allocate a page, load from disk, send a signal to the process, or kill the process.
The kernel also decides which physical pages to evict when memory is tight. This decision is made by the page replacement algorithm. Linux uses a variant of the clock algorithm and LRU approximation, tracking which pages have been recently accessed and preferring to evict pages that have not been used recently. Getting page replacement right matters a lot for system performance under memory pressure.
Every process context switch requires loading the new process's page table base address into the hardware. On x86-64, this is done by writing the address of the top-level page table directory into the CR3 register. This causes the TLB to be flushed (since the translations it cached belong to the previous process), which is one reason context switches have a real performance cost: the new process starts with a cold TLB and pays for TLB misses until its working set warms up. Tagged TLBs on modern hardware can avoid full flushes by tagging entries with a process identifier, but this optimization has limits.
System calls that involve memory (mmap, munmap, brk, mprotect, mlock) modify the kernel's data structures describing the process's virtual memory regions, and may modify the page table itself. These operations have to be performed carefully because they affect hardware state that is live while the process is running.
Memory protection and what mprotect is actually doing
The permissions on a page (readable, writable, executable) are stored in the page table entry. Every entry has bits indicating which operations are permitted. When the hardware translates a virtual address, it also checks whether the access being performed matches the permissions in the entry. An attempt to write to a read-only page, or to execute data from a non-executable page, causes a page fault just like an access to an unmapped page.
The mprotect system call changes the permission bits in page table entries for a range of virtual memory. This is how the kernel implements write protection for code segments (preventing accidental modification of running code), and it is how security features like W^X (write XOR execute, the policy that a page can be writable or executable but not both) are enforced in hardware.
This mechanism is also what makes stack canaries and address space layout randomization work as security mitigations. ASLR randomizes the base addresses of the stack, heap, and shared libraries each time a program runs, so an attacker cannot hard-code addresses in an exploit. The virtual memory system makes this straightforward: the page table mappings simply start at different base addresses each run.
The abstraction and when it leaks
Virtual memory is one of the most successful abstractions in computing. The majority of programmers use it every day without thinking about it, and it works correctly essentially all the time. The abstraction is clean and well-maintained.
But it leaks in specific circumstances, and knowing when it leaks is useful.
Performance anomalies from TLB pressure show up in programs that scatter accesses across huge address spaces or that access many sparsely populated pages in a short time. The fix is usually to compact the working set: access memory in a pattern that keeps the active pages within what the TLB can cover.
Page fault latency matters for programs with real-time requirements. The cost of a page fault is unpredictable from the program's perspective: it might be a quick kernel allocation, or it might require reading from a slow disk. Programs that cannot tolerate latency spikes (audio processing, trading systems, kernel drivers) use mlock to pin pages in RAM and mlockall to prevent any page faults during critical sections.
The mismatch between virtual allocation and physical consumption can mislead profiling. A process with 10GB of virtual address space might only have 500MB of physical pages. Tools like /proc/PID/status on Linux report both: VmSize for virtual, VmRSS for resident (physical). Understanding which number you're looking at matters when debugging memory issues.
Swap can make performance problems invisible until they become catastrophic. A program running fine with 16GB of RAM can suddenly become unusable when a new workload pushes total memory demand past the physical limit and heavy swapping begins. Monitoring RSS (resident set size) rather than virtual size gives a better picture of actual physical memory consumption.
Copy-on-write makes fork behave in ways that surprise people. A parent process with 1GB of resident memory forks, and immediately the parent modifies its entire heap. Every modified page triggers a copy-on-write fault and a page copy. Total physical memory for the parent and child is now nearly 2GB. If you're using fork in a memory-intensive process, patterns of memory modification after fork have real physical memory implications.
What this means for writing software
You do not need to think about virtual memory for most of what you write. The abstraction holds. malloc works. Arrays work. Pointers work. The OS handles the machinery.
But certain classes of problems become much clearer once you have this model.
Understanding why sequential memory access outperforms random access is trivial once you know about cache lines, TLB coverage, and prefetching. The hardware is optimized for the case where physical pages are accessed in order. Random access across a large sparse virtual address space hits TLB misses and cache misses at every step.
Understanding why two processes can share a shared library without duplicating its memory is immediate once you know page tables and physical page sharing. The library code is one set of physical pages, mapped read-only into every process that uses it. Understanding this makes you appreciate why linking dynamically against common libraries costs less physical memory than statically linking everything.
Understanding why a container or virtual machine does not immediately consume all the RAM it is allocated is clear once you know demand paging. Allocating 4GB to a container allocates virtual address space. Physical RAM is only consumed as it is actually written.
Understanding why a segfault on a null pointer dereference happens at address 0x8 rather than 0x0 makes sense once you know that struct field offsets are added to the base pointer: a null pointer to a struct with a field at offset 8 produces an access to virtual address 8, which is in the unmapped null guard region and causes a fault.
The operating system is doing an enormous amount of work on behalf of every program running on a machine, work that is invisible precisely because it is designed to be invisible. Virtual memory is one of the core mechanisms through which that invisibility is maintained. Once you see it, a lot of things that seemed like magic become straightforward engineering, and a lot of things that seemed like inexplicable behavior become obvious.
Top comments (0)