This document summarizes the core concepts of how a CPU handles processes and threads, the mechanism of time slicing along with its trade-offs, and how popular programming languages apply these models.
1. Basic Overview: CPU, Core, Process, and Thread
To understand how a computer works, we go from hardware to software:
- CPU (Central Processing Unit): The central processor. A computer can have one or multiple physical CPUs (sockets).
- Core : A CPU contains multiple Cores. Each Core is an actual physical processing unit.
- Process : The "container" that holds a running program. Each process has its own completely independent memory space and resources.
- Thread : The smallest unit of execution that the Operating System (OS) can schedule. A process contains at least one thread (the main thread) and can contain multiple threads.
Core Truth: The CPU does not run Processes — the CPU runs Threads. A process is just an outer shell. Context switching between Processes is actually switching between Threads belonging to different Processes (which also entails switching memory spaces, making it very heavy).
2. CPU Processing Mechanism and Time Slicing
Each Core can only run exactly 1 Thread at any given time. So what happens when the number of active Threads exceeds the number of Cores?
2.1. True Parallelism
When active Threads ≤ number of Cores:
Each Thread is assigned to a separate Core. Everything runs in true parallel at the same time. There is no waiting.
2.2. Time Slicing and Context Switch
When active Threads > number of Cores (The most common scenario):
The CPU cannot process all of them simultaneously. The Operating System (OS) applies the Time Slicing mechanism.
- The OS divides CPU time into very small slices (called Time Slices or Quantums, usually just a few milliseconds).
- The OS alternately assigns the CPU to Thread A until its time slice runs out, then pauses A, switches to Thread B, then to Thread C, etc. The switching speed is so extremely fast that it gives us the illusion that everything is running "concurrently".
The process of switching from one Thread to another is called a Context Switch. The OS must:
- Save the state (Registers, Program Counter...) of the currently running Thread.
- Load the state of the next Thread to be run.
Below is a Timeline diagram illustrating the cost of Time Slicing interleaved with Context Switches:
Chart Note: The red
[Overhead]blocks represent the "dead time" (cost) where the CPU does no useful work for the application other than unloading the old Thread and loading the new Thread.
3. The Nature of Time Slicing and Trade-offs
Time Slicing solves the multitasking problem, but it's not perfect. It brings massive hidden costs if abused.
3.1. The Context Switch Paradox and Minimal Execution Time
A Context Switch is not free. The act of saving/loading states consumes CPU cycles.
- For the switch to be meaningful, the CPU must spend a minimum amount of time actually processing the logic of the Thread (called Minimal Execution Time ).
- The Problem: When you spawn too many tasks/threads, the amount of time each thread receives in a cycle becomes smaller. If a thread's execution time shrinks and approaches the time taken just to Context Switch, the CPU falls into a state of Thrashing — meaning the CPU is 100% busy just swapping back and forth between threads, while the actual useful work processed is minimal.
3.2. Real-world Trade-offs
When you spawn too many Threads exceeding the CPU's capacity, the system suffers 3 main consequences:
- Increased Latency: The more threads there are, the longer the CPU Run queue. A thread might have to wait a very long time before it's its turn to get a "time slice". This increases the overall latency of the application.
- CPU Overhead: As mentioned above, the CPU wastes more resources managing schedules and Context Switching instead of running your business logic.
- Cache Misses - Extremely Severe: The CPU has ultra-fast caches (L1, L2, L3 Cache). When Thread A is running, A's data is loaded into the Cache. If the CPU is forced to Context Switch to Thread B, B's data will push A's data out of the Cache (Cache Pollution). When Thread A comes back to run, the CPU can no longer find the data in the Cache ( Cache Miss ) and must go down to the RAM to read it again — reading from RAM is hundreds of times slower than Cache.
4. Real-world Application: Operational Models of Languages/Runtimes
Understanding the costs of Threads and Context Switches, backend technologies have chosen different paths:
1. PHP-FPM (Multi-Process / Pre-fork Model)
- Mechanism: Each incoming request is allocated to a separate Process (containing 1 Thread inside).
- Characteristics: Very safe; an error in one request won't kill another request (due to independent memory spaces).
-
Massive Performance Drawbacks:
- Processes are very heavy (consume a lot of RAM).
- Context Switching between Processes is vastly more expensive than between Threads (because it requires swapping Virtual Memory space / TLB flush).
- If there are 1000 concurrent requests, PHP-FPM needs 1000 Processes. The server will run out of RAM and the CPU will immediately suffer from Thrashing due to excessive Context Switching.
-
Limitation: Bottlenecked by the pre-configured number of Worker Processes (e.g.,
pm.max_children = 50). Beyond 50 requests, subsequent requests must wait in a queue.
2. Node.js (Single Thread & Event Loop Model)
- Mechanism: Node.js realizes that Context Switching is the enemy of performance. Therefore, it runs all JavaScript code on a single Main Thread (1 Process).
-
Characteristics:
- Completely eliminates Context Switch costs for the main logic code. CPU isn't wasted, and Cache rarely Misses.
- Handles tens of thousands of connections (I/O) concurrently thanks to the Non-blocking I/O mechanism (combined with epoll/kqueue at the OS level). When there's a File/Network read task, Node.js offloads it to the background (libuv's Thread pool) and moves on to do other things. Once finished, a callback/event notifies it.
- Drawbacks: Because there is only 1 Thread, if you have a heavy computation task (CPU-bound like image compression, a billion-iteration loop), it will block the entire process — other requests will hang waiting until that task completes.
3. Golang (M:N Scheduling & Goroutines Model)
- Mechanism: Go strikes a brilliant balance. It spawns a number of OS Threads (exactly equal to the number of physical CPU Cores to achieve true parallelism). On those OS Threads, it runs tens of thousands, or even millions, of Goroutines.
-
Characteristics (Why Go is top-tier?):
- Ultra-lightweight: A Goroutine is a User-space Thread (a virtual thread managed by the Go Runtime). It only takes about ~2KB of RAM to initialize (compared to 1-2MB for a standard OS Thread).
- Ultra-cheap Context Switch: When Goroutine A is blocked (e.g., calling a DB), the Go Runtime saves A's state and picks Goroutine B to stuff into the OS Thread to continue running. This Context Switch happens at the software level (User-space), without notifying the Operating System , thus the speed is incredibly fast and the cost is a fraction of an OS Context Switch.
- Optimized Cache & CPU: By keeping the number of OS Threads low (equal to Cores), Go completely avoids OS-level Context Switch overload and maximizes CPU Cache utilization.
Summary (The Takeaway)
- CPUs run Threads, not Processes.
- Time Slicing allows running more threads than cores, but brings latency, CPU overhead, and Cache data loss.
- PHP is safe but heavy because it spawns many Processes.
- Node.js uses 1 Thread combined with Asynchronous I/O to dodge Context Switches, excelling at I/O but fearing heavy computation.
- Golang creates ultra-lightweight "virtual threads" (Goroutines) and schedules them on real Cores itself, maximizing both I/O and computation.
Top comments (0)