📺 Prefer to watch? 90-second YouTube Short · 💬 Telegram
Originally published on software-engineer-blog.com.
You run code concurrently all the time. But "concurrent" hides a critical choice: are you spawning separate processes or threads inside the same process? That choice decides whether one crash takes down your entire system or stays contained, and whether you're copying data between isolated worlds or racing to read the same memory.
Mental model: A process is its own house; threads are roommates sharing one.
Processes: Isolation at the Cost of Weight
When you start a process, the operating system hands it its own private address space. That address space is walled off. Your process can't touch another process's memory—the OS enforces it at the CPU level. If your process crashes, it corrupts only its own memory. The kernel cleans it up. Every other process keeps running untouched.
This is why browsers put each tab in its own process. One tab runs malicious JavaScript, spins into an infinite loop, or has a memory leak—that tab's process dies. The rest of your browser lives. You close the dead tab and open a new one. Your other tabs don't even hiccup.
But isolation isn't free. Each process carries:
- Its own copy of the heap, stack, and memory pages
- Its own file descriptor table, open sockets, and kernel resources
- OS overhead to track and protect it
Spawning a process is expensive—milliseconds on modern hardware, but measurably heavier than a thread. And if two processes need to share data, they can't just read the same memory. One process must copy data into a pipe or socket, send it across, and the other process must copy it out and into its own memory. That's overhead on every exchange.
Threads: Speed and Sharing, With a Trap
Threads live inside a single process and share that process's entire memory. The kernel doesn't wall them off from each other. When you spawn a thread, you're not duplicating the heap, the file descriptors, or the kernel state—you're just creating a new stack and registering it with the scheduler.
Spawning a thread is orders of magnitude cheaper than spawning a process. And sharing data is free: both threads read and write the same variables. No copying, no pipes. If you have multi-core hardware and a job that threads can split (parsing a large file, computing in parallel), threads let you put all your cores to work on that one job instantly.
But shared memory is the trap.
When two threads touch the same data without coordination, they collide. One thread reads a 64-bit integer while another thread is writing it—the reader sees a torn write (half the old value, half the new value). Two threads increment a counter simultaneously—both read 5, both write 6, and you lost an increment. One thread modifies a linked list while another traverses it—the traverser hits a dangling pointer and crashes.
To avoid collisions, you add locks (mutexes, semaphores). Thread A locks a resource, does its work, unlocks it. Thread B waits for the lock. But now you've traded one problem for another:
- Deadlock: Thread A holds lock X and waits for lock Y. Thread B holds lock Y and waits for lock X. Both freeze forever.
- Lock contention: Many threads fighting for the same lock spend CPU time spinning, waiting, and context-switching instead of working.
- Fragile logic: A lock protects a section of code, but a developer forgets to acquire it in one place, and you have a race condition that shows up in production only under load.
And crucially: if one thread crashes (null pointer dereference, stack overflow, segmentation fault), it brings down the entire process. No isolation. All threads die with it.
Head-to-Head Comparison
| Property | Process | Thread |
|---|---|---|
| Memory space | Own isolated address space | Shared within one process |
| Startup cost | Heavy (copy memory, kernel state) | Light (just a stack) |
| Data sharing | Copy via pipe/socket | Direct read/write (free but risky) |
| One crash | Stays contained | Kills entire process |
| Synchronization | Message passing (inherently safe) | Locks, mutexes (race conditions, deadlock) |
| Scaling to cores | Hard (separate heaps) | Easy (shared memory) |
A Concrete Example: Parsing a Large File
Using threads:
import threading
results = [] # Shared across threads
lock = threading.Lock()
def parse_chunk(chunk_data, chunk_id):
parsed = parse(chunk_data)
with lock:
results.append((chunk_id, parsed))
threads = []
for i, chunk in enumerate(chunks):
t = threading.Thread(target=parse_chunk, args=(chunk, i))
threads.append(t)
t.start()
for t in threads:
t.join()
print(results)
Each thread parses its chunk and locks before appending to the shared results list. Cheap to spawn, instant sharing—but you must remember the lock or race.
Using processes:
from multiprocessing import Pool
def parse_chunk(chunk_data):
return parse(chunk_data)
with Pool(4) as pool:
results = pool.map(parse_chunk, chunks)
print(results)
Each worker process parses independently. The Pool framework collects results and sends them back. No locks, no races—each process is isolated. But the framework must serialize the chunks, send them over, deserialize, parse, serialize the results, send them back, and deserialize. That copying is the cost.
Why Python Threads Are Different: The GIL
Python's Global Interpreter Lock (GIL) means only one thread can execute Python bytecode at a time. If you spawn 10 threads in Python, they do take turns, but they don't run in true parallel on multi-core systems—the GIL ensures only one holds the interpreter at once. CPU-bound threads don't scale; I/O-bound threads do (one thread blocks on a socket read, another grabs the GIL and runs).
For CPU-bound parallel work in Python, processes (via multiprocessing) sidestep the GIL entirely. Each process has its own interpreter and its own GIL, so they run in true parallel. The cost is process creation overhead and data serialization, but you get real parallelism.
When LLM Inference Matters: Batching Over Threads
If you're serving LLM inference (or any I/O-heavy workload), concurrency is critical. Many requests come in, and you want to batch them and serve them in parallel. At first glance, threads look ideal—cheap, share memory for the model weights.
But shared memory on a model is a trap. If multiple threads try to read from the same model weights while one is loading new weights, you have data races. Most production inference engines (vLLM, Ray, TensorFlow Serving) use process isolation per request batch or per GPU, not threads, because:
- Process isolation prevents one corrupted inference from poisoning the weights.
- Modern inference is I/O-heavy (reading from disk, network requests)—threads excel at this but aren't the bottleneck.
- GPUs naturally parallelize batches, so multi-threading inside a single process adds complexity without proportional gain.
The pattern is: use async I/O (coroutines) to queue many requests cheaply, batch them, and hand them off to a process or GPU worker. Threads between request and batch are overkill; processes between batch and inference are safety.
The Verdict
Reach for threads when: you have fast, in-memory work on shared data and need to use all your cores (numeric computation, graph traversal)—and your language doesn't have a GIL, or your work is I/O-bound. Keep lock sections tiny.
Reach for processes when: you need isolation (crash safety, multi-tenancy), you're doing CPU-bound work in Python, or you're running independent jobs (worker pools, load-balanced services).
Most real systems use both: a main process with a thread pool for I/O, or a process pool running I/O-async inside each worker, or a browser spawning tabs as processes and using threads inside each for UI rendering and JavaScript execution.
The choice isn't one or the other—it's knowing the tradeoff and picking the right tool for the layer you're building.
Watch the 90-second reel for a quick visual breakdown of this exact tradeoff.
Top comments (0)