Python Concurrency — Master Reference

0. The four modules, one sentence each

Module	What it actually is
threading	Multiple OS threads, one process, one GIL — only one thread runs Python bytecode at a time
multiprocessing	Multiple OS processes, each with its own interpreter + own GIL — real parallelism
asyncio	One thread, cooperative scheduling via an event loop — concurrency without parallelism
concurrent.futures	A uniform API (Future, submit, map) sitting on top of threading and multiprocessing — not a fourth concurrency mechanism, a wrapper around the first two

Only two of these give you actual parallelism (simultaneous CPU execution): multiprocessing, and (indirectly) concurrent.futures.ProcessPoolExecutor. Everything else — threading, asyncio, ThreadPoolExecutor — gives you concurrency (interleaving) without parallelism, useful specifically for I/O-bound work.

---

1. Problem → tool decision table

Your workload	Right tool	Why
CPU-bound, pure Python (math, parsing, image processing in Python)	multiprocessing / ProcessPoolExecutor	GIL blocks real parallelism on threads; only separate processes give you multiple cores
I/O-bound, blocking libraries (sync requests, blocking DB driver, file I/O), moderate task count	threading / ThreadPoolExecutor	Blocking calls release the GIL; threads are cheap, no pickling, no IPC
I/O-bound, very high concurrency (thousands of open connections — web scraping, chat servers, API gateways)	asyncio	Threads don't scale to 10k+ (memory/OS scheduler cost); one loop thread can juggle 10k sockets via epoll
Mixed: an asyncio program needs to call one blocking/legacy function	asyncio.to_thread() / loop.run_in_executor()	Offloads just that one blocking call to a thread so it doesn't stall the loop
Need Future/cancel/callback ergonomics over threads or processes, don't want to hand-roll Thread/Process bookkeeping	concurrent.futures (ThreadPoolExecutor/ProcessPoolExecutor)	Same Future API either way, less boilerplate than raw Thread/Process + manual result plumbing

---

2. threading — module summary

Solves: running blocking I/O concurrently within one process without separate processes; sharing memory directly between concurrent units (no serialization).
Cannot: achieve real CPU parallelism (GIL); safely force-kill a running thread.

Construct	Purpose
Thread(target=fn).start() / .join()	Create and run a thread; wait for it to finish
Lock / RLock	Mutual exclusion around shared state
Condition	Wait/notify coordination on shared state
Event	Simple one-shot "has this happened yet" flag, .set()/.wait()
Semaphore	Cap concurrent access to N
Barrier	All N threads wait until all N arrive
threading.local()	Per-thread storage, no locking needed

No Future, no return value from Thread.run() — you must smuggle results out via shared mutable state + a Lock, or use concurrent.futures.ThreadPoolExecutor instead if you want a Future.

Alternative: ThreadPoolExecutor for the same underlying threads, but with pooling, Futures, and less manual bookkeeping. Prefer the executor unless you need fine-grained primitives (Condition, Barrier, Semaphore) that concurrent.futures doesn't expose.

---

3. multiprocessing — module summary

Solves: real CPU parallelism (separate interpreters/GILs); isolation (a crashed worker doesn't take down the parent).
Cannot: share memory by default (must use Value/Array/Manager explicitly); pass unpicklable objects (lambdas, local functions, open sockets/files) across the process boundary; cheaply spin up huge numbers of workers (real OS processes, heavier than threads).

Construct	Purpose	Not to confuse with
Process(target=fn).start() / .join()	Create/run a raw process	threading.Thread (same shape, different cost/isolation)
Pool(n)	Worker pool for parallel map-style work	ThreadPoolExecutor/ProcessPoolExecutor (very similar API, different module)
Pool.map(fn, iterable)	One iterable, one arg per call	Executor.map (variadic iterables, different unpacking rule)
Pool.starmap(fn, iterable_of_tuples)	Unpacks each tuple as multiple args	No equivalent in concurrent.futures — you transpose or wrap manually
Value / Array	Shared memory primitives across processes	threading.local (opposite purpose — per-thread, not shared)
Lock (multiprocessing's own)	Cross-process mutual exclusion	threading.Lock — same name, different implementation, not interchangeable
initializer / initargs (on Pool or Process)	Run setup once per worker at startup, typically to stash a Lock/Value in a module global	—
fork vs spawn (start method)	How child processes are created; affects what's inherited vs must be pickled	—

No create_task here — that name belongs to asyncio. multiprocessing's task-scheduling entry points are Process(...).start() and Pool.map/Pool.apply_async.

Alternative: ProcessPoolExecutor for the same real parallelism with Future ergonomics instead of Pool's own return objects (AsyncResult). Prefer the executor unless you specifically need Pool's starmap, or fine control over Process lifecycle that the executor abstracts away.

---

4. asyncio — module summary

Solves: extremely high I/O concurrency (thousands of simultaneous waits) on a single thread, with no locking needed for state shared between coroutines (only one runs at a time, by definition).
Cannot: get CPU parallelism (still one thread); tolerate any blocking call inside a coroutine without stalling the entire loop; use if your libraries are synchronous-only (needs async-native libraries, or offloading via to_thread).

Construct	Purpose	Not to confuse with
async def	Define a coroutine function	A regular function — calling it returns a coroutine object, does not run the body
await	Yield control to the loop at a cooperative point	.result() (blocking, sync) — await is the async equivalent, only valid inside async def
asyncio.run(main())	Start the event loop, run one coroutine to completion	—
asyncio.create_task(coro)	Schedule a coroutine to run concurrently, starts immediately without waiting	Executor.submit() — conceptually similar ("schedule and get a handle back"), but this is asyncio-only, works on coroutines not arbitrary callables, and returns an asyncio.Task, not a concurrent.futures.Future
asyncio.gather(*coros)	Run several coroutines/tasks concurrently, collect all results	concurrent.futures.wait(..., ALL_COMPLETED) — same idea, different universe
asyncio.TaskGroup	Structured concurrency: run several tasks, auto-cancel siblings on first exception	concurrent.futures has no equivalent — no auto-cancel-siblings there at all
asyncio.sleep(n)	Non-blocking "wait," yields to loop, does not occupy the thread	time.sleep(n) — blocking, freezes the whole loop if called inside a coroutine
asyncio.Lock / asyncio.Event / asyncio.Semaphore	Same coordination concepts as threading's, but await-based and asyncio-only	threading.Lock/Event/Semaphore — not safe or usable across the two worlds interchangeably
asyncio.to_thread(fn, *args)	Run a blocking sync function on a worker thread, await-able from a coroutine	Built on top of concurrent.futures.ThreadPoolExecutor internally
loop.run_in_executor(executor, fn, *args)	Same idea, lets you pass your own executor (thread or process pool)	executor=None uses the loop's lazily-created default ThreadPoolExecutor

Key naming trap: create_task is asyncio's scheduling primitive for coroutines. It has no relationship to multiprocessing.Process or concurrent.futures.submit() beyond the vague shared idea of "start some concurrent unit of work." Different object types come back (Task vs Future vs nothing/AsyncResult), different rules about what you can pass in (coroutine vs arbitrary picklable callable vs arbitrary callable).

Alternative: none, for the "thousands of concurrent I/O waits on one thread" niche — this is asyncio's unique reason to exist. For CPU-bound work reached from inside async code, you must delegate to run_in_executor with a ProcessPoolExecutor — asyncio itself never provides parallelism.

---

5. concurrent.futures — module summary

Solves: one uniform interface (submit, map, Future) over both ThreadPoolExecutor and ProcessPoolExecutor, so switching between thread-based and process-based execution is a one-line change; gives threads a proper Future/return-value/exception-propagation story that raw threading lacks.
Cannot: forcibly cancel/interrupt already-running work (thread or process); auto-cancel sibling tasks when one fails (no TaskGroup/errgroup behavior built in); provide the asyncio-style thousands-of-connections scaling (still bounded by real OS threads/processes).

Construct	Purpose	Not to confuse with
ThreadPoolExecutor(max_workers=n)	Pool of reused threads	threading.Thread (raw, no pooling, no Future)
ProcessPoolExecutor(max_workers=n)	Pool of reused processes	multiprocessing.Pool (very similar, older API, AsyncResult instead of Future, has starmap)
.submit(fn, *args)	One call → one Future, heterogeneous tasks	asyncio.create_task — similar "get a handle back" idea, different object, different world
.map(fn, *iterables)	Same fn across N inputs, ordered results, lazy iterator	multiprocessing.Pool.map(fn, iterable) — single iterable + no auto-unpack; use .starmap there for multi-arg
Future.result() / .exception()	Block for value / block for exception object (no raise)	—
Future.done() / .running() / .cancelled()	State queries	—
Future.cancel()	Works only if still PENDING	Cannot stop RUNNING work — true for both executor types, and for asyncio Task.cancel() too (though that one can interrupt at the next await)
Future.add_done_callback(fn)	Fire fn(future) on completion	Runs in a worker thread (ThreadPoolExecutor) or a result-handling thread in the main process (ProcessPoolExecutor) — never in a worker process
as_completed(futures)	Yield futures one at a time, in completion order	wait() — single blocking call returning a done/not-done split, not an iterator
wait(futures, return_when=...)	One blocking call: ALL_COMPLETED / FIRST_COMPLETED / FIRST_EXCEPTION	FIRST_EXCEPTION only stops waiting early — does not cancel or stop sibling futures
initializer / initargs (on ProcessPoolExecutor)	Same pattern as multiprocessing.Pool's — set up per-worker state (e.g. stash a Lock) once	—
chunksize (on ProcessPoolExecutor.map() only)	Batch items per IPC round-trip, reduce overhead for many cheap calls	Meaningless for ThreadPoolExecutor (no IPC to amortize)

Alternative relationships, summarized:

• vs threading: use the executor unless you need Condition/Barrier/threading.local directly.
• vs multiprocessing.Pool: use the executor for Future ergonomics; drop to Pool.starmap only if you specifically need row-wise multi-arg unpacking without transposing.
• vs asyncio: use concurrent.futures directly for a purely thread/process-based program; use it through to_thread/run_in_executor only when you need to bridge one blocking call out of an otherwise-async program.

---

6. The name-collision cheat sheet (the actual source of your confusion)

If you see...	It belongs to...	Returns...	Do NOT use it with...
Thread(...).start()	threading	nothing (no return value)	—
Process(...).start()	multiprocessing	nothing (no return value)	—
Pool.map(fn, iterable)	multiprocessing	plain list	concurrent.futures — different unpacking rules
Pool.starmap(fn, iter_of_tuples)	multiprocessing only	plain list	concurrent.futures.Executor — no starmap exists there
Executor.submit(fn, *args)	concurrent.futures only	concurrent.futures.Future	—
Executor.map(fn, *iterables)	concurrent.futures only	lazy iterator, zips multiple iterables	multiprocessing.Pool — that .map takes one iterable, no auto-zip
asyncio.create_task(coro)	asyncio only	asyncio.Task (subclass of asyncio.Future)	multiprocessing/concurrent.futures — no such method there, and it only accepts coroutines, never plain callables
asyncio.gather(*coros)	asyncio only	list of results	concurrent.futures.wait/as_completed — parallel concept, different API, different object types
asyncio.Future	asyncio	—	concurrent.futures.Future — similarly named, different class; to_thread/run_in_executor bridge one into the other internally, you don't do it by hand
Lock	exists in three places: threading.Lock, multiprocessing.Lock, asyncio.Lock	—	Never cross-use — a threading.Lock does not protect across processes; a multiprocessing.Lock is not await-able; an asyncio.Lock only makes sense between coroutines on the same loop
to_thread / run_in_executor	asyncio (but internally built on concurrent.futures.ThreadPoolExecutor)	await-able wrapped future	Don't reach for these unless you're inside asyncio code bridging out to a blocking call

Rule of thumb to stop the mixing: if the code has async def/await anywhere, you're in asyncio's vocabulary (create_task, gather, TaskGroup). If it doesn't, and you're using a pool of workers, you're either in bare threading/multiprocessing (Thread/Process/Pool, no Future) or in concurrent.futures (Executor, submit/map, Future) — never both vocabularies for the same call.