Why Use a Thread Pool Instead of Creating Threads Manually?

#multithreading #tradeoff #architecture

When you write multithreaded code, the simplest thing to do is create a new thread every time you need one:

new Thread(() -> Task()).start();

This works fine for small programs. But in a real application that handles many requests, this approach causes serious problems. Let's understand why, and how thread pools solve them.

Problem 1: Creating Threads Is Expensive

Every time you create a thread, the JVM and the OS have to do real work:

Allocate memory for the thread's stack (usually 512KB–1MB)
Register the thread with the OS scheduler
Tear it all down again when the thread finishes

If you create a new thread for every task, you pay this cost again and again. For a handful of tasks, that's fine. For thousands of tasks per second, it adds up fast and slows everything down.

Problem 2: No Limit on How Many Threads Get Created

new Thread() has no idea how many threads your system can actually handle. If your app suddenly gets a burst of traffic, it will happily create thousands of threads — more than your CPU can realistically run at once.

Too many threads means the CPU spends more time switching between them than actually doing work. In the worst case, you run out of memory and the whole application crashes.

Problem 3: No Backpressure

"Backpressure" means having a way to slow down or reject work when the system is overloaded, instead of accepting everything and falling over.

With manual thread creation, there is no such mechanism. Every request just gets its own thread, no matter how busy the system already is.

The Solution: Thread Pools

A thread pool creates a fixed set of worker threads upfront. Instead of creating a new thread per task, tasks are added to a queue, and the existing worker threads pick them up one by one.

ExecutorService pool = Executors.newFixedThreadPool(10);
pool.submit(() -> Task());

This gives you three big advantages:

Threads are reused, not recreated every time — so you avoid the repeated creation/teardown cost.
The number of threads is bounded, so your system doesn't spiral out of control under heavy load.
A queue holds extra tasks when all threads are busy, instead of creating more threads than the system can handle.

Key Tradeoffs to Know

Decision	Options	What to consider
Pool size	Small vs large	CPU-bound work → keep pool size close to number of CPU cores. I/O-bound work (waiting on network/DB) → can use a larger pool since threads spend time waiting, not computing.
Queue size	Small vs large	Small queue = fails fast when overloaded. Large queue = absorbs bursts, but tasks may wait so long they're no longer useful by the time they run.
What happens when the queue is full	Reject / Block / Run on caller's thread	This decides who "pays" for overload the pool, the caller, or the task itself.