Thread Pool

1. The Problem with Creating Threads Per Request

Ever implemented a server that creates a new thread for every request and destroys it when done? This approach has some critical issues you might not immediately notice.

• Performance Issues
  Thread creation is an expensive operation. The OS has to allocate stack memory, set up execution context, and handle various initialization overhead.

• Resource Exhaustion
  When thousands of concurrent requests hit your server, an equal number of threads get created. Memory gets exhausted rapidly, and your system becomes unresponsive.

Thread Pool solves this elegantly.

---

2. Thread Pool

Thread Pool addresses these problems cleanly.

The core idea is simple:

• Pre-create threads and keep them waiting
• Assign work to idle threads when requests come in
• Reuse threads instead of destroying them after work completes

Benefits

1. Performance Boost: Eliminates thread creation overhead
2. Resource Control: Pool size limits concurrent thread count
3. Stability: Thread count stays bounded even during traffic spikes

Think of it like a restaurant: instead of hiring a chef every time a customer orders, you keep N chefs on staff and distribute orders among them.

Architecture

A Thread Pool consists of 3 main components:

1. Worker Threads

• N pre-created threads
• Stay alive, waiting for work

2. Task Queue

• Queue where pending tasks wait
• Must be thread-safe (multiple threads access it)

3. Work Distribution Logic

• Idle threads fetch tasks from the queue
• After completing work, they loop back to check the queue

---

3. Code Example

Using a Thread Pool

// Create Thread Pool
size_t num_threads = std::thread::hardware_concurrency(); // CPU core count
ThreadPool pool(num_threads);

// Insert tasks into queue
for (int i = 0; i < 1000; i++) {
    pool.enqueue([i]() {
        processRequest(requests[i]);
    });
}
// Pool internally maintains only N threads while processing 1000 tasks

Basic Thread Pool Structure

class ThreadPool {
public:
    ThreadPool(size_t threads);
    ~ThreadPool();

    // Add task to queue
    template<class F>
    void enqueue(F&& f);

private:
    std::vector<std::thread> workers;           // Worker threads
    std::queue<std::function<void()>> tasks;    // Task queue

    std::mutex queue_mutex;                     // Protects queue
    std::condition_variable condition;          // Wait/notify threads
    bool stop;
};