DEV Community: Prakash Dass

From TCP Sockets to Thread Pools - Building a Production Grade Web Framework

Prakash Dass — Sun, 09 Nov 2025 12:20:36 +0000

From TCP Sockets to Thread Pools: Building a Production-Grade C++ Web Framework

When you type a URL into your browser and hit enter, something magical happens behind the scenes. Within milliseconds, your request travels across the internet, finds the right server, gets processed, and returns with the content you requested. But have you ever wondered what's really happening inside that web server?

Building a web server from scratch strips away the magic and reveals the elegant engineering underneath. It's where operating system concepts, network programming, and concurrent computing converge into a single, cohesive system. This is the story of NanoHost - a journey from raw TCP sockets to a production-grade C++ web framework.

The Foundation: TCP Sockets

Every web server begins with a socket - a file descriptor that acts as a communication endpoint. When you create a socket, you're asking the operating system to allocate resources for network communication. You bind it to a port, tell it to listen for connections, and suddenly, your program can receive requests from anywhere in the world.

But here's the challenge: the default socket behavior is blocking. When you call accept() to receive a connection, your program freezes until a client connects. This blocking nature creates a fundamental problem: how do you serve thousands of concurrent users when your main thread is stuck waiting?

The answer lies in non-blocking I/O. By configuring sockets with the O_NONBLOCK flag, the accept() call returns immediately, even when no connection is available. Instead of waiting, you check if a connection exists, process it if available, and move on. This simple change transforms your server from a sequential bottleneck into a concurrent powerhouse.

There's another subtle but critical detail: the SO_REUSEADDR socket option. Without it, when you restart your server, the operating system keeps the port in a TIME_WAIT state for about 60 seconds. This means rapid development cycles become painfully slow, and deployment scripts fail unexpectedly. Setting this option is the difference between a development-friendly server and a frustrating one.

The Scalability Problem: Thread Pools

Non-blocking I/O solves one problem but creates another. If your main thread never blocks, it can accept connections rapidly. But who processes them? The naive solution is to spawn a new thread for each connection. This works initially, but it fails catastrophically at scale.

Each thread consumes 2-8 megabytes of stack space. With 10,000 concurrent connections, you're looking at 20-80 gigabytes of memory just for thread stacks. Beyond memory, there's the context-switching overhead - the operating system must constantly save and restore thread states, grinding your CPU to a halt. This is the infamous C10K problem: how do you handle 10,000 concurrent connections?

The thread pool pattern provides the solution. Instead of creating threads on demand, you pre-create a fixed number of worker threads - say, 15 - and maintain a queue of pending tasks. When a connection arrives, you don't create a new thread; you simply enqueue a task. Worker threads continuously pull tasks from this queue, execute them, and return for more.

The elegance is in the synchronization. A mutex protects the shared queue, ensuring thread safety. A condition variable allows worker threads to sleep when no work is available, waking them instantly when a task arrives. Atomic variables track active tasks without locking overhead. This combination delivers maximum concurrency with minimal resource consumption.

Parsing the Protocol: HTTP

Once a connection is accepted and assigned to a worker thread, you're faced with a stream of bytes. These bytes represent an HTTP request, but they're unstructured. Your job is to parse them into meaningful components: method, path, headers, and body.

HTTP is a text-based protocol with a specific structure. The first line contains the method (GET, POST, etc.), the path, and the HTTP version. Following lines contain headers as key-value pairs. An empty line separates headers from the optional body. While this sounds simple, the details matter. Different systems use different line endings. Header values may contain leading whitespace. The body length might be specified in a Content-Length header or be missing entirely.

The parsing strategy is a state machine with three states: parsing the request line, parsing headers, and reading the body. You iterate through the input stream, transitioning between states based on what you encounter. Robust error handling is crucial - malformed requests shouldn't crash your server.

The reverse process - generating HTTP responses - follows a similar pattern. You build the status line, add headers, insert an empty line, and append the body. Static factory methods like HTTPResponse::ok() and HTTPResponse::error() encapsulate this logic, providing a clean API for route handlers.

Routing Requests: Hash Maps and Strategies

A web framework isn't just a server; it's a routing engine. When a request arrives for "/api/users", how do you know which code to execute? The answer is a routing table - a mapping from paths to handler functions.

The simplest implementation uses a hash map. Paths are keys, and handler functions are values. When a request arrives, you look up the path in the map and invoke the corresponding handler. This provides O(1) lookup time, making routing extremely fast even with thousands of routes.

But modern frameworks support multiple routing styles. RESTful routes use HTTP methods and URL paths. RPC-style interfaces send JSON with an "action" field. WebSocket connections require protocol upgrades. How do you handle this diversity?

The Strategy pattern provides the answer. Your dispatcher doesn't commit to a single routing strategy. Instead, it tries multiple strategies in sequence. Is this a JSON POST request with an "action" field? Route it through the action dispatcher. Is it a standard HTTP request with a registered route? Use the REST router. Otherwise, try serving a static file. This cascading approach provides maximum flexibility while maintaining clean code architecture.

Design Patterns in Practice

Great software isn't just about solving problems; it's about solving them elegantly. Design patterns provide proven solutions to recurring challenges.

The Factory pattern encapsulates object creation. Instead of constructing HTTPResponse objects directly, you call factory methods that hide the complexity. The Dependency Injection pattern decouples components. Your AppDispatcher doesn't create a Router; it receives one through its constructor. This makes testing trivial - inject a mock router and verify behavior in isolation.

The Producer-Consumer pattern governs the thread pool. The main thread produces tasks by enqueueing them. Worker threads consume tasks by dequeueing and executing them. A condition variable coordinates this dance, ensuring threads sleep when idle and wake when needed.

Perhaps most importantly, RAII (Resource Acquisition Is Initialization) prevents resource leaks. Locks are automatically released when they go out of scope. Threads are automatically joined in the destructor. File descriptors are closed when objects are destroyed. This automatic resource management eliminates entire categories of bugs.

The Request Lifecycle

Let's trace a single request from arrival to response:

A client connects. The accept() call returns a socket file descriptor.
The main thread enqueues a task to the thread pool and immediately returns to accepting more connections.
A worker thread wakes up, dequeues the task, and begins execution.
The worker reads bytes from the socket using recv(), accumulating them in a buffer.
Once the complete request is received, it's parsed into an HTTPRequest object.
The dispatcher routes the request through appropriate strategies until a handler is found.
The handler executes, performing business logic and generating a response string.
The response string is wrapped in an HTTPResponse object with appropriate status codes and headers.
The formatted HTTP response is sent back to the client using send().
The socket is closed, and the worker thread returns to the pool, ready for the next task.

Throughout this process, the main thread never blocks on slow operations. Worker threads execute independently, processing requests in parallel. Synchronization happens only during queue operations, minimizing contention. This architecture delivers high throughput with low latency.

Building for Production

A production web server requires more than just correct functionality. It needs configuration flexibility, graceful error handling, and operational visibility.

Command-line arguments provide runtime configuration. Port numbers, thread counts, and static file directories shouldn't be hardcoded. Input validation prevents crashes from invalid configurations - port numbers must be between 1 and 65535, thread counts must be positive.

Signal handling enables graceful shutdown. When you press Ctrl+C or send SIGTERM, the server should finish processing active requests, close all sockets, and exit cleanly. This prevents data loss and ensures consistent state.

Extensibility points allow customization without modifying core code. Route handlers are just functions - you can integrate databases, call external APIs, or implement complex business logic. Middleware patterns enable cross-cutting concerns like authentication, logging, and CORS headers.

What We've Built

NanoHost demonstrates that production-grade systems are built on fundamental concepts: sockets for networking, threads for concurrency, mutexes for synchronization, and design patterns for clean architecture. It's pure C++ with no external dependencies, proving that you don't need frameworks to build frameworks.

The performance speaks for itself: 18,500 requests per second with just 15 threads, 5.4 milliseconds average latency, and 45 megabytes of memory usage. These numbers come from understanding how operating systems work and leveraging them effectively.

More importantly, this project reveals the layers of abstraction beneath modern web development. Express.js, Flask, and Django hide these details for productivity, but understanding them makes you a better engineer. You debug smarter, design better APIs, and make informed trade-offs.

Building a web server from scratch isn't just an academic exercise. It's a masterclass in systems programming, concurrent computing, and software architecture.

Explore the full source code: github.com/rprakashdass/nanohost

From TCP Sockets to Thread Pools - Building a Production Grade C++ Web Framework

Prakash Dass — Sun, 09 Nov 2025 08:09:13 +0000

From TCP Sockets to Thread Pools: Building a Production-Grade C++ Web Framework

Note: This is an advanced technical deep-dive into systems programming concepts. We'll explore TCP/IP networking, multithreading primitives, asynchronous request handling, and production-grade architectural patterns in C++.

Introduction
Architecture Overview
TCP/IP Socket Programming
POSIX Thread Pool Implementation
HTTP Protocol Layer
Routing Engine Architecture
Design Patterns & Architectural Decisions
Asynchronous Request Processing Flow
CLI Configuration & Extensibility
Conclusion

Introduction

Modern web servers are complex systems that orchestrate multiple OS-level concepts: socket programming, multithreading, synchronization primitives, and protocol handling. In this article, we'll dissect NanoHost, a lightweight yet powerful C++ web framework that demonstrates these concepts in production-quality code.

What makes this interesting?

Pure C++23 with POSIX compliance
Thread pool pattern for the C10K problem
Non-blocking I/O for high concurrency
Zero external dependencies for core functionality
Express.js-inspired API design

Source Code: github.com/rprakashdass/nanohost

Core Components

Component	Responsibility	OS Concepts Used
Socket Layer	TCP/IP networking	`socket()`, `bind()`, `listen()`, `accept()`
ThreadPool	Concurrent request handling	POSIX threads, mutexes, condition variables
HTTPRequest	Protocol parsing	String processing, state machines
Router	Request dispatching	Hash maps, function pointers
AppDispatcher	Strategy orchestration	Strategy pattern, dependency injection
StaticServer	File serving	File I/O, MIME type detection

TCP/IP Socket Programming

Server Socket Initialization

The foundation of any web server is the socket - an OS-level abstraction for network communication. Let's break down the initialization process:

// 1. Create a TCP socket
int server_socket = socket(AF_INET, SOCK_STREAM, 0);
if (server_socket == -1) {
    std::cerr << "[Error] Couldn't create socket" << std::endl;
    return 1;
}

What's happening here?

AF_INET: Address Family - Internet (IPv4)
SOCK_STREAM: Type - TCP (reliable, connection-oriented)
0: Protocol - Default TCP protocol

Socket Options for Production

// Allow immediate socket reuse (important for rapid restarts)
int opt = 1;
setsockopt(server_socket, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

Why SO_REUSEADDR?
Without this, after stopping your server, the OS keeps the port in TIME_WAIT state for ~60 seconds. This prevents immediate restarts - critical in development and deployment scenarios.

Non-Blocking I/O Configuration

// Configure socket for non-blocking operation
int flags = fcntl(server_socket, F_GETFL, 0);
fcntl(server_socket, F_SETFL, flags | O_NONBLOCK);

Non-blocking vs Blocking I/O:

Blocking I/O	Non-Blocking I/O
Thread waits for data	Returns immediately with EAGAIN
Simple programming model	Requires polling/event loops
Limited concurrency	High concurrency support
One thread per connection	Thousands of connections per thread

Binding and Listening

// Bind socket to address and port
sockaddr_in sockAddr{};
sockAddr.sin_family = AF_INET;
sockAddr.sin_port = htons(port);              // Convert to network byte order
sockAddr.sin_addr.s_addr = INADDR_ANY;        // Listen on all interfaces

bind(server_socket, (sockaddr*)&sockAddr, sizeof(sockAddr));

// Start listening with backlog of 10 pending connections
listen(server_socket, 10);

Network Byte Order:
Different CPU architectures store multi-byte values differently (endianness). htons() (Host TO Network Short) ensures consistent byte ordering across network.

Accepting Connections

while (keepRunning) {
    sockaddr_in client_addr{};
    socklen_t client_len = sizeof(client_addr);

    int client_socket = accept(server_socket, 
                               (sockaddr*)&client_addr, 
                               &client_len);

    if (client_socket == -1) {
        if(errno == EAGAIN || errno == EWOULDBLOCK) {
            // No connection available, sleep briefly to avoid busy-waiting
            std::this_thread::sleep_for(std::chrono::milliseconds(100));
            continue;
        }
        // Handle other errors
    }

    // Enqueue connection to thread pool
    pool.enqueueTask([client_socket, &app]() {
        handleConnection(client_socket, app);
    });
}

Key Design Decision:
The main thread only accepts connections and delegates work to the thread pool. This prevents blocking on slow client operations.

POSIX Thread Pool Implementation

The C10K Problem

Traditional thread-per-connection models fail at scale:

Memory overhead: Each thread consumes ~2-8MB of stack space
Context switching: OS overhead switching between thousands of threads
Thread creation cost: Creating/destroying threads is expensive

Solution: Thread Pool Pattern

Pre-create a fixed number of worker threads that process tasks from a queue.

ThreadPool Design

class ThreadPool {
public:
    ThreadPool(size_t numberOfThreads);
    ~ThreadPool();

    void enqueueTask(std::function<void()> task);
    void waitAll();

private:
    std::atomic<bool> stop;
    std::mutex taskQueueMtx;
    std::condition_variable condition;
    std::queue<std::function<void()>> tasksQueue;
    std::vector<std::thread> workerThreads;

    std::atomic<int> activeTasks;
    std::mutex waitMtx;
    std::condition_variable tasksDoneCondition;

    void workerTask();
};

Initialization: Creating Worker Threads

ThreadPool::ThreadPool(size_t numberOfThreads) 
    : stop(false), activeTasks(0) {

    for(size_t i = 0; i < numberOfThreads; i++) {
        workerThreads.emplace_back([this]() {
            this->workerTask();
        });
    }
}

Lambda Capture [this]:
Captures the thread pool object pointer, allowing worker threads to access member variables safely.

Producer-Consumer Pattern with Condition Variables

void ThreadPool::enqueueTask(std::function<void()> task) {
    {
        std::unique_lock<std::mutex> lock(taskQueueMtx);
        tasksQueue.push(task);
    }  // Lock released here

    condition.notify_one();  // Wake up one sleeping worker
}

Why notify outside the lock?
Reduces contention - the woken thread can immediately acquire the lock without competing with the enqueueing thread.

Worker Thread Event Loop

void ThreadPool::workerTask() {
    while(true) {
        std::function<void()> task;
        {
            std::unique_lock<std::mutex> lock(taskQueueMtx);

            // Wait until: task available OR shutdown requested
            condition.wait(lock, [this]() {
                return stop || !tasksQueue.empty();
            });

            if(stop && tasksQueue.empty()) return;  // Graceful shutdown

            task = std::move(tasksQueue.front());
            tasksQueue.pop();
        }  // Release lock before executing task

        activeTasks++;
        task();  // Execute outside lock - allows concurrent execution
        activeTasks--;

        // Notify waiters if all tasks complete
        if(tasksQueue.empty() && activeTasks == 0) {
            tasksDoneCondition.notify_all();
        }
    }
}

Synchronization Primitives Explained

Mutex (Mutual Exclusion):

std::mutex taskQueueMtx;  // Protects shared queue

Ensures only one thread accesses the queue at a time.

Condition Variable:

std::condition_variable condition;

Efficient thread sleeping/waking mechanism. Threads sleep until signaled, avoiding busy-waiting.

Atomic Variables:

std::atomic<bool> stop;
std::atomic<int> activeTasks;

Lock-free synchronization for simple counters/flags. Hardware-supported atomic operations.

Graceful Shutdown

ThreadPool::~ThreadPool() {
    {
        std::lock_guard<std::mutex> lock(taskQueueMtx);
        stop = true;
    }

    condition.notify_all();  // Wake all workers

    for(auto &worker: workerThreads) {
        if(worker.joinable())
            worker.join();  // Wait for thread to finish
    }
}

RAII (Resource Acquisition Is Initialization):
Destructor automatically cleans up threads - no manual cleanup needed.

HTTP Protocol Layer

HTTP Request Structure

GET /api/users HTTP/1.1          ← Request Line
Host: localhost:11111            ← Headers
Content-Type: application/json
Content-Length: 42

{"user": "john"}                 ← Body

Parsing Implementation

HTTPRequest HTTPRequest::parse(const std::string& rawRequest) {
    HTTPRequest req;
    std::istringstream iss(rawRequest);
    std::string line;

    // 1. Parse Request Line: "GET /path HTTP/1.1"
    std::getline(iss, line);
    std::istringstream firstLine(line);
    firstLine >> req.method >> req.path >> req.version;

    // 2. Parse Headers: "Key: Value"
    while(std::getline(iss, line) && line != "\r") {
        size_t colonPos = line.find(":");
        if(colonPos != std::string::npos) {
            std::string header = line.substr(0, colonPos);
            std::string value = line.substr(colonPos+1);

            // Clean whitespace and carriage returns
            value.erase(std::remove(value.begin(), value.end(), '\r'), 
                       value.end());
            value.erase(0, value.find_first_not_of(" "));

            req.headers[header] = value;
        }
    }

    // 3. Parse Body
    std::string bodyLine, bodyData;
    while(std::getline(iss, bodyLine)) {
        if(!bodyLine.empty() && bodyLine.back() == '\r') {
            bodyLine.pop_back();
        }
        bodyData.append(bodyLine + "\n");
    }
    req.body = bodyData;

    return req;
}

State Machine Approach:
The parser operates in three states:

Request Line → Extract method, path, version
Headers → Parse until empty line
Body → Read remaining data

HTTP Response Generation

class HTTPResponse {
    int statusCode;
    std::string statusMessage;
    std::unordered_map<std::string, std::string> headers;
    std::string body;

public:
    std::string to_string() const {
        std::ostringstream oss;

        // Status line
        oss << "HTTP/1.1 " << statusCode << " " 
            << statusMessage << "\r\n";

        // Headers
        for (const auto& [key, value] : headers) {
            oss << key << ": " << value << "\r\n";
        }

        // Empty line separator
        oss << "\r\n";

        // Body
        oss << body;

        return oss.str();
    }

    static HTTPResponse ok(int code, const std::string& body) {
        return HTTPResponse(code, body);
    }

    static HTTPResponse error(int code, const std::string& message) {
        return HTTPResponse(code, message);
    }
};

Factory Pattern:
Static factory methods provide clean API:

return HTTPResponse::ok(200, jsonData);
return HTTPResponse::error(404, "Not Found");

MIME Type Detection

static const std::unordered_map<std::string, std::string> mimeTypes = {
    { ".html", "text/html" },
    { ".css",  "text/css" },
    { ".js",   "application/javascript" },
    { ".json", "application/json" },
    { ".png",  "image/png" },
    { ".jpg",  "image/jpeg" },
    { ".pdf",  "application/pdf" },
    // ... comprehensive list
};

std::string getMimeType(const std::string& filePath) {
    size_t dotPos = filePath.rfind('.');
    if (dotPos != std::string::npos) {
        std::string ext = filePath.substr(dotPos);
        auto it = mimeTypes.find(ext);
        if (it != mimeTypes.end()) {
            return it->second;
        }
    }
    return "application/octet-stream";  // Default for unknown
}

Performance Note:
Static hash map provides O(1) lookup time. Initialized once, shared across all requests.

Routing Engine Architecture

Dual Routing Strategy

NanoHost supports two routing paradigms:

REST-style routes: Path-based (/api/users)
RPC-style actions: JSON-based ({"action": "getUser"})

Router Implementation

class Router {
public:
    using ActionHandler = std::function<std::string(const std::string& body)>;
    using RouteHandler = std::function<std::string(const std::string& body)>;

    void registerRoute(const std::string& path, RouteHandler handler);
    void registerAction(const std::string& action, ActionHandler handler);

    HTTPResponse route(const std::string& path, 
                      const std::string& body) const;
    HTTPResponse dispatchAction(const std::string& path, 
                               const std::string& body) const;

private:
    std::unordered_map<std::string, ActionHandler> handlers;
    std::unordered_map<std::string, RouteHandler> routes;
};

Route Registration

// REST endpoints
router.registerRoute("/health", [](const std::string&) {
    return R"({"status": "ok", "uptime": "24h"})";
});

router.registerRoute("/users/:id", [](const std::string& body) {
    auto json = nlohmann::json::parse(body);
    return fetchUser(json["id"]);
});

// RPC actions
router.registerAction("greet", [](const std::string& body) {
    auto json = nlohmann::json::parse(body);
    std::string name = json.value("name", "Guest");
    return R"({"message": "Hello, )" + name + R"(!"})";
});

Function Pointers vs Virtual Functions:
We use std::function (type-erased callable) instead of inheritance:

More flexible - accepts lambdas, function pointers, functors
Zero overhead abstraction
No virtual function call overhead

Route Resolution

HTTPResponse Router::route(const std::string& path, 
                           const std::string& body) const {
    auto it = routes.find(path);
    if (it != routes.end()) {
        return HTTPResponse::ok(200, it->second(body));
    }
    return HTTPResponse::error(404, "Route not found");
}

Time Complexity:

find() on unordered_map: O(1) average case
Much faster than regex matching or tree traversal

AppDispatcher: Strategy Orchestration

class AppDispatcher {
    Router& router;
    ActionDispatcher actionDispatcher;

public:
    HTTPResponse HandleRequest(const HTTPRequest& request) {
        const std::string& path = request.path;
        const std::string& method = request.method;

        // Strategy 1: JSON RPC actions
        auto contentTypeIt = request.headers.find("Content-Type");
        if(method == "POST" && 
           contentTypeIt != request.headers.end() && 
           contentTypeIt->second == "application/json") {
            return actionDispatcher.dispatch(request);
        }

        // Strategy 2: REST routes
        auto routeHandler = router.getRoute(path);
        if (routeHandler) {
            return router.route(request.path, request.body);
        }

        // Strategy 3: Static file serving
        return StaticServer::serve(path);
    }
};

Strategy Pattern:
Different handling strategies selected at runtime based on request characteristics.

ActionDispatcher: RPC Handler

HTTPResponse ActionDispatcher::dispatch(const HTTPRequest& request) {
    try {
        json bodyJson = json::parse(request.body);

        if (!bodyJson.contains("action")) {
            return HTTPResponse::error(400, 
                "Missing 'action' in request body");
        }

        std::string action = bodyJson["action"];
        return router.dispatchAction(action, request.body);

    } catch (const json::parse_error& e) {
        return HTTPResponse::error(400, "Invalid JSON");
    } catch (const std::exception& e) {
        return HTTPResponse::error(500, "Internal Server Error");
    }
}

Error Handling:
Comprehensive exception handling prevents server crashes from malformed requests.

Design Patterns & Architectural Decisions

1. Strategy Pattern

Purpose: Select algorithm/behavior at runtime

Implementation:

// Different strategies for different request types
if (isJSON) return actionDispatcher.dispatch(request);
if (hasRoute) return router.route(path, body);
return StaticServer::serve(path);

2. Factory Pattern

Purpose: Encapsulate object creation

Implementation:

HTTPResponse::ok(200, data);     // Success factory
HTTPResponse::error(404, msg);   // Error factory

3. Dependency Injection

Purpose: Decouple components, enable testing

Implementation:

class AppDispatcher {
    Router& router;  // Injected dependency

public:
    AppDispatcher(Router& router) 
        : router(router), 
          actionDispatcher(router) {}
};

Benefits:

Test with mock router
Swap implementations without changing code
Clear dependencies

4. RAII (Resource Acquisition Is Initialization)

Purpose: Automatic resource management

Implementation:

{
    std::unique_lock<std::mutex> lock(taskQueueMtx);
    // Critical section
}  // Lock automatically released

Prevents:

Memory leaks
Resource leaks
Exception-related bugs

5. Producer-Consumer Pattern

Purpose: Decouple work generation from work execution

Implementation:

// Producer (main thread)
pool.enqueueTask([client_socket]() { 
    handleRequest(client_socket); 
});

// Consumer (worker threads)
while(true) {
    wait_for_task();
    execute_task();
}

6. Observer Pattern (Condition Variables)

Purpose: Efficient thread notification

Implementation:

condition.notify_one();  // Wake one waiting thread
condition.notify_all();  // Wake all waiting threads

Asynchronous Request Processing Flow

Complete Request Lifecycle

1. CLIENT CONNECTS
   └─> accept() returns client_socket

2. ENQUEUE TO THREAD POOL
   └─> pool.enqueueTask([client_socket, &app]() { ... })
   └─> Returns immediately (non-blocking)

3. WORKER THREAD PICKS UP TASK
   └─> Waits on condition variable
   └─> Wakes up when task available

4. RECEIVE HTTP DATA
   └─> recv(client_socket, buffer, 4096, 0)
   └─> Reads raw bytes from network

5. PARSE HTTP REQUEST
   └─> HTTPRequest::parse(rawRequest)
   └─> Structured HTTPRequest object

6. ROUTE THROUGH DISPATCHER
   └─> app.HandleRequest(request)
   └─> Selects appropriate strategy

7. EXECUTE HANDLER
   └─> Lambda/function executes business logic
   └─> Returns response string

8. BUILD HTTP RESPONSE
   └─> HTTPResponse::to_string()
   └─> Formats as HTTP protocol

9. SEND TO CLIENT
   └─> send(client_socket, response, size, 0)
   └─> Transmits over network

10. CLEANUP
    └─> close(client_socket)
    └─> Worker thread returns to pool

Concurrency Analysis

Key Points:

Main thread never blocks on request processing
Worker threads execute independently - no coordination needed
Mutex only held during queue operations - minimal contention
Task execution outside locks - maximum parallelism

Performance Characteristics:

// Theoretical maximum throughput
max_throughput = worker_threads * (1 / avg_request_time)

// Example: 15 threads, 10ms avg request time
max_throughput = 15 * (1 / 0.01) = 1,500 requests/second

CLI Configuration & Extensibility

Command-Line Interface

// Default configuration
int port = 11111;
int threadCount = 15;
std::string staticDir = "../public";

// Parse arguments
for (int i = 1; i < argc; ++i) {
    std::string arg = argv[i];
    if (arg == "--port" && i + 1 < argc) {
        port = std::stoi(argv[++i]);
    } else if (arg == "--threads" && i + 1 < argc) {
        threadCount = std::stoi(argv[++i]);
    } else if (arg == "--static" && i + 1 < argc) {
        staticDir = argv[++i];
    }
}

Usage Examples:

# Default settings
./NanoHostApp

# Custom configuration
./NanoHostApp --port 8080 --threads 20 --static ./public

# Production deployment
./NanoHostApp --port 80 --threads 50 --static /var/www/html

Input Validation

// Validate port range
if (port <= 0 || port > 65535) {
    std::cerr << "[Error] Invalid port number: " << port << std::endl;
    return 1;
}

// Validate thread count
if (threadCount <= 0) {
    std::cerr << "[Error] Thread count must be > 0" << std::endl;
    return 1;
}

Extensibility Points

1. Custom Route Handlers

// Simple handler
router.registerRoute("/api/status", [](const std::string&) {
    return R"({"status": "online"})";
});

// Database integration
router.registerRoute("/api/users", [&db](const std::string& body) {
    auto json = nlohmann::json::parse(body);
    return db.query("SELECT * FROM users WHERE id = ?", json["id"]);
});

// Async operations
router.registerRoute("/api/slow", [](const std::string&) {
    std::this_thread::sleep_for(std::chrono::seconds(2));
    return "Completed slow operation";
});

2. Middleware Pattern

// Authentication middleware
auto authMiddleware = [](HTTPRequest& req, HTTPResponse& res) -> bool {
    auto authHeader = req.headers.find("Authorization");
    if (authHeader == req.headers.end()) {
        res = HTTPResponse::error(401, "Unauthorized");
        return false;
    }
    return validateToken(authHeader->second);
};

// Logging middleware
auto loggingMiddleware = [](const HTTPRequest& req) {
    std::cout << "[" << getCurrentTime() << "] " 
              << req.method << " " << req.path << std::endl;
};

// CORS middleware
auto corsMiddleware = [](HTTPResponse& res) {
    res.setHeader("Access-Control-Allow-Origin", "*");
    res.setHeader("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE");
};

3. Custom Dispatchers

// WebSocket dispatcher
class WebSocketDispatcher {
public:
    void handleUpgrade(const HTTPRequest& request, int socket);
    void handleMessage(const std::string& message);
};

// GraphQL dispatcher
class GraphQLDispatcher {
    GraphQLSchema schema;
public:
    HTTPResponse execute(const std::string& query);
};

4. Plugin System

// Plugin interface
class Plugin {
public:
    virtual void onServerStart() = 0;
    virtual void onRequest(HTTPRequest& req) = 0;
    virtual void onResponse(HTTPResponse& res) = 0;
    virtual void onServerStop() = 0;
};

// Plugin manager
class PluginManager {
    std::vector<std::unique_ptr<Plugin>> plugins;
public:
    void registerPlugin(std::unique_ptr<Plugin> plugin);
    void notifyServerStart();
    void notifyRequest(HTTPRequest& req);
};

Conclusion

We've explored the implementation of a production-grade web server, demonstrating:

OS Concepts Applied

TCP/IP Socket Programming: socket(), bind(), listen(), accept()
POSIX Threading: Thread creation, synchronization, condition variables
Non-blocking I/O: fcntl(), O_NONBLOCK, handling EAGAIN
Signal Handling: Graceful shutdown with SIGINT/SIGTERM
File Descriptors: Managing socket and file descriptors
Byte Order Conversion: Network vs host byte order

Design Patterns Used

Strategy Pattern: Request routing strategies
Factory Pattern: Response object creation
Producer-Consumer: Thread pool task queue
Dependency Injection: Loose coupling between components
RAII: Automatic resource management
Observer Pattern: Condition variable notifications

Key Takeaways

Thread pools solve the C10K problem - Pre-allocated threads eliminate creation overhead
Non-blocking I/O enables high concurrency - Don't block the accept loop
Lock-free primitives where possible - Use atomics for simple counters
Design patterns matter - They provide proven solutions to common problems
RAII prevents leaks - Let C++ manage resources automatically

Further Exploration

Next Steps:

Implement epoll/kqueue for true asynchronous I/O
Add TLS/SSL support with OpenSSL
Implement HTTP/2 with multiplexing
Add connection pooling for databases
Build comprehensive middleware system
Implement rate limiting and DDoS protection

Resources:

GitHub Repository:
github.com/rprakashdass/nanohost

About the Author

Prakash Dass R is a systems programmer with a strong foundation in low-level programming, operating system internals, and building high-performance systems. Passionate about artificial intelligence development, bringing expertise in both core systems engineering and modern AI technologies to create efficient, scalable software solutions

Connect:

Website: rprakashdass.in
GitHub: @rprakashdass
LinkedIn: Prakash Dass R

Did you find this article helpful? Consider starring the NanoHost repository on GitHub and sharing this article with your network!

DEV Community: Prakash Dass

From TCP Sockets to Thread Pools - Building a Production Grade Web Framework

From TCP Sockets to Thread Pools: Building a Production-Grade C++ Web Framework

The Foundation: TCP Sockets

The Scalability Problem: Thread Pools

Parsing the Protocol: HTTP

Routing Requests: Hash Maps and Strategies

Design Patterns in Practice

The Request Lifecycle

Building for Production

What We've Built

From TCP Sockets to Thread Pools - Building a Production Grade C++ Web Framework

From TCP Sockets to Thread Pools: Building a Production-Grade C++ Web Framework

Table of Contents

Introduction

Core Components

TCP/IP Socket Programming

Server Socket Initialization

Socket Options for Production

Non-Blocking I/O Configuration

Binding and Listening

Accepting Connections

POSIX Thread Pool Implementation

The C10K Problem

ThreadPool Design

Initialization: Creating Worker Threads

Producer-Consumer Pattern with Condition Variables

Worker Thread Event Loop

Synchronization Primitives Explained

Graceful Shutdown

HTTP Protocol Layer

HTTP Request Structure

Parsing Implementation

HTTP Response Generation

MIME Type Detection

Routing Engine Architecture

Dual Routing Strategy

Router Implementation

Route Registration

Route Resolution

AppDispatcher: Strategy Orchestration

ActionDispatcher: RPC Handler

Design Patterns & Architectural Decisions

1. Strategy Pattern

2. Factory Pattern

3. Dependency Injection

4. RAII (Resource Acquisition Is Initialization)

5. Producer-Consumer Pattern

6. Observer Pattern (Condition Variables)

Asynchronous Request Processing Flow

Complete Request Lifecycle

Concurrency Analysis

CLI Configuration & Extensibility

Command-Line Interface

Input Validation

Extensibility Points

1. Custom Route Handlers

2. Middleware Pattern

3. Custom Dispatchers

4. Plugin System

Conclusion

OS Concepts Applied

Design Patterns Used

Key Takeaways

Further Exploration

About the Author