DEV Community

Cover image for πŸš€How TCP Servers Are Designed to Handle Multiple Requests
mrityunjay saraf
mrityunjay saraf

Posted on β€’ Edited on

14 1 1 1 1

πŸš€How TCP Servers Are Designed to Handle Multiple Requests

πŸ“š Table of Contents

  1. πŸ–₯️ What Is a Server?
  2. πŸ”„ What Is Multi-Threading?
  3. ❓ Why Do We Need a Multi-threaded Server?
  4. πŸ› οΈ How to Design Multi-Threaded Servers?

πŸ–₯️ What Is a Server?

It is the process that listens to the TCP protocol on some port, addresses the client request, and sends the response.

πŸ”„ What Is Multi-Threading?

Multithreading is a programming concept that allows multiple threads of execution to run concurrently within a single process.
Image description

❓ Why Do We Need a Multi-threaded Server?

Let's imagine we have a single-threaded server that will process one request at a time. Now, imagine what if millions of requests come in.
Image description

I've created a server that processes requests one after the other. Each new request waits until the previous one is completed. Take a look at the response times in different terminals:

Response Timing

Now, consider a multithreaded server architecture. It can handle multiple client connections simultaneously using threads. Each incoming connection gets its own thread, enabling the server to serve multiple clients at once.

In our example, all four clients receive their responses concurrently after four seconds. This demonstrates the efficiency of a multithreaded server in handling multiple requests simultaneously.

Image description

πŸ› οΈ How to Design Multi-Threaded Servers?

  • Step 1: Open the Socket to Listen to the Port
    Create the ServerSocket object that will accept the TCP connection from port 1234.
    Image description

  • Step 2: Accept the Client's Connection to That Port
    Accept the client request using the ServerSocket.accept().
    It is a blocking system call meaning it will not move further without accepting the connection.
    ServerSocket.accept() will return the Socket object specific to that client connection, which will be used for reading requests and sending responses to that client.
    Image description

  • Step 3: Create a New Thread for Each Request
    Image description

  • Step 4: Read the Request and Send the Response
    Image description

  • Step 5: Repeat Steps 2 to 4 an Infinite Amount of Time
    Image description

Disclaimer: 🚨 This article aims to explain multithreaded servers. However, there are practical limitations to this design. Creating millions of threads to handle millions of requests can strain system resources. These issues can be addressed by using strategies such as thread pooling.

🌟 Thank you for reading! I appreciate your time and hope you found the article helpful. πŸ“š I'm open to suggestions and feedback, so feel free to reach out. Let's keep exploring together! πŸš€

Image of Timescale

πŸš€ pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applicationsβ€”without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more

Top comments (2)

Collapse
 
mahimarajput profile image
MahimaRajput β€’

Good Job @mjsf1234

Collapse
 
ssd profile image
server side digest β€’

Well written and with code examples

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more