DEV Community: parthlaw

Goroutines: Solving the problem of efficient multithreading

parthlaw — Wed, 01 May 2024 00:48:05 +0000

Introduction

Goroutines are the feature provided by Golang to make concurrent programming simpler and more efficient, eliminating the need to manually manage the threads.
They have a simple syntax:

go functionToRun()

Although it may seem like a traditional multithreading, it is much more efficient than that.

Concept of G, M and P

The document explaining the Goroutines explains the concept of G, M and P orchestration to manage Goroutines efficiently.
G: It refers to Goroutines.
M: It refers to worker threads which executed G i.e. Goroutines.
P: It refers to the processor. It represents a resource which is required to execute Golang code.
To execute a Go code, M must have a P.
The whole process has three main components:

Global Run Queue: Queue of Goroutines to be executed. This queue is global and accessible to all M's and P's.
Local Run Queue: This is also a queue of Goroutines ready for execution. However, this queue is local i.e. per P.
Local Scheduler: The scheduler schedules which G to execute and when. The scheduler is local i.e. per P. The idea behind keeping the scheduler local is to avoid bottlenecks in speed due to the scheduler (if all P's depend on one scheduler).
P-array: Array of all P's available. The size of the array is equal to GOMAXPROCS env variable.

What happens when we submit a Goroutine?

So what happens when we submit a Goroutine? There are two sides to the whole flow. Let's break it down step by step:

Goroutine Side

We submit a Goroutine using go keyword.
The worker thread in which Goroutine is submitted (it can be a main or any other thread) i.e. M has an associated P and a scheduler (comes with P).
Two paths submitted Goroutine i.e. G can take:
- Add it to the local run queue so that the current M picks it up and executes it using P.
- Add it to the global run queue so that any other M picks it up and executes it using the corresponding P.
The decision to submit it to the appropriate queue is taken by the scheduler based on various factors:
- Availability of space in the local run queue.
- Availability of current M i.e. current M is not blocked by any operation such as waiting for I/O. If it is blocked then G is added to the global run queue.
- Priority considerations: The scheduler is preemptive priority-based. If the submitted Goroutine has lower priority than others then it is added to the global run queue.

Diagrammatic illustration from a single M POV

Worker thread Side

When an M is out of work i.e. it has no G waiting to be executed in the local run queue, it immediately starts looking for a G to execute from the global run queue.
In case it does not find any G from the global run queue, it immediately goes into the spinning state.
- Spinning State: It is the state in which the M (worker thread) is constantly looking for work in other P's run queues.
When a thread in a spinning state finds work, it unparks an additional worker-thread to make it in a spinning state. This makes sure that there is at least one spinning M available.
- Parking: An M (or worker thread) is said to be in a parking state when it is put into a suspended or waiting state to conserve resources. In this state, the execution of M is paused until it is unparked.
- Unparking: When the execution of a parked M is resumed, it is said to be in the unparked state.
- Unparking of an M occurs in two scenarios when a G is submitted: i). When there are no spinning M. ii). There is an idle P.
The new spinning M starts looking for work.
If an M does not have an associated P and it is willing to execute some Go code then it pops a P out from the P-array.
When a P is done with executing G, it pops another G from its own list of runnable G's. If the list is empty, it chooses a random P and steals work from it. This is work-stealing algorithm.

The concept of spinning worker thread ensures speed and maximum CPU utilization.

Minimum number of spinning is 1 (as ensured by the algorithm) and the maximum can be GOMAXPROCS env variable.

I tried to represent the flow visually.

Conclusion

The explained G, M and P orchestration is explained in detail in the Design Doc provided in the code explination. You can go through to get a detailed description and more understanding of the algorithm.
I tried to explain the concept in brief. Please point out if there is any inaccuracy in the content.🙌

Object-Oriented Go: Unraveling the Power of OOP in Golang

parthlaw — Fri, 19 Apr 2024 09:24:47 +0000

Object-Oriented Programming

Object Oriented Programming is a paradigm in programming based on the concept of "objects". Objects contain data in the form of properties and methods. It is the most popular style used in programming due to its modularity, code reusability and organisability.
In general naming convention, basic concepts of OOPS include:

Class: A class is a blueprint of an object to be created.
Object: Instance of class containing both data and methods.
Encapsulation: Limiting access to properties, methods or variables in a class for code external to that class.
Inheritance: It allows a class (commonly referred to as subclass) to inherit properties and methods from another class (commonly referred to as parent class).
Polymorphism: Allowing objects of different classes to be treated as objects of a common superclass.
Abstraction: Abstracting a basic concept of multiple classes in a single abstract class. This simplifies logic and makes code more readable.

We won't dive into the details and understanding of OOPS in this article. This article focuses on the implementation of these concepts in GO programming language, since unlike normal programming languages like Java, C++, etc, GO doesn't have the concepts of OOPS directly with common naming conventions. For example, it does not have a class keyword.

OOP in GO

Although GO does not have classes, it allows us to use OOP concepts using structs and interfaces. Let's see how we can use OOP concepts in GO with the examples:

Classes, properties and methods

We can use structs in GO to achieve the same functionality as classes in other programming languages. Structs can have methods and properties.
Let's say we are building a billing application in which we want to define a company:

type Company struct {
    Id string;
    Name string;
    Country string;
}
func newCompany(name string, country string) Company {
    return Company {
        Id: uuid.New().String(),
        Name: name,
        Country: country
    }
}

Go does not have the concept of constructors and classes, so we defined a custom function to return the Company. This would work as a constructor for the Company. We initialize Id in this function.
Now to create an object of type Company, we will call newCompany method like this:

var company Company= newCompany("MyCompany","India")

Note: Function overloading isn't supported in Go. So to create multiple implementations we would have to create multiple functions of different names.

Further, we want to save a company to the database. We can create a method of Company struct, for this purpose:

func(company Company) saveToDatabase() {
    fmt.Println("Saving Company")
}

Encapsulation

Since Go does not concept of classes, objects and access modifiers in it, we cannot implement encapsulation in it like other languages. It has the concept of packages and exported and unexported identifiers.
Let's say we folder structure in our app:

Company is a module. Our models.go file look like this:

package company
import "github.com/google/uuid";
type Company struct {
    Id string;
    Name string;
    Country string;
}
func newCompany(name string, country string) Company {
    return Company {
        Id: uuid.New().String(),
        Name: name,
        Country: country
    }
}

Let's add a property manager which is private. We cannot prevent the code of the same module from accessing this property, however, we can prevent other modules from accessing the property by making the first letter of the name smallcase.

Names with the first letter as capital case are exported and the names with the first letter as small case are limited to the same module. This applies to properties, variables, types and functions.
type Company struct{
  Id string;
  Name string;
  Country string;
  manager string;
}
Inheritance

Although there is no direct concept of Inheritance in go, we can embed base structs into its child structs.
Let's say, we want to define Employee struct also. Since it has the same properties as the company we want it to inherit those properties from Company. We can define it like this:
type Employee struct{
  Company;
  Salary int;
}
To define function newEmployee:
func newEmployee(companyName string,
    companyCountry string, salary int
) Employee {
    company: = newCompany(
        companyName,
        companyCountry)
    return Employee {
        Company: company,
        Salary: salary,
    }
}
We can access the Company parameters or methods directly:
func newEmployee(companyName string,
    companyCountry string, salary int
) Employee {
    company: = newCompany(
        companyName,
        companyCountry)
    employee: = Employee {
        Company: company,
        Salary: salary,
    }
    employee.saveToDatabase();
    return employee
}
Notice we used saveToDatabase method of company with Employee struct object.

Polymorphism

Go provides interfaces for this purpose. A type implementing a function or multiple functions defined in the interface becomes the type of that interface.
We will extend our current example and define another type of VendorCompany. This is the company from which our current company buys raw materials.
Implementation would be similar to Employee. Since all three of them are similar objects and being saved into db in the same table/collection (hypothetically), we would define an interface CompanyInterface so that we treat all three objects as the same type - the interface. Since we have also defined three different types of entities into one object, let's create a method to return the type of entity when called.

model.go

package company
import ("github.com/google/uuid")
type CompanyInterface interface {
    saveToDatabase();
    getType() CompanyEntity;
}
type CompanyEnum string;
const (CompanyType CompanyEnum =
    "Company"; VendorType CompanyEnum =
    "Vendor"; EmployeeType CompanyEnum =
    "Employee";)
type Company struct {
    Id string
    Name string
    Country string
    manager string
}
type Employee struct {
    Company
    Salary int
}
type VendorCompany struct {
    Company
    AccountNumber string
}
func newCompany(name string, country string) Company {
    return Company {
        Id: uuid.New()
            .String(),
        Name: name,
        Country: country
    }
}
func newEmployee(companyName string,
    companyCountry string, salary int
) Employee {
    company: = newCompany(
        companyName,
        companyCountry)
    employee: = Employee {
        Company: company,
        Salary: salary,
    }
    employee.saveToDatabase()
    return employee
}
func newVendor(companyName string,
    companyCountry string,
    accountNumber string) VendorCompany {
    company: = newCompany(
        companyName,
        companyCountry)
    vendor: = VendorCompany {
        Company: company,
        AccountNumber: accountNumber,
    }
    vendor.saveToDatabase()
    return vendor
}

methods.go

package company
import "fmt"
func(c * Company) saveToDatabase() {
    fmt.Println(
        "Saving Company to database..."
    )
}
func(c * Company) getType() CompanyEnum {
    return CompanyType;
}
func(e * Employee) saveToDatabase() {
    fmt.Println(
        "Saving Employee to database..."
    )
}
func(e * Employee) getType() CompanyEnum {
    return EmployeeType;
}
func(v * VendorCompany) saveToDatabase() {
    fmt.Println(
        "Saving VendorCompany to database..."
    )
}
func(v * VendorCompany) getType() CompanyEnum {
    return VendorType;
}

Now lets create a generic function in main.go to print the type of object provided.

func getCompanyNameInLowerCase(company CompanyInterface) {
    fmt.Println(company.getType());
}

We can use this function for all three type of objects:

  employee:= newEmployee("Raju", "India",25)
  company:= newCompany("Lakshmi Chit Funds", "India")
  vendor:= newVendor("Babu Rao","India","abcd1234")
  printCompanyType(&employee)
  printCompanyType(&vendor)
  printCompanyType(&company)

Notice that defining the interface saved us from the three different implementations of printCompanyType for all three types. This practice can improve the code maintainability in the long run.

Abstraction

Similar to all other concepts, Go does not have abstract classes. However, we can make our workaround for these as well.
Let's define a struct CompanyEntity with all Company properties except, we throw a "Not Implemented" error in all struct methods.

struct definition

type CompanyEntity struct {
    Id string
    Name string
    Country string
    manager string
}

methods definition

func(c * CompanyEntity) saveToDatabase() error {
    return errors.New(
        "Not implemented")
}
func(c * CompanyEntity) getType() error {
    return errors.New(
        "Not implemented")
}

This struct can serve as our abstract class, only difference is we can instantiate an object from this struct. However, all the methods are not implemented and can be overwritten by child classes.
We can now make our child structs inherit the properties of this struct.

package company
import (
    "github.com/google/uuid"
)
// Define an interface for types that can be saved to the database
type CompanyInterface interface {
    saveToDatabase();
    getType() CompanyEnum;
}
type CompanyEntity struct {
    Id string
    Name string
    Country string
    manager string
}
type CompanyEnum string;
const (
    CompanyType CompanyEnum = "Company"; VendorType CompanyEnum =
    "Vendor"; EmployeeType CompanyEnum =
    "Employee";
)
type Company struct {
    CompanyEntity
}
type Employee struct {
    CompanyEntity
    Salary int
}
type VendorCompany struct {
    CompanyEntity
    AccountNumber string
}
func newCompanyEntity(name string,
    country string) CompanyEntity {
    return CompanyEntity {
        Id: uuid.New().String(),
        Name: name,
        Country: country,
    }
}
func newCompany(name string, country string) Company {
    companyEntity: =
        newCompanyEntity(name,
            country);
    return Company {
        companyEntity
    }
}
func newEmployee(companyName string,
    companyCountry string, salary int
) Employee {
    companyEntity: =
        newCompanyEntity(
            companyName,
            companyCountry)
    employee: = Employee {
        CompanyEntity: companyEntity,
        Salary: salary,
    }
    employee.saveToDatabase()
    return employee
}
func newVendor(companyName string,
    companyCountry string,
    accountNumber string) VendorCompany {
    companyEntity: =
        newCompanyEntity(
            companyName,
            companyCountry)
    vendor: = VendorCompany {
        CompanyEntity: companyEntity,
        AccountNumber: accountNumber,
    }
    vendor.saveToDatabase()
    return vendor
}

This way we used CompanyEntity struct as an abstract class.

Conclusion

While Go may not have the traditional syntax and constructs associated with Object-Oriented Programming (OOP), it offers powerful features that enable us to achieve similar functionality in a more idiomatic way. By leveraging structs, interfaces, and package organization, Go allows for the implementation of key OOP concepts such as classes, encapsulation, inheritance, polymorphism, and abstraction.
While transitioning from languages with more explicit support for OOP, such as Java or C++, to Go may require some adjustment in thinking, adjusting our thinking to these concepts in Go opens up new possibilities for coding patterns and code structure management.

Thank you for making this far 😇. Please let me know how I can improve on the content and refer me some interesting topics to learn.
Connect me on LinkedIn: https://www.linkedin.com/in/parthlaw

I/O Streams in Java: Understanding Blocking and Non-blocking Paradigms

parthlaw — Sun, 14 Apr 2024 07:03:50 +0000

Introduction

I/O streams are the heart of Input-Output operations. These are the paths through which data flows between a source and a destination.

Input Stream: These are used by a program or an application to read data from a source such as a file, connection, keyboard, etc.
Output Stream: These are used by a program or an application to write data to the destination.

Blocking and Non-Blocking I/O

Basic I/O operations are generally blocking in nature i.e. they block the thread execution until there is some data to read.
For example in a normal HTTP request, a client makes a request to an application which is bind to a certain port (port 80 for HTTP), they first establish a socket connection between them. After the connection is made, the server waits for the client to make a request and then sends the response through the same socket.

In a normal Socket connection, where we want continuous communication between client and server without going through the expensive HTTP request process again and again, we keep the socket connection open for communication. The server waits for the client to send something to respond to it. This means the thread is blocked until the client says something. This SocketServer can be created in Java like this:



ServerSocket serverSocket = new ServerSocket(port)
while (true) {
  Socket socket = serverSocket.accept();
  System.out.println("New client connected: " + socket);

  // ClientHandler.run is the function to infinitely listen 
  //and handle the client messages until the connection closes.

  ClientHandler clientHandler = new ClientHandler(socket);
  clientHandler.run();
}

An example to explain the situation can be: Suppose two friends are playing a game in which they have a pipe to communicate with each other. One friend says something (the client) and the other friend (the server) writes the message down and responds 'ack'. This would look like this:

This is fine for the situation when only two friends want to play the game i.e. only one friend is talking and the other is listening or waiting for him to talk continuously.
The problem arises when multiple friends want to talk and play the game i.e. multiple friends arrive with their respective pipes and want their messages written down on the other end. But since only one person is listening (i.e. friend-2), he cannot wait on the end of each pipe to listen to the messages. One obvious solution for him is to hire multiple people (one person for each pipe) and assign them to the pipes:

This concept is multithreading. In this, you spawn a new thread for each new connection.



ExecutorService executorService = Executors.newSingleThreadExecutor();
ServerSocket serverSocket = new ServerSocket(port)
while (true) {
  Socket socket = serverSocket.accept();
  System.out.println("New client connected: " + socket);

  // ClientHandler.run is the function to infinitely listen 
  and handle the client messages until the connection closes.
  ClientHandler clientHandler = new ClientHandler(socket);
  executorService.submit(clientHandler::run);
}

However, there is a major limitation to this approach as well. Friend 2 cannot keep hiring people for his each friend playing the game. If friends keep coming, there would arrive a point when Friend-2 would be left with no money i.e. a server cannot spawn an infinite number of threads. There needs to be a better solution to this problem.

The problem here is each connection is blocking a thread. That is why we need to spawn a new thread for each new connection. To solve this problem, the non-blocking I/O comes into the picture. This means, that a connection is not blocking a thread and there is some mechanism to go and listen to the connection whenever it wants to send a message.
One of the popular implementations in Java is the use of Channels and Selectors.
In the context of our current example, the idea is to install a recorder which records messages from each pipe separately. Now Friend-2 can just iterate through all the messages and respond to them one by one and process them (write in his notebook). This would eliminate the need for hiring people and friend 2 alone can listen and process messages from his each friend.
Java NIO library provides channel classes for input streams. In our current use-case i.e. for Socket connection, it has ServerSocketChannel class. A channel is just a pathway for data to be transferred (i.e. the pipe). The ServerSocketChannel class allows us to define the connection as non-blocking and register it to a Selector (the Recorder).

Client Side
- The client writes the message data to its outbound buffer.
- The client then attempts to send the data over the network to the server via the channel.
Network Transmission
- The data is taken from the client's outbound buffer and sent to the server over the network.
Server Side
- The transmitted data arrives at the server's network interface and is placed into the network input buffer associated with the server's connection.
- The server's operating system manages this input buffer and makes the data available for the server's application to read.
Server Application:
- When the server's application is ready to read from the channel (by calling channel.read()), it reads data from the network input buffer.
- If data is waiting in the input buffer, the server reads it into its application-level buffer for processing.

You define and register your ServerSocketChannel to a Selector like this:



Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
// configure non blocking nature
serverSocketChannel.configureBlocking(false);
InetSocketAddress inetSocketAddress = new InetSocketAddress(8000);
serverSocketChannel.bind(inetSocketAddress);
System.out.println("Socket Server started on port 8000");

// Now you register the channel with selector with the required interest key (will be explained further).
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

Interest Key is the the key for action that you are interested in, with the channel. In our current context, when we create a new socket server, we are interested in new connections. Interest Key for that action is SelectionKey.OP_ACCEPT.
After creating the ServerSocketChanel, binding it to port 8000 and registering it with a Selector, we can now start handling the connections.



Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
// configure non blocking nature
serverSocketChannel.configureBlocking(false);
InetSocketAddress inetSocketAddress = new InetSocketAddress(8000);
serverSocketChannel.bind(inetSocketAddress);
System.out.println("Socket Server started on port 8000");

// Now you register the channel with selector with the required interest key (will be explained further).
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

while(true) {
 // this is the keys in our selector which are ready to be 
 // processed i.e. new connections which are ready to be 
 // accepted (OP_ACCEPT), messages from connections which are 
 // ready to be read (OP_READ), etc.

 int readyCount = selector.select();
 if(readyCount==0){
  // skip when there is no key to be processed
  continue;
 }
 // get the ready keys set
 Set<SelectionKey> readyKeysSet = selector.selectedKeys();
 // iterate over the readyKeySet
 Iterator iterator = readyKeysSet.iterator();
 // checking if there are any connection requests and the server is ready to accept the connection i.e. OP_ACCEPT is registered.
 while(iterator.hasNext()) {
  SelectionKey key = (SelectionKey) iterator.next();
  iterator.remove();
  if (key.isAcceptable()) {
   System.out.println("Accepting connection");
   // Get the channel for the client connection
   ServerSocketChannel server = (ServerSocketChannel) key.channel();
   // accept the connection
   SocketChannel client = server.accept();
   // configure non blocking behavior for this channel
   client.configureBlocking(false);
   // register the client channel with the same selector. The action we are interested is only read i.e. we only want to listen to the client messages. The SelectonKey is OP_READ.
   SelectionKey clientKey = client.register(selector, SelectionKey.OP_READ); 
  }

  // checking if there is any client wants to send the message 
  // (this would only be true when there is any message and 
  // OP_READ is registered).
  if(key.isReadable()) {
   SocketChannel client = (SocketChannel) key.channel();
   int BUFFER_SIZE = 1024;
   // create a buffer
   ByteBuffer buffer = ByteBuffer.allocate(BUFFER_SIZE);
   try {
    // read data and store it into created buffer
    int bytesRead = client.read(buffer);
    if (bytesRead == -1) {
     System.out.println("Connection close");
     // Connection closed by client
     key.cancel();
     client.close();
     continue;
    }
    // flip to change buffer to read mode from write mode
    buffer.flip();
    // define a byte array of size as the number of bytes in the buffer.
    byte[] receivedBytes = new byte[buffer.remaining()];
    buffer.get(receivedBytes);
    // print the length of byte array
    System.out.println(receivedBytes.length);
    // now immediately after receiving the message we want to write the ack to client. So we register OP_WRITE now so that our server is ready to write to outbound buffer.
    key.interestOpsOr(SelectionKey.OP_WRITE);
   } catch (SocketException e) {
     e.printStackTrace();
     key.cancel();
     client.close();
     continue;
    } catch (Exception e) {
        e.printStackTrace();
    }
  }
  if(key.isWritable()) {
   SocketChannel client = (SocketChannel) key.channel();
   // write to outbound buffer associated with this client's connection
   client.write("ack");
   // immediately remove OP_WRITE interest key, so that server becomes not ready to write.
   key.interestOps(key.interestOps() & ~SelectionKey.OP_WRITE);
  }
 }
}

This way we can handle multiple client connections and their reads and writes in a single thread. As we want to scale and increase the processing time, we should offload our blocking tasks such as processing the read message (processing can include for example saving something to db) to separate threads. This way threads would be killed once these tasks are over. To further optimize it, we can create thread pools for a context of tasks.



ExecutorService executorService = Executors.newFixedThreadPool(n);

This created a pool of maximum n available threads. If there is no thread available, the task will wait in the queue until a thread becomes available.

To further optimize it, we can also create a pool of clients to assign a separate selector to each pool and run the processing of each pool in a separate thread. This way we can increase processing and hence response time for the clients.

Full Java code for the approach with all imports is as follows:



import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.*;
import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MultiThreadedServer {

    private static final int PORT = 8080;
    private static final int MAX_CLIENTS_PER_POOL = 20;
    private static final int MIN_NUM_POOLS = 5; // Number of separate pools

    public static void main(String[] args) throws IOException {
        ExecutorService poolExecutor = Executors.newFixedThreadPool(MIN_NUM_POOLS);

        // Create and open a server socket channel
        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
        serverSocketChannel.socket().bind(new InetSocketAddress(PORT));
        serverSocketChannel.configureBlocking(false);

        // Create a single selector for the server socket channel
        Selector serverSelector = Selector.open();
        serverSocketChannel.register(serverSelector, SelectionKey.OP_ACCEPT);

        // Array to hold selectors for client channels
        Selector[] clientSelectors = new Selector[MIN_NUM_POOLS];
        for (int i = 0; i < MIN_NUM_POOLS; i++) {
            clientSelectors[i] = Selector.open();
            ClientPoolHandler clientPoolHandler = new ClientPoolHandler(clientSelectors[i]);
            poolExecutor.submit(clientPoolHandler::run);
        }

        // Accept and handle client connections in separate threads for each pool
        while (true) {
            serverSelector.select();
            Set<SelectionKey> selectedKeys = serverSelector.selectedKeys();
            Iterator<SelectionKey> keyIterator = selectedKeys.iterator();

            while (keyIterator.hasNext()) {
                SelectionKey key = keyIterator.next();
                keyIterator.remove();

                if (!key.isValid()) {
                    continue;
                }

                if (key.isAcceptable()) {
                    // Accept the connection
                    ServerSocketChannel serverChannel = (ServerSocketChannel) key.channel();
                    SocketChannel clientChannel = serverChannel.accept();
                    boolean clientAdded = false;

                    // Check if any selector has capacity, otherwise create a new one
                    for (Selector selector : clientSelectors) {
                        int numClients = selector.keys().size() - 1; // Subtract 1 for the server channel
                        if (numClients < MAX_CLIENTS_PER_POOL) {
                            clientChannel.configureBlocking(false);
                            clientChannel.register(selector, SelectionKey.OP_READ);
                            clientAdded = true;
                            break;
                        }
                    }

                    // If no selector has capacity, create a new one and spawn a new thread
                    if (!clientAdded) {
                        Selector newSelector = Selector.open();
                        clientChannel.configureBlocking(false);
                        clientChannel.register(newSelector, SelectionKey.OP_READ);
                        ClientPoolHandler clientPoolHandler = new ClientPoolHandler(newSelector); 
                        poolExecutor.submit(clientPoolHandler::run);
                    }
                }
            }
        }
    }

    private static class ClientPoolHandler{
        private Selector selector;

        public ClientPoolHandler(Selector selector) {
            this.selector = selector;
        }

        public void run() {
            try {
                while (true) {
                    selector.select();
                    // Handle selected keys
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

Here we maintain a minimum number of pools running to keep the application available, whenever needed.

Conclusion

Basic I/O operations are typically blocking, causing threads to wait until data is available. However, as the need for scalability arises, especially in scenarios with multiple concurrent connections, blocking I/O becomes impractical due to resource limitations.

The shift towards non-blocking I/O, facilitated by technologies like Java NIO (Non-blocking I/O), offers a more scalable solution. By decoupling connections from threads and employing mechanisms such as Channels and Selectors, non-blocking I/O allows servers to handle numerous connections without exhausting system resources.

In essence, non-blocking I/O enables servers to asynchronously manage multiple connections, improving performance and scalability. By adopting this approach, we can build robust and efficient systems capable of handling high loads while maintaining responsiveness.

Please let me know in the comments, what are the other solutions in Java or in other languages for the problem 🙏.
Let's keep discussing and increasing each other's knowledge.

Thank you for reaching this far. I'm a new grad student from India, and this is my first blog post. I hope you liked the content. Please share in the comments on what I could improve.🙌

Thank you for reading 🙏.