Load balancing for beginners

Keeping in tune with my new years resolution, this week I studied in depth into setting up load balancers and node clustering. Without wasting much time lets dive right in to the notes

Drawing from this popular definition of an API analogy:

Let's then start off by imagining a restaurant with guests, a waiter and the kitchen staff — you know how it goes.

Say this restaurant is huge and we have 4 kitchens in the restaurant each responsible for making amazing meals but specialised in making one delicacy (Chinese, Indian etc). Now for some reason a large chunk of the guests tonight fancy chinese and place around 60 orders, the waiter does his job and takes all orders to kitchen 1 who is specialised in making chinese food - attending to 60+ orders at a go would no doubt put a lot of stress on kitchen 1 but we have to remember that the other kitchens can make chinese as well just not specialised on it.
Wouldn't it make for a better experience for both the chef and the guests if the orders were spread out equally among each kitchen - I mean that would take a chunk of work off kitchen 1 and make sure guests get their food in less time! A win-win case for everyone.

Now the waiter could go about distributing this workload in a couple of ways:

• Spread all orders equally, so if there are 4 requests each kitchen works on 1
• Give the order to the kitchen with the least amount of orders
• Give the order to the kitchen that has always handled the particular guests' order anytime they visit
• Give the order to the kitchen with the hottest stove

There are a chunk of ways to go about it really but for now lets reveal the technologies involved in this analogy:

• The guests are obviously our client devices (web browser, mobile, desktop) 🖥️
• The kitchen is our server/API 🗄️
• The waiter in this scenario is our load balancer ⚖️

You must probably be thinking that the load balancer serves the same purpose as the API since it takes our requests from the client to the processing point/server but you would be a bit off. For a start your request passes through the load balancer (waiter) first before ever making contact with the API/Server (kitchen) and secondly the load balancer as the name implies balances the request load (order) each of your servers have to handle which maximises performance to a great degree.

You see if kitchen 1 kept getting ALL orders, it wouldn't take long before the cooking staff become exhausted and take a break, leaving the guests waiting for much longer to have their orders fulfilled.
Now in this case the waiter is specially trained to reduce overall wait time of the guests by identifying which chef to give an order.

There are a few algorithms a load balancer can employ to determine how to select the server to handle a request we'll look at just the 4 quoted above (which is configurable based on your requirements)

1. Round robin - This just spreads the requests incrementally across each server. If there are 4 requests, 4 servers handle 1 each. If there are 7 requests, only server 8 gets to handle 1
2. Least connections - This just assigns the request to the server with the least request being handled
3. Hashing - This distributes the requests based on a key we define, such as the client IP address or the request URL
4. Least Bandwidth - This algorithm measures traffic in megabits per second (Mbps), sending client requests to the server with the least Mbps of traffic.

You can go over this brilliant article by the good folks at cloudflare to learn more about load balancing algorithms.

Now hopefully we have a firm understanding about the concept of load balancers, but as software engineers we always need to avoid single points of failure and if you've noticed where our single point of failure is in this story kudos to you! and if not let's walk over it together.
You see the waiter could call in sick on a random day and that could quickly disrupt the flow of the restaurant resulting in bad customer experience. Same way our load balancer for whatever reason could fail and then we would more or less be back to square one struggling with a volume of requests. That's why in practice we need to ensure we have fallback load balancers so in a case where our master load balancer experiences downtime, the substitute will be called up and keep the whole process running!

Features of load balancers

Load balancers are quite dynamic and in addition to optimally routing requests to servers they can be used for quite a number of services like:

• Logging
• Encryption
• Caching
• Redirects
• Autoscaling and many more

Load balancers are great and if you decide to use one in your project, here are a few services commonly used in the industry today

• Amazon elastic load balancer
• Digital ocean load balancer
• NGINX
• GCP load balancer

And that will be all for now. Next week I'll dive into the concept of clustering for NodeJS apps and how they can enhance performance of your web apps, until then Ciao!