Vande Matram
Adab has opened a shop selling fruit juices such as carrot juice, orange juice, and sugarcane juice. Adab worked very hard he kept his shop clean. Sugar-free pure juice was served by Adab. He cleaned his juicer daily and used good fruits. He focused on quality and serving his customer to the best of his ability. Due to his hard work, his juice shop becomes successful.
Earlier there were fewer customers so adab used to receive orders, make juice, serve juice and receive payment by himself. These customers could be served in less time.
Now is receiving more customers and his customers have to wait longer. Adab's responsibility is to serve his customer to the best of his ability as he is receiving payment from them.
He wants to reduce the time is the customer has to wait what should he do?
Now, Adab should employ 2 people to make juices and one person to receive orders and payments this will reduce the time the customers have to wait and he will be able to serve more customers.
Adab has created an e-commerce application. The frontend is written in javascript using React and the backend is written in Python using Django. Django app is using Postgres as its database.
Adab has noticed a significant increase in latency as the number of customers increased. What should adab do to reduce the latency of his backend?
[To understand the solution to Adab's Problem you need to understand the concepts given below ]
A load balancer is a component of a distributed system that is responsible to receive requests and distribute them across multiple servers.
Usually, a load balancer is used to distribute HTTP requests among servers. The server that receives a request from the load balancer is called an upstream server.
Diagram
As per the above diagram client send an HTTP request to the load balancer then forwards the request to upstream servers. In the event of an upstream server going down the load balancer will stop forwarding requests and
instead, forward request active to an active upstream server.
Using a load balancer we can distribute HTTP requests over several servers. This increases the serving capacity of our backend, and we can serve more requests
The load balancer also increases the availability of our system as in case of a server going down we can continue serving requests.
How does the load balancer decide which server to forward a request to?
A load balancer should only forward request to upstream services which are up. A load balancer should stop sending requests to those servers which are down and should start forwarding requests to the server when it comes up.
To decide which server to send requests to various algorithms could be used.
Round Robin
In this algorithm, the server cycles through the list of server and send a request to each server one by one.
Consider this example in which we have 3 upstream servers and one load balancer.
Request 1 -> Server 0
Request 2 -> Server 1
Request 3 -> Server 2
Request 4 -> Server 0
Request 5 -> Server 1
Request 6 -> Server 2
Request 7 -> Server 0
Request 8 -> Server 1
Request 9 -> Server 2
Request 10 -> Server 0
There is a variant of round-robin called weighted round-robin in which we give weights to each server and the more the weight the more the requests will be served.
Consider this example in which we have 3 upstream servers and one load balancer.
Weights are as follows:
{
'Server 0': 2,
'Server 1': 1,
'Server 2': 1,
}
Taking the same example as above if we give weight '2' to Server 0 and '1' to Server 1 and Server 2. Then the requests will forwarded as follows:
Request 1 -> Server 0
Request 2 -> Server 0
Request 3 -> Server 1
Request 4 -> Server 2
Request 5 -> Server 0
Request 6 -> Server 0
Request 7 -> Server 1
Request 8 -> Server 2
Request 9 -> Server 0
Request 10 -> Server 0
IP hashing
In this algorithm, the server forwards the request based on IP. The request from the same IP is sent to the same server. The algorithm used to determine which server number to forward requests to is decided by the following formula
server_number_to_send_to = hash(IP) % number_of_servers
Consider the above diagram and the following 10 requests.
Request from IP(127.0.0.1) -> Server 0
Request from IP(127.0.0.2) -> Server 2
Request from IP(127.0.0.3) -> Server 1
Request from IP(127.0.0.1) -> Server 0
Request from IP(127.0.0.2) -> Server 2
Request from IP(127.0.0.3) -> Server 1
Request from IP(127.0.0.1) -> Server 0
Request from IP(127.0.0.2) -> Server 2
Request from IP(127.0.0.3) -> Server 1
Request from IP(127.0.0.1) -> Server 0
...
Nginx (pronounced as Engine X) and Apache Web server are two web servers capable of load balancing.
Kubernetes uses Nginx as its default Load Balancer. Round Robin is the default Load Balancing Algorithm used in Nginx.
Now I want to show you the implementation of load balancing in Nginx.
Assume we have 3 webservers running on 127.0.0.1:8000, 127.0.0.1:8001, 127.0.0.1:8002.
To do Load Balancing on them we are using Nginx. nginx configuration is defined in nginx.conf file. In Ubuntu the path to Nginx.conf is '/etc/nginx/nginx.conf'.
The following code is used for Round Robin Load Balancing.
(Read Comments to understand the code)
events {
worker_connections 768;
}
# Define the http Block
http {
# Tell nginx about our upstream servers and give them the namespace of 'api'
upstream api {
server 127.0.0.1:8000;
server 127.0.0.1:8001;
server 127.0.0.1:8002;
}
# Define the server Block
server {
# Tell nginx to listen for traffic on PORT 80.
listen 80;
# Define the name of server
server_name localhost;
# Forward all requests starting from index('/') endpoint location.
location / {
# Pass the requests to 'api' upstream servers which were defined above.
proxy_pass http://api;
# When passing the request to upstream server also add the Host header to request. The Host header comes from the request that nginx recieved.
proxy_set_header Host $host;
}
}
}
To do IP hashing following Code is used.
events {
worker_connections 768;
}
http {
upstream api {
# Tell nginx we are using ip hashing for 'api' upstream servers.;
ip_hash;
server 127.0.0.1:8000;
server 127.0.0.1:8001;
server 127.0.0.1:8002;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://api;
proxy_set_header Host $host;
}
}
}
We want to do load balancing on them using round-robin on them through Nginx
Now let us discuss the solution to adab's problem.
Adab should use Nginx as his load balancer. And distribute the HTTP requests between 3 webservers running Django applications.
Questions:
Q) Give one use case of load balancing.
A) To distribute HTTP requests among a number of web servers.
Q) Give two benefits of using load balancing.
A) * It increases the availability of our System, as in the event of an upstream server going down requests will be forwarded to other upstream servers.
- It increases the throughput of our System as a higher number of requests can be served.
Q) Give the name of two web servers capable of load balancing.
A) Nginx and Apache Web Server.
Q) Which is the load balancing algorithm used by Nginx by default?
A) Round Robin
Q) Explain IP hashing and round-robin.
A) In Round Robin we cycle through a list of servers turns to send requests.
Requests from the same IP may be forwarded to a different server.
IP hashing we determine the server to which the request should be sent via the following algorithm
hash(IP) % number_of_servers
In IP hashing Requests from Devices having the same IP are forwarded to the same Server.
Q) You are authenticating users using JWT. Should you use IP Hashing or round-robin and why?
A) JWT is stateless. So a request can be handled by any upstream server. We should use Round Robin for the sake of simplicity.
Q) You have two 2 upstream servers running on 127.0.0.1:8000 and 127.0.0.1:8001.
Write Nginx conf to distribute traffic between them via
Round Robin
IP hashing
A) For Round Robin
events {
worker_connections 768;
}
http {
upstream api {
server 127.0.0.1:8000;
server 127.0.0.1:8001;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://api;
proxy_set_header Host $host;
}
}
}
For IP Hashing
events {
worker_connections 768;
}
http {
upstream api {
ip_hash;
server 127.0.0.1:8000;
server 127.0.0.1:8001;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://api;
proxy_set_header Host $host;
}
}
}
Vande Matram
Top comments (0)