DEV Community: Uzair

Load Balancing helps you to scale horizontally by increasing resources (increasing number of servers) while caching enables you to make better use the existing resource you have.

Caching works on the principle that recently requested data is likely to be requested again

Cache is a hardware or a software component which helps in serving the data which is either frequently requested or is resource expensive to compute, So cache stores the response and serves it when asked to do so.

Caches can exist at all levels in architecture, but are often found at the level nearest to the client where they are implemented to return data quickly without taxing downstream levels.

For example:

A client requests image from a server, Initially the server looks through DB for the image and return it to client but however there are the following possibilities

The response can be cached in server side (Eg: in reverse proxy)
The response can be cached client side (Eg: Browser cache, Forward proxy)

This way we are saving the copy of response and client do not have to request the server frequently for the same data, This even decrease the time to load data for client and even decreases load for the server.

If the response for a particular request is saved in cache it is called cache hit or else it is called cache miss

Pros:

Improves read Performance (aka Latency)
Reduce the Load (aka Throughput)

Cons:

Increase complexity of System
Consumes Resources

Cache Invalidation & Eviction

Cache Invalidation:

Updating the cached data is known as cache invalidation. The most common technique used for this purpose is known as TTL (Time To live). A certain time period is defined for cache to live in the memory and after that interval of time cache expires. And new data is requested from server

Cache Eviction:

As cache memory is limited so we have to decide which cached data to keep or not.

Eg: our storage is capable of only storing only 100 keys and currently all of them are filled and now we want to add an additional key so we have to kick out an older cached key so that room for a new one can be made. This process is know as cache Eviction.

Below are some parameters considered while Eviction of a cached key:

FIFO: First In First Out, In this the oldest cached key gets removed so that latest one can be occupied.
LRU: Least Recently Used, In this the cached key which hasn't been used in recent time and was used a long time ago gets discarded.
LFU: Least Frequently Used, In this the cached is used the most less number of times is discarded. Eg: there are 3 keys k1, k2, k3 and they are used 7,3,9 times respectively so k3 will be discarded first as it is used only 3 times which is the least number.

Cache Patterns

Cache Aside pattern:

This is the most commonly used technique in this the cache does not interact with DB or Server it only interacts with Application.

When a client request some data through application, Application looks for it in cache if found then returns it or else request the same to server and then returns to client and also caches it.

Here there was no interaction between cache and server or DB

When the data in DB changes corresponding data in cached memory also have to be modified either through some code by application or through TTL method or combination of both can be used

Pros of cache aside:

Caches only what is needed

Cons of cache Aside:

Cache Miss are Expensive

Read Through pattern:

Here the cache resides between application and server, in this case application never interacts with server or cache.

When a client request some data it is first looked by the application in cache and if not found the cache itself request data to server and returns to the application.

Pros of cache aside:

Caches only what is needed

Cons of cache Aside:

Cache Miss are Expensive

Write Through pattern:

The write-through strategy adds data or updates data in the cache whenever data is written to the database. Because the data in the cache is updated every time it's written to the database, the data in the cache is always current.

Pros of Write Through:

Up to Date Data

Cons of cache Aside:

Writes are Expensive are Expensive
There is a chance that we may add certain data to cache that no one ever reads

Write Behind pattern:

It is similar to write through and just instead of updating cache instantaneously it waits for some time and sends requests in bulk

Pros of Write Through:

No Write penalty

Cons of cache Aside:

Reliability
Lack of consistency

Where Do Cache Resides

Depending on the pattern used for caching and some other parameters cache can be embedded at various levels such as.

Client Side (Like Browser cache or OS Cache)
Forward proxy
Reverse proxy
Application layer itself

Client and Server Side Architecture

Uzair — Tue, 28 Sep 2021 06:04:00 +0000

Who's a Client 🧐

In backend client generally refer to the device through which user is accessing the internet. It could be Laptop, Mobile, Desktop, tablet etc etc. Client contains frontend and the User Interface (UI/UX) 💻

In other words client is a means or door through which a user can interact with backend. A lot of new fancy open source technologies like ReactJs, Angular, HTML, CSS etc etc are used on client side.

What's a Server ❓

Ever service running online needs some kind of computational power to run on and this is provided by server. Server is the place where core logic of our application is executed. You can think it of a computer placed on cloud ☁️

The primary task of a server is to execute the logic based upon the demand by the user.

What's Client Side Architecture ↔️

All the Client, Server architecture are based upon request, response model.

Client sends a request and server responds with a response. 🔭

The above conversation between client and server happens mostly over HTTP protocol which was covered in previous blog post

And speaking from point of view of an modern, advance client server architecture client Sends an request to server mostly through REST-API which was discussed in previous blog post

Types of Client ⚙️

Logical manipulation has to be done on data before returning to client then. The logic may reside on server side and sometimes on client side (depending Upon the service). On this basis Client are classified into two types 🗂️

Thin Client
Thick Client

Thin Client

When Logic that has to be performed on data reside on Server Side then it is classified as Thin Client. It depends on Cloud platform for its features.

Eg: Netflix

Thick Client

When The logic that has to be performed on data resides on server side then it is known as Thick Client. It's function can also run offline. and runs at least some features locally on your device.

Eg: Google Docs (It perform calculations and computational related stuff using local resources but data is stored in database).

Different Tire Architecture (Tier 1, Tier 2, Tier 3)

Now What's a Tier, Tier refers to logical separation between different components of an application or service.

By Components I refers to

Databse
Client or User Interface
Caching
Load Balancer
Servers
etc

There are Mainly three Tire of Cleint - Server architectures

Tier 1 Architecture

This is the most simplest of all the database, server and client all reside at a same machine. 💻

These could be installed and run on a single machine without any external resources.

Eg: when we install an SQL database to practice SQL quires.

Advantages ✔️

No network latency because every component is located on the same machine 🗄️
Highly secure and private as user's data does not leave his/her machine 🔐

Dis-Advantages 🚫

Developer have no control over the software once it is shipped as it is totally disconnected from the outer world, we cannot ship updates and big-fixes anymore, The only way of doing this is by installing a newly release version of the same 🐞
Such software can be easily twerked or reverse engineered ⏪
Performance of Server and Database is directly proportional to the configuration of user's machine. Because the server and Database resides in user's machine itself. 🖥️

Tier 2 Architecture

In this architecture the presentation layer or the frontend as we say runs on client side while Backend Server resides at an external resource. Here database is also included in the server (Means there is no external dedicated DB) 🐳

In two-tier, the application logic is either buried inside the user interface on the client or within the database on the server (or both).

It has better security and scalability scope as compared to Tier 1. Here logic can reside either on client side or side depending on service.

We can either keep the user interface and the business logic on the client or move the business logic to a dedicated backend server, which would make it a three-tier application. We will discuss it next.

Tier 3 Architecture

A three tier architecture is most scalable among the all three. It has three layers

Presentation Layer (PC, Tablet, Laptop or any sort of UI Screen)
Application layer (server)
A dedicated Database

User Interface, Server and DB all three resides on different machines and thus are physically separated.

Here Logic resides at server side and data is stored at database. This architecture can integrate data from different resources

Let's take a simple blog website for example User Interface or frontend as we say is written in HTML and CSS, Server is Built up in NodeJS and Express or Django or any other backend framework and DB could be MongoDB or SQL etc, etc
This is a typical example of 3 Tire architecture.

N- Tier Architecture

In some cases even Tier 3 architecture isn't sufficient to handle traffic so load balancer, caching layer, etc are introduced on top of our tradational server and DB

Let's Look what these components are

Cache
Message queues for asynchronous behavior
Load balancers
Search servers for searching through massive amounts of data
Components involved in processing massive amounts of data
Components running heterogeneous tech commonly known as web services such as machine learning models, or some security services etc etc

All the above represented components are very complex and requires a separate post on each

All big software services like Facebook, Amazon, Apple Services, Netflix, Google Services , Instagram, Uber, Airbnb etc etc have a N-Tier Architecture.

N-Tier Architecture is also known as distributed service or distributed applications.
Below is a very rough representation of N Tier Architecture, We'll make more concise and detailed as we proceed

Do Drop a Like and Share This if you find it useful 💜

Load Balancer - 101

Uzair — Sun, 19 Sep 2021 06:57:49 +0000

Okay, Now What's a Load-Balancer?? 🙄

As the name suggest it has something to do with balancing the load, but here in backend system design architecture load generally refers to traffic or amount of incoming requests 📶

So yeah, going by the name it manages traffic to servers so that they may not get overloaded by traffic ⚖️

Imagine Facebook or Instagram, such a huge giant they are aren't they now let us assume all of their traffic and backend process is handled by a single server, not even a super computer can do such wonders 😂
So definitely they must have multiple server spread across the globe 🌍 now how to decide which request goes to which server 🤨 And how to make sure that a particular server is not stormed by lot of requests, while other servers sits idle and smile at each other 😆

This is the time when load balancer comes for rescue🦸
It does the job of deciding which request goes to which server, and thus solving the above mentioned problems ✔️

Let's see this in detail 🕵️

When we talk about a server its not a single server, actually there are lot of server sitting behind load balancer, Load Balancer is a device (hardware) or may be a software (ex: Ngnix)
But when software is avalible then who will spend money on hardware 💰
So a software load balancer is widely used 🎢

In the above image servers could be web server or application server, and server may further communicate with database etc etc

A load balancer acts as a traffic cop 👮 sitting in front of servers and routing clients across all servers with equal load 📶 This ensures that no single server have too much load which can lead to degradation of service or degradation of server's health 👨‍⚕️

A load balancer divides load uniformly across all servers, in the above scenario every server would receive 25% of the load as there are 4 servers. ♾️
When client resolves URL into an IP address through DNS Lookup it basically gives the Load Balancer's IP not server's IP.
Requests are measured in QPS (Query per second)
This Adds extra security 🔐 as servers are not publicly exposed and remains in a private network. If a client tries to directly access servers it cant do so without going through load balancer.

Load balancing Algorithms 🍖

The algorithm in a particular load balancer decide which server will be receiving the next incoming request

The Algorithms are as follow 👇

Round Robin
weighted Round Robin
Least Connections
Least Response Time
IP Hash

Lets take a look at all of them in detail 🕵️‍♂️

Round Robin ⭕

Requests are sequentially sent to 1st, 2nd 3rd,.... till last server and then next request is sent back to the first server and this process is continues in a circular manner 🔁

In the above case requests will be sent in this manner:

S1
S2
S3
S4
S1
S2
S3

And So on...

Weighted Round Robin 🏋️

One Drawback in Round Robin is that it assume that all the servers have same computational power and there's no way we can assign more requests to the one with more power and less request to the one with less power. 🚦

To counter this we assign weights to each server and based on that weight requests are redirected 💪

Least Connections Algo ↔️

Request are sent to server with least number of open connections ⚖️

In the above representation at this particular instance S3 has the least number of connections so the next incoming request will be received by server S3 according to Least Connection Algo

Least Response Time Algo 🔭

Request are sent to the server with fewest active connection and lowest average response time ↔️

Here the avg response time is calculated by multiplying number of connections and response time

In the first case S2 and S3 both have same avg response time so in that case either of S2 or S3 will receive the request.

But in case 2 S3 has the least avg response time hence it will receive the request

IP Hashing Algo #️⃣

Here IP of the client is used to determine the server to which request has to be redirected 🥢

An high level explanation would be:

Some logic is written in a function and an IP address is passed to that function, based on that logic which resides inside the function it return the server to which request has to be redirected for that particular IP address 👩🏻‍🔧

Placement of Load Balancers 🛤️

Based on certain criteria Load balancer can be placed at Layer 4 or Layer 7 in the OSI Model of a product 🧘‍♂️

An OSI Model consist of 7 layers which are 👇

Its Similar to layers in a Cake 🥮 😁

Layer 4 Load Balancers:

The Layer 4 Load balancer are placed on the 4th Layer of OSI model i.e Transport Layer

They don't have access to data and decision to redirect request to a server is purely done on the basis on data.

Layer 7 Load Balancers:

Layer 7 Load Balancer are placed on Application layer and have access to incoming as well outgoing request and data and decision is made up on its basis, Layer 7 LB is very very much complex :/

At Last an high level architecture of load balancer can be depicted as:

Happy Load Balancing 🚀

System Design - 101

Uzair — Sun, 19 Sep 2021 06:05:59 +0000

Hey There!!
Welcome to System Design 101

Find Your way to the topic which excites the most!!!
NOTE: It is recommended to follow the topics in order if you have no prior knowledge in system design