📐System Design for Beginners

#systemdesign #beginners #architecture

What is System Design?

System design is the process of developing and planning the architecture, components, modules, and interfaces for creating complex software, hardware, or information systems.

There is the good tutorial about System Design in here

In this article I want to show how to apply SD in practice -> step by step.

Let's try to create a system design for the Dating Service (analog of Tinder, Bumble, Badoo, etc.)

1️⃣ Requirements Gathering Process.
In this step, let's consider Functional vs. Non Functional Requirements

Functional requirements:

the user can register in the application

the user can create/edit a personal profile

users can receive notifications when receiving dating offers or when two users are automatically matched

the user can accept/reject dating offers

the user can view a list of other users according to their preferences and their location

the user can upload photos/small videos

administrators should have their own functionality for moderation

Non-functional requirements:

System responsiveness (so that the user can quickly view the list of potential candidates for dating)

High availability (as the load increases, the application must scale and be available, otherwise users will go to another similar service)

Consistency (when accepting/rejecting offers between users)

Provide monitoring with alerts for the system

2️⃣ Capacity Estimation

Metrics for Capacity Estimation

100M - Monthly Active Users (MAU)

10M - Daily Active Users (DAU)

about 10M - registered users (REG_USERS)

about 3MB(megabyte) - user data (USER_DATA): messages, photos/small videos, profile data, etc.

let's assume that the ratio of writing to reading data is 1 to 10 (RATIO_WRITE_READ)

on average one user performs about 50 operations (OPERS)

86400 - number of seconds in a day (NUM_SEC_DAY)

365 - number of days in the year (DAYS_OF_YAER)

Estimated load indicators
Requests Per Second (RPS) or Queries Per Second (QPS):

RPS_READ = OPERS*REG_USERS/NUM_SEC_DAY = 50*10M/86400 = 5800 - RPS for reading

RPS_WRITE = RPS_READ/RATIO_WRITE_READ = 5800/10 = 580 - RPS for writing data

Network Bandwidth (NB)

NB = USER_DATA*RPS_WRITE = 3MB*580 ~ 2000 MB/sec (2GB/sec)

NB_PER_YEAR = NB*NUM_SEC_DAY*DAYS_OF_YAER = 2GB/sec * 86400sec * 365days ~ 65PB + 20% (with Overhead) ~ 80PB(petabyte)

Storage load (SL)

SL = DAU*USER_DATA = 10M*3MB = 30TB(terabyte) ~ 30TB + 20% (with Overhead) ~ 40TB

Calculation of cloud infrastructure capacity

let's assume that one instance can handle with about 500 RPS: NUMBER_INST = RPS_READ(Peak RPS)/500 = 5800/500 + 20%(with Overhead) ~ 15 instances;

Instances for Network Bandwidth: Convert to Gbps (125 MB/sec = 1 Gbps) NB = 2000/125 ~ 16 Gbps; if each instance can handle 1 Gbps, then NUMBER_INST_NB = 16/1 ~ 20 instances (with Overhead);

Storage, if each storage instance can handle 5TB of data, then NUMBER_INST_STOR = SL/5 = 40/5 = 8 instances. If you implement 2x replication for fault tolerance, you need twice the storage: 8*2 = 16 ~ 20 instances (with Overhead);

How much does it cost?💵

Network Bandwidth (~0.1$ for 1GB): 80PB = 80,000,000GB*0.1$ = 8M$

SSD storage (~300$ for 1TB): 40TB * 300$ = 12,000$

3️⃣ High-Level Design

By high-level design we will mean a design that minimizes the basic scenarios of interaction with the system, but does not reflect all the complexity of interaction between the large number of real components and does not take into account the load, under which the system will be in a productive environment.

In this step, let's draw the scheme of components for our system using Excalidraw.

To utilize the following components of SD

From this library

Through those components, let's build the next scheme.

4️⃣ Component design
Previously, we looked at the high-level components that most systems will consist of.
At the same time, we often hid all the logic inside the application behind one logical "square". Let's try to delve into the details of what can actually happen under the hood.
And we can separate our scheme into two parts: