DEV Community

Cover image for System Design: Capacity - Traffic Estimation for Instagram & Twitter
ZeeshanAli-0704
ZeeshanAli-0704

Posted on

1

System Design: Capacity - Traffic Estimation for Instagram & Twitter

Article related to Capacity Calculations
How I Calculate Capacity for Systems Design - 2

Instagram Calculations

1. Daily Active Users (DAU) and Posts

  • Total users: 600 million
  • Daily Active Users (DAU): 200 million
  • Assume Percentage of DAU posting 2 posts each day: 25%

Write Request

Total posts per day= 200 million x 25% x 2 = 100 million posts/day - 100M posts/day

Total posts per second = 100 million posts/seconds in a day
= 100M/100K = 1000 / seconds

Read Request

  • Views per day:

Total views per day = 200 million x 100 reads = 20 billion
views/day

Total views per seconds = 20 billion views / seconds in a day = 200K/seconds

Overall Traffic Estimations:

Write per second: 1000/seconds
Read per second: 200K/seconds

2. Storage Calculation for One Day

let's stick to the above calculation where 100M write requests we have.

Breaking them into Image Post & Reel Post only for now.

Here, you can also consider another post request like like, comment, share etc..

Images

  • Percentage of posts that are images: 70% of 100M

So,

  • Number of image posts per day:

Image posts per day = 70% times 100 million = 70 million

  • Storage per image: 200 KB

  • Total storage for images:

Storage for images = 70 million x 200 KB = 14,000,000,000 KB = 14,000 TB

Reels

  • Percentage of posts that are reels: 30% so,
  • Number of reel posts per day:

Reel posts per day = 30% times 100 million = 30 million

  • Storage per reel: 2 MB
  • Total storage for reels:

Storage for reels = 30 million x 2 MB = 60,000,000 MB = 60,000 TB

Total Storage Needed for One Day

  • Total storage:

Total storage = Storage for images + Storage for reels

Total storage = 14,000 TB (images) + 60,000 TB (reels) = 74,000 TB = 74 petabytes (PB)

3. Storage Calculation for Five Years

  • Assuming 365 days per year:
  • Storage needed for one day: 74 PB
  • Storage needed for five years:

Total storage for five years = 74 PB times 365 days times 5 years

Total storage for five years = 135,350 PB = 135.35 ~ 130 exabytes (EB)

4. Bandwidth Calculation

1. Incoming Data Bandwidth

  • Total data coming to the server per day: 74 PB
  • Convert to Mbps:

74 PB = 74 x 1000 x 1000 x 1000 MB

Data per day = 74,000,000,000 MB

Data per second = 74,000,000,000 MB / 86400 seconds ~ 856,481.48 MBps ~ 836.67 GBps ~ 800 GB/sec

2. Outgoing Data Bandwidth

  • Views per day: 20 billion

Assuming 50% for images and 50% for reels:

Image Posts Assumption

  • Each image post: 200 KB
  • Total outgoing data for images:

20 billion × 0.5 × 200 KB = 2,000,000,000,000 KB = 2,000,000,000 MB = 2,000 TB

Reels View Assumption

  • Each reel post: 2 MB
  • The user watches 1 out of 5 reels:

20 billion × 0.5 × 1/5 × 2 MB = 4,000,000,000 MB = 4,000 TB

Total Outgoing Data

2,000 TB (image posts) + 4,000 TB (reels) = 6,000 TB

Convert to GBps:

6,000 TB = 6,000,000 GB

Data per second = 6,000,000 GB / 86,400 seconds ≈ 69.44 GBps ~ 70GBps

Summary

  • Daily Storage Requirement: 74 PB
  • Five-Year Storage Requirement: 135 EB
  • Incoming Bandwidth Requirement: ~856,481.48 MBps (or ~800 GBps)
  • Outgoing Bandwidth Requirement: ~69.44 GBps (or ~70 GBps)

Twitter Calculations

Twitter's Traffic Estimations and storage requirements:

Assumptions:

  1. 300 million monthly active users (MAU): This is the total number of unique users who use Twitter at least once a month.

  2. 50% of users use Twitter daily: This implies that half of the monthly active users are active on Twitter every day.

  3. Users post 2 tweets per day on average.

  4. 10% of tweets contain media: Media can include images, videos, etc.

  5. Data is stored for 5 years: This is the retention period for storing the tweets and media data.

Estimations:

Daily Active Users (DAU):

DAU = 300 million x 50% = 150 million

Tweets QPS (Queries per Second):

  • Total tweets per day:

Total tweets / day = 150 million users x 2 tweets per user = 300 million tweets

  • Tweets per second (assuming tweets are uniformly distributed throughout the day):

Tweets / second = 300 million tweets x 24 hours x 3600 seconds ~ 3472 tweets per second

approx 3500 tweets per second (rounded for simplicity)

  • Peak QPS: Often, systems need to handle peak loads, which can be higher than the average. Assuming the peak QPS is double the average: text Peak QPS = 2 x 3500 approx 7000 tweets per second

Media Storage Estimation:

Average tweet sizes:

  • tweet_id: 64 bytes
  • text: 140 bytes
  • media: 1 MB (only for tweets containing media, which is 10%)

Daily media storage:

Daily media storage = 150 million users x 2 tweets per user x 10% x 1 MB

300 million tweets x 10% x 1 MB ~
30 million tweets with media x 1 MB ~
30 TB per day ~ 30TB/Day

5-year media storage:

5-year media storage = 30 TB per day x 365 days x 5 years
30 TB/day x 1825 days

54,750 TB

Approx 55 PB (rounded for simplicity, where 1 PB = 1000 TB)

Summary:

  1. Daily Active Users (DAU): 150 million
  2. Tweets QPS:
    • Average: ~3500 tweets per second
    • Peak: ~7000 tweets per second
  3. Media Storage:
    • Daily: 30 TB
    • 5-year: ~55 PB

These estimations provide an understanding of the scale at which Twitter operates in terms of query load and storage requirements.

Thanks
Zeeshan

Article related to Capacity Calculations
How I Calculate Capacity for Systems Design - 2

More Details:

Get all articles related to system design
Hastag: SystemDesignWithZeeshanAli

Git: https://github.com/ZeeshanAli-0704/SystemDesignWithZeeshanAli

Sentry blog image

How I fixed 20 seconds of lag for every user in just 20 minutes.

Our AI agent was running 10-20 seconds slower than it should, impacting both our own developers and our early adopters. See how I used Sentry Profiling to fix it in record time.

Read more

Top comments (0)

Image of Docusign

🛠️ Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more