Requirements & Back-of-the-Envelope Calculation

Overview

This article covers how to size a system before building it: first gathering functional and non-functional requirements, then running back-of-the-envelope calculations (QPS, storage, bandwidth, cache, and server count) through a worked example.

Why estimation matters

• Using estimations we can go from "build a url shortener" to concrete numbers:
  • Queries per second (QPS)
  • storage/year
  • bandwidth
  • server count
• And with that you can spot bottlenecks before they turn into a problem.
• It provides us the ability to think about feasibility and scale, and drives capacity planning: how many servers, how much cache, how many DB shards, how much network bandwidth...

Before talking about estimates

Before estimating anything, pin down the requirements: what the system must do (functional) and how it must behave (non-functional).

Functional Requirements

• Describes what the system does, features from the user's perspective. "Ask yourself what can a user do in my app?"
  • Twitter: post a tweet, follow another user, timeline, like/comment a tweet.
  • Youtube: upload a video, play back a video, search for videos, like/share a video, recommendation feed.
• As you can see these behaviors map directly to APIs and core use cases.
• Trying to do everything, and losing time-to-market on your core features, is terrible.

How to gather Functional Requirements?

• Think about the system, talk with stakeholders, and teammates.
• List primary usecases && Identify the actors: "In this uber-like app, clients request a driver and drivers can accept or reject clients. Will clients be able to rate drivers? And vice-versa?"
• Think about edge scenarios to consider: "What happens if no driver accepts the request? Can a client cancel after a driver already accepted? What if the driver cancels mid-ride, or the app loses connection during the trip?"

Non-Functional Requirements

• Describes how the system behaves, the quality attributes and constraints: performance, reliability, and user experience.
• The architectural trade-offs live here (e.g. read vs write optimized, strong vs eventual consistency) and also estimation targets (QPS, storage, latency, budgets).
• Always attach numbers, don't be vague and say "the system must be fast and highly available", pin concrete targets to each attribute "p99 latency < 200ms" and "99.9% availability".

Some Important Categories of Non-Functional Requirements:

• Scalability: ability to handle growth in users/traffic/data.
  • 1M daily active users; 500M tweets/day; 10k requests/sec
  • "Do we need geographical distribution or just a local region?"
• Performance (latency and throughput): speed and responsiveness.
  • p99 response < 200ms; sub-second search queries
• Availability: Fraction of time the system is up.
  • 99.9% (three nines) vs 99.99% (four nines) uptime.
• Reliability: Correct operation, fault tolerance, no single point of failure (SPOF).
  • Survive one AZ/node failure and still return the right responses.
  • Redundant app servers, db replica, automatic failover, etc.
  • Automatic failover = the system detects when a component failed and switches to a healthy backup on its own.
• Durability: Data survives failures, no data loss.
  • No committed write ever lost; e.g. AWS S3 is designed for 11 9s of object durability (99.999999999%)
• Consistency: how up-to-date reads are across replicas.
  • Strong (banking balances) vs eventual within a few seconds (social media posts).
• Security and Privacy: Protection of data and access.
  • Auth, encryption in transit + at rest, rate limiting, GDPR compliance.
  • encryption at rest = data is encrypted while stored. (disk, database, backups)
  • encryption in transit = data is encrypted while moving between two points. (HTTPS)
• Maintainability/Monitoring: Ease of change; Logging and monitoring.
  • Can also be non-functional requirements.

How to gather Non-Functional Requirements?

• Generally developers only think about functional requirements and it only turns into a problem when software is shipped to production and users start to face slow responses and loadings.
• Simple: question yourself or stakeholders about project's non-functional requirements.
  • How many daily active users?
  • Requests per second?
  • Most requests under 200ms (p99 <200ms)?
  • Can this data be eventually consistent within a few seconds?
  • Is there an uptime target or no SPOF requirement (more than one server instance, DB, etc)?
  • Do we have any reliability constraint, if one server goes down should we have two or more? Geographically distributed or same region?
  • Any GDPR concerns, or special security/privacy requirements?
  • Can users see stale data for a few seconds as a trade-off to reach the required faster responses?
  • Is there a budget or cost ceiling constraining the design?
  • Is it read-heavy or write-heavy, and how big are peaks vs average?
• Obs: while talking with non-technical people we could ask the same questions in a different style to gather non-functional requirements.

Back-of-the-Envelope Estimation

• Quick, rough calculations to size a system in minutes. It's not meant to be exact, it's just a good approximation to check feasibility and find bottlenecks.
• Generally it uses rules of thumb, general guidelines or principles based on experience and observation, not 100% precise but helpful for back-of-the-envelope calculations.
• First of all, you must know basic measures used for these quick calculations.

Data Volume Units (Powers of Two)

• 2^10 ~= 1 Thousand -> 1 KB (Kilobyte)
• 2^20 ~= 1 Million -> 1 MB (Megabyte)
• 2^30 ~= 1 Billion -> 1 GB (Gigabyte)
• 2^40 ~= 1 Trillion -> 1 TB (Terabyte)
• 2^50 ~= 1 Quadrillion -> 1 PB (Petabyte)

Latency numbers every programmer should know (Peter Norvig and Jeff Dean)

• These numbers are dated, but the point isn't accuracy, it's the relative ratios / orders of magnitude between tiers (memory vs SSD vs disk vs network).
• Memory is fast, disk is slow, network across regions is slowest (avoid disk and cross-region hops on the hot path).
• If you need to send data across the network, compress it.
• Caching in memory can turn a 30ms disk read into a ~100ns lookup.
• A cross-continent round trip (~150ms) dominates everything else (put data near users).

• Table from: ByteByteGo

• ms = millisecond (10^-3 seconds)

• µs = microsecond (10^-6 seconds)

• ns = nanosecond (10^-9 seconds)

Estimation example

1. Estimating write QPS: start from daily active users (DAU) * actions per user per day, then convert to per-second (divide by the seconds in one day 86400)

• 100k daily active users (DAU)
• each user posts 3 photos per day (a photo post is a write)

writes per day = 100,000 * 3 = 300,000 photos/day
write QPS = 300,000 / 86,400 = (3 * 10^5) / (8.64 * 10^4) = 30 / 8.64 ~= 3
(it doesn't need to be exact, it is an approximation)

• write QPS = 3

2. Estimate peak QPS: multiply average QPS by a peak factor (2x-10x) to plan for spikes

PEAK write QPS = 2 * 3 = 6

3. Estimate read vs write QPS: apply the read/write ratio (this system is read-heavy, 100:1)

read QPS = 100 * 3 = 300
PEAK read QPS = 2 * 300 = 600

4. Estimate storage = objects per day * size per object * retention period * overhead factor

• overhead factor = metadata, indexes, replication, etc... (e.g. double overhead = 2)
• assumptions: 3mb avg photo size, 4 years retention, overhead factor = 2, writes = 300,000 photos/day (from step 1)

storage = 300,000 * 3mb * 365 * 4 * 2 ~= 2.6PB

5. Estimate bandwidth = QPS * payload size (split into ingress = writes, egress = reads)

ingress (writes) = write QPS * size = 3 * 3mb = 9 mb/s
egress (reads) = read QPS * size = 300 * 3mb = 900 mb/s
(peak 600 * 3mb = 1.8 gb/s)

• reads dominate (read-heavy), use peak QPS for the peak value

6. Estimate memory/cache: use a rule of thumb -> Pareto principle (cache the hot 20% that serves 80% of reads).

cache = 20% of daily data = (300,000 * 3mb) * 0.2 = 900 GB * 0.2 = 180 GB

7. Estimate server count = required QPS / QPS a single server can handle (+redundancy for safety)

• the dominant load is reads, so size on PEAK read QPS = 600
• assume 1 server handles ~1,000 QPS

600 / 1,000 < 1 -> 1 server already handles the load
Server count = 1 + 1 = 2
(the +1 is for redundancy/availability, so there is no
single point of failure if one goes down)

What the numbers told us: it's a read-heavy service (100:1), so favor caching + read replicas and a CDN for media. Storage is large and media-dominated (~2.6PB over 4 years), so use object storage with tiering, not a single DB. Egress dominates bandwidth (~900 mb/s, peak ~1.8 gb/s), which is why media should go through a CDN. The request load itself is small (peak 600 read QPS), so 1 app server already handles it (plus 1 more for redundancy, 2 total) alongside a cache tier.

Conclusion

• Gathering requirements, knowing about functional and non-functional requirements, and doing back-of-the-envelope calculations are important skills that every system architect must have.
• Even if you didn't know these before this reading, if you already did that instinctively you were on the right path.
• The biggest mistake of some developers is trying to develop a system when they don't know its functional and non-functional requirements. It will certainly be a mess and full of issues.
  • Developing the wrong system well is worse than developing the right system poorly.
• Obviously what we've discussed is just the start. The most important part is experience: doing and repeating.

References

• System Design Primer (Donne Martin, GitHub)
• Back-of-the-Envelope Estimation (ByteByteGo)
• Teach Yourself Programming in Ten Years (Peter Norvig, the latency numbers table)
• How to Gather Requirements in System Design Interviews (DesignGurus)