Designing Data-Intensive Applications

Core Principles
This highlights three fundamental pillars:

1. Reliability

• Tolerating hardware & software faults
• Human error

Explanation:
A system is reliable if it keeps working correctly even when things go wrong.

This includes:

• Server crashes or disk failures
• Bugs in the code
• Mistakes made by developers or users

Example: If a database node crashes, the system should still serve users without losing data.

1. Scalability

• Measuring load & performance
• Latency percentiles, throughput

Explanation:
Scalability is about handling growth (more users, more data, more requests).

Key ideas:

• Load: How much work the system handles
• Latency: How long requests take
• Throughput: How many requests are processed

Example: Your app works fine with 1,000 users — can it handle 1 million?

1. Maintainability

Operability, simplicity & evolvability

Explanation:
Maintainability means the system is easy to:

• Operate (monitor, debug, deploy)
• Understand (clean code, clear structure)
• Change (add features without breaking everything)

Example: A well-structured modular monolith (like what you’re building) is easier to maintain than messy, tightly coupled code.

When designing systems, you should always balance:

• Reliability
• Scalability
• Maintainability

These are your “north star” principles.

---

Data-intensive means a system where data is the main challenge, not just the code or logic.

A data-intensive application is one that:

• Handles large amounts of data
• Focuses on storing, processing, and moving data efficiently
• Cares a lot about the performance, scalability, and reliability of data

What makes an app “data-intensive”?

1. Large volume of data
   Millions of users, records, or transactions

2. High read/write operations
   Constant requests to the database

3. Data processing & transformation
   Filtering, aggregating, and analyzing data

4. Data consistency & reliability matter
   You can’t lose or corrupt data

---

Data-Intensive vs Compute-Intensive

Data-Intensive
Managing data (storage, DB, caching, scaling)

Compute-Intensive
Heavy calculations (AI, video rendering, scientific computing)