Reading DDIA, Day 1: Data Systems, Reliability, and Why a Fault Isn't a Failure

Deepak Verma — Wed, 14 Oct 2026 02:48:40 +0000

I've started reading Designing Data-Intensive Applications (DDIA), and I'm going to write up what I learn each day as I go. My original handwritten notes go on GitHub, and this is the cleaned-up version.

Day 1 covers the building blocks of data systems, the questions to ask when you design one, and the first big topic: reliability.

The building blocks

Most data-intensive applications are built from a few standard pieces:

Databases: store data so that they, or another application, can find it again later.
Caches: remember the result of an expensive operation, to speed up reads.
Search indexes: let users search data by keyword or filter it in various ways.
Stream processing: send a message to another process, to be handled asynchronously.
Batch processing: periodically crunch a large amount of accumulated data.

The lines between these categories are blurry:

Some datastores are also used as message queues. Example: Redis.
Some message queues come with database-like durability guarantees. Example: Apache Kafka.

A few more tools that show up again and again:

Memcached: an application-managed caching layer.
Elasticsearch / Solr: full-text search servers.

Questions to ask when designing a data system or service

How do you make sure data stays correct and complete, even when things go wrong internally?
How do you keep performance consistently good for clients, even when parts of your system are degraded?
How do you scale to handle an increase in load?
What does a good API for the service look like?

The three concerns that matter most

Reliability
Scalability
Maintainability

Today was all about the first one.

Reliability

For software, reliability means:

The application performs the function the user expected.
It can tolerate the user making mistakes or using the software in unexpected ways.
Its performance is good enough for the required use case, under the expected load and data volume.
The system prevents unauthorised access and abuse.

In one line: it should work correctly, even when things go wrong.

Faults vs. failures (remember this one)

The things that can go wrong are called faults. Systems that anticipate faults and can cope with them are called fault-tolerant or resilient.
A fault is not the same as a failure.
- A fault is one component of the system deviating from its spec.
- A failure is when the system as a whole stops providing the required service to the user.

📌 On my reading list: Netflix's Chaos Monkey.

Types of faults

1. Hardware faults

A hard disk crashes
RAM becomes faulty
The power grid has a blackout
Someone unplugs the wrong network cable

Related term: MTTF (Mean Time To Failure).

2. Software errors

A software bug that causes every instance of an application server to crash when given a particular bad input.
- Example: the leap second on June 30, 2012 made many applications hang at the same time, because of a bug in the Linux kernel.
A runaway process that uses up a shared resource: CPU time, memory, disk space, or network bandwidth.
A service the system depends on slows down, becomes unresponsive, or starts returning corrupted responses.
Cascading failures, where a small fault in one component triggers a fault in another component, which then triggers more faults.

Day 1 takeaway

A reliable system isn't one where nothing goes wrong. It's one where faults happen and the system as a whole keeps working anyway.

📓 Original handwritten notes: Github link

See you tomorrow for Day 2. 👋

DEV Community: Deepak Verma