Ask an engineer to design a database and they reach for the tools they know: rows, a B-tree index, a mix of reads, updates, and deletes on individually addressable records. That instinct is correct for most systems and wrong for this one. A time-series database is built around a workload so lopsided that almost every general-purpose assumption inverts. Writes arrive constantly and in enormous volume, they almost always carry a timestamp near now, and once a point lands it is essentially never updated. Reads are heavily skewed toward recent data. Queries are range scans and time-bucketed aggregations, not point lookups. The whole design bends around those facts, and every interesting decision in it falls out of taking them seriously.
This is the long version. We will start from the shape of the workload, build up the series data model and why it matters, walk the write path from the write-ahead log through the in-memory head to immutable on-disk blocks, take apart the compression that makes months of history affordable, be precise about the one failure mode that actually kills these systems in production, and then work through downsampling, retention, and the query engine that turns a tag filter into a fast aggregation. By the end you should be able to defend, in front of a skeptical staff engineer, why a row store struggles here and where a purpose-built engine earns its keep. One note on numbers up front: every throughput, cardinality, and ratio figure here is an industry-typical range or an illustration, with one exception. The Gorilla figures of roughly 1.37 bytes per point and about a 12x reduction come from Facebook's published paper, and I cite them as such.
The workload is the whole argument
Start with what the data actually looks like. A point is tiny: a timestamp and usually a single float, sometimes with a little metadata. A single fleet of servers or IoT devices can push millions of these per second, each one describing CPU usage, request latency, a sensor reading, or a price at an instant. The points arrive in a relentless stream, almost always in timestamp order, and they are immutable once written. Nobody goes back and edits the CPU reading from four minutes ago.
Time-series workloads are append-heavy on writes, hot on recent data for reads, and immutable once a point lands.
Reads have their own bias. Dashboards refresh every few seconds and query recent windows, so the vast majority of read traffic touches the newest slice of data. When someone does ask a long-range question, they almost never want every raw point; they want a trend, a rate, a percentile bucketed into intervals. The dominant read is not "give me this one value" but "give me p99 latency per service over the last six hours, grouped into one-minute buckets," answered in well under a second.
Hold those three facts together, because the rest of the design is just their consequences. Writes vastly outnumber and outpace reads. Data is immutable. Access is overwhelmingly recent and aggregated. A storage engine tuned for exactly this can make choices a general database cannot, and that is the entire reason these systems exist as a separate category.
The series is the unit, and cardinality is the price
The fundamental object in a time-series database is not a row, it is a series. A series is identified by a metric name plus a set of key-value tags, and each series is an ordered stream of points. So http_requests_total with tags service=checkout, region=us-east, and status=500 is one series, and changing any tag value gives you a different one. This model is powerful because it lets you slice and aggregate a metric along any labeled dimension at query time. It is also a loaded gun, and understanding why is the difference between someone who has operated these systems and someone who has only read about them.
A series is a metric name plus a set of tags; the number of distinct tag combinations is the cardinality, and it is the real scaling limit.
The number of active series is called cardinality, and it equals the product of the cardinalities of all the tags on a metric. A metric with a service tag of 50 values and a region tag of 10 values has 500 series, which is nothing. Add an endpoint tag with 200 values and you are at 100,000. Now attach a tag whose values are unbounded, a user id, a full URL with query parameters, a request id, a container name that changes on every deploy, and cardinality does not grow, it explodes, because every new value mints a brand-new series that must be tracked for the whole retention window. Keep this word in mind. We will come back to it as the central failure mode, because it is the one thing this workload punishes hardest.
Why a relational row store fights you
Before building the specialized thing, it is worth being precise about why the general thing struggles, because the answer names every design decision that follows.
A relational engine stores data row by row and indexes it with a B-tree. That layout interleaves the timestamp, the value, and every tag column together on disk. A query that only wants the value over a time range still drags every other column through memory and cache, wasting I/O on data it will discard. The B-tree itself suffers under sustained high-rate inserts, because a stream of new series and new points keeps touching and rewriting index pages, producing write amplification and fragmentation exactly when you need ingestion to stay cheap. Values are stored in fixed-width, uncompressed form, which leaves enormous compression on the table, and we are about to see just how compressible this data really is. And retention in a row store means deleting billions of individual rows, which generates dead tuples, vacuum pressure, and more index churn, rather than simply dropping a file.
You can bolt time-series behavior onto a relational engine. TimescaleDB does precisely this, turning a PostgreSQL table into a hypertable that automatically partitions into time-bounded chunks and adds columnar compression on older chunks. That it works is real proof the relational base can be adapted. That it requires exactly time partitioning, columnar layout, and compression to be added on top is also the whole argument for why a native design has those properties from the start.
The write path: WAL, head block, immutable blocks
Here is the pattern at the heart of nearly every modern time-series database, and it is worth internalizing because it recurs everywhere. Recent data lives in fast, mutable memory. Historical data lives in tightly packed, read-only, time-bounded files. The write path is the machinery that moves data from the first state to the second with a durability guarantee in between.
Every point is appended to a write-ahead log, buffered in an in-memory head block, then flushed to immutable on-disk blocks.
When a point arrives at a storage node, the first thing that happens is a cheap sequential append to a write-ahead log. This is the durability contract: if the process crashes before anything else, the point is not lost, because on restart the node replays the WAL to rebuild its in-memory state. Only after the log append does the point get inserted into the in-memory head block, a mutable structure holding the most recent window of samples for every active series. The head is what serves the freshest reads, and it is where incoming points accumulate before they are ever compressed.
The head does not grow forever. On a time boundary, Prometheus for example cuts on roughly two-hour boundaries, the accumulated data is compressed and written out as an immutable block covering that fixed time range. Once that block is safely on disk, the WAL segments it covered are redundant and get truncated. This separation is what buys you both properties at once. The head absorbs a high, effectively random write rate in memory with a fast durability guarantee from the log, while historical data settles into files that are cheap to scan, trivially cacheable because they never change, safe to replicate, and cheap to delete wholesale later.
There is one wrinkle worth naming because interviewers probe it. The head and the block encoders are optimized for near-now, in-order writes. A point that arrives far behind the current time does not fit that assumption cleanly, so many systems historically rejected out-of-order data outright, and those that support it route it through a separate, more expensive path. This is not an accident or a missing feature so much as a deliberate trade: optimizing for the common case, ordered and recent, is where the throughput and compression wins come from.
Compression is a design goal, not an optimization
In most databases compression is a knob you turn later. In a time-series database it is load-bearing, because it is the entire difference between keeping a week of data and keeping a year for the same money. And the data compresses astonishingly well once you exploit its structure, which is exactly what Facebook's Gorilla paper set out to do.
Timestamps compress with delta-of-delta encoding; float values compress with XOR against the previous value, the Gorilla scheme.
Two properties do the work. First, timestamps within a series are usually regularly spaced, say every fifteen seconds. Storing each 64-bit timestamp raw is absurd when they are so predictable, so instead you store the delta between consecutive timestamps, and then the delta-of-delta, the change in that delta. For perfectly regular sampling the delta-of-delta is zero, which encodes into a single bit, and small jitter encodes into a handful of bits. A long run of evenly spaced timestamps collapses to almost nothing.
Second, consecutive values within a series usually change slowly, and many metrics are near-constant or trending gently. Gorilla encodes each float by XORing it with the previous value. When two values are identical the XOR is zero, one bit. When they are close, the XOR has long runs of leading and trailing zero bits, so you store only the meaningful middle bits plus a short header saying where they sit. Put the two encoders together and, on real Facebook production data, points went from 16 bytes each down to about 1.37 bytes, roughly a 12x reduction. That is the number that makes months of retention economically possible.
The caveat is the same one from the write path. This bit-packed encoding assumes in-order, append-only writes within a block, which is another reason these systems prefer near-now ordered ingestion and treat backfill as a special case. The compression and the write-path design reinforce each other.
Cardinality is the failure mode that actually kills you
Now we return to the word to remember, because if you take one thing from this piece into an interview, make it this. High cardinality is the single most important thing to get right, and the failure it causes is the one that most cleanly separates people who have run these systems from people who have not.
High cardinality, driven by unbounded tag values, is the central failure mode; per-series state grows with series count, not point rate.
Recall that cardinality is the number of distinct active series and equals the product of the cardinalities of all the tags. The damage concentrates in exactly the parts of the system that hold per-series state. The in-memory head keeps structures for every active series. The inverted tag index, which we will meet properly in a moment, stores a posting list entry for every label value. Both of those grow with the number of series, not with the number of points. That is the crucial inversion: a time-series database scales primarily on active series, not on points per second. You can push more points into a bounded set of series more or less forever, but every new series costs memory that does not come back until the data ages out.
So a cardinality explosion does not show up as a gradual slowdown. It shows up as the ingestion node running out of memory and crashing, or as a query that used to touch a thousand series now having to touch ten million and grinding to a halt. The standard defenses all target cardinality directly and they are worth reciting because they are the answer to "how would you keep this stable." Never put unbounded or high-churn values, ids, raw URLs, emails, timestamps, into tags. Aggregate or drop high-cardinality labels at ingest through relabeling. Enforce per-metric and per-tenant series limits so one bad deploy cannot take down the cluster. And move genuinely high-cardinality dimensions into logs or traces, where per-event detail belongs, rather than forcing them into a metrics model that charges you forever for each distinct value. A candidate who volunteers the cardinality trap unprompted is signaling real operational scars, and that is exactly what the question is designed to find.
Downsampling and retention make the economics work
Raw high-resolution data is both expensive to keep and expensive to query over long ranges. A query for the last year does not want to read billions of raw points per series, and you do not want to pay to store them at full resolution forever. Two coordinated mechanisms solve both problems.
Downsampling rolls raw points up to 1-minute and 1-hour aggregates, and tiered retention keeps each resolution for a different length of time.
Downsampling precomputes lower-resolution versions of the data. A background rollup job reads raw points and, for each series and each coarser bucket, one minute and then one hour, stores a small set of aggregates: typically min, max, sum, count, and last, often with a percentile sketch alongside. A year-long query then reads a few thousand hourly rollup points per series instead of billions of raw ones, which is what keeps long-range dashboards fast. Retention then governs how long each resolution tier lives, for example raw for fifteen days, one-minute rollups for ninety days, one-hour rollups for two years. Because data is partitioned into time-bounded blocks, enforcing retention is a cheap file-level delete of whole expired blocks, not a row-by-row scan.
Two subtleties matter and both make good interview material. First, the aggregation choice has to be mathematically correct. You cannot average pre-averaged data across buckets and get the right answer, which is exactly why rollups store sums and counts, so an average can be recombined correctly, and why percentiles demand mergeable sketches rather than a single stored p99 value that cannot be meaningfully combined. Second, downsampling is lossy by design. Once raw points age out you can no longer investigate a spike at full resolution or compute an aggregate you never precomputed. The trade is therefore permanent and worth tuning per metric: pay to keep raw data for ad hoc high-resolution forensics, or pay far less to keep only rollups and give up that option.
The query engine: inverted index plus scatter-gather
The last piece is the read path, and it starts from a problem. Queries select series by tag matchers, not by primary key, so the engine needs to turn a predicate like service=checkout and status matches 5xx into an exact set of series quickly, without scanning sample data for series it does not need. The right structure is borrowed straight from search engines: an inverted index.
An inverted index turns tag matchers into a series set, then the query fans out to shards and gathers partial aggregates.
For every label key-value pair, service=checkout, status=500, region=eu-west, the index keeps a posting list, the sorted set of series ids that carry that pair. A tag query resolves by looking up the posting lists for the matching pairs and intersecting or unioning them to produce the precise series set to read. Notice that this index is another place cardinality bites first: its size grows with the number of distinct label values, so a high-cardinality label bloats the index before it bloats anything else.
Once the series set is known, the engine reads the relevant time range from the head block and any overlapping on-disk blocks, choosing the coarsest resolution tier that still satisfies the requested step so it does the least work possible. On top of the raw samples it applies the operations a generic SQL engine does not have built in: group by time to bucket points into fixed intervals, rate and increase to turn monotonically increasing counters into per-second rates while correctly handling counter resets, aggregations like sum, avg, min, and max across series, and quantile functions computed from mergeable sketches so they can be combined across shards and rollups.
That last point is what makes distribution work. In a sharded deployment, series are spread across nodes by hashing series identity, and a read runs as scatter-gather: a coordinator sends the query to every shard that owns matching series, each shard computes a partial aggregate locally to minimize how much data crosses the network, and the coordinator merges the partials into the final result. Pushing the aggregation down to where the data lives, rather than shipping raw points to a central node, is the same principle that shows up in every well-designed distributed query system.
The trade-offs worth arguing about
A good design answer is not a list of components, it is a set of defended choices, and this problem is rich in the kind where there is no single right answer.
Push versus pull ingestion is the classic one. In a pull model the database scrapes a metrics endpoint on each target on a fixed schedule, which gives the server control over load, makes a target being down an immediate and explicit signal because the scrape fails, and makes redundant scrapers trivial, but it cannot easily reach targets behind NAT or capture short-lived jobs, so pull systems add a push gateway for batch work. In a push model clients send data in, which handles ephemeral and event-like producers naturally and works through any network path, but it exposes the database to uncontrolled write bursts and turns "is the client alive" into an ambiguous question. Pull fits controlled, service-oriented fleets; push fits heterogeneous or ephemeral producers; large platforms often support both.
The engine choice is another. You can reuse a log-structured merge tree, the family behind Cassandra and RocksDB, which gives you a proven, general engine and existing operational knowledge quickly, at the cost of weaker time-series compression because it does not know timestamps are monotonic and values change slowly. Or you build a purpose-built engine with per-series columnar layout and delta-of-delta and XOR encoding, which is bespoke code to maintain but wins the roughly 10x storage advantage. Dedicated systems almost always pick the specialized path because the compression win is large enough to pay for the engineering. And underneath it all sits the durability trade: acknowledging each point only after quorum replication and fsync maximizes safety but caps throughput, so because one lost fifteen-second sample rarely matters, most time-series databases lean hard toward throughput in a way a transactional database never would.
Where to go from here
The thing worth keeping is the method, not the trivia. A time-series database is fast and cheap because it refuses to be general. It gives up arbitrary updates, cross-series joins, and unbounded cardinality, and in exchange it gets columnar per-series layout, compression that turns 16 bytes into a fraction of one, time partitioning that makes retention a file delete, and a query engine that answers recent-range aggregations in milliseconds. Every one of those is the same move: take the lopsided shape of the workload seriously and let the design fall out of it. Understand which of those properties your own problem actually needs, and the build-versus-adapt decision stops being a guess.
I teach time-series databases and the rest of system design this way, from first principles with real diagrams and the trade-offs that only surface in production, as an interactive course at systemdesign.academy. The foundation lessons are free and need no signup.
Read the free lessons: https://systemdesign.academy
Top comments (0)