Disk structures are durable, but updating them for every write is expensive.
LioranDB uses memtables to absorb writes before flushing them to the on-disk B+ tree.
What is a memtable?
A memtable is an ordered in-memory map.
In LioranDB, each entry contains either:
Value(bytes)
or:
Tombstone
A tombstone represents a deletion.
The memtable also tracks:
- Approximate memory usage
- Minimum LSN
- Maximum LSN
- Entry count
- Put count
- Delete count
The write path
A simplified write path looks like this:
Application write
↓
WAL durability
↓
Mutable memtable
↓
Immutable memtable queue
↓
Background flush
↓
Disk B+ tree
The active mutable memtable accepts new writes.
When it crosses a size limit, the engine rotates it into an immutable memtable.
That immutable table is no longer modified and can safely be flushed in the background.
Why ordered maps?
LioranDB uses an ordered map for memtable entries.
This helps because the flush process can emit keys in sorted order, which is friendly to the B+ tree and bulk-write paths.
It also simplifies range merging between:
- Mutable data
- Immutable data
- On-disk pages
Backpressure
Background flushing cannot be allowed to fall behind forever.
LioranDB therefore tracks limits such as:
Maximum immutable memtables
Maximum immutable bytes
Partition-wide queue limits
Maximum writer stall duration
If the disk cannot drain the backlog quickly enough, the foreground write path slows down.
That may sound undesirable, but controlled backpressure is much safer than consuming memory until the process dies.
A memtable is not merely a cache.
It is a pressure valve between CPU-speed writes and disk-speed persistence.
Without that valve, the engine would either become slow on every commit or dangerously accumulate unbounded work.
Built by Swaraj Puppalwar under Lioran Group.
Links:
Top comments (0)