When an application inserts a document, the database cannot immediately rewrite every affected disk page and wait for all of them to be flushed.
That would be painfully slow.
LioranDB therefore uses a Write-Ahead Log, commonly called a WAL.
The core rule
Before a write is acknowledged as durable, its log record must be persisted.
Transaction
↓
Append changes to WAL
↓
fsync WAL
↓
Acknowledge commit
↓
Flush data pages later
If the process crashes before the data pages are written, the WAL can replay the committed operation.
Group commit
Calling fsync for every individual transaction would destroy throughput.
LioranDB groups multiple transactions into one durability batch.
A batch can be triggered by:
- Enough pending bytes
- Enough waiting transactions
- A timeout
- Strict durability mode
- Shutdown
- A follow-up durability generation
Conceptually:
Txn A ─┐
Txn B ─┼─→ One WAL append and fsync
Txn C ─┘
All transactions in that durable range can then be released together.
Durability modes
The engine supports multiple WAL profiles:
Strict
Balanced
HighThroughput
UnsafeBenchmark
Balanced is the default.
It groups commits over a very small time window while still syncing before acknowledging them.
UnsafeBenchmark can acknowledge data in the operating system cache without an fsync, but it exists for benchmarking, not production durability.
Why LSNs matter
Every WAL record has a Log Sequence Number, or LSN.
The engine tracks values such as:
Next LSN
Durable LSN
Checkpoint LSN
Pending WAL bytes
Oldest waiter age
These values tell the engine which operations are durable, which pages are behind, and which log segments may eventually be reclaimed.
A WAL turns disk latency from a wall into a queue.
The hard part is not merely appending bytes. It is coordinating batching, fsync, checkpoints, waiters, backpressure, and crash recovery without lying to the caller about durability.
That is where database engineering becomes properly spicy.
LioranDB is created by Swaraj Puppalwar at Lioran Group.
More information:
Top comments (0)