A WAL can recover recent writes, but replaying an ever-growing log on every startup would eventually become unacceptable.
Databases therefore create checkpoints.
What does a checkpoint mean?
A checkpoint advances the durable on-disk state.
Conceptually:
WAL contains operations up to LSN 1,000,000
Data pages are flushed up to LSN 900,000
Checkpoint LSN becomes 900,000
The engine now knows that older WAL records below the safe reclaim boundary may no longer be required.
The manifest
LioranDB stores critical engine metadata in a MANIFEST file.
It includes values such as:
Page size
Next page ID
Next LSN
Checkpoint LSN
Freelist root
Table root pages
Without this metadata, the engine would not know where its tables begin or which log range is already reflected on disk.
Atomic replacement
Updating metadata in place is dangerous.
A crash halfway through the write could leave a partially written manifest.
LioranDB instead follows an atomic replacement flow:
Encode new manifest
↓
Write temporary file
↓
fsync temporary file
↓
Rename temporary file over MANIFEST
↓
Sync directory entry
The rename allows the old or new complete manifest to exist, instead of a half-manifest chimera crawling out of a power failure.
Bounded checkpoint slices
Checkpointing can also compete with foreground writes for disk bandwidth.
LioranDB supports limits based on:
- Bytes per checkpoint slice
- Wall-clock time per slice
- Dirty-byte targets
- WAL quiet periods
- Periodic full checkpoints
The checkpoint can yield and continue later rather than monopolizing storage.
A checkpoint is not simply “save database now.”
It is a coordinated agreement between WAL durability, dirty pages, metadata publication, index state, and log reclamation.
Getting that agreement wrong is how databases lose acknowledged data.
Getting it right is how startup stays fast without compromising recovery.
LioranDB is developed by Swaraj Puppalwar at Lioran Group.
Explore:
Top comments (0)