A single global database lock is an excellent way to turn a 24-core machine into a decorative heater.
LioranDB uses partitions to scale work across CPU cores.
Independent engine partitions
A partitioned LioranDB node runs multiple independent storage-engine instances.
Each partition has its own:
- WAL
- Checkpoint pipeline
- Flush pipeline
- On-disk directory
- Memtables
- B+ tree state
The directory layout resembles:
data/
└── partitions/
├── 0000/
├── 0001/
├── 0002/
└── 0003/
Routing keys
Keys are assigned to partitions using a deterministic hash.
Conceptually:
partition = hash(key) % partition_count
The same key always reaches the same partition.
A write therefore touches only the engine responsible for that key instead of contending on one global structure.
Parallel startup
Partitions are also opened in parallel.
LioranDB tracks startup metrics such as:
- Total wall-clock open time
- Sum of partition open times
- Maximum partition time
- P50 and P95 open time
- Files opened
- Bytes read
- Peak parallel partition workers
This distinction is useful.
If four partitions each take five seconds but open concurrently, the serial sum is twenty seconds while wall-clock startup is closer to five.
Scans are different
Exact-key operations route to one partition.
A full range scan may need to scan every partition and merge the results into sorted order.
That means partitioning improves point-operation scalability but makes global ordered operations more complex.
This is a standard distributed-systems tradeoff appearing inside a single machine.
From multi-core to multi-node
The partition abstraction is also the foundation for future node-level distribution.
A partition can eventually be:
Owned by one node
Replicated to followers
Moved to another node
Routed through a cluster client
Partitioning is therefore more than a performance optimization.
It is the seam where a single-node database begins evolving into a distributed one.
LioranDB is developed by Swaraj Puppalwar at Lioran Group.
Explore:
Top comments (0)