DEV Community

Cover image for Building a Log-Structured Merge Tree in Go

Building a Log-Structured Merge Tree in Go

Justin Ethier on December 15, 2022

Introduction Last year I read more challenging projects every programmer should try and decided to take a deep dive into researching sol...
Collapse
 
yoyo_programming profile image
yoyo

Why did you use a skip list for the memtable instead of the built-in map in Go?

Collapse
 
justinethier profile image
Justin Ethier • Edited

Go's built-in map is not thread-safe, so mutexes must be used for concurrent access. For high performance it is better to use a data structure that is better suited for concurrent access, such as a skip list.

Collapse
 
dmast3r profile image
Sourabh Khandelwal

Could you get inspiration from what Java's ConcurrentHashMap does? Essentially, it uses a technique called lock striping—instead of holding a single lock on the entire hash map, different locks are held depending on the bucket. As far as I can recall, Java uses 16 different locks spread across the entire hash map. This helps reduce contention a bit. Furthermore, it has the concept of read-write locks, so multiple readers can acquire the lock simultaneously, as long as no other thread is writing.

However, thinking about this again, the SkipList with an RWMutex probably works similarly. The striping behavior would come from the SkipList's inherent hierarchical structure, while the RWMutex could allow multiple readers to hold the lock simultaneously.

Collapse
 
zhiquan profile image
hu • Edited

from your GetFunction on search disk,you check index files range all levels especially higher level. since files in same level whose level bigger than 0 in sorted order, why need check one by one?