In this post, we are going to implement a very simple client persistence key-value store with RocksDB. The idea is to have the simplest possible implementation as an initial reference.
You can find the link with full source code for this post in the references section in the end of this post.
From the RocksDB documentation: RocksDB is a persistent and embeddable key-value store for fast storage environments. Its open-source and was created and is still maintained by Facebook and it's optimized for fast, low latency storage such as flash drives and high-speed disk drives.
Let's create our example, first step is to generate an initial SpringBoot Application with basic REST endpoints, navigate to start.spring.io and create an application, select Java 11 adding actuator and web as dependencies to start with, if you have never done so, check out this post where you'll find out how to do it in very simple steps. Let's call this app rocksdbBootApp.
Once the app is created the next step is to add Rocksjava as a dependency, here you see a snippet of how to do it using maven.
<properties>
<java.version>11</java.version>
<rocksdb.version>5.5.1</rocksdb.version>
</properties>
<dependency>
<groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId>
<version>${rocksdb.version}</version>
</dependency>
Let's now create an interface with the signature of the basic operations we want, KeyValueRepository.
package io.stockgeeks.repository;
public interface KeyValueRepository<K, V> {
void save(K key, V value);
V find(K key);
void delete(K key);
}
Nice, we have basic initial operations to save, find and delete entries, let's implement these basic operations, the most important thing to notice in the following fragments of code is that RocksDB works with bytes so it's all transformed to byte array and back when interacting with it's APIs, because we're using Strings here in this simple example we can simply use the getBytes to transform a String into a byte array and build it back using String(byte[] bytes) constructor. If you need to serialize objects you can use spring SerializationUtils or if you're not using Spring there's also a SerializationUtils class in apache-commons API that can be used.
First we implement the interface and we declare 1 constant with the local storage name which will be reflected in a directory in the file system where RocksDB will keep the data structures, we also define the java File reference which points to the final folder structure and the RocksDB reference we're going to use, we mark the class as a @Repository with the Spring annotation and add logging using lombok @Slf4j annotation:
@Slf4j
@Repository
public class RocksDBRepositoryImpl implements KeyValueRepository<String, String> {
private final static String NAME = "first-db";
File dbDir;
RocksDB db;
Let's now create an initialization method that will initialize the RocksDB file system structure and it's configurations and get it ready for interactions.
We use a @PostConstruct annotation so this fragment will be executed after the application starts. We then make sure the file system directory structure that we need is created and we open the RocksDB after the initialization, if you follow the code here as it is the file system structure where it will be maintained is under: /tmp/rocks-db/first-db, you will probably need to adjust the code paths a bit if you're using a Windows environment.
Note: Java has a universal path
@PostConstruct
void initialize() {
RocksDB.loadLibrary();
final Options options = new Options();
options.setCreateIfMissing(true);
dbDir = new File("/tmp/rocks-db", NAME);
try {
Files.createDirectories(dbDir.getParentFile().toPath());
Files.createDirectories(dbDir.getAbsoluteFile().toPath());
db = RocksDB.open(options, dbDir.getAbsolutePath());
} catch(IOException | RocksDBException ex) {
log.error("Error initializng RocksDB, check configurations and permissions, exception: {}, message: {}, stackTrace: {}",
ex.getCause(), ex.getMessage(), ex.getStackTrace());
}
log.info("RocksDB initialized and ready to use");
}
With the application ready to go we can now have the basic operations methods implemented, starting with the operation to save, nothing really special about it, just a simple straightforward implementation, we log the method call and we use the RocksDB.save method converting the Strings passed as aforementioned, in case of any exceptions we simply log the error in this case:
@Override
public synchronized void save(String key, String value) {
log.info("save");
try {
db.put(key.getBytes(), value.getBytes());
} catch (RocksDBException e) {
log.error("Error saving entry in RocksDB, cause: {}, message: {}", e.getCause(), e.getMessage());
}
}
The find operation is also straightforward, the only important thing to notice is that we check the result for null avoiding a NullpointerException during runtime in case the passed in key does not exist and the aforementioned conversion from byte array to String using the String constructor.
@Override
public String find(String key) {
log.info("find");
String result = null;
try {
byte[] bytes = db.get(key.getBytes());
if(bytes == null) return null;
result = new String(bytes);
} catch (RocksDBException e) {
log.error("Error retrieving the entry in RocksDB from key: {}, cause: {}, message: {}", key, e.getCause(), e.getMessage());
}
return result;
}
The deletion follows the same pattern and its implementation is straightforward.
@Override
public void delete(String key) {
log.info("delete");
try {
db.delete(key.getBytes());
} catch (RocksDBException e) {
log.error("Error deleting entry in RocksDB, cause: {}, message: {}", e.getCause(), e.getMessage());
}
}
}
Now that we have the basic implementation in place we're good to create an API to complete the example in a way that is more interactive, so let's create simple API using Spring REST, the API simply pass in the received values to the RocksDB implementation exposing it using basic HTTP method calls and handle the results returning 200 or 204 when applicable, please check Spring Boot Crash Course for more details, explanation, and links about building a basic API with SpringBoot if you feel like you need further references:
public class RocksApi {
private final KeyValueRepository<String, String> rocksDB;
public RocksApi(KeyValueRepository<String, String> rocksDB) {
this.rocksDB = rocksDB;
}
@PostMapping("/{key}")
public ResponseEntity<String> save(@PathVariable("key") String key, @RequestBody String value) {
log.info("RocksApi.save");
rocksDB.save(key, value);
return ResponseEntity.ok(value);
}
@GetMapping("/{key}")
public ResponseEntity<String> find(@PathVariable("key") String key) {
log.info("RocksApi.find");
String result = rocksDB.find(key);
if(result == null) return ResponseEntity.noContent().build();
return ResponseEntity.ok(result);
}
@DeleteMapping("/{key}")
public ResponseEntity<String> delete(@PathVariable("key") String key) {
log.info("RocksApi.delete");
rocksDB.delete(key);
return ResponseEntity.ok(key);
}
Now with the application ready you can build the project with maven mvn clean package and then run it mvn spring-boot:run, once the application starts you can test the basic operations using the API we just created using curl, the options used in the curl command below are to print headers and the payload from the requests so you can see a bit better what's going on:
Add entries: curl -v -H "Content-Type: text/plain" -X POST http://localhost:8080/api/rocks/1 -d mypersistedvalue, if you add a new entry with the same key(the 1 in the path) it will automatically override the previous one, notice the key can be any string in this simple implementation, here we're just using 1 as key for the example which is parsed as a String.
Get entries: curl -iv -X GET -H "Content-Type: text/plain" http://localhost:8080/api/rocks/1
Delete entries: curl -X DELETE http://localhost:8080/api/rocks/1
Done! This was a very simple initial reference meant to get you started, if you ever need to use a client-side persistent cache and decide RocksDB is a good option, this is surely just the tip of it, RocksDB gives you many other possible options when you need to customize your cache, you can set filters(like a BloomFilter) you can configure LRU caches and many other options are available, check out their full documentation and references on the Reference section below for further details.
Note: Kafka uses RocksDB by default when doing Streams and using KTables or GlobalKTables.
References
Clone the source code from github rocksdbBootApp : git clone git@github.com:stockgeeks/rocksdbBootApp.git
RocksJava documentation and project
Header Photo by Christopher Gower on Unsplash
Top comments (3)
Nice article!, Hows this compared to Redis?
Greatly simplifying based on their most common usage: Redis is in memory while RocksDB is a Key / value store with persistence based on log-structured merge trees. So picking one of the m really depend on your use case.
Nice article, Thank you very much for this