What is the best distributed key-value database? I want to choose a database by user experience, not by the website that thinks their database is the best.
The database must meet these requirements:
- Easy to setup
- A HTTP API or a NodeJS / Javascript driver
- Scalable to hundreds of TeraBytes
Top comments (1)
If you truly need to scale to hundreds of terabytes, you can use any S3-like cloud object storage provider (actual S3, Google Cloud Storage, DigitalOcean Spaces, etc) as a KV store with virtually unlimited scale.
If you want something you can run locally: NATS KV store. NodeJS driver. Running the server only requires two commands:
brew install nats-serverandnats-server -js. Caveat: values must be under 1MB each in the server's default config or if you plan on using it with a global cluster like NGS.NATS KV store was modeled after
etcdwhich, last I checked, does not meet your "easy to setup" criterion.If you don't actually need hundreds of terabytes (and I'd be curious why you think you do), you can use Redis. It's the single most popular KV store in existence, there's a driver for nearly every language, and it can still theoretically scale to hundreds of terabytes if your keys are evenly distributed. It theoretically supports up to 16384 nodes in a single cluster, so if you have that many nodes each with 512GB of RAM, you can store roughly 8 PB of data in it.