DEV Community: HotKV

Redis benchmark: HotKV vs Valkey and Dragonfly in a three-node cluster

HotKV Ltd — Thu, 05 Nov 2026 19:00:00 +0000

This Redis benchmark runs HotKV, Redis, Valkey and Dragonfly as three-node clusters on AWS. Our earlier benchmarks ran each server as one instance on a whole machine. That layout suits HotKV, which uses every core in one process. Redis and Valkey run commands on one thread per process, and they are commonly scaled across cores with many processes, usually as a cluster. So we measured every product the way a cluster deploys it.

In short. In this Redis benchmark on three AWS servers, HotKV had the highest throughput on 8 of 9 commands: 92.3 million GETs per second against 57.3 million for Redis, 52.9 million for Valkey and 22.5 million for Dragonfly. Redis and Valkey were ahead on APPEND. The load generators were near their limit on some commands, so the figures are lower bounds.

Why we ran this Redis benchmark as a cluster

A single-instance benchmark, like the one in our launch article, asks how much one server process can do with a whole machine. That is a reasonable question for HotKV and Dragonfly, which use all cores in one process. It says less about Redis and Valkey. Their commands run on one thread per process (I/O threads can offload network work), so a 48-vCPU server is normally used by running dozens of processes.

We do not want our numbers to depend on a layout that suits one architecture better than another. A cluster is common ground. All four products can run as one, and for Redis and Valkey it is a usual way to use many cores. We ran this benchmark to see how the comparison looks when each product is deployed that way.

What we tested

Every product ran as a cluster on the same three servers, one product at a time, driven by the same two load generators. The sections below give the machines, each layout and the procedure.

Machines

Servers. Three c8in.12xlarge instances (Intel Xeon 6975P-C, 48 vCPUs, 75 Gbit/s network) in one AWS cluster placement group in eu-south-2a, running Ubuntu 24.04. Every server process was pinned to CPUs 0 to 39.
Load generators. One c8in.48xlarge running 16 memtier_benchmark processes of 12 threads each, and one c8in.12xlarge running 4 processes of 12 threads. They were started together and the operations per second of all processes were added up.
Tuning. All five machines got the same host tuning: the repository's tune_server.sh, receive packet steering over all CPUs, the network card's receive ring at its maximum, a larger softirq budget and a listen backlog of 65,535.

Servers and layout

HotKV. One process per server with 40 threads, in cluster mode (3 primaries). The build is 0.3.0 plus a change that adds the pipeline-prefetch setting, and it ran with pipeline-prefetch yes. The setting is off by default. It looks ahead in a client's pipeline and asks the CPU for the memory the next commands will read.
Redis 8.10.3 and Valkey 9.1.3. 40 single-thread processes per server, so 120 primaries, in cluster mode. Each process had a core to itself. Their own batch prefetching was left at its default.
Dragonfly 2.0.2. One process per server with 40 threads, in cluster mode, built from source.
Persistence. Off on every server.

Workload and procedure

The data set was 5,000,000 keys with 256-byte values, and pipeline depth was 100. memtier_benchmark ran in cluster mode, so each client talks to the right node. Each command ran for 15 seconds, twice, and we report the median of the two. The data set was loaded again, identically, before every run, so a command never ran on top of the previous command's changes. The scripts and every run are in the benchmark repository.

Results

Millions of operations per second, median of two runs. The highest value in each row is in bold.

Command	HotKV	Redis 8.10.3	Valkey 9.1.3	Dragonfly 2.0.2	HotKV vs Redis	HotKV vs Valkey
GET	92.3	57.3	52.9	22.5	1.61x	1.74x
SET	76.1	55.0	48.5	25.2	1.38x	1.57x
SETNX	86.1	60.7	57.0	27.5	1.42x	1.51x
INCR	119.2	118.4	104.6	30.8	1.01x	1.14x
SADD	51.7	40.4	33.8	22.7	1.28x	1.53x
RPUSH	47.1	29.3	29.0	18.7	1.61x	1.62x
HSET	38.4	28.8	27.2	18.4	1.33x	1.41x
LPUSH	31.1	23.7	24.0	16.9	1.31x	1.30x
APPEND	35.0	41.7	40.9	10.4	0.84x	0.86x

Against Dragonfly, HotKV was ahead on all nine commands, from 1.85 times (LPUSH) to 4.09 times (GET).

How to read the ratios

A throughput figure from a load test is a floor when the load generators are the limit. If a load generator is at 100 percent CPU, the server may have been able to take more. The figure then shows only that it took at least this much. A ratio between two floors is not exact, because either side could be higher.

We flag every figure where the main load generator ran above 85 percent CPU. These were GET, SETNX and INCR for HotKV, and GET, SET, SETNX, INCR, SADD, HSET and APPEND for Redis and Valkey. On those figures, read the ratio as a rough size and not a precise one.

On SADD and HSET, HotKV's figure is not flagged but Redis's and Valkey's are. Their true throughput may be higher, and HotKV's lead smaller than shown. The same applies to APPEND, where it would make their lead over HotKV larger than shown.

RPUSH and LPUSH have no flag for any server, so they give the cleanest comparison. HotKV was 1.3 to 1.6 times ahead of Redis and Valkey on them.

What the results show

Three things stand out in the table, and the first is about Redis and Valkey rather than HotKV.

Redis and Valkey scale well as clusters

With 120 single-thread processes, Redis served 57 million GETs per second and 118 million INCRs per second across three servers. That confirms the cluster layout suits them. It is also why HotKV's margin here is smaller than in our single-instance runs.

Those runs compared a design that uses all cores in one process with products that were not run in the layout they are commonly scaled in. This Redis benchmark gives Redis and Valkey that layout. HotKV's lead over them narrows to between 1.0 and 1.7 times on eight commands.

HotKV had the highest throughput on 8 of 9 commands

The lead was widest on GET and RPUSH against Redis (1.6 times) and on GET against Valkey (1.7 times). On RPUSH and LPUSH the load generators had room for every server, so those figures are limited by the servers. On INCR, HotKV and Redis were level within 1 percent, and the load generators, not the servers, set the limit for all three.

Redis and Valkey were ahead on APPEND

HotKV served 35.0 million APPENDs per second against 41.7 million for Redis and 40.9 million for Valkey. We have not found the reason. APPEND grows the dataset by about 10 GB per second across the cluster, so the run is also a test of memory allocation. A 30-second APPEND run ran the servers out of memory.

We profiled HotKV on APPEND and saw page faults, TLB shootdowns and lock contention. We have not shown which of them accounts for the gap.

Limitations

These are HotKV Ltd's measurements of the versions and configuration above. They are not a claim about any other version, configuration or workload of these products.

The test setup

Load generators near their limit. Many figures are lower bounds, as the section on reading the ratios explains. HotKV's APPEND figure is not one of them, because the load generators were at about 30 percent CPU with it.
Layouts we chose. We ran Redis and Valkey as one single-thread process per core and did not tune either product beyond that. Other layouts or tuning could change their figures. If you think a setting here disadvantages a product, tell us and we will check it and publish the result.
Two runs, one region. Each figure is the median of two 15-second runs on one set of machines in one region. We did not measure variation between machines or days.

What was not measured

Latency. We measured throughput only here. A separate Redis latency benchmark measures latency.
Persistence, replication and failover. All servers ran with persistence off and no replicas.
Other setups. Three servers and one workload shape. A larger cluster, other value sizes or other pipeline depths can change the ratios.

Redis, Valkey and Dragonfly are trademarks of their respective owners. They are named only to identify the software we measured, and this implies no sponsorship, endorsement or affiliation.

Reproduce it

The sweep script, the exact command sequence, the host tuning and every raw result are in the benchmark repository. The sweep script is harness/cluster_bench_sweep.sh. A chart of the same figures is on the performance page.

Try it yourself

HotKV Cloud runs HotKV clusters for you. Start on HotKV Cloud, or talk to sales to benchmark a cluster against your own workload.

Frequently asked questions

Why run a Redis benchmark as a cluster?

Redis and Valkey run commands on one thread per process, so a single process cannot use a large machine. They are commonly scaled across cores by running many processes as a cluster. A cluster benchmark measures each product in a layout like that, so the comparison does not rest on a single-instance deployment alone.

Did HotKV have the highest throughput in cluster mode?

On 8 of 9 commands, yes. HotKV led Redis by 1.0 to 1.6 times and Valkey by 1.1 to 1.7 times, and Dragonfly by 1.85 to 4.09 times. Redis and Valkey were ahead on APPEND. On INCR, HotKV and Redis were level within 1 percent, and the load generators set the limit.

Why were Redis and Valkey ahead on APPEND?

We do not know yet. APPEND grows the dataset by about 10 GB per second across the cluster, so the run also tests memory allocation. A profile of HotKV showed page faults, TLB shootdowns and lock contention, but we have not shown which of them accounts for the gap.

What is the pipeline-prefetch setting?

It makes HotKV look ahead in a client's pipeline and ask the CPU for the memory the next commands will read, so those reads overlap. It changes no reply. It is off by default, and HotKV ran with it on in this benchmark.

Can I reproduce this Redis benchmark?

Yes. The sweep script, the exact command sequence, the host tuning and every raw result are in the benchmark repository, and the same figures are charted on the performance page. You need three servers, one or more load generators and the four products installed by the harness.

Does this replace the single-instance results?

No. The two answer different questions. A single-instance run shows what one server process does with a whole machine. A cluster run shows what three machines do when each product is laid out as a cluster. Both are on the performance page.

Redis latency benchmark: HotKV vs Valkey and Dragonfly

HotKV Ltd — Thu, 05 Nov 2026 07:00:00 +0000

This Redis latency benchmark measures how long HotKV, Redis, Valkey and Dragonfly take to answer at a fixed request rate. Our earlier benchmarks measured throughput, and a reader rightly pointed out that throughput is not everything. So we held the load constant, raised it step by step, and recorded the median (p50) and the 99th percentile (p99) at each step. We also measured pub/sub delivery time.

In short. In this Redis latency benchmark on AWS, HotKV and Dragonfly both held 1.6 million GETs per second, Redis with 8 I/O threads held about 0.8 million, Valkey with 8 I/O threads about 0.4 million, and single-thread Redis and Valkey about 0.2 million. From 200,000 to 1.6 million requests per second HotKV's p99 stayed between 0.59 and 0.76 ms, against 0.71 to 1.48 ms for Dragonfly. At light load the multi-threaded products tied on p99, and pub/sub delivery was a tie.

Why we ran a Redis latency benchmark at a fixed rate

A throughput test pushes a server as hard as it will go and reports how much it served. That says little about how long one request waits. A latency test fixes how many requests per second the clients send, and measures the time each takes. The server is then at the same load as every other product, so the times can be compared.

This kind of test has a catch. memtier_benchmark is a closed-loop tool, so a client waits for a reply before it sends the next request. If a server cannot keep up, the client simply sends fewer requests, and the latency of the ones it did send looks better than it should. We therefore count a step as sustained only when the achieved rate is at least 97 percent of the rate we asked for. Steps that fail the test are marked "not sustained" in the tables and their latency is not shown.

What we tested

Machines

Servers and load generator. c8a.12xlarge instances (AMD EPYC 9R45, 48 cores, no SMT, 96 GiB, 22.5 Gbit/s network) in one AWS cluster placement group in eu-south-2a, running Ubuntu 24.04. One instance ran the load generator and one ran the server.
One product at a time. Each product ran alone on the server, so nothing overlapped and nothing shared the network. Every server process was pinned to CPUs 0 to 39, with persistence off and protected mode off.

Products

HotKV 0.3.0 with 40 threads.
Dragonfly 2.0.2 with 40 proactor threads.
Redis 8.10.3 and Valkey 9.1.3, each run twice, once in the default single-thread setup and once with --io-threads 8.

The launch line and the live CONFIG GET output of every run are saved with its results.

Workload and procedure

The data set was 1,000,000 keys with 256-byte values, and pipeline depth was 1, so each request waits for its own reply. The load was GET only, and a mix of 1 write to 10 reads. Each step ran for 20 seconds, twice, and we report the mean of the two runs. memtier_benchmark used a per-connection rate limit and recorded an HDR histogram. We ran two connection counts.

816 connections (24 threads of 34 connections), for rates from 20,000 to 2.4 million requests per second.
48 connections (4 threads of 12 connections), for rates up to 300,000 requests per second. Above that, 48 connections cannot reach the rate, because each connection can only send as fast as it gets replies.

Results

The GET tables come first, one for each connection count, followed by pub/sub.

At 816 connections

Each cell is p50 / p99 in milliseconds, for GET. "NS" means not sustained, and the rate it did reach is in brackets. A dash means the sweep for that product had already stopped.

Requests/s	HotKV	Dragonfly	Redis 1 thread	Redis 8 I/O threads	Valkey 1 thread	Valkey 8 I/O threads
19,584	0.38 / 1.81	0.48 / 1.90	1.63 / 5.29	0.68 / 2.07	1.59 / 5.13	0.76 / 2.63
99,552	0.24 / 0.76	0.27 / 1.76	2.60 / 5.12	0.40 / 0.98	2.65 / 5.13	1.71 / 3.19
199,920	0.25 / 0.59	0.25 / 1.34	2.91 / 5.60	0.55 / 1.11	3.02 / 5.81	1.11 / 2.15
399,840	0.27 / 0.63	0.27 / 1.48	NS (258k)	0.62 / 1.23	NS (259k)	1.24 / 1.49
799,680	0.28 / 0.75	0.28 / 1.26	-	0.68 / 1.18	-	NS (670k)
1,199,520	0.28 / 0.67	0.28 / 1.05	-	NS (1,078k)	-	-
1,599,360	0.28 / 0.64	0.28 / 0.71	-	-	-	-
2,399,856	NS (2,284k)	NS (2,281k)	-	-	-	-

The mixed workload gave the same picture. HotKV's p99 was 0.54 to 0.75 ms from 200,000 to 1.6 million requests per second, and the points each product could sustain were the same as for GET. All of it is in the results folder.

How to read the single-thread rows

The 1.6 to 3 ms medians of single-thread Redis and Valkey at 816 connections are not their steady-state latency. With a rate limit, memtier_benchmark makes all 816 connections fire at the same moment. A single-thread server has to work through that burst of 816 requests one after another, so the median of a request includes its place in the queue. We ran the 48-connection test below to remove this effect.

At 48 connections

The cells have the same meaning as above, for GET only.

Requests/s	HotKV	Dragonfly	Redis 1 thread	Redis 8 I/O threads	Valkey 1 thread	Valkey 8 I/O threads
9,984	0.14 / 0.54	0.14 / 0.37	0.21 / 0.89	0.17 / 0.92	0.21 / 0.91	0.20 / 0.81
49,968	0.10 / 0.28	0.11 / 0.28	0.21 / 0.41	0.13 / 0.28	0.21 / 0.43	0.21 / 0.44
99,984	0.09 / 0.28	0.10 / 0.28	0.20 / 0.40	0.13 / 0.28	0.20 / 0.38	0.19 / 0.35
199,968	0.08 / 0.27	0.08 / 0.27	0.20 / 0.35	0.10 / 0.27	0.21 / 0.36	0.11 / 0.28
300,000	0.08 / 0.27	0.08 / 0.27	NS (231k)	0.09 / 0.26	NS (233k)	0.09 / 0.26

Pub/sub delivery time

We measured the time from a publish to its delivery to a subscriber, with 64 publishers, 100 subscribers and a 128-byte payload, using pubsub-sub-bench. Each message went to one subscriber. Cells are p50 / p99 in milliseconds.

Deliveries/s	HotKV	Dragonfly	Redis 1 thread	Redis 8 I/O threads	Valkey 1 thread	Valkey 8 I/O threads
about 21,000	0.10 / 0.71	0.10 / 0.65	0.11 / 0.74	0.11 / 0.62	0.11 / 0.76	0.12 / 0.78
about 107,000	0.10 / 1.02	0.12 / 0.92	0.10 / 0.80	0.11 / 0.63	0.10 / 0.82	0.10 / 0.86
about 215,000	0.11 / 1.20	0.13 / 1.13	0.51 / 1.03	0.20 / 0.73	0.57 / 1.07	0.12 / 1.22

All products delivered in the same range. At the highest load the single-thread products had a higher median, 0.51 and 0.57 ms against 0.11 ms for HotKV.

We also had to patch pubsub-sub-bench. It compared a publish time in milliseconds with a receive time in microseconds. The patch is in the results folder.

What the results show

The tables show a lead for HotKV and Dragonfly at high load, a tie at light load, and a ceiling for single-thread Redis and Valkey.

HotKV and Dragonfly kept a low tail to 1.6 million requests per second

Both sustained 1.6 million GETs per second at 816 connections, with a median of 0.28 ms. HotKV's p99 was between 0.59 and 0.76 ms from 200,000 requests per second up. Dragonfly's was between 0.71 and 1.48 ms, and the two were close at 1.6 million (0.64 against 0.71 ms). Redis and Valkey with I/O threads stopped sustaining the rate at lower loads, and single-thread Redis and Valkey stopped at about 260,000.

At light load the multi-threaded products tie

At 48 connections, up to 300,000 requests per second, HotKV, Dragonfly and Redis with 8 I/O threads all had a p99 of about 0.27 ms. HotKV had the lowest median, 0.08 to 0.09 ms from 100,000 requests per second, with Dragonfly at 0.08 to 0.10 ms. A difference of 0.01 ms is within what other parts of the setup can cause, so we do not claim it as a win.

Single-thread Redis and Valkey ran out at about 230,000 a second

At 48 connections they had a median of about 0.2 ms and a p99 of 0.35 to 0.43 ms, and they did not sustain 300,000 requests per second (about 231,000 and 233,000). That is the ceiling of one thread on this hardware and workload. It is why Redis and Valkey are usually run as many processes or as a cluster, as in our cluster benchmark.

Limitations

These are HotKV Ltd's measurements of the versions and configuration above. They are not a claim about any other version, configuration or workload of these products.

The test setup

One server, one load generator. Two repeats per step, on one set of machines on one date. We did not measure variation between machines or days.
Layouts we chose. Redis and Valkey ran in their default single-thread setup and with --io-threads 8, and we did not tune either beyond that. Other settings could change their figures. If you think a setting here disadvantages a product, tell us and we will check it and publish the result.
The tool's burst effect. At 816 connections, the medians of single-thread products are inflated by the way memtier_benchmark rate limiting works, as explained above.

What was not measured

Data. Larger or skewed values, memory per key, and persistence. Persistence was off for every server.
Modes. Cluster mode, sharded pub/sub, TLS, replication and failover.
Pub/sub fan-out. One message to many subscribers.

Redis, Valkey and Dragonfly are trademarks of their respective owners. They are named only to identify the software we measured, and this implies no sponsorship, endorsement or affiliation.

Reproduce it

The sweep scripts, the launch line and live configuration of every server, the pubsub-sub-bench patch and every raw result are in the benchmark repository. The latency script is harness/lat_sweep.sh, and the pub/sub script is harness/ps_sweep.sh. The raw folder holds one archive per test with the memtier_benchmark JSON, the HDR histograms and the CPU use of the server for every run.

Try it yourself

HotKV Cloud runs HotKV for you. Start on HotKV Cloud, or talk to sales to benchmark HotKV against your own workload.

Frequently asked questions

Is HotKV faster than Redis in latency?

Under load, at the rates we tested, yes. At 816 connections and 200,000 requests per second HotKV had a p99 of 0.59 ms, Redis with 8 I/O threads 1.11 ms and single-thread Redis 5.60 ms. At light load with 48 connections the multi-threaded products tied on p99, at about 0.27 ms.

Why did you hold the request rate fixed?

At a fixed rate every product carries the same load, so the latencies can be compared. A throughput test shows how much a server serves at its limit, which says little about how long one request waits.

What does "not sustained" mean?

The product achieved less than 97 percent of the request rate we asked for. memtier_benchmark is a closed-loop tool, so an overloaded server makes the clients send less and the latency of the remaining requests looks better than it is. We show no latency for those steps.

How did pub/sub delivery time compare?

At the three loads we tested, all products delivered in the same range, with a median of about 0.1 ms at the two lower loads. The table above has every figure.

Can I reproduce this Redis latency benchmark?

Yes. The scripts, the configurations and every raw result are in the benchmark repository. You need two servers on the same network, a load generator, and the four products.

Vector search: HotKV vs Redis 8 vector sets, from 1 to 100 clients

HotKV Ltd — Thu, 05 Nov 2026 01:39:46 +0000

Vector search sits underneath many AI features: retrieval for RAG, semantic caching, agent memory. Redis 8 ships vector sets, a beta data type, built into the server, so it is the obvious comparison. We loaded the same OpenAI embeddings into HotKV and into Redis 8.10.2, ran the same 1,000 queries against each with 1 to 100 concurrent client connections, and measured how accurate the answers were, how many queries per second each engine served, and how long each query took.

In short. On 999,000 vectors, at a minimum recall of 95%, HotKV served 1.4 times the queries per second of Redis with one client and 7.9 times with 100 clients (51,146 against 6,510). At 99% recall the ratios were 1.8 and 5.4. HotKV's median latency was lower at every client count. Redis's throughput levelled off by 32 clients, while HotKV's kept growing to 100. HotKV loaded the vectors in 188 s against 935 s for Redis, which was pinned to four vCPUs for its load, and its recall was higher at every search depth. This is one dataset on one machine, the HotKV build is an unreleased development build, and we did not measure Redis's Query Engine or its 8-bit mode, so read it as a measurement, not a ranking.

What we tested

Data

The first 999,000 rows of dbpedia-entities-openai-1M by Kumar Shivendu and Nirant Kasliwal (MIT license), and, for a smaller index, the first 100,000 rows. The rows are DBpedia entities embedded with OpenAI text-embedding-ada-002, 1,536 dimensions. The last 1,000 rows are the queries, so no row appears in both sets. The answer key is the exact top 10 nearest neighbours of every query, computed by brute force in 64-bit floating point independently of either engine, so both are graded against the same ground truth.

Engines and settings

Engines. HotKV commit c8dafd37 (VSIM.CREATEINDEX, VSIM.SET, VSIM.SEARCH), an unreleased development build compiled with default release flags, not -march=native. Redis 8.10.2 built from its release tag with a stock make (VADD, VSIM). Both build HNSW graphs.
Index settings. Cosine similarity, HNSW with M=16 and ef_construction=200 on both, set explicitly. They are also each engine's own defaults.
Precision. Full-precision float32 vectors on both. Redis defaults to 8-bit quantization, so it ran with NOQUANT.
Machine. One Azure Standard_D160alds_v7 (AMD EPYC 9V45, 160 vCPUs). Both servers were pinned to the same 128 vCPUs, the load generators to the other 32, all over loopback. HotKV ran 128 threads and Redis its default I/O threads. We tried 1, 4, 8 and 16 and the setting made no consistent difference.
Load generator. One process per client connection, each keeping one request outstanding, with the requests built before the timed part. The busiest load-generator process used 8% of one CPU in any run, so the client never set the limit.

Procedure

For search depths (ef) of 40, 80, 150 and 300, and 500 on the larger index, we ran the 1,000 queries repeatedly with 1, 2, 4, 8, 16, 32, 50 and 100 clients: one second of warm-up, then four seconds counted, three times each, with the order of the two engines alternating. We report the median. Recall was measured separately, over one connection. HotKV was loaded over 32 connections in every run. The Redis indexes were built once, over 32 connections with VADD ... CAS, saved, and loaded for the measurements. The scripts and every trial are in the benchmark repository.

Results

Matched recall targets

The two engines do not reach the same recall at the same ef, so we compare throughput at shared minimum recall targets: for each target, the fastest measured search depth that reaches it. The depths and recalls reached are in the tables, because they differ a little. This uses measured depths, not interpolation. Queries per second on 999,000 vectors:

Recall@10 target	Clients	HotKV	Redis 8.10.2	HotKV vs Redis	Median latency
95% or better	1	1,328 (ef 80, 0.970)	946 (ef 80, 0.962)	1.40x	0.77 / 1.07 ms
95% or better	8	10,342	5,531	1.87x	0.79 / 1.43 ms
95% or better	16	18,500	7,265	2.55x	0.87 / 2.15 ms
95% or better	50	37,508	6,825	5.50x	1.33 / 7.17 ms
95% or better	100	51,146	6,510	7.86x	1.91 / 13.49 ms
99% or better	1	507 (ef 300, 0.993)	287 (ef 500, 0.994)	1.77x	1.99 / 3.31 ms
99% or better	16	6,961	2,322	3.00x	2.30 / 6.70 ms
99% or better	50	12,393	2,483	4.99x	4.02 / 18.91 ms
99% or better	100	13,843	2,583	5.36x	7.11 / 37.65 ms

Median latency is HotKV / Redis. The same pattern held for a 90% target: 90,003 against 7,973 queries per second at 100 clients (11.3 times), both at ef 40.

Every search depth

ef	HotKV recall	Redis recall	HotKV, 1 client	Redis, 1 client	HotKV, 50 clients	Redis, 50 clients	HotKV, 100 clients	Redis, 100 clients
40	0.940	0.923	2,108	1,259	63,897	8,061	90,003	7,973
80	0.970	0.962	1,328	946	37,508	6,825	51,146	6,510
150	0.987	0.980	875	636	21,912	5,241	26,122	4,942
300	0.993	0.990	507	396	12,393	3,454	13,843	3,402
500	0.997	0.994	338	287	7,688	2,483	8,843	2,583

Queries per second on 999,000 vectors. On 100,000 vectors the pattern is the same. At 95% recall HotKV served 1.5 times Redis's queries per second with one client (1,816 against 1,214) and 8.9 times with 100 (63,247 against 7,119).

How much of this is noise

Each cell is the median of three trials. Half of the cells had a spread between their lowest and highest trial of 1.3% of the median or less, nine in ten 7% or less, and the widest was 26% (Redis, 100,000 vectors, ef 300, 50 clients). The report lists every trial.

Recall was measured once per index build. Three independent builds of each engine on 100,000 vectors differed by up to 0.013 at the same depth, so HotKV's higher recall is clear up to ef 300 and within build-to-build variation at ef 500 on the smaller index.

Loading and memory

Loaded over 32 connections each, HotKV took 188 seconds for the 999,000 vectors (5,325 per second) and Redis 935 seconds (1,069 per second), including the time the load generator spent formatting each vector. On 100,000 vectors it was 15 seconds against 57. Redis loaded faster with CAS than without in a check on 20,000 vectors (2,279 against 1,488 vectors per second), so we used it. The Redis load ran in an earlier step with Redis pinned to four vCPUs while other checks ran on the machine; HotKV loaded alone on 128 vCPUs. Treat the Redis loading time as a lower bound on its speed, not a like-for-like comparison.

After loading 999,000 vectors the HotKV process held 6.77 GB and the Redis process 7.01 GB.

What limits Redis 8.10.2 here

Redis vector sets run each search on its own thread, but the HNSW code has a fixed number of thread slots (32, set when Redis is compiled), and vectors are added on the main thread. In our runs, from 32 clients up, Redis kept between 6 and 30 of the machine's vCPUs busy and its throughput had levelled off. HotKV kept about 92 vCPUs busy at 100 clients on ef 80 and its throughput was still growing. That is how each engine behaves on this machine, not something the benchmark configured.

Limitations

These are HotKV Ltd's measurements of the versions and configuration above, not a claim about every Redis deployment.

The test setup

A development build. Commit c8dafd37 is a development build, not a release.
One dataset, one machine. 1,536-dimension OpenAI embeddings, top 10 only, float32, cosine. Other dimensions, data or hardware can give different ratios.
The client shares the machine. The load generators ran on separate vCPUs but shared the memory system and the last-level cache with the servers.
Persistence off. Neither server wrote snapshots or an append-only file.

What was not measured

Redis's 8-bit mode. Redis stores 8-bit quantized vectors by default, which uses less memory. We compared float32 against float32.
The Redis Query Engine. Redis also offers vector search through FT.CREATE and FT.SEARCH. We tested vector sets only, so the results say nothing about the Query Engine.
Redis loading and reload. The Redis indexes were built in an earlier run with Redis pinned to four vCPUs, saved, and reloaded for the measurements, so the Redis load times are a lower bound and are not comparable like for like with HotKV's. Searching a reloaded index was about 10 percent slower than a freshly built one in a check on 20,000 vectors, so Redis's query figures may be low by about that much.

Try it yourself

Vector search is built into the same server as HotKV's prompt cache, and the same vector index backs its semantic cache and agent memory. Vector search is included on every HotKV Cloud plan. The prompt cache, semantic cache and agent memory come with the Performance plan and above. Start on HotKV Cloud, or talk to sales to benchmark it against your own embeddings.