DEV Community: Fariz Prawira

How redis-packhash Helps You Leverage Listpack Hashes in Redis

Fariz Prawira — Tue, 28 Jul 2026 14:00:00 +0000

In my earlier post, I showed that storing a million small values as Redis keys is a waste of memory. It's because every key has its own overhead. To fix that, we need to group the keys into hashes and keep them small enough that Redis will keep them in the compact listpack encoding. In my benchmark with 1M keys before, a 53.8 MB memory usage went down to 6.9 MB. I call it an 87% discount.

The catch is you have to watch a lot of things. You decide how many hashes to split the keys across, and you need to spread the keys evenly across them. Those are some parts of the bookkeeping that you need to watch in order to get listpack savings.

So I built redis-packhash, a client agnostic, no dependency npm package. You tell it how many keys you expect to store and that's it. It will decide how many hashes to use and spread every key across them evenly, behind a simple get/set.

Install

npm install redis-packhash

No runtime dependencies. It works with ioredis, node-redis, and other Redis clients that expose hset, hget, and hdel.

Quickstart

Bring your Redis client and hand it to PackHash, and everything after that is a simple get/set.

import { PackHash } from "redis-packhash";
import Redis from "ioredis";

// Store each user's locale, one small value per user.
const locales = new PackHash(new Redis(), {
  namespace: "user:locale",
  expectedKeys: 1_000_000,
});

const userId = "9f8c4a2b-1e77-4c3d-bd21-7a0e5f6c8d90";
// Values are strings.
await locales.set(userId, "en-US");

const locale = await locales.get(userId);
//=> "en-US"

That's it. Initialize the store with your expectedKeys, and you are ready to go.
You need another dataset with a different use case? Spin up a new instance. Each one takes an optional namespace that prefixes its Redis top level keys, so they will not collide.

The rest of the API

redis-packhash only accepts string keys. You can pass a UUID, an email, or anything as long as it's string. It runs every key through the FNV-1a hash function and buckets on the result, so the spread stays even whatever your ids look like. Values also need to be string. You pick the serialization, so it never silently reshapes your data.

const logins = new PackHash(redis, {
  namespace: "user:logins",
  expectedKeys: 1_000_000,
});

await logins.set(userId, String(42)); // stringify going in

const raw = await logins.get(userId);
const count = raw ? parseInt(raw, 10) : 0; // and parse coming out => 42

Beyond get and set, you also get has and del, plus mset and mget for batches.

await locales.has("4821"); //=> false

await locales.del("4822"); //=> false

await locales.mset([
  ["4821", "en-US"],
  ["4822", "fr-FR"],
]);

const found = await locales.mget(["4821", "4822", "9999"]);
found.get("4821"); //=> "en-US"
found.get("9999"); //=> null

Sizing is IMPORTANT

The one decision that you make at the very beginning is IMPORTANT. Failed to fill the correct input means you are not save anything. That's the expectedKeys param, which is how many keys you expected to store. Set it too low and buckets overflow and the encoding fallback to a hashtable, which means losing the saving entirely. Set it too high and you get more buckets than you need, which means cost you a little extra overhead. When in doubt, estimate high.

const locales = new PackHash(redis, {
  namespace: "user:locale",
  expectedKeys: 1_000_000,
});

locales.buckets; //=> 2605 total buckets

WARNING: changing expectedKeys on a store that already holds data will silently orphans every existing data. You either can migrate it manually before changing it, or spin a new instance if match your case.

Limits that you need to honor

Listpack has two limits, and it will fallback from listpack to hashtable the moment it crosses either one:

Setting (Redis 7.0+)	Redis default	Description
`hash-max-listpack-entries`	512	Number of fields in the hash
`hash-max-listpack-value`	64 bytes	Length of any single field name or value

expectedKeys handles the first limit for you by compute the bucket count needed to achieve below the default 512. But the second limit is on you, at least for now.
The 64 bytes limit applies to the field name and the value separately, not shared between them, and your top level key is not included.

// Case #1: Value over 64 byte
const key = "9f8c4a2b-1e77-4c3d-bd21-7a0e5f6c8d90";
const value = JSON.stringify({
  locale: "en-US",
  timezone: "America/New_York",
  dateFormat: "MM/DD/YYYY",
});

await locales.set(key, value);
// HSET user:locale:2312  "<36-byte key>"  <74-byte value>
// bucket user:locale:2312 falls back to hashtable

// Case #2: Key over 64 byte
const key = "tenant:acme-corp/region:eu-west-1/user:9f8c4a2b-1e77-4c3d-bd21-7a0e5f6c8d90";
const value = "en-US";
await locales.set(key, value );
// HSET user:locale:85  "<75-byte key>"  "<5-byte value>"
// bucket user:locale:85 falls back to hashtable

The entry limit, and your Redis version

Rather than assuming your limits, just ask your own server:

127.0.0.1:6379> CONFIG GET hash-max-listpack-*
1) "hash-max-listpack-entries"
2) "512"
3) "hash-max-listpack-value"
4) "64"

Before Redis 7.0 it was hash-max-ziplist-* and the encoding was ziplist.

If you change your default hash-max-listpack-entries or it has different value in your server for whatever reason, you can set it manually using maxListpackEntries option.

const locales = new PackHash(redis, {
  namespace: "user:locale",
  expectedKeys: 1_000_000,
  maxListpackEntries: 256, // match your config
});

That is the whole thing

For now, that's what this library supports. I've some things in mind to improve it, and if do too, I'm more than happy to hear them.

Cut your Redis memory usage by up to 90%

Fariz Prawira — Wed, 24 Jun 2026 06:07:24 +0000

If you treat your Redis as a bag of single String keys, I have great news for you. By utilizing listpack, you can cut your Redis memory usage by up to 90% without dropping a single byte of your actual data. Let's assume your key is look like this user:1203:blocked 1, one key per user blocked status. It works, but most of your stored data in the memory is not your assigned data. There is plenty of overhead in a single key.

Why all those keys are expensive

When you store your data by running SET user:1203:blocked 1, it feels like you are storing 1 byte of data. But that's not true. In fact, most of the stored data is not yours, it's from Redis itself.

Here's what a single key really cost you:

Redis wrap your value using redisObject, which is about 16 bytes before your actual data.
They key itself is not your typical String. It has a small header of its own.
Every key will lives in Redis's main dict entry, that mean each key needs to reserve more memory there (roughly 24 bytes).
All of those above will also get rounds up by the memory allocator to a fixed size.

One key is harmless, but remember that your keys are expanding as your users grow.

How to save the memory

I assume your keys are like these:

SET user:10001:blocked 1
SET user:15089:blocked 1
...
SET user:10201:external_id AI27X
SET user:10590:external_id JO91P
...

One key per use case per user. With a million of users, that is a million keys that have per-key overhead.

To reduce the number of keys that live in Redis's main dict entry, we can group them into hashes and bucket them by ID range, so the hash stays small:

HSET user:blocked:1 0001 1
HSET user:blocked:1 5089 1
...
HSET user:external_id:1 0201 AI27X
HSET user:external_id:1 0590 JO91P
...

The bucket is the user ID divided by 1000, so one hash hold thousand users. To read a user you compute the bucket and use HGET like usual:

HGET user:blocked:1 0001

Same data, but a million top level keys collapse into thousand.

The proof

Don't put your faith on theory alone, let's test it your self.

First run, one key per user (baseline):

redis-cli FLUSHALL
redis-cli INFO memory | grep "used_memory:"
seq 1 1000000 | awk '{v=($1%7==0); print "SET user:"$1":blocked " v}' | redis-cli --pipe
redis-cli INFO memory | grep "used_memory:"

Result:

Second run, 1000 users per bucketed hash using hashtable:

redis-cli FLUSHALL
redis-cli CONFIG SET hash-max-listpack-entries 1
redis-cli INFO memory | grep "used_memory:"
seq 1 1000000 | awk '{v=($1%7==0); b=int(($1-1)/1000); print "HSET user:blocked:"b" "$1" "v}' | redis-cli --pipe
redis-cli INFO memory | grep "used_memory:"
redis-cli OBJECT ENCODING user:blocked:0

Result:

Third run, 1000 users per bucketed hash using listpack:

redis-cli FLUSHALL
redis-cli CONFIG SET hash-max-listpack-entries 1024
redis-cli INFO memory | grep "used_memory:"
seq 1 1000000 | awk '{v=($1%7==0); b=int(($1-1)/1000); print "HSET user:blocked:"b" "$1" "v}' | redis-cli --pipe
redis-cli INFO memory | grep "used_memory:"
redis-cli OBJECT ENCODING user:blocked:0

Result:

Layout	Encoding	Used memory	Savings vs baseline
1M individual keys	`string`	53.8 MB	—
1M in 1000-field hashes	`hashtable`	46.1 MB	≈ 14%
1M in 1000-field hashes	`listpack`	6.9 MB	≈ 87%

That's the up to 90% we talked about.

What makes the hash cheap

It's obviously because we eliminate the per value redisObject. We also merge a repeated key. user:10001:blocked shrinks into hash key 0001, and the user:blocked:1 become bucket key, paid once per thousand.
But that's not mean just because we eliminate small portion of overhead, it suddenly cuts the memory significantly. It not vanish, it relocates. The hash also has its own overhead. Now you need to pay for a dictEntry on the hash, SDS for the field and the value, and also lose the shared integer trick on the value. Sum it up and it's close to a zero. That is why the hashtable only saved us ≈ 14%.
Then comes the listpack to the spotlight. The per-field dictEntry cost is gone, the bucket array cost is gone, the SDS cost for every field and every value is replace by a compact one. Now most of the overheads are gone and you can free most of your memory.

The catch

Listpack is an optimization Redis applies on hash when the hash stays small, and it goes away the moment your hash grow past either one of two limits: hash-max-listpack-entries (default 512) and hash-max-listpack-value (default 64). The first limits how many fields a hash can hold, and the second limit the size of the value or the field (it has separate limit for both not sum).
Cross either one and Redis will turn your listpack into hashtable without warning. Even from a single value that is one byte too long, your hash will flip. The change is also one way, meaning that if you manage to update your value to under the limit, it will not go back to listpack.

The tradeoffs

There is no freebie, only tradeoffs:

Read and write to a listpack is linier O(n), that's why we need to keep it small.
Separate key can have separate TTL, but when you merge them into hash, there is no per field TTL. Redis 7.4 added per field TTL (using HEXPIRE), but they carry their own memory cost and turn your listpack into listpackEx (at least not hashtable).
More complex code to manage.

Listpack is not a new tool you need to install, not a dependency to add, not even a switch to flip. It is already there, built in your Redis. You just need to tune your code to utilize it. Find the cases that fit and your memory usage drops several fold. Lower memory means lower bills.