DEV Community: JeongSeop Byeon

Stop Defaulting to LRU: How I Built a 15.5M ops/sec S3-FIFO Cache for Node.js by Hacking V8

JeongSeop Byeon — Wed, 29 Jul 2026 09:15:16 +0000

Whenever we need an in-memory cache in Node.js, 99% of us do the exact same thing: npm install lru-cache. It’s muscle memory. LRU (Least Recently Used) has been the industry standard for decades, and for good reason—it’s intuitive and it works.
But if you are running a high-throughput backend, relying on LRU might be quietly bottlenecking your performance and degrading your cache hit ratios.
Recently, I went down a rabbit hole after reading the acclaimed 2023 SOSP paper: "FIFO queues are all you need for cache eviction". The paper proves that a new algorithm called S3-FIFO (using three simple FIFO queues) completely outperforms complex LRU/LFU hybrids. It solves LRU's biggest flaw: Cache Pollution caused by "one-hit wonders" (like a massive DB scan or web crawler traffic knocking out all your hot data).
High-performance storage engines and CDNs outside JavaScript have already adopted S3-FIFO. But the Node.js ecosystem was strangely quiet because we lacked a production-ready, ultra-optimized implementation.
So, I decided to build s3fifo. But I didn't just want to port the algorithm—I wanted to push the V8 engine to its absolute limit.

Here is how I squeezed 15.5M ops/sec out of JavaScript, and why you should probably audit your cache packages today.

The Problem: High-Level Abstractions Hurt Performance

If you implement a cache in Node.js using naive JS Objects or Arrays, V8 engine overhead will destroy your throughput.
Typical LRU implementations allocate a { key, value, prev, next } node object on every single set() operation to manage a Doubly Linked List. Under high-throughput traffic, creating millions of transient wrapper objects triggers frequent, expensive Garbage Collection (GC) pauses.

To build a cache that outpaces industry defaults, I had to drop down to systems-level memory patterns within JavaScript.

Geeky V8 & Systems Optimizations

1. Zero-GC Flat Parallel Arrays & Slot Recycling

s3fifo allocates zero wrapper node objects at runtime. Instead, it uses Flat Parallel Arrays:

#keys: (string | undefined)[]
#values: (V | undefined)[]
#meta: Uint32Array (Flags & Generation ID)
#starts / #ttls: Float64Array Available slots are tracked and recycled using a contiguous Uint32Array stack (#freeSlots). Insertion and eviction are simple O(1) index shifts on flat arrays. Zero dynamic object allocations = Zero GC pressure.

// Slot index stack recycling: O(1) with zero GC
const slot = this.#freeSlots[this.#freeSlotTop--]!;
this.#keys[slot] = key;
this.#values[slot] = value;

2. Bit-Packed Metadata & Generation Tracking (32-bit Integers)

Instead of storing state flags inside JS objects, metadata for every slot is packed into a single 32-bit integer inside #meta (Uint32Array):

Lower 5 bits: State flags (FREQ counter 0-3, RESIDENT, STALE, FREED)
Upper 27 bits: Generation ID (increments every time a slot is recycled)

const META_FREQ_MSK  = 0b00011;
const META_STALE_MSK = 0b00100;
const META_RESI_MSK  = 0b01000;
const META_FREED_MSK = 0b10000;
// Read frequency counter via bitwise AND
const freq = meta & META_FREQ_MSK;

This packs all operational state into just 4 bytes per slot while staying immune to V8 hidden class shape transitions.

3. O(1) Zombie Detection via 64-bit Bit Packing

In S3-FIFO, the Ghost queue (#G) tracks items evicted from the main cache to give them a second chance if accessed again. But what happens if a slot is recycled for a new key while the old reference is still sitting in the Ghost queue (the ABA problem)?
Instead of using pointers or hashmap lookups, s3fifo packs the slot_index AND its generation_id into a single 64-bit float in the Float64Array Ghost ring buffer:

// Pack generation ID and slot index into one Float64
const packedForG = gen * SHIFT_ADDR + evictedSlot;
// Upon popping from Ghost queue:
const ghostOrZombie = packed % SHIFT_ADDR;
const poppedGen = Math.floor(packed / SHIFT_ADDR);
const currentGen = (this.#meta[ghostOrZombie]! >> 5) & GEN_MASK;
// If generations don't match, it's a "Zombie" (recycled slot) -> Discard in O(1)!
const isZombie = poppedGen !== currentGen;

4. Bitwise Operations Over Modulo Arithmetic

Ring buffers require index wrapping ((index + 1) % capacity). However, the modulo operator (%) is computationally expensive in hot execution loops.
s3fifo enforces internal ring buffer sizes to always be a power of 2. This allows replacing modulo operations with lightning-fast bitwise AND (&):

// Standard Ring Buffer (Slow Modulo)
this.tail = (this.tail + 1) % this.capacity;
// S3-FIFO Ring Buffer (Fast Bitwise Masking)
this.tail = (this.tail + 1) & this.mask;

5. Coarse-Grained Timestamp Caching for TTL

Calling `performance.now()` or `Date.now()` on every hit in a multi-million-ops loop adds measurable system call overhead. `s3fifo` maintains an internal `#cacheNow` timestamp updated via `setInterval` at a configurable resolution (`ttlResolution`, default 100ms) with `.unref()`, completely avoiding event loop blockage while eliminating syscall overhead.

📊 Benchmark: Hit Rate & Throughput vs lru-cache

Tested on a Zipfian distribution (skew 0.99, working set of 100,000 keys) comparing s3fifo v1.0 to Node's popular lru-cache:

Average Hit Rate (%)

Cache Size (% of Pool)	lru-cache	s3fifo
1%	48.90%	58.30%
5%	65.00%	71.10%
10%	72.30%	76.40%
25%	82.10%	82.70%
50%	89.00%	86.30%

Average Throughput (ops/sec)

Cache Size (% of Pool)	lru-cache	s3fifo
1%	10.8M	15.5M
5%	10.8M	14.4M
10%	10.2M	14.3M
25%	10.3M	13.5M
50%	10.3M	14.7M

Production Ready & Keyv Integration

You don't need to rewrite your application to try this out.

I've created `keyv-s3fifo`, which has been officially merged into the Keyv ecosystem as a first-class storage adapter. If you already use Keyv, you can switch your cache engine to S3-FIFO with a single line of code!

Conclusion

LRU isn’t dead, but defaulting to it without benchmarking is leaving major performance on the table. If your Node.js application handles large-scale traffic, web crawlers, or high-volume database queries, S3-FIFO will protect your hot data from cache pollution while delivering higher throughput.
Stop defaulting. Start benchmarking.
Check out the code, run the benchmarks on your machine, and if you find it useful, I’d love a ⭐️ on GitHub!
👉 GitHub: s3fifo

👉 npm: s3fifo

👉 npm: keyv-s3fifo

s3fifo 1.0: Zero-Allocation S3-FIFO Cache for Node.js is Ready for Production

JeongSeop Byeon — Thu, 23 Jul 2026 16:24:28 +0000

Disclaimer: As a non-native English speaker, I used AI to help structure and polish this article.

A few dayss ago, I shared Implementing a Zero-Allocation S3-FIFO Cache in Node.js—an exploratory v0.1 release demonstrating how the S3-FIFO (Simple and Scalable Scan-Resistant FIFO) caching algorithm can be implemented using pre-allocated TypedArrays to eliminate Garbage Collection (GC) pressure in Node.js.

Today, after extensive stress testing, architectural hardening, and 100% unit test coverage, I'm excited to announce the release of s3fifo v1.0.0! 🚀

Quick Recap: Why S3-FIFO + Zero Allocation?

Traditional LRU (Least Recently Used) caches suffer from two major production drawbacks:

Cache Pollution by One-Hit Wonders: Scans or sequential queries flood the cache with items accessed only once, evicting your high-frequency working set.
Garbage Collection Overhead: Naive object-based cache implementations allocate nodes dynamically on every set(), triggering frequent GC pauses under high-throughput workloads. S3-FIFO solves cache pollution by organizing cache entries into three lightweight queues:
Small (S): Filters out one-hit wonders quickly (typically ~10% of total capacity).
Main (M): Holds multi-access, high-frequency items (~90% of capacity).
Ghost (G): Remembers evicted keys from Small to instantly promote them to Main if re-requested. s3fifo achieves zero dynamic object allocation during hot get/set cycles by backing these queues with contiguous TypedArrays (Uint32Array, Float64Array) and reusable index pools. --- ## What's New in v1.0.0? (Production Readiness) While v0.1 focused on core algorithm speed, v1.0.0 delivers all the developer ergonomics, lifecycle tools, and safety guarantees required for mission-critical Node.js microservices. ### 1. 💾 Cold-Start Persistence (dump & load) Prevent Database Thundering Herd / Cache Stampede during container restarts or deployments. s3fifo 1.0 supports serializing active resident entries alongside their original creation timestamps and remaining TTL:

import fs from "node:fs";
import { S3Fifo } from "s3fifo";
const cache = new S3Fifo<string>({ max: 10000 });
// 1. Export active cache items (optionally filter out temporary keys)
const dumpData = cache.dump((key, value) => !key.startsWith("temp:"));
fs.writeFileSync("cache-snapshot.json", JSON.stringify(dumpData));
// 2. On server startup / pre-warming: restore cache state instantly
const snapshot = JSON.parse(fs.readFileSync("cache-snapshot.json", "utf-8"));
cache.load(snapshot);

2. ♻️ Safe Resource Lifecycle (`dispose` Callback)

When cache items are evicted, overwritten, or cleared, you often need to release external resources (e.g., closing file descriptors, destroying DB handles, or tracking eviction metrics).

const cache = new S3Fifo<Buffer>({
  max: 500,
  dispose: (key, buffer, reason) => {
    console.log(`Key ${key} removed due to: ${reason}`); // 'evict' | 'set' | 'delete' | 'clear'
    // Safely free native memory or resources
  },
});

🛡️ Re-Entrancy Protection: dispose callbacks are safely deferred until the cache operation completes, preventing internal state corruption if a callback invokes cache.set() or cache.delete() recursively.

3. 🔍 Side-Effect-Free Inspection (peek)

Need to check a cached value for logging, health checks, or monitoring without bumping frequency counters or altering eviction status? peek() allows pure, side-effect-free reading:

// Does not modify S3-FIFO frequency bit fields or TTL timestamps
const val = cache.peek("user:1001");

4. 🔄 Standard ES6 Iterators & Map API

s3fifo 1.0 integrates seamlessly with JavaScript's native iteration protocols:

// Standard JS Map style iterators
for (const [key, value] of cache) {
  console.log(key, value);
}
const keys = Array.from(cache.keys());
const values = Array.from(cache.values());
const entries = Array.from(cache.entries());
cache.forEach((val, key) => {
  /* ... */
});

5. 🧹 Leak-Free Lifecycle Teardown (`close`)

In serverless, hot-reloading (HMR), or test environments, background timers can prevent process termination. The close() method clears active background TTL tickers, releases memory buffers, and prevents memory leaks:

// Clean teardown on shutdown
cache.close();
console.log(cache.isClosed); // true

📊 Benchmark: Hit Rate & Throughput vs lru-cache

Tested on a Zipfian distribution (skew 0.99, working set of 100,000 keys) comparing s3fifo v1.0 to Node's popular lru-cache:

Average Hit Rate (%)

Cache Size (% of Pool)	lru-cache	s3fifo
1%	48.90%	58.30%
5%	65.00%	71.10%
10%	72.30%	76.40%
25%	82.10%	82.70%
50%	89.00%	86.30%

Average Throughput (ops/sec)

Cache Size (% of Pool)	lru-cache	s3fifo
1%	10.8M	15.5M
5%	10.8M	14.4M
10%	10.2M	14.3M
25%	10.3M	13.5M
50%	10.3M	14.7M

Key Takeaways:

Up to +9.4% higher hit rate when cache capacity is small relative to dataset size (1%–10% range)—ideal for front-end database caches or high-concurrency microservices.

- ~40% higher throughput (~14.5M–15.5M ops/sec vs ~10.5M ops/sec) due to zero object allocations during `get`/`set` operations.

Get Started

Install via npm:

npm install s3fifo

Basic usage:

import { S3Fifo } from "s3fifo";
const cache = new S3Fifo<string>({
  max: 1000,
  ttl: 60000, // 60s global TTL
});
cache.set("session:abc", "user_data");
console.log(cache.get("session:abc")); // 'user_data'
console.log(cache.size); // 1

Links & Feedback

📦 NPM: s3fifo
🐙 GitHub Repository: BJS-kr/s3fifo If you're looking for an in-memory cache alternative in Node.js with scan-resistance and minimal GC footprint, give s3fifo a try! Bug reports, feedback, and GitHub stars are greatly appreciated! 🙏

Implementing a Zero-Allocation S3-FIFO Cache in Node.js

JeongSeop Byeon — Mon, 20 Jul 2026 13:22:32 +0000

Disclaimer: As a non-native English speaker, I used AI to help translate and structure this article.

Implementing a Zero-Allocation S3-FIFO Cache in Node.js

When building a Node.js backend and reaching for an in-memory cache, lru-cache is the undisputed default choice for almost everyone—and for good reason. It is robust, feature-rich, and heavily proven in production.

However, as datasets grow, LRU algorithms naturally struggle with "one-hit wonders"—items that are requested once and never again. In a standard LRU cache, these items push out frequently accessed data, polluting the cache and dropping your hit rate prematurely.

The S3-FIFO (Simple and Scalable Scan-Resistant FIFO) caching algorithm addresses this fundamental flaw. Rather than being a niche solution, S3-FIFO is a general-purpose caching algorithm that uses a three-queue system to efficiently filter out one-hit wonders.

I built s3fifo to bring this algorithm to the Node.js ecosystem as a highly viable alternative to LRU. Beyond just implementing the algorithm, I built it with a strict performance goal: zero-allocation.

When should you consider `s3fifo`?

High throughput requirements: You need maximum operations per second.
Strict memory constraints: You want to avoid the Garbage Collection (GC) pauses typically associated with traditional object-based caches.
Sparse data access (Low cache coverage): Your total dataset is massive, and your cache can only afford to hold a small fraction of it (e.g., < 10%). In these cases, S3-FIFO's resistance to cache pollution significantly outperforms LRU.

Familiar API

Before diving into the internals, the API is designed to be a drop-in replacement for most standard Maps or LRU implementations:

import { S3Fifo } from "s3fifo";

const cache = new S3Fifo({ max: 1000, ttl: 60000 });
cache.set("key", "value");
cache.get("key"); 
cache.delete("key");

Here is a brief look at the implementation details and the technical reasoning behind them.

1. Avoiding Object Allocation Overhead

Traditional cache implementations often represent items as objects (e.g., linked list nodes with next and prev pointers). While this is simple to implement, creating and destroying thousands of objects per second adds continuous pressure to the V8 Garbage Collector (GC).

To minimize GC pauses, s3fifo uses pre-allocated flat arrays:

this.#meta = new Uint8Array(totalSize);
this.#keys = new Array(totalSize);
this.#values = new Array(totalSize);
this.#starts = new Float64Array(totalSize);
this.#ttls = new Float64Array(totalSize);

Once the cache is instantiated, no new objects are allocated for structural management. Inserting a new item simply writes primitive values and references to an available index in these typed arrays.

2. Array-based Ring Buffers

S3-FIFO requires three queues: Small (S), Main (M), and Ghost (G).
Using standard Array.push() and Array.shift() would be disastrous for performance because shift() runs in O(N) time.

Instead, the queues are implemented as fixed-size Ring Buffers. By enforcing queue sizes to be strict powers of 2, we can replace relatively expensive modulo operations (%) with ultra-fast bitwise AND (&):

// Pushing to a ring buffer
this.container[this.tail] = address;
this.tail = (this.tail + 1) & this.mask; // Bitwise modulo

3. The "Ghost" Ring via Bitwise Flags

The Ghost ring is a core concept in S3-FIFO. It tracks the keys of items recently evicted from the Small ring. If a Ghost key is requested again, it gets promoted directly to the Main ring.

Storing actual keys in a separate queue would require additional memory management. To solve this efficiently, s3fifo tracks states using a single byte (Uint8) in the #meta array:

const META_FREQ_MSK   = 0b00000011; // Lower 2 bits: Frequency (0-3)
const META_STALE_MSK  = 0b00000100; // 3rd bit: Is deleted?
const META_RESI_MSK   = 0b00010000; // 5th bit: Is resident? (1=S/M, 0=Ghost)

When an item is evicted to the Ghost ring, its META_RESI_MSK bit is flipped to 0, and its value reference is cleared for the GC. Structurally, it becomes a "Ghost" without requiring a separate physical node or array shift.

4. O(1) Lazy Deletion

Removing an item from the middle of an array-based ring buffer typically requires shifting elements, which is an O(N) operation.

To avoid this, delete() operations are handled lazily:

this.#meta[address] |= META_STALE_MSK;
this.#addressIndex.delete(key);

The item's META_STALE_MSK bit is flipped to 1, and it is removed from the Map index. The physical slot remains in the ring buffer. When the ring's head pointer eventually reaches this slot during natural eviction, the algorithm notices the stale bit and safely cleans it up in O(1) time.

5. Lazy TTL Evaluation

Supporting Time-To-Live (TTL) in caches often introduces overhead, either through background sweeping timers (setInterval) or complex priority queues.

s3fifo avoids background timers entirely. Expiration is evaluated lazily on get() and lazily cleaned up during natural ring eviction. This ensures that even with complex per-item TTL requirements, the cache remains entirely zero-allocation and lock-free from background processes.

6. Benchmark Results

Here is a comparison with lru-cache using a Zipfian distribution (skew=0.99, pool=100,000) on an AMD Ryzen 7 7700.

Average Hit Rate (%)

Cache Size (% of Pool)	lru-cache	s3fifo
1%	48.90%	58.32%
5%	65.01%	71.14%
25%	82.10%	82.67%

Average Throughput (ops/sec)

Cache Size (% of Pool)	lru-cache	s3fifo
1%	10.0M	15.3M
10%	8.7M	12.5M
50%	9.7M	14.6M

In workloads with low cache coverage (sparse environments), s3fifo provides noticeable improvements in both hit rate and throughput.

Conclusion

While lru-cache is a fantastic general-purpose default, s3fifo offers a modern, scan-resistant alternative with a zero-allocation architecture.

Although the library is fully typed and rigorously tested with 100% test coverage, it is still currently in its v0.1.x stage. I plan to add more features (like async fetching and broader configuration options) in the future.

If you find this approach interesting, give it a try! Contributors, feedback, and PRs are incredibly welcome to help mature the project.

GitHub: https://github.com/BJS-kr/s3fifo
NPM: https://www.npmjs.com/package/s3fifo

References

S3-FIFO Website: https://s3fifo.com/
Original Paper (SOSP '23): FIFO Queues are All You Need for Cache Eviction

DEV Community: JeongSeop Byeon

Stop Defaulting to LRU: How I Built a 15.5M ops/sec S3-FIFO Cache for Node.js by Hacking V8

Here is how I squeezed 15.5M ops/sec out of JavaScript, and why you should probably audit your cache packages today.

The Problem: High-Level Abstractions Hurt Performance

To build a cache that outpaces industry defaults, I had to drop down to systems-level memory patterns within JavaScript.

Geeky V8 & Systems Optimizations

1. Zero-GC Flat Parallel Arrays & Slot Recycling

2. Bit-Packed Metadata & Generation Tracking (32-bit Integers)

3. O(1) Zombie Detection via 64-bit Bit Packing

4. Bitwise Operations Over Modulo Arithmetic

5. Coarse-Grained Timestamp Caching for TTL

📊 Benchmark: Hit Rate & Throughput vs lru-cache

Average Hit Rate (%)

Average Throughput (ops/sec)

Production Ready & Keyv Integration

I've created keyv-s3fifo, which has been officially merged into the Keyv ecosystem as a first-class storage adapter. If you already use Keyv, you can switch your cache engine to S3-FIFO with a single line of code!

Conclusion

s3fifo 1.0: Zero-Allocation S3-FIFO Cache for Node.js is Ready for Production

Quick Recap: Why S3-FIFO + Zero Allocation?

2. ♻️ Safe Resource Lifecycle (dispose Callback)

3. 🔍 Side-Effect-Free Inspection (peek)

4. 🔄 Standard ES6 Iterators & Map API

5. 🧹 Leak-Free Lifecycle Teardown (close)

📊 Benchmark: Hit Rate & Throughput vs lru-cache

Average Hit Rate (%)

Average Throughput (ops/sec)

Key Takeaways:

- ~40% higher throughput (~14.5M–15.5M ops/sec vs ~10.5M ops/sec) due to zero object allocations during get/set operations.

Get Started

Links & Feedback

Implementing a Zero-Allocation S3-FIFO Cache in Node.js

Implementing a Zero-Allocation S3-FIFO Cache in Node.js

When should you consider s3fifo?

Familiar API

1. Avoiding Object Allocation Overhead

2. Array-based Ring Buffers

3. The "Ghost" Ring via Bitwise Flags

4. O(1) Lazy Deletion

5. Lazy TTL Evaluation

6. Benchmark Results

Average Hit Rate (%)

Average Throughput (ops/sec)

Conclusion

References

I've created `keyv-s3fifo`, which has been officially merged into the Keyv ecosystem as a first-class storage adapter. If you already use Keyv, you can switch your cache engine to S3-FIFO with a single line of code!

2. ♻️ Safe Resource Lifecycle (`dispose` Callback)

3. 🔍 Side-Effect-Free Inspection (`peek`)

5. 🧹 Leak-Free Lifecycle Teardown (`close`)

- ~40% higher throughput (~14.5M–15.5M ops/sec vs ~10.5M ops/sec) due to zero object allocations during `get`/`set` operations.

When should you consider `s3fifo`?