DEV Community: Kareem

Stop sending every cache read to Redis: Why Multi-Tier Caching is the Future of Node.js

Kareem — Fri, 21 Aug 2026 16:52:49 +0000

Most backend architectures look like this:

App Server (Node.js) ──[ 1–5 ms network hop ]──► Redis ──[ JSON.parse ]──► Response

While Redis is fast, doing network hops and JSON serialization on every single cache read costs tens of thousands in cloud bills and introduces tail latency under load.

What if your Node.js cache had:

L1 RAM (Sub-microsecond V8 Heap): Zero network hop, 0 ns decode time with direct object reference caching.
L1.5 NVMe Disk Spill: Fast local disk fallback instead of evicting to the void when RAM gets full.
L2 Redis / Valkey Tier: Distributed multi-instance sync with AES-256 encryption and Brotli compression.
WASM Bloom Filter & Count-Min Sketch: Gating cold misses and burst frequency tracking with WebAssembly.

That is why I built TriCache.

⚡ What Makes It Different?

┌─────────────────────────────────────────────────────────────────────────────┐
│ TriCache 3-Tier Caching Engine                                              │
│                                                                             │
│  [ Next.js "use cache" | NestJS @Cacheable | Prisma withTriCache | Hono ]   │
│                                     │                                       │
│                    ┌────────────────▼───────────────┐                       │
│                    │  L1 Smart Memory (0 ns decode) │ ◄── 4.54M ops/sec     │
│                    └────────────────┬───────────────┘                       │
│                                     │ (Eviction Spill)                      │
│                    ┌────────────────▼───────────────┐                       │
│                    │  L1.5 NVMe Disk (Atomic Async) │ ◄── 500 MB Local Pool │
│                    └────────────────┬───────────────┘                       │
│                                     │ (Miss Promotion)                      │
│                    ┌────────────────▼───────────────┐                       │
│                    │  L2 Redis / Valkey Backplane   │ ◄── Cluster / Streams │
│                    └────────────────────────────────┘                       │
└─────────────────────────────────────────────────────────────────────────────┘

1. Zero-Allocation WebAssembly Bloom Filter

Cold cache misses are screened through an inlined WebAssembly double-hashing Bloom filter before touching disk or Redis. By pre-allocating the staging memory buffer, .add() executes in 220 ns (4.54 Million ops/sec) with 0 JavaScript garbage collection pauses.

2. O(1) Generational Tag Invalidation

Instead of $O(N)$ Redis KEYS or SMEMBERS deletions, TriCache uses atomic generational version counters. Tag invalidations take 605 ns (1.65 Million ops/sec) across single or batch tags.

3. Thundering Herd Coalescing

10,000 concurrent coroutines requesting the same cold key collapse into exactly 1 upstream fetch call.

🛠️ Plug-and-Play with Your Favorite Stack

Next.js 16 & React 19 RSC Streams

// next.config.mjs
export default {
  cacheHandler: require.resolve('tricache/next'),
};

Prisma ORM

import { PrismaClient } from '@prisma/client';
import { withTriCache } from 'tricache/prisma';
import { CacheService } from 'tricache';

const cache = CacheService.create();
const prisma = new PrismaClient().$extends(withTriCache({ cache }));

// Automatically cached + auto-invalidates on mutations:
const users = await prisma.user.findMany({ where: { role: 'ADMIN' }, cache: { ttl: 300 } });

NestJS Decorators

@Injectable()
export class UserService {
  @Cacheable({ ttl: 300, tags: ['users'] })
  async getUser(id: string) {
    return this.userRepo.findById(id);
  }

  @CacheEvict({ tags: ['users'] })
  async updateUser(id: string, data: UpdateDto) {
    return this.userRepo.update(id, data);
  }
}

Universal HTTP Middleware & 304 ETags (Express / Fastify / Hono)

import { fastifyCache } from 'tricache/http';

app.get('/api/products', { preHandler: fastifyCache({ cache, ttl: 60 }) }, handler);

📊 Live Benchmark Highlights

Operation	Throughput	Latency
L1 RAM Hot Read (0-copy)	621,100 ops/sec	1.61 µs
Generational Tag Invalidation	1,650,000 ops/sec	605 ns
WASM Bloom Filter Insert	4,540,000 ops/sec	220 ns
Distributed Mutex Lock	341,300 ops/sec	2.93 µs

📦 Try It Out

TriCache is 100% open-source under MIT, with 485 automated chaos & resilience tests.

npm install tricache
# or try the CLI:
npx tricache inspect

🔗 GitHub: https://github.com/Kareem411/TriCache
📦 npm: https://www.npmjs.com/package/tricache

I’d love to hear your thoughts, feedback, and edge cases in the comments! 🚀

How We Built a 3-Tier Node.js Cache That Hits 2.8M ops/sec (And Never Drops a Key on Eviction)

Kareem — Wed, 27 May 2026 07:56:29 +0000

Caching in Node.js usually forces a frustrating compromise. You either stick with a basic in-memory Map (or LRU-cache) that risks running your process out of memory (OOM) under heavy loads, or you accept the network latency floor and serialization overhead of a standalone Redis/Valkey instance.

When building high-throughput systems, neither option feels entirely right. A localhost Redis round-trip is fast, but it’s still bounded by networking stacks.

We built tricache to eliminate this compromise completely. It’s an open-source, three-tier caching engine for Node.js designed to maximize raw single-threaded throughput while implementing protective guardrails like automated disk spilling, thundering-herd prevention, and an integrated WebAssembly Bloom filter.

The result? 2.81 million read operations per second from a single thread—over 100× faster than a local Redis round-trip—without the risk of unbounded RAM growth.

Here is a look under the hood at how it works and the architectural decisions that made it happen.

The 3-Tier Architecture

To achieve high hit rates without risking OOM crashes, tricache splits data management into three distinct layers:

L1 (Smart Memory Cache): A blazing-fast, process-level memory layer built on top of a highly optimized V8 Map.
L1.5 (Local Disk Spill): When the L1 memory cap (l1MaxBytes) or entry cap is reached, evicted entries aren't simply deleted. They spill over to a local, high-speed NVMe disk directory using optimized msgpackr binary serialization.
L2 (Distributed Backplane): The shared remote tier (Redis or Valkey) used for multi-instance distributed sync.

Core Technical Features & Innovations

1. Advanced Hybrid Eviction via Count-Min Sketch

Standard LRU (Least Recently Used) caching fails catastrophically during sudden scan floods or data bursts, where long-resident, high-value keys get wiped out by single-access keys.

tricache implements a hybrid eviction algorithm that scores keys based on a combination of Priority Score + Access Frequency + Remaining TTL. To track access frequency without introducing a massive memory footprint, we integrated a Count-Min Sketch using a tiny 4 KB Uint16Array (4 rows × 512 columns).

Every time a key is queried or set, it passes through the sketch. When L1 needs to evict data, it uses reservoir sampling to pick an eviction candidate in a single $O(1)$ pass, ensuring a 78% survival rate for high-frequency keys during benchmark flood tests.

2. Inlined WASM Bloom Filter

Before hitting a synchronous Map lookup or checking the filesystem/Redis on a cache miss, queries pass through an ultra-lightweight WASM Bloom filter.

The filter is optimized with $k=7$ hash probes. At a rated capacity of roughly 18,000 entries, it maintains a false-positive rate of just 1%. If the filter says a key does not exist, it is a guaranteed miss, allowing tricache to skip heavy lookups entirely. The 562-byte WASM binary is fully inlined as a Base64 string—meaning zero filesystem access overhead at boot time.

3. Native Thundering-Herd Prevention

When a highly contested key expires under heavy traffic, hundreds of concurrent requests usually hit the underlying database at the exact same moment.

tricache avoids this with a native Inflight Promise Registry. No matter how many concurrent callers request an expired or missing key, the user-supplied fetchFn fires exactly once. All other concurrent requests hook into that single pending promise, protecting your database or upstream APIs from collapsing under sudden load.

4. Enterprise-Grade Resiliency Out of the Box

Stale-While-Revalidate (SWR) & Stale-If-Error: Instantly serves stale data to the client while revalidating the data asynchronously in the background. If the upstream database fails during revalidation, staleIfError kicks in to extend the stale entry's life, keeping your app fully operational during upstream outages.
OOM Guard: A background timer polls heapUsed and heapTotal. If memory utilization breaches a set threshold (e.g., 85%), an emergency eviction routine triggers immediately, clearing out the coldest 20% of L1 memory before the Node.js process crashes.
At-Rest Encryption: Supports automated AES-256-GCM authenticated encryption for L2 values, disk spill files, and cold-start snapshots, complete with zero-downtime key rotation support.

Performance Benchmarks

All metrics were captured executing on a single Node.js thread (with synchronous paths completely un-awaited):

L1 Memory Cache Throughput

Operation	Throughput	Latency	Mechanics
`get` (Hot Hit)	2.81 M/s	~356 ns	Bloom filter → Map lookup → Return
`get` (Cold Miss)	7.14 M/s	~140 ns	Bloom filter gates early return
`delete` (Exact Key)	5.36 M/s	~186 ns	Synchronous Map deletion

End-to-End Service Performance

Operation	Throughput	Latency
`get` (L1 Warm Hit)	2.03 M/s	~491 ns
`get` (SWR Stale Serve)	1.78 M/s	~562 ns
`get` (Miss + `fetchFn`)	13.7 K/s	~73 µs

Quick Start: Getting Started in 30 Seconds

Getting started requires zero initial configuration. Sensible production defaults apply automatically.

npm install tricache
# or pnpm add tricache

import { CacheService } from 'tricache';

// Initialize the process-level singleton
const cache = CacheService.create({
  namespace: 'my-app',
  redisHost: process.env.REDIS_HOST, // Omit in dev to automatically fallback to L1/Disk only
});

// Fetch-or-Get with an automatic database fallback and a 5-minute TTL
const user = await cache.get(
  `user:${userId}`,
  () => db.users.findById(userId),
  300, // TTL in seconds
  { swr: 30 } // Seamless Stale-While-Revalidate for 30 seconds
);

Advanced Group Invalidation with Tags

You can tag items on creation and invalidate entire collections across L1, local disk, and distributed Redis nodes simultaneously:

// Add items to the cache with descriptive tags
await cache.set(`product:1`, productData, 300, undefined, { tags: ['catalog'] });
await cache.set(`product:2`, alternativeData, 300, undefined, { tags: ['catalog'] });

// Atomically evict all catalog entries everywhere
await cache.invalidateTag('catalog');

Open Source & Contributing

tricache is fully open-source, licensed under the MIT license, and built entirely using TypeScript. If you are building high-performance Node.js services or want to dive into the codebase (especially src/smart-memory-cache.ts), check out the repository!

👉 Check out the repo on GitHub: github.com/Kareem411/TriCache

Feedback, bug reports, and performance optimizations via Pull Requests are always welcome! Let me know what caching bottlenecks you're currently facing in your architecture below.