Ashwani Arya

Posted on Oct 31

Caching Systems: A Complete Guide

#systemdesign #webdev #distributedsystems #programming

A comprehensive, beginner-friendly guide to understanding and implementing caching systems

📚 Introduction: What is Caching?

The Real-World Analogy

Imagine you're a student studying for exams:

Without Caching:

Every time you need information, you walk to the library (10 minutes away)
Find the book on the shelf
Read the page you need
Walk back home
Repeat this for EVERY piece of information you need

With Caching:

First time: Walk to library, photocopy the important pages
Keep those photocopies on your desk at home
Next time you need that info: Just look at your desk (5 seconds!)
Only go back to the library if you need something NEW

This is exactly what caching does in software!

The Kitchen Pantry Analogy

Think of caching like organizing your kitchen:

🏪 Grocery Store (Database)    →  Slow, but has EVERYTHING
   ↓
🚗 Drive & Shop (Network Call) →  Takes time & effort
   ↓
🏠 Pantry (Cache)              →  Fast, has what you use OFTEN
   ↓
👨‍🍳 Cooking (Your App)           →  Instant access!

You don't drive to the store every time you need salt. You keep frequently-used items in your pantry. That's caching!

What is Caching (Technical Definition)

Caching is the process of storing copies of data in a temporary storage location (the "cache") so that future requests for that data can be served faster.

Key Concept:

Original Data Source (slow) → Cache (fast copy) → Your Application (blazing fast!)

Why Do We Cache?

Speed ⚡
- Reading from cache: ~1-10 milliseconds
- Reading from database: ~50-500 milliseconds
- Reading from external API: ~500-5000 milliseconds
Reduced Load 📉
- Fewer database queries
- Less network traffic
- Lower server costs
Better User Experience 😊
- Instant responses
- Works offline (in some cases)
- Smoother interactions
Reliability 🛡️
- Still works if the original source is slow
- Backup data if source is temporarily down

Visual: How Caching Works

┌─────────────────────────────────────────────────────────────┐
│                    REQUEST FOR DATA                         │
└────────────────────┬────────────────────────────────────────┘
                     │
                     ▼
          ┌──────────────────────┐
          │   Is it in CACHE?    │
          └──────────┬───────────┘
                     │
         ┌───────────┴───────────┐
         │                       │
         ▼                       ▼
    ┌────────┐            ┌──────────┐
    │  YES!  │            │    NO    │
    │ (HIT)  │            │  (MISS)  │
    └────┬───┘            └─────┬────┘
         │                      │
         │                      ▼
         │          ┌────────────────────────┐
         │          │ Fetch from Original    │
         │          │ Source (DB/API/Server) │
         │          └───────────┬────────────┘
         │                      │
         │                      ▼
         │          ┌────────────────────────┐
         │          │   Store in Cache       │
         │          │   for next time        │
         │          └───────────┬────────────┘
         │                      │
         └──────────┬───────────┘
                    │
                    ▼
         ┌────────────────────┐
         │   Return Data to   │
         │   Application      │
         └────────────────────┘

The Basic Flow

First Request (Cache Miss):

User requests product info
  → Check cache: NOT FOUND ❌
  → Fetch from database (slow: 200ms)
  → Store in cache
  → Return to user
Total time: 200ms

Second Request (Cache Hit):

User requests same product info
  → Check cache: FOUND! ✅
  → Return from cache (fast: 2ms)
Total time: 2ms  (100x faster!)

Simple Example: Before vs After

WITHOUT Caching:

async function getProductDetails(productId) {
  // Every call goes to the database
  const product = await database.query('SELECT * FROM products WHERE id = ?', [productId]);
  return product;
}

// User clicks 5 times → 5 database queries (slow!)

WITH Caching:

const cache = {}; // Simple in-memory cache

async function getProductDetails(productId) {
  // Check cache first
  if (cache[productId]) {
    console.log('Cache HIT! 🎯');
    return cache[productId]; // Super fast!
  }

  console.log('Cache MISS 😞');
  // Not in cache, get from database
  const product = await database.query('SELECT * FROM products WHERE id = ?', [productId]);

  // Store in cache for next time
  cache[productId] = product;

  return product;
}

// User clicks 5 times → 1 database query + 4 cache hits (much faster!)

🎯 Fundamental Principles of Good Caching

1. Cache Hit vs Cache Miss

These are the two outcomes every cache lookup can have:

Cache Hit ✅

The data you're looking for IS in the cache
Fast response
Success!

Cache Miss ❌

The data you're looking for is NOT in the cache
Must fetch from original source
Slower, but necessary

Visual Representation:

Request → Cache Check
              │
    ┌─────────┴─────────┐
    │                   │
    ▼                   ▼
  HIT ✅              MISS ❌
  (Fast!)            (Fetch from source)

Cache Hit Ratio is a key metric:

Hit Ratio = (Cache Hits / Total Requests) × 100%

Good cache: 80-95% hit ratio
Poor cache: <50% hit ratio

Example:

const cacheStats = {
  hits: 0,
  misses: 0
};

function getCachedData(key) {
  if (cache.has(key)) {
    cacheStats.hits++;
    console.log(`Hit ratio: ${(cacheStats.hits / (cacheStats.hits + cacheStats.misses) * 100).toFixed(1)}%`);
    return cache.get(key);
  }

  cacheStats.misses++;
  const data = fetchFromDatabase(key);
  cache.set(key, data);
  return data;
}

2. Cache Invalidation: The Hardest Problem

"There are only two hard things in Computer Science: cache invalidation and naming things."
— Phil Karlton

What is Cache Invalidation?

Cache invalidation is the process of removing or updating stale (outdated) data from the cache.

The Problem:

Time 0:00 → User gets product price: $100 (from cache)
Time 0:30 → Admin updates price to $80 (in database)
Time 0:35 → User refreshes page: Still shows $100! (from cache)
            ⚠️ STALE DATA! The cache doesn't know about the update!

Visual: The Staleness Problem

Database:        Cache:           User Sees:
$100             $100             $100 ✓
  ↓                ↓                ↓
$80 (updated!)   $100 (old!)      $100 ❌ WRONG!

Common Invalidation Strategies:

Time-Based (TTL - Time To Live)

   // Cache expires after 5 minutes
   cache.set('product:123', data, { ttl: 300 }); // 300 seconds = 5 min

Event-Based

   // When product is updated, remove from cache
   function updateProduct(id, newData) {
     database.update(id, newData);
     cache.delete(`product:${id}`); // Invalidate immediately!
   }

Version-Based

   // Cache key includes version
   cache.set('product:123:v2', data);
   // When updated, use new version
   cache.set('product:123:v3', newData);

3. Temporal and Spatial Locality

These are the two principles that make caching effective:

Temporal Locality ⏰

"If data is accessed once, it will likely be accessed again soon"
Example: User viewing their profile page repeatedly

// User views their profile 10 times in a session
// Without cache: 10 database queries
// With cache: 1 database query + 9 cache hits

Spatial Locality 📍

"If data is accessed, nearby/related data will likely be accessed soon"
Example: When viewing product #123, user often views related products #124, #125

// Prefetch related products
function getProduct(id) {
  const product = cache.get(id) || fetchFromDB(id);

  // Prefetch related products (spatial locality!)
  product.relatedIds.forEach(relatedId => {
    if (!cache.has(relatedId)) {
      prefetchProduct(relatedId); // Load in background
    }
  });

  return product;
}

4. TTL (Time To Live)

TTL defines how long data should stay in cache before expiring.

Visual: TTL Lifecycle

Data Added      TTL Starts       Still Valid       Expired!
   │               │                 │                │
   ▼               ▼                 ▼                ▼
[Store] ────────> [5 min] ────────> [3 min] ────────> [0 min]
Cache: ✅          Cache: ✅         Cache: ✅         Cache: ❌
                                                     (Auto-deleted)

Choosing the Right TTL:

// Static content (rarely changes) → Long TTL
cache.set('logo.png', data, { ttl: 86400 }); // 24 hours

// Semi-static content → Medium TTL
cache.set('product-list', data, { ttl: 300 }); // 5 minutes

// Dynamic content → Short TTL
cache.set('stock-count', data, { ttl: 10 }); // 10 seconds

// User session → Session-based TTL
cache.set('user-session', data, { ttl: 1800 }); // 30 minutes

Trade-offs:

TTL	Pros	Cons
Long (hours/days)	Fewer database calls, faster	Risk of stale data
Short (seconds/minutes)	Fresher data	More database calls
No TTL (cache forever)	Maximum speed	Stale data unless manually invalidated

5. Cache Size Management

Caches have limited space. What happens when the cache is full?

Eviction Policies (How to decide what to remove):

LRU (Least Recently Used) 🏆 Most Common

Remove the item that hasn't been used for the longest time

Cache: [A, B, C, D, E] (full)
Access pattern: A, B, C, A, B
Need to add F → Remove D (least recently used)
New Cache: [A, B, C, E, F]

LFU (Least Frequently Used)

Remove the item that's been accessed the fewest times

Item A: accessed 10 times
Item B: accessed 2 times
Item C: accessed 15 times
Need space → Remove B (least frequently used)

FIFO (First In, First Out)

Remove the oldest item (like a queue)

Cache: [A, B, C, D, E] (full)
Need to add F → Remove A (first in)
New Cache: [B, C, D, E, F]

Visual: LRU in Action

class LRUCache {
  constructor(maxSize = 100) {
    this.cache = new Map();
    this.maxSize = maxSize;
  }

  get(key) {
    if (!this.cache.has(key)) return null;

    // Move to end (mark as recently used)
    const value = this.cache.get(key);
    this.cache.delete(key);
    this.cache.set(key, value);

    return value;
  }

  set(key, value) {
    // Remove if exists (will re-add at end)
    if (this.cache.has(key)) {
      this.cache.delete(key);
    }

    // If full, remove oldest (first item)
    if (this.cache.size >= this.maxSize) {
      const firstKey = this.cache.keys().next().value;
      this.cache.delete(firstKey);
      console.log(`Evicted ${firstKey} (LRU)`);
    }

    this.cache.set(key, value);
  }
}

// Usage
const cache = new LRUCache(3); // Max 3 items
cache.set('A', 1);
cache.set('B', 2);
cache.set('C', 3);
cache.set('D', 4); // Cache full! Evicts 'A'
// Cache now: [B, C, D]

6. Cache Coherence

When you have multiple caches, how do you keep them in sync?

The Problem:

User Cache (Browser):  price = $100
CDN Cache:             price = $100
Server Cache:          price = $100
Database:              price = $80 (just updated!)

⚠️ Three layers of stale data!

Solution Strategies:

Write-Through - Update all caches immediately
Cache Versioning - Use version numbers in cache keys
Cache Tags/Groups - Invalidate related caches together
Pub/Sub - Broadcast invalidation events

// Pub/Sub example
eventBus.on('product:updated', (productId) => {
  // Invalidate in all cache layers
  browserCache.delete(`product:${productId}`);
  serverCache.delete(`product:${productId}`);
  cdnCache.purge(`/api/product/${productId}`);
});

Visual: Complete Cache Lifecycle

┌─────────────────────────────────────────────────────────────┐
│                    CACHE LIFECYCLE                          │
└─────────────────────────────────────────────────────────────┘

1. DATA REQUESTED
   │
   ▼
2. CHECK CACHE
   │
   ├─→ HIT ✅ → Return immediately (fast!)
   │            │
   │            └─→ Update access time (for LRU)
   │
   └─→ MISS ❌ → 3. FETCH FROM SOURCE (slow)
                  │
                  ▼
               4. STORE IN CACHE
                  │
                  ├─→ Check cache size
                  │   │
                  │   └─→ If full: EVICT (LRU/LFU/FIFO)
                  │
                  └─→ Set TTL timer
                      │
                      ▼
               5. SERVE TO USER
                  │
                  ▼
               6. WAIT FOR...
                  │
                  ├─→ TTL expires → DELETE from cache
                  │
                  ├─→ Manual invalidation → DELETE from cache
                  │
                  └─→ Next access → Go to step 2

🗂️ Types of Caching

Caching exists at many different layers of your application. Let's explore each type!

Caching by Location

1. Browser Cache 🌐

Your web browser stores files locally on your computer.

What gets cached:

HTML pages
CSS stylesheets
JavaScript files
Images, fonts, videos
API responses (with proper headers)

Visual: Browser Cache Flow

User visits website
    ↓
Browser checks cache
    ↓
┌───────────────┬─────────────────┐
│  FOUND ✅     │  NOT FOUND ❌   │
│  (Use cached) │  (Download)     │
└───────┬───────┴────────┬────────┘
        │                │
        │                ▼
        │      ┌──────────────────┐
        │      │  Download from   │
        │      │  Server          │
        │      └────────┬─────────┘
        │               │
        │               ▼
        │      ┌──────────────────┐
        │      │  Store in Cache  │
        │      └────────┬─────────┘
        │               │
        └───────────────┘
                │
                ▼
        Display to User

Example: HTTP Headers

// Server tells browser to cache for 1 hour
response.headers = {
  'Cache-Control': 'public, max-age=3600',
  'ETag': 'abc123' // Version identifier
};

// Browser automatically caches!
// Next request: No network call needed for 1 hour

2. CDN Cache 🌍

CDN (Content Delivery Network) stores copies of your files on servers around the world.

How it works:

User in Tokyo                    User in New York
     │                                │
     ▼                                ▼
CDN Server (Tokyo)           CDN Server (New York)
     │                                │
     └────────────┬───────────────────┘
                  │
                  ▼
          Origin Server (California)

Benefits:

Faster: Users download from nearby servers
Reduced Load: Origin server handles fewer requests
Reliability: If origin is down, CDN might still serve cached content

Real-World Example:

// Without CDN: Everyone downloads from your server
<img src="https://yourserver.com/logo.png"> // Slow for distant users

// With CDN: Downloads from nearest location
<img src="https://cdn.yoursite.com/logo.png"> // Fast everywhere!

3. Application Cache (Frontend) 💻

This is cache YOU control in your JavaScript code.

Storage Options:

Storage Type	Size Limit	Persistence	Use Case
Memory (Variables)	RAM limit	Until page refresh	Temporary, fast access
SessionStorage	~5-10MB	Until tab closes	Per-tab data
LocalStorage	~5-10MB	Forever (until cleared)	User preferences, tokens
IndexedDB	~50MB+	Forever	Large datasets, offline apps
Cache API	Varies	Forever	Service Workers, PWAs

Example: localStorage

// Simple key-value cache
function getCachedUserProfile(userId) {
  const cacheKey = `user:${userId}`;

  // Try to get from localStorage
  const cached = localStorage.getItem(cacheKey);

  if (cached) {
    const { data, timestamp } = JSON.parse(cached);

    // Check if still valid (5 minutes)
    if (Date.now() - timestamp < 5 * 60 * 1000) {
      console.log('Cache hit! 🎯');
      return data;
    }
  }

  // Cache miss - fetch from API
  console.log('Cache miss 😞');
  const data = await fetchUserProfile(userId);

  // Store in cache
  localStorage.setItem(cacheKey, JSON.stringify({
    data,
    timestamp: Date.now()
  }));

  return data;
}

4. Server-Side Cache 🖥️

Cache on your backend server.

Common Tools:

Redis (most popular)
Memcached
In-memory objects

Example: Redis Cache

// Node.js with Redis
const redis = require('redis');
const client = redis.createClient();

async function getProduct(productId) {
  const cacheKey = `product:${productId}`;

  // Check Redis cache
  const cached = await client.get(cacheKey);
  if (cached) {
    console.log('Redis cache hit!');
    return JSON.parse(cached);
  }

  // Fetch from database
  const product = await database.query('SELECT * FROM products WHERE id = ?', [productId]);

  // Store in Redis (expire in 1 hour)
  await client.setex(cacheKey, 3600, JSON.stringify(product));

  return product;
}

5. Database Cache 🗄️

Databases have their own internal caching.

Query Result Cache:

-- First query: Slow (reads from disk)
SELECT * FROM products WHERE category = 'electronics';
-- Time: 250ms

-- Same query again: Fast (from cache)
SELECT * FROM products WHERE category = 'electronics';
-- Time: 5ms

Buffer Pool:

Keeps frequently accessed data in RAM
Automatically managed by the database

Caching by Strategy

Now let's look at different patterns for how caching works:

1. Cache-Aside (Lazy Loading) 💤

Most common pattern!

You manually check cache, and load data if missing.

Application → Check Cache → Found? Yes → Return data
                    │
                    └─→ Not Found? → Fetch from DB
                                          │
                                          ▼
                                   Store in Cache
                                          │
                                          ▼
                                    Return data

Code Example:

async function getUser(userId) {
  // 1. Try cache first
  let user = cache.get(userId);

  // 2. If not in cache, load from DB
  if (!user) {
    user = await database.getUserById(userId);
    // 3. Store in cache for next time
    cache.set(userId, user, { ttl: 3600 });
  }

  return user;
}

Pros:

Simple to implement
Only caches what's actually used
Resilient (cache failure doesn't break app)

Cons:

First request is always slow (cache miss)
Extra code in your application

2. Read-Through 📖

Cache sits between your app and database. Cache handles loading automatically.

Application → Ask Cache → Cache checks itself
                              │
                    ┌─────────┴──────────┐
                    │                    │
                Found in cache      Not found
                    │                    │
                    │                    ▼
                    │            Load from Database
                    │                    │
                    │                    ▼
                    │            Store in self
                    │                    │
                    └────────┬───────────┘
                             │
                             ▼
                      Return to Application

Code Example:

// Cache handles loading (you don't!)
class ReadThroughCache {
  constructor(dataLoader) {
    this.cache = new Map();
    this.dataLoader = dataLoader; // Function to load data
  }

  async get(key) {
    // Check cache
    if (this.cache.has(key)) {
      return this.cache.get(key);
    }

    // Not in cache - load using provided loader
    const data = await this.dataLoader(key);
    this.cache.set(key, data);
    return data;
  }
}

// Usage
const userCache = new ReadThroughCache(
  (userId) => database.getUserById(userId)
);

// Just ask cache - it handles everything!
const user = await userCache.get(123);

Pros:

Cleaner application code
Cache logic centralized

Cons:

More complex to set up
Cache is a single point of failure

3. Write-Through ✍️

When you update data, write to BOTH cache and database at the same time.

Application writes data
         │
         ▼
    ┌────────┐
    │ Cache  │ ← Updated immediately
    └────────┘
         │
         ▼
    ┌────────┐
    │   DB   │ ← Also updated
    └────────┘
         │
         ▼
   Return success

Code Example:

async function updateUser(userId, newData) {
  // 1. Update database
  await database.updateUser(userId, newData);

  // 2. Update cache immediately
  cache.set(userId, newData, { ttl: 3600 });

  // Both are in sync! ✅
  return newData;
}

Pros:

Cache always in sync with database
No stale data
Next read is fast (already cached)

Cons:

Writes are slower (two operations)
Wasted cache space if data isn't read soon

4. Write-Back (Write-Behind) 📝

Write to cache immediately, update database later (async).

Application writes data
         │
         ▼
    ┌────────┐
    │ Cache  │ ← Updated immediately
    └────────┘
         │
         └─→ Success returned to app (fast!)

    (Later, in background)
         │
         ▼
    ┌────────┐
    │   DB   │ ← Updated asynchronously
    └────────┘

Code Example:

const writeQueue = [];

async function updateUser(userId, newData) {
  // 1. Update cache immediately
  cache.set(userId, newData, { ttl: 3600 });

  // 2. Queue database write for later
  writeQueue.push({ userId, newData });

  // 3. Return immediately (fast!)
  return newData;
}

// Background process flushes queue
setInterval(async () => {
  while (writeQueue.length > 0) {
    const { userId, newData } = writeQueue.shift();
    await database.updateUser(userId, newData);
  }
}, 5000); // Every 5 seconds

Pros:

Super fast writes
Reduced database load (batch updates)

Cons:

Risk of data loss (if cache crashes before DB update)
Complex to implement
Cache and DB temporarily out of sync

5. Write-Around 🔄

Write directly to database, skip cache. Cache loads on next read.

Application writes data
         │
         ▼
    ┌────────┐
    │ Cache  │ ← NOT updated (deleted if exists)
    └────────┘
         │
         ▼
    ┌────────┐
    │   DB   │ ← Updated directly
    └────────┘

Code Example:

async function updateUser(userId, newData) {
  // 1. Update database directly
  await database.updateUser(userId, newData);

  // 2. Remove from cache (if exists)
  cache.delete(userId);

  // Next read will load fresh data from DB
  return newData;
}

Pros:

Simple
Avoids caching data that's rarely read
No risk of stale data after writes

Cons:

First read after write is slow (cache miss)

Visual: Strategy Comparison

┌──────────────┬─────────────┬─────────────┬──────────────┬──────────────┐
│   Strategy   │ Read Speed  │ Write Speed │ Consistency  │  Complexity  │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Cache-Aside  │   Fast ⚡   │   Medium    │    Good      │     Low      │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Read-Through │   Fast ⚡   │   Medium    │    Good      │    Medium    │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Write-Through│   Fast ⚡   │   Slow 🐢   │  Excellent ✅│    Medium    │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Write-Back   │   Fast ⚡   │  Very Fast⚡│    Risky ⚠️ │     High     │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Write-Around │ Medium/Slow │   Medium    │  Excellent ✅│     Low      │
└──────────────┴─────────────┴─────────────┴──────────────┴──────────────┘

Choosing the Right Strategy

Use Cache-Aside when:

You want simple, flexible caching
You're starting out with caching
Reads are much more common than writes

Use Read-Through when:

You want cleaner code
Cache can be a proxy for data access

Use Write-Through when:

Data consistency is critical
Reads are very frequent after writes
You can tolerate slower writes

Use Write-Back when:

Write performance is critical
You can handle complexity
Small data loss risk is acceptable

Use Write-Around when:

Writes are very common
Data is rarely read after writing
You want to avoid cache pollution

⚠️ Why Caching is One of the Hardest Problems in Computer Science

The Famous Quote

"There are only two hard things in Computer Science: cache invalidation and naming things."
— Phil Karlton

Let's understand WHY caching is so difficult, with real-world scenarios that show what can go wrong.

Problem #1: Cache Invalidation Timing ⏰

The Challenge: Knowing WHEN to remove or update cached data.

Scenario: The Stale Price Problem

// 9:00 AM - Customer views product
const product = cache.get('product:123');
// Shows: $100 ✅

// 9:15 AM - Admin updates price in database
database.update('products', { id: 123, price: 80 });
// Database now has $80
// But cache still has $100! ❌

// 9:20 AM - Another customer views product
const product = cache.get('product:123');
// Still shows: $100 ❌ WRONG!
// Should show: $80

// Result: Customer sees wrong price for 1+ hours until cache expires!

Why This is Hard:

The database doesn't automatically tell the cache it changed
Multiple servers might each have their own cache
You can't know which data will change, when

Real Consequence:

E-commerce: Customer buys at wrong price → lost revenue or angry customer
Banking: Shows wrong account balance → regulatory issues
Social media: Shows deleted post → privacy violation

Problem #2: The Distributed Cache Nightmare 🌐

The Challenge: Keeping multiple caches in sync across different servers.

Visual: Multi-Layer Cache Problem

User Browser Cache:    Price = $100
       ↓
CDN Cache:             Price = $100
       ↓
Server 1 Cache:        Price = $100
Server 2 Cache:        Price = $80  (just updated!)
Server 3 Cache:        Price = $100
       ↓
Database:              Price = $80  (source of truth)

Problem: 4 out of 5 caches are wrong! 😱

Real Scenario:

// Server 1 updates the product
async function updateProduct(id, newPrice) {
  await database.update(id, { price: newPrice });
  serverCache.set(id, { price: newPrice }); // Only updates Server 1's cache!
}

// User request goes to Server 2
// Server 2's cache still has old price!
// Different users see different prices depending on which server they hit!

Why This is Hard:

Can't easily communicate between all cache layers
Network delays mean updates don't arrive simultaneously
Each layer might have different expiration times
CDN caches might be in different countries

Problem #3: Cache Stampede (Thundering Herd) 🐘🐘🐘

The Challenge: When cache expires, thousands of requests hit the database at once.

Visual: The Stampede

Cache Item:     [Alive]  →  [Alive]  →  [EXPIRES!]  →  [Chaos]
Requests:         Fast       Fast         Database     Database
                 (Hit)      (Hit)        overloaded!  overloaded!
                                              │
                                              ▼
                              ┌───────────────────────────────┐
                              │  1000 requests all miss cache │
                              │  All query database at once!  │
                              │  Database crashes! 💥         │
                              └───────────────────────────────┘

Real Example:

// Popular item cached for 1 hour
cache.set('trending-product', data, { ttl: 3600 });

// After 1 hour, cache expires
// Next second: 10,000 users try to view the product
// ALL 10,000 requests miss the cache
// ALL 10,000 requests hit the database simultaneously
// Database can't handle it → crashes or becomes very slow

// Timeline:
// 10:00:00 - Cache expires
// 10:00:00 - Request 1: Cache miss → Query DB
// 10:00:00 - Request 2: Cache miss → Query DB
// 10:00:00 - Request 3: Cache miss → Query DB
// ... (10,000 more requests)
// 10:00:05 - Database timeout!

Solution: Request Coalescing

const pendingRequests = new Map();

async function getProductSafe(productId) {
  const cached = cache.get(productId);
  if (cached) return cached;

  // Check if another request is already fetching this
  if (pendingRequests.has(productId)) {
    console.log('Waiting for existing request...');
    return pendingRequests.get(productId); // Reuse the pending request!
  }

  // First request - fetch from DB
  const promise = database.getProduct(productId)
    .then(data => {
      cache.set(productId, data);
      pendingRequests.delete(productId);
      return data;
    });

  pendingRequests.set(productId, promise);
  return promise;
}

// Now if 10,000 requests come at once, only 1 database query happens!

Why This is Hard:

Need to coordinate between simultaneous requests
Race conditions are tricky to handle
Different for every programming language/framework

Problem #4: Memory Management 💾

The Challenge: Caches use RAM, and RAM is limited.

The Tradeoff:

More Cache:
  ✅ Faster responses
  ✅ Higher hit ratio
  ❌ More memory usage
  ❌ Risk of running out of memory
  ❌ Slower evictions

Less Cache:
  ✅ Less memory usage
  ❌ More cache misses
  ❌ More database load

Real Problem:

// Your server has 8GB RAM
// Operating system needs: 2GB
// Application code needs: 2GB
// Available for cache: 4GB

// Each cached item: ~1MB
// Can cache: ~4,000 items

// But you have 1,000,000 products!
// Can only cache 0.4% of them!

// Question: Which 4,000 products should you cache?
// Answer: The most frequently accessed (LRU)
// Problem: How do you know which are most frequent?

Memory Leak Example:

// BAD: Cache grows forever!
const cache = {};

function addToCache(key, value) {
  cache[key] = value; // Never removes old items!
}

// After 1 month of running:
// Cache has millions of items
// Using 10GB of RAM
// Server crashes! 💥

Good Version with Size Limit:

class BoundedCache {
  constructor(maxItems = 1000) {
    this.cache = new Map();
    this.maxItems = maxItems;
  }

  set(key, value) {
    if (this.cache.size >= this.maxItems) {
      // Remove oldest item (first in Map)
      const firstKey = this.cache.keys().next().value;
      this.cache.delete(firstKey);
    }
    this.cache.set(key, value);
  }
}

Why This is Hard:

Need to constantly monitor memory usage
Deciding what to evict is complex
Different data sizes make it unpredictable
Memory fragmentation over time

Problem #5: Cache Coherence in Distributed Systems 🌍

The Challenge: Multiple servers, each with their own cache.

Scenario: The Split Brain

User A → Server 1 → Cache 1 → Database
                       ↓
                   price: $100

User B → Server 2 → Cache 2 → Database
                       ↓
                   price: $80

Same product, different prices! 😱

Why does this happen?

Step 1: Both servers cache product at $100
  Server 1 Cache: $100 ✅
  Server 2 Cache: $100 ✅

Step 2: Price updated to $80 via Server 1
  Server 1 Cache: $80 ✅ (invalidated & refreshed)
  Server 2 Cache: $100 ❌ (doesn't know about update!)

Step 3: Users see different prices!
  User on Server 1: $80
  User on Server 2: $100

Solution Attempt 1: Short TTL

// Cache for only 30 seconds
cache.set('product', data, { ttl: 30 });

// Pro: Caches sync within 30 seconds
// Con: More database load, cache misses every 30 seconds

Solution Attempt 2: Cache Invalidation Bus

// When Server 1 updates data, broadcast to all servers
function updateProduct(id, newData) {
  database.update(id, newData);

  // Tell ALL servers to invalidate cache
  messageQueue.publish('invalidate', { type: 'product', id });
}

// Each server listens for invalidation messages
messageQueue.subscribe('invalidate', (message) => {
  cache.delete(`product:${message.id}`);
});

// Pro: All caches stay in sync
// Con: Complex infrastructure (need message queue)
// Con: What if a server is down when message is sent?

Why This is Hard:

Network is unreliable (messages can be lost)
Servers can be in different data centers
Race conditions (update + invalidate arrive out of order)
No guarantee all servers receive the message

Problem #6: Stale Data Cascades 🌊

The Challenge: Cached data that depends on other cached data.

Scenario:

// User profile (cached for 1 hour)
cache.set('user:123', {
  name: 'Alice',
  teamId: 456
}, { ttl: 3600 });

// Team info (cached for 1 hour)
cache.set('team:456', {
  name: 'Engineering',
  members: [123, 234, 345]
}, { ttl: 3600 });

// Problem: User leaves team
database.removeUserFromTeam(123, 456);

// Now we have:
// cache['user:123'].teamId = 456 ❌ (should be null)
// cache['team:456'].members includes 123 ❌ (should be removed)

// Need to invalidate BOTH caches!
// But how do you know they're related?

Visual: Dependency Web

Product
  ├─→ Category (cached)
  ├─→ Brand (cached)
  ├─→ Reviews (cached)
  │    └─→ User profiles (cached)
  └─→ Related products (cached)
       └─→ Their categories (cached)

If product changes, need to invalidate 6+ caches!
But which ones? 😰

Why This is Hard:

Need to track dependencies between cached items
Invalidating one cache might require invalidating dozens more
Easy to miss a dependency
Overhead of tracking can be expensive

Problem #7: Testing is Nearly Impossible 🧪

The Challenge: Cache bugs only appear under specific timing conditions.

Why Testing is Hard:

// This test will PASS, but code is buggy!
test('getCachedProduct works', async () => {
  const product = await getCachedProduct(123);
  expect(product.name).toBe('Widget');
  // ✅ Test passes
});

// Real world:
// - What if cache expires between check and use?
// - What if two requests race?
// - What if database updates during cache read?
// - What if cache server disconnects?

// You need to test:
// ❌ Cache hit
// ❌ Cache miss  
// ❌ Cache expiration during request
// ❌ Cache failure/unavailable
// ❌ Stale data
// ❌ Race conditions
// ❌ Cache stampede
// ❌ Memory limits
// ❌ Distributed cache sync

Real Bug Example:

// Looks fine, but has a race condition!
async function getUser(id) {
  let user = cache.get(id);

  if (!user) {
    user = await database.getUser(id);
    cache.set(id, user);
  }

  return user;
}

// Race condition:
// Request A: Check cache → Miss → Start DB query
// Request B: Check cache → Miss → Start DB query
// Request A: DB returns → Set cache
// Request B: DB returns → Set cache
// Result: 2 database queries instead of 1!

Why This is Hard:

Timing bugs are non-deterministic
Only appear under load
Can't reliably reproduce in tests
Require complex mocking of cache behavior

The Fundamental Paradox 🤔

The ultimate problem with caching:

To make your app faster, you add caching.
But caching adds complexity.
Complexity causes bugs.
Debugging takes time.
Time makes development slower.

You added caching to go faster...
But now everything is slower! 😅

The Tradeoffs:

No Cache:
  ✅ Simple, predictable
  ✅ Always correct data
  ✅ Easy to test
  ❌ Slow
  ❌ High database load

With Cache:
  ✅ Fast
  ✅ Reduced database load
  ❌ Complex
  ❌ Risk of stale data
  ❌ Hard to test
  ❌ More things to monitor
  ❌ More things to go wrong

Why We Still Use Caching

Despite all these problems, caching is essential because:

Speed Matters - Users expect instant responses
Scale Requires It - Can't hit database for every request
Cost Savings - Cheaper than buying more database servers
Better UX - Fast apps are delightful to use

The key is: Use caching wisely, understand the tradeoffs, and plan for invalidation from day one!

✅❌ Do's and Don'ts of Caching

✅ DO: Cache Frequently Accessed Data

Good:

// User profile - accessed on every page
cache.set(`user:${userId}`, userProfile, { ttl: 3600 });

// Popular product - viewed by thousands
cache.set(`product:${trendingId}`, product, { ttl: 600 });

Don't:

// One-time password reset token - only used once!
cache.set(`reset-token:${token}`, data, { ttl: 3600 });
// ❌ Waste of cache space!

Rule: Only cache data that will be requested multiple times.

✅ DO: Set Appropriate TTL

Good:

// Static content - long TTL
cache.set('site-logo', logo, { ttl: 86400 }); // 24 hours

// Semi-dynamic - medium TTL
cache.set('product-list', products, { ttl: 300 }); // 5 minutes

// Real-time - short TTL
cache.set('stock-level', count, { ttl: 10 }); // 10 seconds

Don't:

// Everything with same TTL
cache.set('logo', data, { ttl: 60 });
cache.set('user-balance', data, { ttl: 60 });
cache.set('news-feed', data, { ttl: 60 });
// ❌ Different data has different freshness needs!

✅ DO: Use Meaningful Cache Keys

Good:

cache.set(`user:profile:${userId}`, data);
cache.set(`product:${productId}:v2`, data); // includes version
cache.set(`cart:${userId}:${sessionId}`, data);

Don't:

cache.set('data', data); // ❌ Too generic!
cache.set('u123', data); // ❌ Not clear what 'u' means
cache.set(productId.toString(), data); // ❌ No namespace

Pattern: {type}:{identifier}:{sub-identifier}

✅ DO: Handle Cache Misses Gracefully

Good:

async function getProduct(id) {
  try {
    const cached = await cache.get(id);
    if (cached) return cached;

    // Fallback to database
    const product = await database.getProduct(id);

    // Try to cache, but don't fail if caching fails
    try {
      await cache.set(id, product, { ttl: 300 });
    } catch (cacheError) {
      console.warn('Cache set failed:', cacheError);
      // Continue anyway - we have the data!
    }

    return product;
  } catch (error) {
    // If cache.get() fails, go straight to database
    return database.getProduct(id);
  }
}

Don't:

async function getProduct(id) {
  const cached = await cache.get(id); // If this throws, entire function fails!
  if (cached) return cached;
  return database.getProduct(id);
}
// ❌ No error handling - cache failure breaks everything!

✅ DO: Invalidate on Updates

Good:

async function updateProduct(id, newData) {
  // 1. Update database
  await database.update(id, newData);

  // 2. Invalidate cache
  await cache.delete(`product:${id}`);

  // 3. Optionally, pre-warm cache
  await cache.set(`product:${id}`, newData, { ttl: 300 });

  return newData;
}

Don't:

async function updateProduct(id, newData) {
  await database.update(id, newData);
  // ❌ Forgot to invalidate cache!
  // Users will see old data until TTL expires!
  return newData;
}

❌ DON'T: Cache Everything

Bad:

// Caching EVERYTHING
function cacheAll() {
  cache.set('timestamp', Date.now());  // ❌ Changes every millisecond!
  cache.set('random', Math.random());  // ❌ Should be random, not cached!
  cache.set('uuid', generateUUID());   // ❌ Should be unique!
}

Good:

// Cache strategically
function cacheSelectively() {
  // ✅ Cache data that's expensive to compute and doesn't change often
  cache.set('exchange-rates', rates, { ttl: 3600 });
  cache.set('trending-products', products, { ttl: 300 });
}

Rule: Cache only data that is:

Expensive to fetch/compute
Accessed frequently
Doesn't change too often

❌ DON'T: Cache Sensitive Data Without Encryption

Bad:

// ❌ Storing sensitive data in plain text
cache.set(`user:${userId}`, {
  password: 'plaintextpassword',
  creditCard: '1234-5678-9012-3456',
  ssn: '123-45-6789'
});

Good:

// ✅ Only cache non-sensitive data
cache.set(`user:${userId}`, {
  name: 'Alice',
  email: 'alice@example.com',
  preferences: { theme: 'dark' }
});

// Or encrypt if you must cache sensitive data
cache.set(`secure:${userId}`, encrypt(sensitiveData));

❌ DON'T: Cache Without Size Limits

Bad:

// ❌ Unlimited cache - memory leak!
const cache = {};

function addToCache(key, value) {
  cache[key] = value; // Grows forever!
}

Good:

// ✅ Bounded cache with eviction
class LRUCache {
  constructor(maxSize = 1000) {
    this.cache = new Map();
    this.maxSize = maxSize;
  }

  set(key, value) {
    if (this.cache.size >= this.maxSize) {
      const firstKey = this.cache.keys().next().value;
      this.cache.delete(firstKey);
    }
    this.cache.set(key, value);
  }
}

❌ DON'T: Ignore Cache Metrics

Bad:

// ❌ No monitoring
function get(key) {
  return cache.get(key) || fetchFromDB(key);
}

Good:

// ✅ Track cache performance
const metrics = {
  hits: 0,
  misses: 0,
  errors: 0
};

function get(key) {
  try {
    const value = cache.get(key);
    if (value) {
      metrics.hits++;
      return value;
    }
    metrics.misses++;
    return fetchFromDB(key);
  } catch (error) {
    metrics.errors++;
    throw error;
  }
}

// Log metrics periodically
setInterval(() => {
  const hitRate = (metrics.hits / (metrics.hits + metrics.misses)) * 100;
  console.log(`Cache hit rate: ${hitRate.toFixed(1)}%`);
}, 60000);

❌ DON'T: Cache User-Specific Data Globally

Bad:

// ❌ Caching user data without user ID
cache.set('current-user', userData);
// What if two users access the same server?
// They'll get each other's data!

Good:

// ✅ Always include user ID in key
cache.set(`user:${userId}:profile`, userData);
cache.set(`cart:${userId}`, cartData);

❌ DON'T: Cache Errors

Bad:

async function getProduct(id) {
  try {
    const cached = cache.get(id);
    if (cached) return cached;

    const product = await api.getProduct(id); // Might throw error
    cache.set(id, product); // ❌ Caches even if product is an error!
    return product;
  } catch (error) {
    cache.set(id, error); // ❌ Caching errors!
    throw error;
  }
}

Good:

async function getProduct(id) {
  const cached = cache.get(id);
  if (cached) return cached;

  try {
    const product = await api.getProduct(id);
    // ✅ Only cache on success
    if (product && !product.error) {
      cache.set(id, product);
    }
    return product;
  } catch (error) {
    // ✅ Don't cache errors
    throw error;
  }
}

Decision Tree: Should I Cache This?

                    Should I cache this data?
                             │
                             ▼
                   Is it accessed frequently?
                             │
                    ┌────────┴────────┐
                    │                 │
                   NO                YES
                    │                 │
                    ▼                 ▼
              DON'T CACHE    Is it expensive to fetch?
                                     │
                            ┌────────┴────────┐
                            │                 │
                           NO                YES
                            │                 │
                            ▼                 ▼
                      MAYBE CACHE     Does it change often?
                      (if very                │
                       frequent)     ┌────────┴────────┐
                                    │                 │
                                   YES                NO
                                    │                 │
                                    ▼                 ▼
                            Use SHORT TTL      ✅ CACHE IT!
                            or don't cache    (with appropriate TTL)

💻 Frontend Caching: Deep Dive

Now let's focus specifically on caching in frontend applications (JavaScript/Browser).

Frontend Storage Options Comparison

┌───────────────┬──────────┬─────────────┬────────────┬──────────────┐
│  Storage      │   Size   │ Persistence │   Speed    │  Use Case    │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ Variables     │  RAM     │ Page reload │ Fastest ⚡ │ Temp data    │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ SessionStorage│ ~5-10MB  │ Tab closes  │ Very Fast  │ Per-session  │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ LocalStorage  │ ~5-10MB  │ Forever     │ Very Fast  │ Preferences  │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ IndexedDB     │ ~50MB+   │ Forever     │ Fast       │ Large data   │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ Cache API     │ Varies   │ Forever     │ Fast       │ PWA, offline │
└───────────────┴──────────┴─────────────┴────────────┴──────────────┘

Simple Example: In-Memory Cache

Basic implementation:

class SimpleFrontendCache {
  constructor() {
    this.cache = new Map();
  }

  set(key, value, ttlSeconds = 300) {
    const expiresAt = Date.now() + (ttlSeconds * 1000);
    this.cache.set(key, { value, expiresAt });
  }

  get(key) {
    const item = this.cache.get(key);

    if (!item) return null;

    // Check if expired
    if (Date.now() > item.expiresAt) {
      this.cache.delete(key);
      return null;
    }

    return item.value;
  }

  delete(key) {
    this.cache.delete(key);
  }

  clear() {
    this.cache.clear();
  }
}

// Usage
const cache = new SimpleFrontendCache();

// Cache API response
async function fetchUserProfile(userId) {
  const cacheKey = `user:${userId}`;

  // Check cache first
  const cached = cache.get(cacheKey);
  if (cached) {
    console.log('✅ Cache hit!');
    return cached;
  }

  // Fetch from API
  console.log('❌ Cache miss, fetching from API...');
  const response = await fetch(`/api/users/${userId}`);
  const user = await response.json();

  // Store in cache (5 minutes)
  cache.set(cacheKey, user, 300);

  return user;
}

Complex Example: Multi-Layer Cache with localStorage

Advanced implementation with persistence:

class AdvancedFrontendCache {
  constructor(options = {}) {
    this.memoryCache = new Map(); // Fast, temporary
    this.useLocalStorage = options.useLocalStorage !== false;
    this.maxMemoryItems = options.maxMemoryItems || 100;
    this.namespace = options.namespace || 'cache';

    // Cleanup expired items on startup
    this.cleanup();
  }

  /**
   * Generate namespaced key
   */
  _getKey(key) {
    return `${this.namespace}:${key}`;
  }

  /**
   * Set value in cache
   */
  set(key, value, options = {}) {
    const ttl = options.ttl || 300; // Default 5 minutes
    const persist = options.persist !== false;
    const expiresAt = Date.now() + (ttl * 1000);

    const cacheItem = {
      value,
      expiresAt,
      createdAt: Date.now()
    };

    // Store in memory
    if (this.memoryCache.size >= this.maxMemoryItems) {
      // Evict oldest item (LRU)
      const firstKey = this.memoryCache.keys().next().value;
      this.memoryCache.delete(firstKey);
    }
    this.memoryCache.set(key, cacheItem);

    // Optionally persist to localStorage
    if (persist && this.useLocalStorage) {
      try {
        const storageKey = this._getKey(key);
        localStorage.setItem(storageKey, JSON.stringify(cacheItem));
      } catch (error) {
        console.warn('localStorage.setItem failed:', error);
        // Quota exceeded or other error - continue without persisting
      }
    }
  }

  /**
   * Get value from cache
   */
  get(key) {
    // Try memory cache first (fastest)
    let item = this.memoryCache.get(key);

    // If not in memory, try localStorage
    if (!item && this.useLocalStorage) {
      try {
        const storageKey = this._getKey(key);
        const stored = localStorage.getItem(storageKey);
        if (stored) {
          item = JSON.parse(stored);
          // Restore to memory cache
          this.memoryCache.set(key, item);
        }
      } catch (error) {
        console.warn('localStorage.getItem failed:', error);
      }
    }

    if (!item) return null;

    // Check expiration
    if (Date.now() > item.expiresAt) {
      this.delete(key);
      return null;
    }

    return item.value;
  }

  /**
   * Delete item from cache
   */
  delete(key) {
    this.memoryCache.delete(key);

    if (this.useLocalStorage) {
      try {
        const storageKey = this._getKey(key);
        localStorage.removeItem(storageKey);
      } catch (error) {
        console.warn('localStorage.removeItem failed:', error);
      }
    }
  }

  /**
   * Clear all cache
   */
  clear() {
    this.memoryCache.clear();

    if (this.useLocalStorage) {
      try {
        // Remove all items with our namespace
        const keysToRemove = [];
        for (let i = 0; i < localStorage.length; i++) {
          const key = localStorage.key(i);
          if (key && key.startsWith(this.namespace + ':')) {
            keysToRemove.push(key);
          }
        }
        keysToRemove.forEach(key => localStorage.removeItem(key));
      } catch (error) {
        console.warn('localStorage cleanup failed:', error);
      }
    }
  }

  /**
   * Clean up expired items
   */
  cleanup() {
    if (!this.useLocalStorage) return;

    try {
      const now = Date.now();
      const keysToRemove = [];

      for (let i = 0; i < localStorage.length; i++) {
        const key = localStorage.key(i);
        if (key && key.startsWith(this.namespace + ':')) {
          try {
            const item = JSON.parse(localStorage.getItem(key));
            if (item && now > item.expiresAt) {
              keysToRemove.push(key);
            }
          } catch (e) {
            // Invalid JSON, remove it
            keysToRemove.push(key);
          }
        }
      }

      keysToRemove.forEach(key => localStorage.removeItem(key));
      console.log(`Cleaned up ${keysToRemove.length} expired cache items`);
    } catch (error) {
      console.warn('Cache cleanup failed:', error);
    }
  }

  /**
   * Get cache statistics
   */
  getStats() {
    let localStorageCount = 0;
    let totalSize = 0;

    if (this.useLocalStorage) {
      for (let i = 0; i < localStorage.length; i++) {
        const key = localStorage.key(i);
        if (key && key.startsWith(this.namespace + ':')) {
          localStorageCount++;
          const value = localStorage.getItem(key);
          totalSize += value ? value.length : 0;
        }
      }
    }

    return {
      memoryItems: this.memoryCache.size,
      localStorageItems: localStorageCount,
      estimatedSize: totalSize + ' bytes'
    };
  }
}

// Usage Example
const cache = new AdvancedFrontendCache({
  namespace: 'myapp',
  maxMemoryItems: 50,
  useLocalStorage: true
});

// Simple cache
async function getUser(userId) {
  const key = `user:${userId}`;
  const cached = cache.get(key);

  if (cached) return cached;

  const response = await fetch(`/api/users/${userId}`);
  const user = await response.json();

  // Cache for 5 minutes, persist to localStorage
  cache.set(key, user, { ttl: 300, persist: true });

  return user;
}

// Cache with request deduplication (prevent stampede)
const pendingRequests = new Map();

async function getProduct(productId) {
  const key = `product:${productId}`;

  // Check cache
  const cached = cache.get(key);
  if (cached) return cached;

  // Check if request is already pending
  if (pendingRequests.has(key)) {
    console.log('Request already pending, waiting...');
    return pendingRequests.get(key);
  }

  // Make request
  const promise = fetch(`/api/products/${productId}`)
    .then(res => res.json())
    .then(product => {
      cache.set(key, product, { ttl: 600 });
      pendingRequests.delete(key);
      return product;
    })
    .catch(error => {
      pendingRequests.delete(key);
      throw error;
    });

  pendingRequests.set(key, promise);
  return promise;
}

// Cleanup on page unload
window.addEventListener('beforeunload', () => {
  cache.cleanup();
});

// Log stats
console.log('Cache stats:', cache.getStats());

Real-World Example: Product Listing with Cache

class ProductCache {
  constructor() {
    this.cache = new AdvancedFrontendCache({
      namespace: 'products',
      maxMemoryItems: 100
    });
  }

  /**
   * Get product list with caching
   */
  async getProducts(filters = {}) {
    // Create cache key from filters
    const cacheKey = `list:${JSON.stringify(filters)}`;

    const cached = this.cache.get(cacheKey);
    if (cached) {
      console.log('✅ Returning cached product list');
      return cached;
    }

    console.log('⏳ Fetching products from API...');
    const queryParams = new URLSearchParams(filters);
    const response = await fetch(`/api/products?${queryParams}`);
    const products = await response.json();

    // Cache for 2 minutes
    this.cache.set(cacheKey, products, { ttl: 120 });

    return products;
  }

  /**
   * Get single product
   */
  async getProduct(productId) {
    const cacheKey = `product:${productId}`;

    const cached = this.cache.get(cacheKey);
    if (cached) {
      console.log(`✅ Returning cached product ${productId}`);
      return cached;
    }

    console.log(`⏳ Fetching product ${productId} from API...`);
    const response = await fetch(`/api/products/${productId}`);
    const product = await response.json();

    // Cache for 10 minutes
    this.cache.set(cacheKey, product, { ttl: 600 });

    return product;
  }

  /**
   * Invalidate product cache when updated
   */
  invalidateProduct(productId) {
    this.cache.delete(`product:${productId}`);
    // Also clear list caches (they might contain this product)
    this.clearListCache();
  }

  /**
   * Clear all list caches
   */
  clearListCache() {
    // In a real app, you'd track list cache keys
    // For simplicity, we'll clear everything starting with "list:"
    console.log('🗑️ Clearing product list caches');
  }
}

// Usage in React/Vue component
const productCache = new ProductCache();

async function loadProducts() {
  try {
    const products = await productCache.getProducts({ category: 'electronics' });
    displayProducts(products);
  } catch (error) {
    console.error('Failed to load products:', error);
  }
}

🧠 Mental Models & Visual Thinking

Building the right mental model helps you understand caching intuitively.

Mental Model #1: The Library Card Catalog

┌─────────────────────────────────────────────────────────┐
│                      THE LIBRARY                        │
│                                                         │
│  🗄️  Card Catalog (Cache)                             │
│      │                                                  │
│      ├─ Quick lookups                                  │
│      ├─ Limited space                                  │
│      └─ Points to books                                │
│                                                         │
│  📚  Shelves (Database)                                │
│      │                                                  │
│      ├─ All books stored here                         │
│      ├─ Slow to browse                                │
│      └─ Permanent storage                             │
└─────────────────────────────────────────────────────────┘

Finding a book:
1. Check card catalog (fast!) ✅
2. If not listed, search shelves (slow) 🐌
3. Add card for next time 📝

Key Insight: Cache is an index, not the source of truth.

Mental Model #2: Your Brain's Memory

Working Memory (Cache)          Long-term Memory (Database)
┌─────────────┐                ┌──────────────────┐
│  7±2 items  │                │  Unlimited items │
│  Instant    │                │  Slow recall     │
│  Temporary  │                │  Permanent       │
└─────────────┘                └──────────────────┘
      ↓                                 ↓
What you're        ←──────────  Everything you've
thinking                       learned
about NOW

Your brain does this naturally:

Recent info = fast to recall (cached!)
Old info = need to "reload" from long-term memory
Frequently used info = stays in working memory

Apply to code:

Recently accessed data → keep in cache
Rarely used data → fetch when needed
Frequently accessed → cache with long TTL

Mental Model #3: The Notepad System

📱 Phone (Your App)
    │
    ▼
📋 Notepad (Cache)           ← Fast to check
    │                           Limited pages
    │                           Temporary
    ├─ Today's meetings
    ├─ Important phone numbers
    └─ Quick notes

    │ (If not on notepad)
    ▼

🗄️ Filing Cabinet (Database) ← Slow to search
                                Everything stored
                                Permanent

How you use your notepad:

✅ Write down things you'll need soon
✅ Cross out outdated info (invalidation!)
✅ Tear out old pages when full (eviction!)
❌ Don't write down everything
❌ Don't trust outdated notes

Same with caching!

Visual #1: Cache Layers (Onion Model)

                    ┌─────────┐
                    │  USER   │
                    └────┬────┘
                         │
           ┌─────────────┼─────────────┐
           │             │             │
           │   BROWSER CACHE (10ms)   │
           │                           │
           └─────────────┬─────────────┘
                         │
           ┌─────────────┼─────────────┐
           │             │             │
           │     CDN CACHE (50ms)      │
           │                           │
           └─────────────┬─────────────┘
                         │
           ┌─────────────┼─────────────┐
           │             │             │
           │   SERVER CACHE (100ms)    │
           │                           │
           └─────────────┬─────────────┘
                         │
           ┌─────────────┼─────────────┐
           │             │             │
           │    DATABASE (500ms)       │
           │   [Source of Truth]       │
           └───────────────────────────┘

Each layer:
- Faster than the next
- Smaller than the next
- Closer to the user

Strategy: Hit the closest/fastest layer first!

Visual #2: Time-Based Cache Flow

Timeline View of a Cached Item:

0:00        1:00        2:00        3:00        4:00        5:00
│           │           │           │           │           │
│           │           │           │           │           │
│  Add      │  Hit      │  Hit      │  Hit      │  Hit      │  Expired
│  to       │  ✅       │  ✅       │  ✅       │  ✅       │  ❌
│  Cache    │           │           │           │           │  
│           │           │           │           │           │
└───────────┴───────────┴───────────┴───────────┴───────────┴──────→

TTL = 5 minutes

After expiration:
5:01 → Cache Miss → Fetch from DB → Cache again (restart cycle)

Visual #3: Cache Hit Rate Impact

Scenario: 1000 requests, different hit rates

90% Hit Rate (Good!)
━━━━━━━━━━━━━━━━━━━  Cache Hits (900) - Fast! ⚡
══                     Cache Misses (100) - Slow 🐌

50% Hit Rate (Poor)
━━━━━━━━━━            Cache Hits (500) - Fast! ⚡
══════════            Cache Misses (500) - Slow 🐌

10% Hit Rate (Terrible!)
━━                     Cache Hits (100) - Fast! ⚡
══════════════════    Cache Misses (900) - Slow 🐌

Goal: Achieve 80%+ hit rate for meaningful performance improvement

Visual #4: Cache Size vs Performance

          Performance ▲
                      │
                      │         ┌─────────────
                      │        /
                      │       /
                      │      /     Diminishing
                      │     /      Returns
                      │    /
                      │   /
                      │  /
                      │ /
                      │/
──────────────────────┴──────────────────────► Cache Size

Sweet Spot: 
- Not too small (poor hit rate)
- Not too large (memory waste, slow evictions)
- Just right (good hit rate, manageable size)

Typical: Cache 10-20% of your data for 80-90% hit rate
(Pareto Principle applies!)

Visual #5: Stale Data Timeline

Database Value:   $100 ──────────────► $80
Cache Value:      $100 ────────────────────────────► $100 (STALE!)

                  ├─────────────────┬──────────────────────────┤
                  Synchronized      Out of Sync (Dangerous!)

Solutions:
1. TTL:          $100 ──────X  (expire) → Fetch $80
2. Invalidate:   $100 ──────X  (delete) → Fetch $80  
3. Update:       $100 ──────→  $80 (write-through)

The Cache Decision Matrix

                    Change Frequency
                    │
         Rarely     │     Often
        Changes     │    Changes
                    │
─────────────────────┼─────────────────────
         │          │          │
  HIGH   │   ⭐⭐⭐  │   ⭐⭐    │  HIGH
         │  PERFECT │  SHORT   │
 Access  │  TO      │  TTL     │  Access
 Freq    │  CACHE   │  OK      │  Freq
         │          │          │
─────────┼──────────┼──────────┼─────────
         │          │          │
  LOW    │   ⭐     │    ❌    │  LOW
         │  MAYBE   │  DON'T   │
 Access  │  CACHE   │  CACHE   │  Access
 Freq    │  IF SLOW │          │  Freq
         │  TO FETCH│          │
─────────┴──────────┴──────────┴─────────

Use this matrix to decide what to cache!

🛠️ Practical Implementation Guide

Step 1: Audit Your Application

Questions to answer:

What are your slowest operations?

   // Measure everything
   console.time('fetchProducts');
   const products = await fetchProducts();
   console.timeEnd('fetchProducts');
   // fetchProducts: 450ms  ← Candidate for caching!

What data is accessed most frequently?

   // Track access patterns
   const accessLog = {};
   function logAccess(resource) {
     accessLog[resource] = (accessLog[resource] || 0) + 1;
   }

   // After a day:
   // { 'user-profile': 1000, 'product-123': 850, 'settings': 5 }
   //    ↑ Cache this!        ↑ Cache this!      ↑ Don't cache

How often does your data change?
- Every second → Don't cache or very short TTL
- Every minute → Short TTL (30-60s)
- Every hour → Medium TTL (5-15 min)
- Every day → Long TTL (hours)
- Rarely → Very long TTL (days)

Step 2: Choose Your Caching Strategy

Decision Tree:

Are you building frontend or backend?
         │
    ┌────┴────┐
    │         │
Frontend   Backend
    │         │
    │         └─► Server Cache (Redis, Memcached)
    │
    └─► Storage type?
         │
    ┌────┴─────────────┐
    │                  │
Temporary         Persistent
(In-memory)      (localStorage/IndexedDB)

Frontend recommendations:

Temporary session data → In-memory Map
User preferences → localStorage
Large datasets → IndexedDB
API responses → In-memory + localStorage

Step 3: Implement Basic Caching

Start simple, then optimize:

// Phase 1: Simple in-memory cache
const cache = new Map();

async function getData(id) {
  if (cache.has(id)) return cache.get(id);
  const data = await fetchFromAPI(id);
  cache.set(id, data);
  return data;
}

// Phase 2: Add TTL
function cacheWithTTL(id, data, ttl = 300) {
  const expiresAt = Date.now() + ttl * 1000;
  cache.set(id, { data, expiresAt });
}

// Phase 3: Add metrics
const metrics = { hits: 0, misses: 0 };
function getFromCache(id) {
  const item = cache.get(id);
  if (item && Date.now() < item.expiresAt) {
    metrics.hits++;
    return item.data;
  }
  metrics.misses++;
  return null;
}

// Phase 4: Add size limits (use LRU class from earlier)

Step 4: Handle Invalidation

Three approaches:

1. Time-based (Easiest)

// Just let it expire
cache.set(key, data, { ttl: 300 });

2. Event-based (Best for critical data)

// Invalidate on update
function updateProduct(id, newData) {
  database.update(id, newData);
  cache.delete(`product:${id}`);
}

3. Hybrid (Most common)

// TTL as backup, events for immediate updates
cache.set(key, data, { ttl: 3600 }); // 1 hour fallback

eventBus.on('product-updated', (id) => {
  cache.delete(`product:${id}`); // Immediate invalidation
});

Step 5: Monitor & Optimize

Essential metrics:

class CacheWithMetrics {
  constructor() {
    this.cache = new Map();
    this.metrics = {
      hits: 0,
      misses: 0,
      sets: 0,
      deletes: 0,
      errors: 0,
      totalSize: 0
    };
  }

  get(key) {
    const value = this.cache.get(key);
    if (value) {
      this.metrics.hits++;
      return value;
    }
    this.metrics.misses++;
    return null;
  }

  set(key, value) {
    this.metrics.sets++;
    this.metrics.totalSize += JSON.stringify(value).length;
    this.cache.set(key, value);
  }

  getReport() {
    const total = this.metrics.hits + this.metrics.misses;
    return {
      hitRate: ((this.metrics.hits / total) * 100).toFixed(2) + '%',
      totalRequests: total,
      cacheSize: this.cache.size,
      estimatedMemory: (this.metrics.totalSize / 1024).toFixed(2) + ' KB'
    };
  }
}

// Log report every minute
setInterval(() => {
  console.log('Cache Report:', cache.getReport());
}, 60000);

What to watch:

Hit Rate: Should be >80%
Memory Usage: Should be stable (not growing)
Cache Size: Should have a limit
Errors: Should be near 0

Step 6: Test Your Cache

Key scenarios to test:

// Test 1: Cache hit
test('should return cached data', () => {
  cache.set('key1', 'value1');
  expect(cache.get('key1')).toBe('value1');
});

// Test 2: Cache miss
test('should return null on miss', () => {
  expect(cache.get('nonexistent')).toBeNull();
});

// Test 3: Expiration
test('should expire after TTL', async () => {
  cache.set('key1', 'value1', { ttl: 1 }); // 1 second
  await sleep(1100); // Wait 1.1 seconds
  expect(cache.get('key1')).toBeNull();
});

// Test 4: Size limit
test('should evict old items when full', () => {
  const cache = new LRUCache(2);
  cache.set('a', 1);
  cache.set('b', 2);
  cache.set('c', 3); // Should evict 'a'
  expect(cache.get('a')).toBeNull();
  expect(cache.get('b')).toBe(2);
  expect(cache.get('c')).toBe(3);
});

// Test 5: Invalidation
test('should invalidate on update', async () => {
  await cache.set('product:1', { price: 100 });
  await updateProduct(1, { price: 80 });
  expect(cache.get('product:1')).toBeNull();
});

Step 7: Debug Cache Issues

Common problems and solutions:

Problem 1: Stale data

// Symptom: Users see old data
// Solution: Check TTL and invalidation

// Debug:
console.log('Cached value:', cache.get(key));
console.log('Database value:', await db.get(key));
console.log('Cache age:', Date.now() - cacheItem.createdAt);

Problem 2: Memory leak

// Symptom: Memory usage grows over time
// Solution: Add size limits and cleanup

// Debug:
console.log('Cache size:', cache.size);
console.log('Memory used:', process.memoryUsage().heapUsed / 1024 / 1024, 'MB');

Problem 3: Low hit rate

// Symptom: Cache hit rate < 50%
// Solution: Increase TTL or cache more data

// Debug:
const hitRate = metrics.hits / (metrics.hits + metrics.misses);
console.log('Hit rate:', (hitRate * 100).toFixed(1) + '%');
console.log('Most missed keys:', getMostMissedKeys());

Problem 4: Race conditions

// Symptom: Multiple DB queries for same data
// Solution: Request deduplication

// Add this:
const pending = new Map();

async function getData(key) {
  if (pending.has(key)) {
    return pending.get(key); // Reuse pending request
  }
  const promise = fetchData(key);
  pending.set(key, promise);
  try {
    const data = await promise;
    pending.delete(key);
    return data;
  } catch (error) {
    pending.delete(key);
    throw error;
  }
}

Performance Checklist

Before deploying your cache:

[ ] Cache hit rate is >80%
[ ] Memory usage is stable (not growing)
[ ] Cache has size limit
[ ] TTL is appropriate for each data type
[ ] Invalidation works correctly
[ ] Error handling prevents cache failures from breaking app
[ ] Metrics are logged for monitoring
[ ] Tests cover hit, miss, expiration, and eviction
[ ] Documentation explains what's cached and why
[ ] Fallback to source works if cache fails

🎓 Summary: Key Takeaways

The Golden Rules

Cache Wisely
- Only cache frequently accessed data
- Only cache data that's expensive to fetch
- Don't cache everything!
Plan for Invalidation
- Always set a TTL (even if it's long)
- Delete cache on updates
- Monitor for stale data
Limit Cache Size
- Use LRU or similar eviction
- Monitor memory usage
- Don't let cache grow unbounded
Handle Failures
- Cache failure shouldn't break your app
- Always have fallback to source
- Log errors but continue
Measure Everything
- Track hit rate
- Monitor memory
- Log cache operations
- Adjust based on data

When to Use Caching

✅ Good candidates:

API responses
Database queries
Computed values
User profiles
Product catalogs
Static content

❌ Poor candidates:

One-time data
Real-time data (unless very short TTL)
Sensitive data (without encryption)
Random/unique values
Frequently changing data

Final Wisdom

"Premature optimization is the root of all evil" - Donald Knuth

Start simple:

Build your app without caching
Identify bottlenecks with real data
Add caching where it makes the biggest impact
Measure improvements
Iterate

Remember: Caching adds complexity. Only add it when the performance benefit outweighs the complexity cost!

📚 Further Reading

Happy Caching! 🚀

Remember: The best cache is the one that's invisible to your users - it just makes everything faster!