A comprehensive, beginner-friendly guide to understanding and implementing caching systems
📚 Introduction: What is Caching?
The Real-World Analogy
Imagine you're a student studying for exams:
Without Caching:
- Every time you need information, you walk to the library (10 minutes away)
- Find the book on the shelf
- Read the page you need
- Walk back home
- Repeat this for EVERY piece of information you need
With Caching:
- First time: Walk to library, photocopy the important pages
- Keep those photocopies on your desk at home
- Next time you need that info: Just look at your desk (5 seconds!)
- Only go back to the library if you need something NEW
This is exactly what caching does in software!
The Kitchen Pantry Analogy
Think of caching like organizing your kitchen:
🏪 Grocery Store (Database) → Slow, but has EVERYTHING
↓
🚗 Drive & Shop (Network Call) → Takes time & effort
↓
🏠 Pantry (Cache) → Fast, has what you use OFTEN
↓
👨🍳 Cooking (Your App) → Instant access!
You don't drive to the store every time you need salt. You keep frequently-used items in your pantry. That's caching!
What is Caching (Technical Definition)
Caching is the process of storing copies of data in a temporary storage location (the "cache") so that future requests for that data can be served faster.
Key Concept:
Original Data Source (slow) → Cache (fast copy) → Your Application (blazing fast!)
Why Do We Cache?
-
Speed ⚡
- Reading from cache: ~1-10 milliseconds
- Reading from database: ~50-500 milliseconds
- Reading from external API: ~500-5000 milliseconds
-
Reduced Load 📉
- Fewer database queries
- Less network traffic
- Lower server costs
-
Better User Experience 😊
- Instant responses
- Works offline (in some cases)
- Smoother interactions
-
Reliability 🛡️
- Still works if the original source is slow
- Backup data if source is temporarily down
Visual: How Caching Works
┌─────────────────────────────────────────────────────────────┐
│ REQUEST FOR DATA │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ Is it in CACHE? │
└──────────┬───────────┘
│
┌───────────┴───────────┐
│ │
▼ ▼
┌────────┐ ┌──────────┐
│ YES! │ │ NO │
│ (HIT) │ │ (MISS) │
└────┬───┘ └─────┬────┘
│ │
│ ▼
│ ┌────────────────────────┐
│ │ Fetch from Original │
│ │ Source (DB/API/Server) │
│ └───────────┬────────────┘
│ │
│ ▼
│ ┌────────────────────────┐
│ │ Store in Cache │
│ │ for next time │
│ └───────────┬────────────┘
│ │
└──────────┬───────────┘
│
▼
┌────────────────────┐
│ Return Data to │
│ Application │
└────────────────────┘
The Basic Flow
First Request (Cache Miss):
User requests product info
→ Check cache: NOT FOUND ❌
→ Fetch from database (slow: 200ms)
→ Store in cache
→ Return to user
Total time: 200ms
Second Request (Cache Hit):
User requests same product info
→ Check cache: FOUND! ✅
→ Return from cache (fast: 2ms)
Total time: 2ms (100x faster!)
Simple Example: Before vs After
WITHOUT Caching:
async function getProductDetails(productId) {
// Every call goes to the database
const product = await database.query('SELECT * FROM products WHERE id = ?', [productId]);
return product;
}
// User clicks 5 times → 5 database queries (slow!)
WITH Caching:
const cache = {}; // Simple in-memory cache
async function getProductDetails(productId) {
// Check cache first
if (cache[productId]) {
console.log('Cache HIT! 🎯');
return cache[productId]; // Super fast!
}
console.log('Cache MISS 😞');
// Not in cache, get from database
const product = await database.query('SELECT * FROM products WHERE id = ?', [productId]);
// Store in cache for next time
cache[productId] = product;
return product;
}
// User clicks 5 times → 1 database query + 4 cache hits (much faster!)
🎯 Fundamental Principles of Good Caching
1. Cache Hit vs Cache Miss
These are the two outcomes every cache lookup can have:
Cache Hit ✅
- The data you're looking for IS in the cache
- Fast response
- Success!
Cache Miss ❌
- The data you're looking for is NOT in the cache
- Must fetch from original source
- Slower, but necessary
Visual Representation:
Request → Cache Check
│
┌─────────┴─────────┐
│ │
▼ ▼
HIT ✅ MISS ❌
(Fast!) (Fetch from source)
Cache Hit Ratio is a key metric:
Hit Ratio = (Cache Hits / Total Requests) × 100%
Good cache: 80-95% hit ratio
Poor cache: <50% hit ratio
Example:
const cacheStats = {
hits: 0,
misses: 0
};
function getCachedData(key) {
if (cache.has(key)) {
cacheStats.hits++;
console.log(`Hit ratio: ${(cacheStats.hits / (cacheStats.hits + cacheStats.misses) * 100).toFixed(1)}%`);
return cache.get(key);
}
cacheStats.misses++;
const data = fetchFromDatabase(key);
cache.set(key, data);
return data;
}
2. Cache Invalidation: The Hardest Problem
"There are only two hard things in Computer Science: cache invalidation and naming things."
— Phil Karlton
What is Cache Invalidation?
Cache invalidation is the process of removing or updating stale (outdated) data from the cache.
The Problem:
Time 0:00 → User gets product price: $100 (from cache)
Time 0:30 → Admin updates price to $80 (in database)
Time 0:35 → User refreshes page: Still shows $100! (from cache)
⚠️ STALE DATA! The cache doesn't know about the update!
Visual: The Staleness Problem
Database: Cache: User Sees:
$100 $100 $100 ✓
↓ ↓ ↓
$80 (updated!) $100 (old!) $100 ❌ WRONG!
Common Invalidation Strategies:
- Time-Based (TTL - Time To Live)
// Cache expires after 5 minutes
cache.set('product:123', data, { ttl: 300 }); // 300 seconds = 5 min
- Event-Based
// When product is updated, remove from cache
function updateProduct(id, newData) {
database.update(id, newData);
cache.delete(`product:${id}`); // Invalidate immediately!
}
- Version-Based
// Cache key includes version
cache.set('product:123:v2', data);
// When updated, use new version
cache.set('product:123:v3', newData);
3. Temporal and Spatial Locality
These are the two principles that make caching effective:
Temporal Locality ⏰
- "If data is accessed once, it will likely be accessed again soon"
- Example: User viewing their profile page repeatedly
// User views their profile 10 times in a session
// Without cache: 10 database queries
// With cache: 1 database query + 9 cache hits
Spatial Locality 📍
- "If data is accessed, nearby/related data will likely be accessed soon"
- Example: When viewing product #123, user often views related products #124, #125
// Prefetch related products
function getProduct(id) {
const product = cache.get(id) || fetchFromDB(id);
// Prefetch related products (spatial locality!)
product.relatedIds.forEach(relatedId => {
if (!cache.has(relatedId)) {
prefetchProduct(relatedId); // Load in background
}
});
return product;
}
4. TTL (Time To Live)
TTL defines how long data should stay in cache before expiring.
Visual: TTL Lifecycle
Data Added TTL Starts Still Valid Expired!
│ │ │ │
▼ ▼ ▼ ▼
[Store] ────────> [5 min] ────────> [3 min] ────────> [0 min]
Cache: ✅ Cache: ✅ Cache: ✅ Cache: ❌
(Auto-deleted)
Choosing the Right TTL:
// Static content (rarely changes) → Long TTL
cache.set('logo.png', data, { ttl: 86400 }); // 24 hours
// Semi-static content → Medium TTL
cache.set('product-list', data, { ttl: 300 }); // 5 minutes
// Dynamic content → Short TTL
cache.set('stock-count', data, { ttl: 10 }); // 10 seconds
// User session → Session-based TTL
cache.set('user-session', data, { ttl: 1800 }); // 30 minutes
Trade-offs:
| TTL | Pros | Cons |
|---|---|---|
| Long (hours/days) | Fewer database calls, faster | Risk of stale data |
| Short (seconds/minutes) | Fresher data | More database calls |
| No TTL (cache forever) | Maximum speed | Stale data unless manually invalidated |
5. Cache Size Management
Caches have limited space. What happens when the cache is full?
Eviction Policies (How to decide what to remove):
LRU (Least Recently Used) 🏆 Most Common
Remove the item that hasn't been used for the longest time
Cache: [A, B, C, D, E] (full)
Access pattern: A, B, C, A, B
Need to add F → Remove D (least recently used)
New Cache: [A, B, C, E, F]
LFU (Least Frequently Used)
Remove the item that's been accessed the fewest times
Item A: accessed 10 times
Item B: accessed 2 times
Item C: accessed 15 times
Need space → Remove B (least frequently used)
FIFO (First In, First Out)
Remove the oldest item (like a queue)
Cache: [A, B, C, D, E] (full)
Need to add F → Remove A (first in)
New Cache: [B, C, D, E, F]
Visual: LRU in Action
class LRUCache {
constructor(maxSize = 100) {
this.cache = new Map();
this.maxSize = maxSize;
}
get(key) {
if (!this.cache.has(key)) return null;
// Move to end (mark as recently used)
const value = this.cache.get(key);
this.cache.delete(key);
this.cache.set(key, value);
return value;
}
set(key, value) {
// Remove if exists (will re-add at end)
if (this.cache.has(key)) {
this.cache.delete(key);
}
// If full, remove oldest (first item)
if (this.cache.size >= this.maxSize) {
const firstKey = this.cache.keys().next().value;
this.cache.delete(firstKey);
console.log(`Evicted ${firstKey} (LRU)`);
}
this.cache.set(key, value);
}
}
// Usage
const cache = new LRUCache(3); // Max 3 items
cache.set('A', 1);
cache.set('B', 2);
cache.set('C', 3);
cache.set('D', 4); // Cache full! Evicts 'A'
// Cache now: [B, C, D]
6. Cache Coherence
When you have multiple caches, how do you keep them in sync?
The Problem:
User Cache (Browser): price = $100
CDN Cache: price = $100
Server Cache: price = $100
Database: price = $80 (just updated!)
⚠️ Three layers of stale data!
Solution Strategies:
- Write-Through - Update all caches immediately
- Cache Versioning - Use version numbers in cache keys
- Cache Tags/Groups - Invalidate related caches together
- Pub/Sub - Broadcast invalidation events
// Pub/Sub example
eventBus.on('product:updated', (productId) => {
// Invalidate in all cache layers
browserCache.delete(`product:${productId}`);
serverCache.delete(`product:${productId}`);
cdnCache.purge(`/api/product/${productId}`);
});
Visual: Complete Cache Lifecycle
┌─────────────────────────────────────────────────────────────┐
│ CACHE LIFECYCLE │
└─────────────────────────────────────────────────────────────┘
1. DATA REQUESTED
│
▼
2. CHECK CACHE
│
├─→ HIT ✅ → Return immediately (fast!)
│ │
│ └─→ Update access time (for LRU)
│
└─→ MISS ❌ → 3. FETCH FROM SOURCE (slow)
│
▼
4. STORE IN CACHE
│
├─→ Check cache size
│ │
│ └─→ If full: EVICT (LRU/LFU/FIFO)
│
└─→ Set TTL timer
│
▼
5. SERVE TO USER
│
▼
6. WAIT FOR...
│
├─→ TTL expires → DELETE from cache
│
├─→ Manual invalidation → DELETE from cache
│
└─→ Next access → Go to step 2
🗂️ Types of Caching
Caching exists at many different layers of your application. Let's explore each type!
Caching by Location
1. Browser Cache 🌐
Your web browser stores files locally on your computer.
What gets cached:
- HTML pages
- CSS stylesheets
- JavaScript files
- Images, fonts, videos
- API responses (with proper headers)
Visual: Browser Cache Flow
User visits website
↓
Browser checks cache
↓
┌───────────────┬─────────────────┐
│ FOUND ✅ │ NOT FOUND ❌ │
│ (Use cached) │ (Download) │
└───────┬───────┴────────┬────────┘
│ │
│ ▼
│ ┌──────────────────┐
│ │ Download from │
│ │ Server │
│ └────────┬─────────┘
│ │
│ ▼
│ ┌──────────────────┐
│ │ Store in Cache │
│ └────────┬─────────┘
│ │
└───────────────┘
│
▼
Display to User
Example: HTTP Headers
// Server tells browser to cache for 1 hour
response.headers = {
'Cache-Control': 'public, max-age=3600',
'ETag': 'abc123' // Version identifier
};
// Browser automatically caches!
// Next request: No network call needed for 1 hour
2. CDN Cache 🌍
CDN (Content Delivery Network) stores copies of your files on servers around the world.
How it works:
User in Tokyo User in New York
│ │
▼ ▼
CDN Server (Tokyo) CDN Server (New York)
│ │
└────────────┬───────────────────┘
│
▼
Origin Server (California)
Benefits:
- Faster: Users download from nearby servers
- Reduced Load: Origin server handles fewer requests
- Reliability: If origin is down, CDN might still serve cached content
Real-World Example:
// Without CDN: Everyone downloads from your server
<img src="https://yourserver.com/logo.png"> // Slow for distant users
// With CDN: Downloads from nearest location
<img src="https://cdn.yoursite.com/logo.png"> // Fast everywhere!
3. Application Cache (Frontend) 💻
This is cache YOU control in your JavaScript code.
Storage Options:
| Storage Type | Size Limit | Persistence | Use Case |
|---|---|---|---|
| Memory (Variables) | RAM limit | Until page refresh | Temporary, fast access |
| SessionStorage | ~5-10MB | Until tab closes | Per-tab data |
| LocalStorage | ~5-10MB | Forever (until cleared) | User preferences, tokens |
| IndexedDB | ~50MB+ | Forever | Large datasets, offline apps |
| Cache API | Varies | Forever | Service Workers, PWAs |
Example: localStorage
// Simple key-value cache
function getCachedUserProfile(userId) {
const cacheKey = `user:${userId}`;
// Try to get from localStorage
const cached = localStorage.getItem(cacheKey);
if (cached) {
const { data, timestamp } = JSON.parse(cached);
// Check if still valid (5 minutes)
if (Date.now() - timestamp < 5 * 60 * 1000) {
console.log('Cache hit! 🎯');
return data;
}
}
// Cache miss - fetch from API
console.log('Cache miss 😞');
const data = await fetchUserProfile(userId);
// Store in cache
localStorage.setItem(cacheKey, JSON.stringify({
data,
timestamp: Date.now()
}));
return data;
}
4. Server-Side Cache 🖥️
Cache on your backend server.
Common Tools:
- Redis (most popular)
- Memcached
- In-memory objects
Example: Redis Cache
// Node.js with Redis
const redis = require('redis');
const client = redis.createClient();
async function getProduct(productId) {
const cacheKey = `product:${productId}`;
// Check Redis cache
const cached = await client.get(cacheKey);
if (cached) {
console.log('Redis cache hit!');
return JSON.parse(cached);
}
// Fetch from database
const product = await database.query('SELECT * FROM products WHERE id = ?', [productId]);
// Store in Redis (expire in 1 hour)
await client.setex(cacheKey, 3600, JSON.stringify(product));
return product;
}
5. Database Cache 🗄️
Databases have their own internal caching.
Query Result Cache:
-- First query: Slow (reads from disk)
SELECT * FROM products WHERE category = 'electronics';
-- Time: 250ms
-- Same query again: Fast (from cache)
SELECT * FROM products WHERE category = 'electronics';
-- Time: 5ms
Buffer Pool:
- Keeps frequently accessed data in RAM
- Automatically managed by the database
Caching by Strategy
Now let's look at different patterns for how caching works:
1. Cache-Aside (Lazy Loading) 💤
Most common pattern!
You manually check cache, and load data if missing.
Application → Check Cache → Found? Yes → Return data
│
└─→ Not Found? → Fetch from DB
│
▼
Store in Cache
│
▼
Return data
Code Example:
async function getUser(userId) {
// 1. Try cache first
let user = cache.get(userId);
// 2. If not in cache, load from DB
if (!user) {
user = await database.getUserById(userId);
// 3. Store in cache for next time
cache.set(userId, user, { ttl: 3600 });
}
return user;
}
Pros:
- Simple to implement
- Only caches what's actually used
- Resilient (cache failure doesn't break app)
Cons:
- First request is always slow (cache miss)
- Extra code in your application
2. Read-Through 📖
Cache sits between your app and database. Cache handles loading automatically.
Application → Ask Cache → Cache checks itself
│
┌─────────┴──────────┐
│ │
Found in cache Not found
│ │
│ ▼
│ Load from Database
│ │
│ ▼
│ Store in self
│ │
└────────┬───────────┘
│
▼
Return to Application
Code Example:
// Cache handles loading (you don't!)
class ReadThroughCache {
constructor(dataLoader) {
this.cache = new Map();
this.dataLoader = dataLoader; // Function to load data
}
async get(key) {
// Check cache
if (this.cache.has(key)) {
return this.cache.get(key);
}
// Not in cache - load using provided loader
const data = await this.dataLoader(key);
this.cache.set(key, data);
return data;
}
}
// Usage
const userCache = new ReadThroughCache(
(userId) => database.getUserById(userId)
);
// Just ask cache - it handles everything!
const user = await userCache.get(123);
Pros:
- Cleaner application code
- Cache logic centralized
Cons:
- More complex to set up
- Cache is a single point of failure
3. Write-Through ✍️
When you update data, write to BOTH cache and database at the same time.
Application writes data
│
▼
┌────────┐
│ Cache │ ← Updated immediately
└────────┘
│
▼
┌────────┐
│ DB │ ← Also updated
└────────┘
│
▼
Return success
Code Example:
async function updateUser(userId, newData) {
// 1. Update database
await database.updateUser(userId, newData);
// 2. Update cache immediately
cache.set(userId, newData, { ttl: 3600 });
// Both are in sync! ✅
return newData;
}
Pros:
- Cache always in sync with database
- No stale data
- Next read is fast (already cached)
Cons:
- Writes are slower (two operations)
- Wasted cache space if data isn't read soon
4. Write-Back (Write-Behind) 📝
Write to cache immediately, update database later (async).
Application writes data
│
▼
┌────────┐
│ Cache │ ← Updated immediately
└────────┘
│
└─→ Success returned to app (fast!)
(Later, in background)
│
▼
┌────────┐
│ DB │ ← Updated asynchronously
└────────┘
Code Example:
const writeQueue = [];
async function updateUser(userId, newData) {
// 1. Update cache immediately
cache.set(userId, newData, { ttl: 3600 });
// 2. Queue database write for later
writeQueue.push({ userId, newData });
// 3. Return immediately (fast!)
return newData;
}
// Background process flushes queue
setInterval(async () => {
while (writeQueue.length > 0) {
const { userId, newData } = writeQueue.shift();
await database.updateUser(userId, newData);
}
}, 5000); // Every 5 seconds
Pros:
- Super fast writes
- Reduced database load (batch updates)
Cons:
- Risk of data loss (if cache crashes before DB update)
- Complex to implement
- Cache and DB temporarily out of sync
5. Write-Around 🔄
Write directly to database, skip cache. Cache loads on next read.
Application writes data
│
▼
┌────────┐
│ Cache │ ← NOT updated (deleted if exists)
└────────┘
│
▼
┌────────┐
│ DB │ ← Updated directly
└────────┘
Code Example:
async function updateUser(userId, newData) {
// 1. Update database directly
await database.updateUser(userId, newData);
// 2. Remove from cache (if exists)
cache.delete(userId);
// Next read will load fresh data from DB
return newData;
}
Pros:
- Simple
- Avoids caching data that's rarely read
- No risk of stale data after writes
Cons:
- First read after write is slow (cache miss)
Visual: Strategy Comparison
┌──────────────┬─────────────┬─────────────┬──────────────┬──────────────┐
│ Strategy │ Read Speed │ Write Speed │ Consistency │ Complexity │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Cache-Aside │ Fast ⚡ │ Medium │ Good │ Low │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Read-Through │ Fast ⚡ │ Medium │ Good │ Medium │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Write-Through│ Fast ⚡ │ Slow 🐢 │ Excellent ✅│ Medium │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Write-Back │ Fast ⚡ │ Very Fast⚡│ Risky ⚠️ │ High │
├──────────────┼─────────────┼─────────────┼──────────────┼──────────────┤
│ Write-Around │ Medium/Slow │ Medium │ Excellent ✅│ Low │
└──────────────┴─────────────┴─────────────┴──────────────┴──────────────┘
Choosing the Right Strategy
Use Cache-Aside when:
- You want simple, flexible caching
- You're starting out with caching
- Reads are much more common than writes
Use Read-Through when:
- You want cleaner code
- Cache can be a proxy for data access
Use Write-Through when:
- Data consistency is critical
- Reads are very frequent after writes
- You can tolerate slower writes
Use Write-Back when:
- Write performance is critical
- You can handle complexity
- Small data loss risk is acceptable
Use Write-Around when:
- Writes are very common
- Data is rarely read after writing
- You want to avoid cache pollution
⚠️ Why Caching is One of the Hardest Problems in Computer Science
The Famous Quote
"There are only two hard things in Computer Science: cache invalidation and naming things."
— Phil Karlton
Let's understand WHY caching is so difficult, with real-world scenarios that show what can go wrong.
Problem #1: Cache Invalidation Timing ⏰
The Challenge: Knowing WHEN to remove or update cached data.
Scenario: The Stale Price Problem
// 9:00 AM - Customer views product
const product = cache.get('product:123');
// Shows: $100 ✅
// 9:15 AM - Admin updates price in database
database.update('products', { id: 123, price: 80 });
// Database now has $80
// But cache still has $100! ❌
// 9:20 AM - Another customer views product
const product = cache.get('product:123');
// Still shows: $100 ❌ WRONG!
// Should show: $80
// Result: Customer sees wrong price for 1+ hours until cache expires!
Why This is Hard:
- The database doesn't automatically tell the cache it changed
- Multiple servers might each have their own cache
- You can't know which data will change, when
Real Consequence:
- E-commerce: Customer buys at wrong price → lost revenue or angry customer
- Banking: Shows wrong account balance → regulatory issues
- Social media: Shows deleted post → privacy violation
Problem #2: The Distributed Cache Nightmare 🌐
The Challenge: Keeping multiple caches in sync across different servers.
Visual: Multi-Layer Cache Problem
User Browser Cache: Price = $100
↓
CDN Cache: Price = $100
↓
Server 1 Cache: Price = $100
Server 2 Cache: Price = $80 (just updated!)
Server 3 Cache: Price = $100
↓
Database: Price = $80 (source of truth)
Problem: 4 out of 5 caches are wrong! 😱
Real Scenario:
// Server 1 updates the product
async function updateProduct(id, newPrice) {
await database.update(id, { price: newPrice });
serverCache.set(id, { price: newPrice }); // Only updates Server 1's cache!
}
// User request goes to Server 2
// Server 2's cache still has old price!
// Different users see different prices depending on which server they hit!
Why This is Hard:
- Can't easily communicate between all cache layers
- Network delays mean updates don't arrive simultaneously
- Each layer might have different expiration times
- CDN caches might be in different countries
Problem #3: Cache Stampede (Thundering Herd) 🐘🐘🐘
The Challenge: When cache expires, thousands of requests hit the database at once.
Visual: The Stampede
Cache Item: [Alive] → [Alive] → [EXPIRES!] → [Chaos]
Requests: Fast Fast Database Database
(Hit) (Hit) overloaded! overloaded!
│
▼
┌───────────────────────────────┐
│ 1000 requests all miss cache │
│ All query database at once! │
│ Database crashes! 💥 │
└───────────────────────────────┘
Real Example:
// Popular item cached for 1 hour
cache.set('trending-product', data, { ttl: 3600 });
// After 1 hour, cache expires
// Next second: 10,000 users try to view the product
// ALL 10,000 requests miss the cache
// ALL 10,000 requests hit the database simultaneously
// Database can't handle it → crashes or becomes very slow
// Timeline:
// 10:00:00 - Cache expires
// 10:00:00 - Request 1: Cache miss → Query DB
// 10:00:00 - Request 2: Cache miss → Query DB
// 10:00:00 - Request 3: Cache miss → Query DB
// ... (10,000 more requests)
// 10:00:05 - Database timeout!
Solution: Request Coalescing
const pendingRequests = new Map();
async function getProductSafe(productId) {
const cached = cache.get(productId);
if (cached) return cached;
// Check if another request is already fetching this
if (pendingRequests.has(productId)) {
console.log('Waiting for existing request...');
return pendingRequests.get(productId); // Reuse the pending request!
}
// First request - fetch from DB
const promise = database.getProduct(productId)
.then(data => {
cache.set(productId, data);
pendingRequests.delete(productId);
return data;
});
pendingRequests.set(productId, promise);
return promise;
}
// Now if 10,000 requests come at once, only 1 database query happens!
Why This is Hard:
- Need to coordinate between simultaneous requests
- Race conditions are tricky to handle
- Different for every programming language/framework
Problem #4: Memory Management 💾
The Challenge: Caches use RAM, and RAM is limited.
The Tradeoff:
More Cache:
✅ Faster responses
✅ Higher hit ratio
❌ More memory usage
❌ Risk of running out of memory
❌ Slower evictions
Less Cache:
✅ Less memory usage
❌ More cache misses
❌ More database load
Real Problem:
// Your server has 8GB RAM
// Operating system needs: 2GB
// Application code needs: 2GB
// Available for cache: 4GB
// Each cached item: ~1MB
// Can cache: ~4,000 items
// But you have 1,000,000 products!
// Can only cache 0.4% of them!
// Question: Which 4,000 products should you cache?
// Answer: The most frequently accessed (LRU)
// Problem: How do you know which are most frequent?
Memory Leak Example:
// BAD: Cache grows forever!
const cache = {};
function addToCache(key, value) {
cache[key] = value; // Never removes old items!
}
// After 1 month of running:
// Cache has millions of items
// Using 10GB of RAM
// Server crashes! 💥
Good Version with Size Limit:
class BoundedCache {
constructor(maxItems = 1000) {
this.cache = new Map();
this.maxItems = maxItems;
}
set(key, value) {
if (this.cache.size >= this.maxItems) {
// Remove oldest item (first in Map)
const firstKey = this.cache.keys().next().value;
this.cache.delete(firstKey);
}
this.cache.set(key, value);
}
}
Why This is Hard:
- Need to constantly monitor memory usage
- Deciding what to evict is complex
- Different data sizes make it unpredictable
- Memory fragmentation over time
Problem #5: Cache Coherence in Distributed Systems 🌍
The Challenge: Multiple servers, each with their own cache.
Scenario: The Split Brain
User A → Server 1 → Cache 1 → Database
↓
price: $100
User B → Server 2 → Cache 2 → Database
↓
price: $80
Same product, different prices! 😱
Why does this happen?
Step 1: Both servers cache product at $100
Server 1 Cache: $100 ✅
Server 2 Cache: $100 ✅
Step 2: Price updated to $80 via Server 1
Server 1 Cache: $80 ✅ (invalidated & refreshed)
Server 2 Cache: $100 ❌ (doesn't know about update!)
Step 3: Users see different prices!
User on Server 1: $80
User on Server 2: $100
Solution Attempt 1: Short TTL
// Cache for only 30 seconds
cache.set('product', data, { ttl: 30 });
// Pro: Caches sync within 30 seconds
// Con: More database load, cache misses every 30 seconds
Solution Attempt 2: Cache Invalidation Bus
// When Server 1 updates data, broadcast to all servers
function updateProduct(id, newData) {
database.update(id, newData);
// Tell ALL servers to invalidate cache
messageQueue.publish('invalidate', { type: 'product', id });
}
// Each server listens for invalidation messages
messageQueue.subscribe('invalidate', (message) => {
cache.delete(`product:${message.id}`);
});
// Pro: All caches stay in sync
// Con: Complex infrastructure (need message queue)
// Con: What if a server is down when message is sent?
Why This is Hard:
- Network is unreliable (messages can be lost)
- Servers can be in different data centers
- Race conditions (update + invalidate arrive out of order)
- No guarantee all servers receive the message
Problem #6: Stale Data Cascades 🌊
The Challenge: Cached data that depends on other cached data.
Scenario:
// User profile (cached for 1 hour)
cache.set('user:123', {
name: 'Alice',
teamId: 456
}, { ttl: 3600 });
// Team info (cached for 1 hour)
cache.set('team:456', {
name: 'Engineering',
members: [123, 234, 345]
}, { ttl: 3600 });
// Problem: User leaves team
database.removeUserFromTeam(123, 456);
// Now we have:
// cache['user:123'].teamId = 456 ❌ (should be null)
// cache['team:456'].members includes 123 ❌ (should be removed)
// Need to invalidate BOTH caches!
// But how do you know they're related?
Visual: Dependency Web
Product
├─→ Category (cached)
├─→ Brand (cached)
├─→ Reviews (cached)
│ └─→ User profiles (cached)
└─→ Related products (cached)
└─→ Their categories (cached)
If product changes, need to invalidate 6+ caches!
But which ones? 😰
Why This is Hard:
- Need to track dependencies between cached items
- Invalidating one cache might require invalidating dozens more
- Easy to miss a dependency
- Overhead of tracking can be expensive
Problem #7: Testing is Nearly Impossible 🧪
The Challenge: Cache bugs only appear under specific timing conditions.
Why Testing is Hard:
// This test will PASS, but code is buggy!
test('getCachedProduct works', async () => {
const product = await getCachedProduct(123);
expect(product.name).toBe('Widget');
// ✅ Test passes
});
// Real world:
// - What if cache expires between check and use?
// - What if two requests race?
// - What if database updates during cache read?
// - What if cache server disconnects?
// You need to test:
// ❌ Cache hit
// ❌ Cache miss
// ❌ Cache expiration during request
// ❌ Cache failure/unavailable
// ❌ Stale data
// ❌ Race conditions
// ❌ Cache stampede
// ❌ Memory limits
// ❌ Distributed cache sync
Real Bug Example:
// Looks fine, but has a race condition!
async function getUser(id) {
let user = cache.get(id);
if (!user) {
user = await database.getUser(id);
cache.set(id, user);
}
return user;
}
// Race condition:
// Request A: Check cache → Miss → Start DB query
// Request B: Check cache → Miss → Start DB query
// Request A: DB returns → Set cache
// Request B: DB returns → Set cache
// Result: 2 database queries instead of 1!
Why This is Hard:
- Timing bugs are non-deterministic
- Only appear under load
- Can't reliably reproduce in tests
- Require complex mocking of cache behavior
The Fundamental Paradox 🤔
The ultimate problem with caching:
To make your app faster, you add caching.
But caching adds complexity.
Complexity causes bugs.
Debugging takes time.
Time makes development slower.
You added caching to go faster...
But now everything is slower! 😅
The Tradeoffs:
No Cache:
✅ Simple, predictable
✅ Always correct data
✅ Easy to test
❌ Slow
❌ High database load
With Cache:
✅ Fast
✅ Reduced database load
❌ Complex
❌ Risk of stale data
❌ Hard to test
❌ More things to monitor
❌ More things to go wrong
Why We Still Use Caching
Despite all these problems, caching is essential because:
- Speed Matters - Users expect instant responses
- Scale Requires It - Can't hit database for every request
- Cost Savings - Cheaper than buying more database servers
- Better UX - Fast apps are delightful to use
The key is: Use caching wisely, understand the tradeoffs, and plan for invalidation from day one!
✅❌ Do's and Don'ts of Caching
✅ DO: Cache Frequently Accessed Data
Good:
// User profile - accessed on every page
cache.set(`user:${userId}`, userProfile, { ttl: 3600 });
// Popular product - viewed by thousands
cache.set(`product:${trendingId}`, product, { ttl: 600 });
Don't:
// One-time password reset token - only used once!
cache.set(`reset-token:${token}`, data, { ttl: 3600 });
// ❌ Waste of cache space!
Rule: Only cache data that will be requested multiple times.
✅ DO: Set Appropriate TTL
Good:
// Static content - long TTL
cache.set('site-logo', logo, { ttl: 86400 }); // 24 hours
// Semi-dynamic - medium TTL
cache.set('product-list', products, { ttl: 300 }); // 5 minutes
// Real-time - short TTL
cache.set('stock-level', count, { ttl: 10 }); // 10 seconds
Don't:
// Everything with same TTL
cache.set('logo', data, { ttl: 60 });
cache.set('user-balance', data, { ttl: 60 });
cache.set('news-feed', data, { ttl: 60 });
// ❌ Different data has different freshness needs!
✅ DO: Use Meaningful Cache Keys
Good:
cache.set(`user:profile:${userId}`, data);
cache.set(`product:${productId}:v2`, data); // includes version
cache.set(`cart:${userId}:${sessionId}`, data);
Don't:
cache.set('data', data); // ❌ Too generic!
cache.set('u123', data); // ❌ Not clear what 'u' means
cache.set(productId.toString(), data); // ❌ No namespace
Pattern: {type}:{identifier}:{sub-identifier}
✅ DO: Handle Cache Misses Gracefully
Good:
async function getProduct(id) {
try {
const cached = await cache.get(id);
if (cached) return cached;
// Fallback to database
const product = await database.getProduct(id);
// Try to cache, but don't fail if caching fails
try {
await cache.set(id, product, { ttl: 300 });
} catch (cacheError) {
console.warn('Cache set failed:', cacheError);
// Continue anyway - we have the data!
}
return product;
} catch (error) {
// If cache.get() fails, go straight to database
return database.getProduct(id);
}
}
Don't:
async function getProduct(id) {
const cached = await cache.get(id); // If this throws, entire function fails!
if (cached) return cached;
return database.getProduct(id);
}
// ❌ No error handling - cache failure breaks everything!
✅ DO: Invalidate on Updates
Good:
async function updateProduct(id, newData) {
// 1. Update database
await database.update(id, newData);
// 2. Invalidate cache
await cache.delete(`product:${id}`);
// 3. Optionally, pre-warm cache
await cache.set(`product:${id}`, newData, { ttl: 300 });
return newData;
}
Don't:
async function updateProduct(id, newData) {
await database.update(id, newData);
// ❌ Forgot to invalidate cache!
// Users will see old data until TTL expires!
return newData;
}
❌ DON'T: Cache Everything
Bad:
// Caching EVERYTHING
function cacheAll() {
cache.set('timestamp', Date.now()); // ❌ Changes every millisecond!
cache.set('random', Math.random()); // ❌ Should be random, not cached!
cache.set('uuid', generateUUID()); // ❌ Should be unique!
}
Good:
// Cache strategically
function cacheSelectively() {
// ✅ Cache data that's expensive to compute and doesn't change often
cache.set('exchange-rates', rates, { ttl: 3600 });
cache.set('trending-products', products, { ttl: 300 });
}
Rule: Cache only data that is:
- Expensive to fetch/compute
- Accessed frequently
- Doesn't change too often
❌ DON'T: Cache Sensitive Data Without Encryption
Bad:
// ❌ Storing sensitive data in plain text
cache.set(`user:${userId}`, {
password: 'plaintextpassword',
creditCard: '1234-5678-9012-3456',
ssn: '123-45-6789'
});
Good:
// ✅ Only cache non-sensitive data
cache.set(`user:${userId}`, {
name: 'Alice',
email: 'alice@example.com',
preferences: { theme: 'dark' }
});
// Or encrypt if you must cache sensitive data
cache.set(`secure:${userId}`, encrypt(sensitiveData));
❌ DON'T: Cache Without Size Limits
Bad:
// ❌ Unlimited cache - memory leak!
const cache = {};
function addToCache(key, value) {
cache[key] = value; // Grows forever!
}
Good:
// ✅ Bounded cache with eviction
class LRUCache {
constructor(maxSize = 1000) {
this.cache = new Map();
this.maxSize = maxSize;
}
set(key, value) {
if (this.cache.size >= this.maxSize) {
const firstKey = this.cache.keys().next().value;
this.cache.delete(firstKey);
}
this.cache.set(key, value);
}
}
❌ DON'T: Ignore Cache Metrics
Bad:
// ❌ No monitoring
function get(key) {
return cache.get(key) || fetchFromDB(key);
}
Good:
// ✅ Track cache performance
const metrics = {
hits: 0,
misses: 0,
errors: 0
};
function get(key) {
try {
const value = cache.get(key);
if (value) {
metrics.hits++;
return value;
}
metrics.misses++;
return fetchFromDB(key);
} catch (error) {
metrics.errors++;
throw error;
}
}
// Log metrics periodically
setInterval(() => {
const hitRate = (metrics.hits / (metrics.hits + metrics.misses)) * 100;
console.log(`Cache hit rate: ${hitRate.toFixed(1)}%`);
}, 60000);
❌ DON'T: Cache User-Specific Data Globally
Bad:
// ❌ Caching user data without user ID
cache.set('current-user', userData);
// What if two users access the same server?
// They'll get each other's data!
Good:
// ✅ Always include user ID in key
cache.set(`user:${userId}:profile`, userData);
cache.set(`cart:${userId}`, cartData);
❌ DON'T: Cache Errors
Bad:
async function getProduct(id) {
try {
const cached = cache.get(id);
if (cached) return cached;
const product = await api.getProduct(id); // Might throw error
cache.set(id, product); // ❌ Caches even if product is an error!
return product;
} catch (error) {
cache.set(id, error); // ❌ Caching errors!
throw error;
}
}
Good:
async function getProduct(id) {
const cached = cache.get(id);
if (cached) return cached;
try {
const product = await api.getProduct(id);
// ✅ Only cache on success
if (product && !product.error) {
cache.set(id, product);
}
return product;
} catch (error) {
// ✅ Don't cache errors
throw error;
}
}
Decision Tree: Should I Cache This?
Should I cache this data?
│
▼
Is it accessed frequently?
│
┌────────┴────────┐
│ │
NO YES
│ │
▼ ▼
DON'T CACHE Is it expensive to fetch?
│
┌────────┴────────┐
│ │
NO YES
│ │
▼ ▼
MAYBE CACHE Does it change often?
(if very │
frequent) ┌────────┴────────┐
│ │
YES NO
│ │
▼ ▼
Use SHORT TTL ✅ CACHE IT!
or don't cache (with appropriate TTL)
💻 Frontend Caching: Deep Dive
Now let's focus specifically on caching in frontend applications (JavaScript/Browser).
Frontend Storage Options Comparison
┌───────────────┬──────────┬─────────────┬────────────┬──────────────┐
│ Storage │ Size │ Persistence │ Speed │ Use Case │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ Variables │ RAM │ Page reload │ Fastest ⚡ │ Temp data │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ SessionStorage│ ~5-10MB │ Tab closes │ Very Fast │ Per-session │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ LocalStorage │ ~5-10MB │ Forever │ Very Fast │ Preferences │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ IndexedDB │ ~50MB+ │ Forever │ Fast │ Large data │
├───────────────┼──────────┼─────────────┼────────────┼──────────────┤
│ Cache API │ Varies │ Forever │ Fast │ PWA, offline │
└───────────────┴──────────┴─────────────┴────────────┴──────────────┘
Simple Example: In-Memory Cache
Basic implementation:
class SimpleFrontendCache {
constructor() {
this.cache = new Map();
}
set(key, value, ttlSeconds = 300) {
const expiresAt = Date.now() + (ttlSeconds * 1000);
this.cache.set(key, { value, expiresAt });
}
get(key) {
const item = this.cache.get(key);
if (!item) return null;
// Check if expired
if (Date.now() > item.expiresAt) {
this.cache.delete(key);
return null;
}
return item.value;
}
delete(key) {
this.cache.delete(key);
}
clear() {
this.cache.clear();
}
}
// Usage
const cache = new SimpleFrontendCache();
// Cache API response
async function fetchUserProfile(userId) {
const cacheKey = `user:${userId}`;
// Check cache first
const cached = cache.get(cacheKey);
if (cached) {
console.log('✅ Cache hit!');
return cached;
}
// Fetch from API
console.log('❌ Cache miss, fetching from API...');
const response = await fetch(`/api/users/${userId}`);
const user = await response.json();
// Store in cache (5 minutes)
cache.set(cacheKey, user, 300);
return user;
}
Complex Example: Multi-Layer Cache with localStorage
Advanced implementation with persistence:
class AdvancedFrontendCache {
constructor(options = {}) {
this.memoryCache = new Map(); // Fast, temporary
this.useLocalStorage = options.useLocalStorage !== false;
this.maxMemoryItems = options.maxMemoryItems || 100;
this.namespace = options.namespace || 'cache';
// Cleanup expired items on startup
this.cleanup();
}
/**
* Generate namespaced key
*/
_getKey(key) {
return `${this.namespace}:${key}`;
}
/**
* Set value in cache
*/
set(key, value, options = {}) {
const ttl = options.ttl || 300; // Default 5 minutes
const persist = options.persist !== false;
const expiresAt = Date.now() + (ttl * 1000);
const cacheItem = {
value,
expiresAt,
createdAt: Date.now()
};
// Store in memory
if (this.memoryCache.size >= this.maxMemoryItems) {
// Evict oldest item (LRU)
const firstKey = this.memoryCache.keys().next().value;
this.memoryCache.delete(firstKey);
}
this.memoryCache.set(key, cacheItem);
// Optionally persist to localStorage
if (persist && this.useLocalStorage) {
try {
const storageKey = this._getKey(key);
localStorage.setItem(storageKey, JSON.stringify(cacheItem));
} catch (error) {
console.warn('localStorage.setItem failed:', error);
// Quota exceeded or other error - continue without persisting
}
}
}
/**
* Get value from cache
*/
get(key) {
// Try memory cache first (fastest)
let item = this.memoryCache.get(key);
// If not in memory, try localStorage
if (!item && this.useLocalStorage) {
try {
const storageKey = this._getKey(key);
const stored = localStorage.getItem(storageKey);
if (stored) {
item = JSON.parse(stored);
// Restore to memory cache
this.memoryCache.set(key, item);
}
} catch (error) {
console.warn('localStorage.getItem failed:', error);
}
}
if (!item) return null;
// Check expiration
if (Date.now() > item.expiresAt) {
this.delete(key);
return null;
}
return item.value;
}
/**
* Delete item from cache
*/
delete(key) {
this.memoryCache.delete(key);
if (this.useLocalStorage) {
try {
const storageKey = this._getKey(key);
localStorage.removeItem(storageKey);
} catch (error) {
console.warn('localStorage.removeItem failed:', error);
}
}
}
/**
* Clear all cache
*/
clear() {
this.memoryCache.clear();
if (this.useLocalStorage) {
try {
// Remove all items with our namespace
const keysToRemove = [];
for (let i = 0; i < localStorage.length; i++) {
const key = localStorage.key(i);
if (key && key.startsWith(this.namespace + ':')) {
keysToRemove.push(key);
}
}
keysToRemove.forEach(key => localStorage.removeItem(key));
} catch (error) {
console.warn('localStorage cleanup failed:', error);
}
}
}
/**
* Clean up expired items
*/
cleanup() {
if (!this.useLocalStorage) return;
try {
const now = Date.now();
const keysToRemove = [];
for (let i = 0; i < localStorage.length; i++) {
const key = localStorage.key(i);
if (key && key.startsWith(this.namespace + ':')) {
try {
const item = JSON.parse(localStorage.getItem(key));
if (item && now > item.expiresAt) {
keysToRemove.push(key);
}
} catch (e) {
// Invalid JSON, remove it
keysToRemove.push(key);
}
}
}
keysToRemove.forEach(key => localStorage.removeItem(key));
console.log(`Cleaned up ${keysToRemove.length} expired cache items`);
} catch (error) {
console.warn('Cache cleanup failed:', error);
}
}
/**
* Get cache statistics
*/
getStats() {
let localStorageCount = 0;
let totalSize = 0;
if (this.useLocalStorage) {
for (let i = 0; i < localStorage.length; i++) {
const key = localStorage.key(i);
if (key && key.startsWith(this.namespace + ':')) {
localStorageCount++;
const value = localStorage.getItem(key);
totalSize += value ? value.length : 0;
}
}
}
return {
memoryItems: this.memoryCache.size,
localStorageItems: localStorageCount,
estimatedSize: totalSize + ' bytes'
};
}
}
// Usage Example
const cache = new AdvancedFrontendCache({
namespace: 'myapp',
maxMemoryItems: 50,
useLocalStorage: true
});
// Simple cache
async function getUser(userId) {
const key = `user:${userId}`;
const cached = cache.get(key);
if (cached) return cached;
const response = await fetch(`/api/users/${userId}`);
const user = await response.json();
// Cache for 5 minutes, persist to localStorage
cache.set(key, user, { ttl: 300, persist: true });
return user;
}
// Cache with request deduplication (prevent stampede)
const pendingRequests = new Map();
async function getProduct(productId) {
const key = `product:${productId}`;
// Check cache
const cached = cache.get(key);
if (cached) return cached;
// Check if request is already pending
if (pendingRequests.has(key)) {
console.log('Request already pending, waiting...');
return pendingRequests.get(key);
}
// Make request
const promise = fetch(`/api/products/${productId}`)
.then(res => res.json())
.then(product => {
cache.set(key, product, { ttl: 600 });
pendingRequests.delete(key);
return product;
})
.catch(error => {
pendingRequests.delete(key);
throw error;
});
pendingRequests.set(key, promise);
return promise;
}
// Cleanup on page unload
window.addEventListener('beforeunload', () => {
cache.cleanup();
});
// Log stats
console.log('Cache stats:', cache.getStats());
Real-World Example: Product Listing with Cache
class ProductCache {
constructor() {
this.cache = new AdvancedFrontendCache({
namespace: 'products',
maxMemoryItems: 100
});
}
/**
* Get product list with caching
*/
async getProducts(filters = {}) {
// Create cache key from filters
const cacheKey = `list:${JSON.stringify(filters)}`;
const cached = this.cache.get(cacheKey);
if (cached) {
console.log('✅ Returning cached product list');
return cached;
}
console.log('⏳ Fetching products from API...');
const queryParams = new URLSearchParams(filters);
const response = await fetch(`/api/products?${queryParams}`);
const products = await response.json();
// Cache for 2 minutes
this.cache.set(cacheKey, products, { ttl: 120 });
return products;
}
/**
* Get single product
*/
async getProduct(productId) {
const cacheKey = `product:${productId}`;
const cached = this.cache.get(cacheKey);
if (cached) {
console.log(`✅ Returning cached product ${productId}`);
return cached;
}
console.log(`⏳ Fetching product ${productId} from API...`);
const response = await fetch(`/api/products/${productId}`);
const product = await response.json();
// Cache for 10 minutes
this.cache.set(cacheKey, product, { ttl: 600 });
return product;
}
/**
* Invalidate product cache when updated
*/
invalidateProduct(productId) {
this.cache.delete(`product:${productId}`);
// Also clear list caches (they might contain this product)
this.clearListCache();
}
/**
* Clear all list caches
*/
clearListCache() {
// In a real app, you'd track list cache keys
// For simplicity, we'll clear everything starting with "list:"
console.log('🗑️ Clearing product list caches');
}
}
// Usage in React/Vue component
const productCache = new ProductCache();
async function loadProducts() {
try {
const products = await productCache.getProducts({ category: 'electronics' });
displayProducts(products);
} catch (error) {
console.error('Failed to load products:', error);
}
}
🧠 Mental Models & Visual Thinking
Building the right mental model helps you understand caching intuitively.
Mental Model #1: The Library Card Catalog
┌─────────────────────────────────────────────────────────┐
│ THE LIBRARY │
│ │
│ 🗄️ Card Catalog (Cache) │
│ │ │
│ ├─ Quick lookups │
│ ├─ Limited space │
│ └─ Points to books │
│ │
│ 📚 Shelves (Database) │
│ │ │
│ ├─ All books stored here │
│ ├─ Slow to browse │
│ └─ Permanent storage │
└─────────────────────────────────────────────────────────┘
Finding a book:
1. Check card catalog (fast!) ✅
2. If not listed, search shelves (slow) 🐌
3. Add card for next time 📝
Key Insight: Cache is an index, not the source of truth.
Mental Model #2: Your Brain's Memory
Working Memory (Cache) Long-term Memory (Database)
┌─────────────┐ ┌──────────────────┐
│ 7±2 items │ │ Unlimited items │
│ Instant │ │ Slow recall │
│ Temporary │ │ Permanent │
└─────────────┘ └──────────────────┘
↓ ↓
What you're ←────────── Everything you've
thinking learned
about NOW
Your brain does this naturally:
- Recent info = fast to recall (cached!)
- Old info = need to "reload" from long-term memory
- Frequently used info = stays in working memory
Apply to code:
- Recently accessed data → keep in cache
- Rarely used data → fetch when needed
- Frequently accessed → cache with long TTL
Mental Model #3: The Notepad System
📱 Phone (Your App)
│
▼
📋 Notepad (Cache) ← Fast to check
│ Limited pages
│ Temporary
├─ Today's meetings
├─ Important phone numbers
└─ Quick notes
│ (If not on notepad)
▼
🗄️ Filing Cabinet (Database) ← Slow to search
Everything stored
Permanent
How you use your notepad:
- ✅ Write down things you'll need soon
- ✅ Cross out outdated info (invalidation!)
- ✅ Tear out old pages when full (eviction!)
- ❌ Don't write down everything
- ❌ Don't trust outdated notes
Same with caching!
Visual #1: Cache Layers (Onion Model)
┌─────────┐
│ USER │
└────┬────┘
│
┌─────────────┼─────────────┐
│ │ │
│ BROWSER CACHE (10ms) │
│ │
└─────────────┬─────────────┘
│
┌─────────────┼─────────────┐
│ │ │
│ CDN CACHE (50ms) │
│ │
└─────────────┬─────────────┘
│
┌─────────────┼─────────────┐
│ │ │
│ SERVER CACHE (100ms) │
│ │
└─────────────┬─────────────┘
│
┌─────────────┼─────────────┐
│ │ │
│ DATABASE (500ms) │
│ [Source of Truth] │
└───────────────────────────┘
Each layer:
- Faster than the next
- Smaller than the next
- Closer to the user
Strategy: Hit the closest/fastest layer first!
Visual #2: Time-Based Cache Flow
Timeline View of a Cached Item:
0:00 1:00 2:00 3:00 4:00 5:00
│ │ │ │ │ │
│ │ │ │ │ │
│ Add │ Hit │ Hit │ Hit │ Hit │ Expired
│ to │ ✅ │ ✅ │ ✅ │ ✅ │ ❌
│ Cache │ │ │ │ │
│ │ │ │ │ │
└───────────┴───────────┴───────────┴───────────┴───────────┴──────→
TTL = 5 minutes
After expiration:
5:01 → Cache Miss → Fetch from DB → Cache again (restart cycle)
Visual #3: Cache Hit Rate Impact
Scenario: 1000 requests, different hit rates
90% Hit Rate (Good!)
━━━━━━━━━━━━━━━━━━━ Cache Hits (900) - Fast! ⚡
══ Cache Misses (100) - Slow 🐌
50% Hit Rate (Poor)
━━━━━━━━━━ Cache Hits (500) - Fast! ⚡
══════════ Cache Misses (500) - Slow 🐌
10% Hit Rate (Terrible!)
━━ Cache Hits (100) - Fast! ⚡
══════════════════ Cache Misses (900) - Slow 🐌
Goal: Achieve 80%+ hit rate for meaningful performance improvement
Visual #4: Cache Size vs Performance
Performance ▲
│
│ ┌─────────────
│ /
│ /
│ / Diminishing
│ / Returns
│ /
│ /
│ /
│ /
│/
──────────────────────┴──────────────────────► Cache Size
Sweet Spot:
- Not too small (poor hit rate)
- Not too large (memory waste, slow evictions)
- Just right (good hit rate, manageable size)
Typical: Cache 10-20% of your data for 80-90% hit rate
(Pareto Principle applies!)
Visual #5: Stale Data Timeline
Database Value: $100 ──────────────► $80
Cache Value: $100 ────────────────────────────► $100 (STALE!)
├─────────────────┬──────────────────────────┤
Synchronized Out of Sync (Dangerous!)
Solutions:
1. TTL: $100 ──────X (expire) → Fetch $80
2. Invalidate: $100 ──────X (delete) → Fetch $80
3. Update: $100 ──────→ $80 (write-through)
The Cache Decision Matrix
Change Frequency
│
Rarely │ Often
Changes │ Changes
│
─────────────────────┼─────────────────────
│ │ │
HIGH │ ⭐⭐⭐ │ ⭐⭐ │ HIGH
│ PERFECT │ SHORT │
Access │ TO │ TTL │ Access
Freq │ CACHE │ OK │ Freq
│ │ │
─────────┼──────────┼──────────┼─────────
│ │ │
LOW │ ⭐ │ ❌ │ LOW
│ MAYBE │ DON'T │
Access │ CACHE │ CACHE │ Access
Freq │ IF SLOW │ │ Freq
│ TO FETCH│ │
─────────┴──────────┴──────────┴─────────
Use this matrix to decide what to cache!
🛠️ Practical Implementation Guide
Step 1: Audit Your Application
Questions to answer:
- What are your slowest operations?
// Measure everything
console.time('fetchProducts');
const products = await fetchProducts();
console.timeEnd('fetchProducts');
// fetchProducts: 450ms ← Candidate for caching!
- What data is accessed most frequently?
// Track access patterns
const accessLog = {};
function logAccess(resource) {
accessLog[resource] = (accessLog[resource] || 0) + 1;
}
// After a day:
// { 'user-profile': 1000, 'product-123': 850, 'settings': 5 }
// ↑ Cache this! ↑ Cache this! ↑ Don't cache
-
How often does your data change?
- Every second → Don't cache or very short TTL
- Every minute → Short TTL (30-60s)
- Every hour → Medium TTL (5-15 min)
- Every day → Long TTL (hours)
- Rarely → Very long TTL (days)
Step 2: Choose Your Caching Strategy
Decision Tree:
Are you building frontend or backend?
│
┌────┴────┐
│ │
Frontend Backend
│ │
│ └─► Server Cache (Redis, Memcached)
│
└─► Storage type?
│
┌────┴─────────────┐
│ │
Temporary Persistent
(In-memory) (localStorage/IndexedDB)
Frontend recommendations:
- Temporary session data → In-memory Map
- User preferences → localStorage
- Large datasets → IndexedDB
- API responses → In-memory + localStorage
Step 3: Implement Basic Caching
Start simple, then optimize:
// Phase 1: Simple in-memory cache
const cache = new Map();
async function getData(id) {
if (cache.has(id)) return cache.get(id);
const data = await fetchFromAPI(id);
cache.set(id, data);
return data;
}
// Phase 2: Add TTL
function cacheWithTTL(id, data, ttl = 300) {
const expiresAt = Date.now() + ttl * 1000;
cache.set(id, { data, expiresAt });
}
// Phase 3: Add metrics
const metrics = { hits: 0, misses: 0 };
function getFromCache(id) {
const item = cache.get(id);
if (item && Date.now() < item.expiresAt) {
metrics.hits++;
return item.data;
}
metrics.misses++;
return null;
}
// Phase 4: Add size limits (use LRU class from earlier)
Step 4: Handle Invalidation
Three approaches:
1. Time-based (Easiest)
// Just let it expire
cache.set(key, data, { ttl: 300 });
2. Event-based (Best for critical data)
// Invalidate on update
function updateProduct(id, newData) {
database.update(id, newData);
cache.delete(`product:${id}`);
}
3. Hybrid (Most common)
// TTL as backup, events for immediate updates
cache.set(key, data, { ttl: 3600 }); // 1 hour fallback
eventBus.on('product-updated', (id) => {
cache.delete(`product:${id}`); // Immediate invalidation
});
Step 5: Monitor & Optimize
Essential metrics:
class CacheWithMetrics {
constructor() {
this.cache = new Map();
this.metrics = {
hits: 0,
misses: 0,
sets: 0,
deletes: 0,
errors: 0,
totalSize: 0
};
}
get(key) {
const value = this.cache.get(key);
if (value) {
this.metrics.hits++;
return value;
}
this.metrics.misses++;
return null;
}
set(key, value) {
this.metrics.sets++;
this.metrics.totalSize += JSON.stringify(value).length;
this.cache.set(key, value);
}
getReport() {
const total = this.metrics.hits + this.metrics.misses;
return {
hitRate: ((this.metrics.hits / total) * 100).toFixed(2) + '%',
totalRequests: total,
cacheSize: this.cache.size,
estimatedMemory: (this.metrics.totalSize / 1024).toFixed(2) + ' KB'
};
}
}
// Log report every minute
setInterval(() => {
console.log('Cache Report:', cache.getReport());
}, 60000);
What to watch:
- Hit Rate: Should be >80%
- Memory Usage: Should be stable (not growing)
- Cache Size: Should have a limit
- Errors: Should be near 0
Step 6: Test Your Cache
Key scenarios to test:
// Test 1: Cache hit
test('should return cached data', () => {
cache.set('key1', 'value1');
expect(cache.get('key1')).toBe('value1');
});
// Test 2: Cache miss
test('should return null on miss', () => {
expect(cache.get('nonexistent')).toBeNull();
});
// Test 3: Expiration
test('should expire after TTL', async () => {
cache.set('key1', 'value1', { ttl: 1 }); // 1 second
await sleep(1100); // Wait 1.1 seconds
expect(cache.get('key1')).toBeNull();
});
// Test 4: Size limit
test('should evict old items when full', () => {
const cache = new LRUCache(2);
cache.set('a', 1);
cache.set('b', 2);
cache.set('c', 3); // Should evict 'a'
expect(cache.get('a')).toBeNull();
expect(cache.get('b')).toBe(2);
expect(cache.get('c')).toBe(3);
});
// Test 5: Invalidation
test('should invalidate on update', async () => {
await cache.set('product:1', { price: 100 });
await updateProduct(1, { price: 80 });
expect(cache.get('product:1')).toBeNull();
});
Step 7: Debug Cache Issues
Common problems and solutions:
Problem 1: Stale data
// Symptom: Users see old data
// Solution: Check TTL and invalidation
// Debug:
console.log('Cached value:', cache.get(key));
console.log('Database value:', await db.get(key));
console.log('Cache age:', Date.now() - cacheItem.createdAt);
Problem 2: Memory leak
// Symptom: Memory usage grows over time
// Solution: Add size limits and cleanup
// Debug:
console.log('Cache size:', cache.size);
console.log('Memory used:', process.memoryUsage().heapUsed / 1024 / 1024, 'MB');
Problem 3: Low hit rate
// Symptom: Cache hit rate < 50%
// Solution: Increase TTL or cache more data
// Debug:
const hitRate = metrics.hits / (metrics.hits + metrics.misses);
console.log('Hit rate:', (hitRate * 100).toFixed(1) + '%');
console.log('Most missed keys:', getMostMissedKeys());
Problem 4: Race conditions
// Symptom: Multiple DB queries for same data
// Solution: Request deduplication
// Add this:
const pending = new Map();
async function getData(key) {
if (pending.has(key)) {
return pending.get(key); // Reuse pending request
}
const promise = fetchData(key);
pending.set(key, promise);
try {
const data = await promise;
pending.delete(key);
return data;
} catch (error) {
pending.delete(key);
throw error;
}
}
Performance Checklist
Before deploying your cache:
- [ ] Cache hit rate is >80%
- [ ] Memory usage is stable (not growing)
- [ ] Cache has size limit
- [ ] TTL is appropriate for each data type
- [ ] Invalidation works correctly
- [ ] Error handling prevents cache failures from breaking app
- [ ] Metrics are logged for monitoring
- [ ] Tests cover hit, miss, expiration, and eviction
- [ ] Documentation explains what's cached and why
- [ ] Fallback to source works if cache fails
🎓 Summary: Key Takeaways
The Golden Rules
-
Cache Wisely
- Only cache frequently accessed data
- Only cache data that's expensive to fetch
- Don't cache everything!
-
Plan for Invalidation
- Always set a TTL (even if it's long)
- Delete cache on updates
- Monitor for stale data
-
Limit Cache Size
- Use LRU or similar eviction
- Monitor memory usage
- Don't let cache grow unbounded
-
Handle Failures
- Cache failure shouldn't break your app
- Always have fallback to source
- Log errors but continue
-
Measure Everything
- Track hit rate
- Monitor memory
- Log cache operations
- Adjust based on data
When to Use Caching
✅ Good candidates:
- API responses
- Database queries
- Computed values
- User profiles
- Product catalogs
- Static content
❌ Poor candidates:
- One-time data
- Real-time data (unless very short TTL)
- Sensitive data (without encryption)
- Random/unique values
- Frequently changing data
Final Wisdom
"Premature optimization is the root of all evil" - Donald Knuth
Start simple:
- Build your app without caching
- Identify bottlenecks with real data
- Add caching where it makes the biggest impact
- Measure improvements
- Iterate
Remember: Caching adds complexity. Only add it when the performance benefit outweighs the complexity cost!
📚 Further Reading
Happy Caching! 🚀
Remember: The best cache is the one that's invisible to your users - it just makes everything faster!
Top comments (0)