A cache stampede is one of those bugs nobody sees in a load test and everybody sees at the first real traffic peak. A popular entry expires, a hundred requests notice in the same second, and all hundred recompute it. With a database query that is annoying. With a language model behind the cache it becomes slow and expensive, and in the worst case Amazon Bedrock throttles you exactly when traffic is highest.
In my article on the semantic cache for Bedrock I deliberately left this topic out and wrote that I would only add a short lock under high load. This is the follow-up: I will show you how to prevent a cache stampede with Lambda, ElastiCache for Valkey and a few lines of TypeScript. At the end you have a runnable project with AWS CDK, tests that need no AWS account and a simple load test to measure the effect yourself.
In short
When a frequently read cache entry expires, exactly one request should recompute it, not all of them at once. You get there with a short lock via SET NX in Valkey, an old value that the other requests receive in the meantime, and spread-out expiry times. The biggest pitfall is releasing the lock: it may only be deleted while it still belongs to you.
What you will learn in this article
- What a cache stampede is and why it hurts so much in front of Bedrock
- Why in-memory tricks do not help in Lambda
- Lock, stale value, XFetch and jitter in one small TypeScript function
- A safe lock release with a token and a Lua script or DELIFEQ
- How to verify with a load test that the value is really computed only once
What is a cache stampede?
The term describes a load spike that the cache itself triggers. While an entry is valid, the cache protects the system behind it. The moment it expires, that protection disappears for everyone at once. Every request arriving now sees a cache miss and starts the expensive computation. The pattern is also known as “dogpile”, and it is closely related to the “thundering herd”, where many waiting processes start running simultaneously in response to a single event.
My example for this article is the fictional bookshop “Seitenwind”. Its home page shows a “pick of the day” per genre, written by a language model on Amazon Bedrock. The recommendation stays in the cache for ten minutes. That sounds harmless until a newsletter goes out on Saturday morning and 300 people click “crime” at the same time. If the entry expires right then, in the worst case 300 identical model calls land at Bedrock.
Why this hurts so much in front of a language model
Three things come together. First, a model call takes seconds instead of milliseconds, so the window in which other requests see the empty cache is long. Second, every call costs tokens, and you pay for 300 identical answers 300 times. Third, Bedrock has quotas for requests and tokens per minute. A stampede can use them up, and then even requests that have nothing to do with the pick of the day start to fail.
Why in-memory tricks do not help in Lambda
In a classic Node.js server you would simply coalesce identical requests inside the process: the first one starts the computation, all others attach to the same promise. That does not work in Lambda. In the standard model, an execution environment handles only one request at a time. If 300 requests arrive in parallel, Lambda starts many environments, each with its own memory. Coordination therefore has to happen in a place everyone can see. In our case, that place is the cache itself.
Strategies against a cache stampede
A few building blocks help against a cache stampede, and they combine well. The table shows what I typically use in serverless projects and what for.
| Building block | What it does | When I use it |
|---|---|---|
| Lock (SET NX) | Only the request that gets the lock recomputes | Whenever a computation is expensive |
| Stale value | Other requests get the previous answer in the meantime | When a slightly outdated answer is acceptable |
| Early refresh (XFetch) | Single requests recompute shortly before expiry | For very frequently read keys |
| Jitter on the TTL | Expiry times are spread randomly | Whenever many keys are written at the same time |
| External refresh | A schedule recomputes values in advance | For a few well-known keys |
A lock with SET NX
At the core is a single command: SET lock:pick:v1:crime <token> NX PX 20000. NX means the key is only written if it does not exist yet. PX sets the lifetime in milliseconds. If Valkey answers OK, you hold the lock. If it answers with nothing, someone else already has it. The lifetime is your safety net: if the Lambda function crashes in the middle of the computation, the lock disappears on its own after 20 seconds.
The value of the lock is a random token. That sounds like a detail, yet it is the most important part. If your computation takes longer than the lifetime, the lock expires and a second request grabs a new one. If the first request then blindly deletes it with DEL, it removes the lock of the second one. That is why the key is only deleted when the stored value still matches your own token. Valkey 9.0 added the DELIFEQ command for this; on older versions you use a small Lua script.
Serving the stale value
A lock alone prevents duplicate computation, but the other requests then have to wait. It is better to keep the entry longer than it is fresh. I therefore store two times: freshUntil inside the entry and a longer TTL for the key in Valkey. After ten minutes the pick counts as stale, but it stays stored for another hour. Anyone who does not get the lock during that time immediately serves the old pick. Waiting only happens when there is really nothing in the cache, for example right after a deployment.
Early refresh with XFetch
In 2015, Vattani, Chierichetti and Lowenstein described a method known as XFetch. The idea: every request rolls the dice on whether to recompute the entry before it expires. Long before expiry the chance is tiny; shortly before, it rises sharply. The formula is surprisingly short: recompute if now − delta · beta · ln(random) ≥ expiry. delta is the duration of the last computation, beta a factor that moves the refresh earlier or later. According to the authors, beta = 1 works well in practice.
Its charm is that slow computations are automatically refreshed earlier. A model call taking three seconds gets a bigger head start than a query taking 50 milliseconds. Combined with the lock, XFetch makes sure the entry is usually refreshed before it ever becomes stale.
Jitter on the expiry time
The fourth building block is the simplest. If you write five genres into the cache at the same time after a deployment, a fixed TTL makes them expire at the same time too. A few percent of randomness on the lifetime spreads the expirations. You might know the principle from exponential backoff with jitter, where it is used for retries for the same reason.
🎬 Cache stampede with a lock: only one request asks the model – this animated diagram only runs on my site: watch it on sascha-ms.de
Prevent a cache stampede: step by step with Lambda and Valkey
Now it gets practical. We build an HTTP API with a Lambda function that reads the pick of the day from Valkey and rewrites it via Bedrock when needed. This time I describe the infrastructure with AWS CDK in TypeScript.
Architecture of the example: the Lambda function asks the cache first. Only the lock holder calls Bedrock.
Prerequisites
- Node.js 22 or newer and an AWS account with CDK bootstrapped in your region
- Docker, because the native Valkey GLIDE library is installed inside the Lambda build image
- An ElastiCache Serverless cache for Valkey 9 with a public endpoint and IAM authentication. The semantic cache article shows step by step how to create it.
- Model access in Bedrock, in this example Amazon Nova Lite via the EU inference profile
Step 1: Create the project
Create an empty folder and install the dependencies. The versions in the example repository are the ones I tested with.
mkdir bookshop-picks && cd bookshop-picks
npm init -y
npm pkg set type=module
npm install @aws-sdk/client-bedrock-runtime @valkey/valkey-glide aws-cdk-lib constructs
npm install -D typescript@5.9 tsx vitest aws-cdk esbuild @types/node@24 @types/aws-lambda
mkdir -p src lib bin test
Step 2: The logic against the cache stampede
The heart of it is the getOrCompute function. It does not know Valkey at all, only a small interface with four methods, so you can test it without a network. Read the code once from top to bottom: fresh hit, try the lock, compute and store on success, otherwise serve the stale value or wait briefly.
// src/stampede.ts
import { randomUUID } from "node:crypto";
/** Minimal cache interface: Valkey in production, an in-memory fake in tests. */
export interface CacheStore {
get(key: string): Promise<string | null>;
/** SET key value PX ttlMs */
set(key: string, value: string, ttlMs: number): Promise<void>;
/** SET key value NX PX ttlMs – true if this caller wrote the key */
setIfAbsent(key: string, value: string, ttlMs: number): Promise<boolean>;
/** Deletes the key only if it still holds the given value (lock release). */
deleteIfEquals(key: string, value: string): Promise<boolean>;
}
/** What we store per key: the value plus the data XFetch needs. */
export interface CacheEntry {
value: string;
/** How long the last computation took (ms) – "delta" in XFetch */
computeMs: number;
/** From this point on the entry counts as stale (ms since epoch) */
freshUntil: number;
}
export type Source = "hit" | "computed" | "stale" | "waited";
export interface Result {
value: string;
source: Source;
}
export interface StampedeOptions {
/** How long an entry stays fresh */
freshMs: number;
/** How long a stale entry may still be served while someone refreshes it */
staleMs: number;
/** Lock lifetime: must be longer than the slowest computation */
lockMs: number;
/** How long callers without the lock wait for the result at most */
maxWaitMs: number;
/** Polling interval while waiting */
pollMs: number;
/** XFetch factor: 1 is a good start, > 1 refreshes earlier */
beta: number;
/** Random variation of freshMs (0.1 = ±10 %) */
jitter: number;
now?: () => number;
random?: () => number;
sleep?: (ms: number) => Promise<void>;
}
export const defaultOptions: StampedeOptions = {
freshMs: 10 * 60_000,
staleMs: 60 * 60_000,
lockMs: 20_000,
maxWaitMs: 3_000,
pollMs: 150,
beta: 1,
jitter: 0.1,
};
/** Thrown when the value is not ready within maxWaitMs. */
export class CacheBusyError extends Error {
constructor(public readonly key: string) {
super(`Value for ${key} is being computed, please retry shortly`);
this.name = "CacheBusyError";
}
}
/**
* XFetch (Vattani et al. 2015): the closer the expiry, the more likely
* a single caller refreshes early. random() lies in (0, 1].
*/
export function shouldRefreshEarly(
entry: CacheEntry,
now: number,
beta: number,
random: number,
): boolean {
const safeRandom = Math.min(Math.max(random, Number.MIN_VALUE), 1);
return now - entry.computeMs * beta * Math.log(safeRandom) >= entry.freshUntil;
}
/** Spreads expiry times so that keys written together do not expire together. */
export function withJitter(ms: number, jitter: number, random: number): number {
return Math.round(ms * (1 + jitter * (random * 2 - 1)));
}
export function parseEntry(raw: string | null): CacheEntry | undefined {
if (!raw) return undefined;
try {
const entry = JSON.parse(raw) as CacheEntry;
return typeof entry.value === "string" && typeof entry.freshUntil === "number"
? entry
: undefined;
} catch {
return undefined; // broken entries count as a miss
}
}
/**
* Reads a value from the cache. Only the caller holding the lock computes it;
* everyone else gets the stale value or waits briefly for the new one.
*/
export async function getOrCompute(
store: CacheStore,
key: string,
compute: () => Promise<string>,
options: Partial<StampedeOptions> = {},
): Promise<Result> {
const o = { ...defaultOptions, ...options };
const now = o.now ?? Date.now;
const random = o.random ?? Math.random;
const sleep = o.sleep ?? ((ms: number) => new Promise<void>((r) => setTimeout(r, ms)));
const entry = parseEntry(await store.get(key));
if (entry && !shouldRefreshEarly(entry, now(), o.beta, random())) {
return { value: entry.value, source: "hit" };
}
const lockKey = `lock:${key}`;
const token = randomUUID();
const gotLock = await store.setIfAbsent(lockKey, token, o.lockMs);
if (gotLock) {
try {
const started = now();
const value = await compute();
const computeMs = Math.max(1, now() - started);
const freshMs = withJitter(o.freshMs, o.jitter, random());
const fresh: CacheEntry = { value, computeMs, freshUntil: now() + freshMs };
await store.set(key, JSON.stringify(fresh), freshMs + o.staleMs);
return { value, source: "computed" };
} finally {
await store.deleteIfEquals(lockKey, token);
}
}
// Someone else is computing. A stale value is better than waiting.
if (entry) return { value: entry.value, source: entry.freshUntil > now() ? "hit" : "stale" };
const deadline = now() + o.maxWaitMs;
while (now() < deadline) {
await sleep(o.pollMs);
const ready = parseEntry(await store.get(key));
if (ready) return { value: ready.value, source: "waited" };
}
throw new CacheBusyError(key);
}
Two details are easy to miss. The finally block releases the lock even when Bedrock throws an error; without it, everyone else would wait until the lock expires. In addition, broken entries count as a miss instead of an error. A format change after a deployment therefore does not take the page down.
Step 3: The Valkey adapter with a safe release
This adapter translates the interface into commands for Valkey GLIDE, the official client of the Valkey project. Each execution environment creates the connection once and then reuses it. For the release I use a Lua script, because it runs on every Valkey version and I could also test it locally against Redis OSS.
// src/valkey-store.ts
import { GlideClusterClient, Script, ServiceType, TimeUnit } from "@valkey/valkey-glide";
import type { CacheStore } from "./stampede.js";
// Deletes the lock only if it still holds our token. redis.call also works on
// Valkey (and on Redis OSS for local tests). On Valkey 9 or later
// the single command DELIFEQ does the same.
const releaseScript = new Script(`
if redis.call("get", KEYS[1]) == ARGV[1] then
return redis.call("del", KEYS[1])
end
return 0`);
let pending: Promise<GlideClusterClient> | undefined;
/** One client per Lambda execution environment; GLIDE renews the IAM token itself. */
export function getClient(): Promise<GlideClusterClient> {
pending ??= GlideClusterClient.createClient({
addresses: [{ host: required("CACHE_HOST"), port: 6379 }],
useTLS: true,
requestTimeout: 500,
credentials: {
username: required("CACHE_USER"),
iamConfig: {
clusterName: required("CACHE_NAME").toLowerCase(),
service: ServiceType.Elasticache,
region: required("AWS_REGION"),
},
},
}).catch((error: unknown) => {
pending = undefined; // try again on the next call
throw error;
});
return pending;
}
export function valkeyStore(client: GlideClusterClient): CacheStore {
return {
async get(key) {
const value = await client.get(key);
return value === null ? null : value.toString();
},
async set(key, value, ttlMs) {
await client.set(key, value, { expiry: { type: TimeUnit.Milliseconds, count: ttlMs } });
},
async setIfAbsent(key, value, ttlMs) {
const result = await client.set(key, value, {
conditionalSet: "onlyIfDoesNotExist",
expiry: { type: TimeUnit.Milliseconds, count: ttlMs },
});
return result === "OK";
},
async deleteIfEquals(key, value) {
const deleted = await client.invokeScript(releaseScript, { keys: [key], args: [value] });
return deleted === 1;
},
};
}
export function required(name: string): string {
const value = process.env[name];
if (!value) throw new Error(`Environment variable ${name} is missing`);
return value;
}
If your cache runs Valkey 9 or newer, you can also implement deleteIfEquals with client.customCommand(["DELIFEQ", key, value]). That saves the script but ties you to the new version.
Step 4: Bedrock call and handler
The model call is deliberately simple. What matters is that it is passed in as a function, so getOrCompute decides whether it happens at all.
// src/recommend.ts
import { BedrockRuntimeClient, ConverseCommand } from "@aws-sdk/client-bedrock-runtime";
import { required } from "./valkey-store.js";
const bedrock = new BedrockRuntimeClient({});
const SYSTEM_PROMPT = `You write the "pick of the day" for the fictional bookshop "Seitenwind".
Recommend one well-known, real book for the given genre in at most 80 words.
Do not invent prices, stock levels or quotes.`;
/** The expensive call we want to protect: takes seconds and costs tokens. */
export async function writeRecommendation(genre: string): Promise<string> {
const response = await bedrock.send(
new ConverseCommand({
modelId: required("MODEL_ID"),
system: [{ text: SYSTEM_PROMPT }],
messages: [{ role: "user", content: [{ text: `Genre: ${genre}` }] }],
inferenceConfig: { maxTokens: 300, temperature: 0.7 },
}),
);
const text = (response.output?.message?.content ?? [])
.map((block) => block.text ?? "")
.join("")
.trim();
if (!text) throw new Error("Empty answer from the model");
return text;
}
The handler checks the genre against a fixed list. That is more than input validation: every new key is a potential model call, and an open list would be an invitation to drain your Bedrock budget with made-up genres. If the function finds no value within three seconds, it answers with 503 and Retry-After instead of calling Bedrock itself.
// src/handler.ts
import type { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from "aws-lambda";
import { writeRecommendation } from "./recommend.js";
import { CacheBusyError, getOrCompute, type CacheStore } from "./stampede.js";
import { getClient, valkeyStore } from "./valkey-store.js";
// A fixed list keeps the number of cache keys (and Bedrock calls) small.
export const GENRES = ["crime", "fantasy", "nonfiction", "romance", "scifi"] as const;
export function isGenre(value: string | undefined): value is (typeof GENRES)[number] {
return GENRES.includes(value as (typeof GENRES)[number]);
}
export async function respond(
genre: string | undefined,
store: CacheStore,
compute: (genre: string) => Promise<string>,
): Promise<APIGatewayProxyResultV2> {
if (!isGenre(genre)) {
return json(400, { message: `Unknown genre. Allowed: ${GENRES.join(", ")}` });
}
try {
const result = await getOrCompute(store, `pick:v1:${genre}`, () => compute(genre), {
freshMs: Number(process.env.FRESH_SECONDS ?? 600) * 1000,
staleMs: Number(process.env.STALE_SECONDS ?? 3600) * 1000,
lockMs: 20_000,
maxWaitMs: 3_000,
});
// One log line per request: lets you count the sources in Logs Insights
console.log(JSON.stringify({ genre, source: result.source }));
return json(200, { genre, recommendation: result.value, source: result.source });
} catch (error) {
if (error instanceof CacheBusyError) {
return json(503, { message: "Recommendation is being prepared, please retry." }, { "retry-after": "2" });
}
throw error;
}
}
export async function handler(event: APIGatewayProxyEventV2): Promise<APIGatewayProxyResultV2> {
const store = valkeyStore(await getClient());
return respond(event.pathParameters?.genre, store, writeRecommendation);
}
function json(statusCode: number, body: unknown, headers: Record<string, string> = {}) {
return {
statusCode,
headers: { "content-type": "application/json", ...headers },
body: JSON.stringify(body),
};
}
Step 5: The CDK stack
The stack creates the function, the permissions for elasticache:Connect and Bedrock, and the HTTP API. Watch the function timeout: 25 seconds is longer than the 20-second lock. If it were shorter, Lambda could stop the function while it still holds the lock, and the other requests would wait needlessly for a value that never arrives.
// lib/bookshop-picks-stack.ts
import { ArnFormat, CfnOutput, Duration, Stack, type StackProps } from "aws-cdk-lib";
import { HttpApi, HttpMethod } from "aws-cdk-lib/aws-apigatewayv2";
import { HttpLambdaIntegration } from "aws-cdk-lib/aws-apigatewayv2-integrations";
import * as iam from "aws-cdk-lib/aws-iam";
import * as lambda from "aws-cdk-lib/aws-lambda";
import { NodejsFunction, OutputFormat } from "aws-cdk-lib/aws-lambda-nodejs";
import * as logs from "aws-cdk-lib/aws-logs";
import type { Construct } from "constructs";
export interface BookshopPicksProps extends StackProps {
/** Name of an existing ElastiCache Serverless cache (Valkey) with a public endpoint */
cacheName: string;
cacheHost: string;
/** ElastiCache user with IAM authentication */
cacheUser?: string;
modelId?: string;
/** Bundle in the Lambda build image (default). Tests switch this off. */
dockerBundling?: boolean;
}
export class BookshopPicksStack extends Stack {
constructor(scope: Construct, id: string, props: BookshopPicksProps) {
super(scope, id, props);
const cacheUser = props.cacheUser ?? "default.iam-user";
const modelId = props.modelId ?? "eu.amazon.nova-lite-v1:0";
const picksFn = new NodejsFunction(this, "PicksFn", {
entry: "src/handler.ts",
handler: "handler",
runtime: lambda.Runtime.NODEJS_24_X,
architecture: lambda.Architecture.X86_64,
memorySize: 512,
timeout: Duration.seconds(25), // longer than lockMs (20 s) plus the cache calls
logGroup: new logs.LogGroup(this, "PicksLogs", { retention: logs.RetentionDays.TWO_WEEKS }),
environment: {
CACHE_NAME: props.cacheName,
CACHE_HOST: props.cacheHost,
CACHE_USER: cacheUser,
MODEL_ID: modelId,
FRESH_SECONDS: "600",
STALE_SECONDS: "3600",
},
bundling: {
format: OutputFormat.CJS,
// GLIDE ships a native library: install it inside the Lambda build image
nodeModules: ["@valkey/valkey-glide"],
forceDockerBundling: props.dockerBundling ?? true,
},
});
picksFn.addToRolePolicy(
new iam.PolicyStatement({
actions: ["elasticache:Connect"],
resources: [
this.formatArn({ service: "elasticache", resource: "serverlesscache", resourceName: props.cacheName, arnFormat: ArnFormat.COLON_RESOURCE_NAME }),
this.formatArn({ service: "elasticache", resource: "user", resourceName: cacheUser, arnFormat: ArnFormat.COLON_RESOURCE_NAME }),
],
}),
);
picksFn.addToRolePolicy(
new iam.PolicyStatement({
actions: ["bedrock:InvokeModel"],
resources: [
`arn:aws:bedrock:${this.region}:${this.account}:inference-profile/${modelId}`,
"arn:aws:bedrock:*::foundation-model/amazon.nova-lite-v1:0",
],
}),
);
const api = new HttpApi(this, "PicksApi");
api.addRoutes({
path: "/picks/{genre}",
methods: [HttpMethod.GET],
integration: new HttpLambdaIntegration("PicksIntegration", picksFn),
});
new CfnOutput(this, "ApiUrl", { value: api.apiEndpoint });
}
}
// bin/app.ts
import { App } from "aws-cdk-lib";
import { BookshopPicksStack } from "../lib/bookshop-picks-stack.js";
const app = new App();
new BookshopPicksStack(app, "BookshopPicksStack", {
env: { region: process.env.CDK_DEFAULT_REGION ?? "eu-central-1", account: process.env.CDK_DEFAULT_ACCOUNT },
cacheName: app.node.tryGetContext("cacheName") ?? "bookshop-cache",
cacheHost: app.node.getContext("cacheHost"),
});
The cdk.json file only contains the line { "app": "npx tsx bin/app.ts" }.
Step 6: Test without AWS
For the tests, the repository contains a MemoryStore that mimics SET NX PX in memory. The most important test sends 50 requests at once to an empty cache and checks that Bedrock was asked exactly once.
// test/stampede.test.ts (excerpt)
it("computes only once for 50 parallel requests on an empty cache", async () => {
const store = new MemoryStore(Date.now);
const compute = vi.fn(async () => {
await wait(50);
return "The Name of the Rose";
});
const results = await Promise.all(
Array.from({ length: 50 }, () =>
getOrCompute(store, "pick:crime", compute, { pollMs: 5, maxWaitMs: 1000, jitter: 0 }),
),
);
expect(compute).toHaveBeenCalledTimes(1);
expect(results.filter((r) => r.source === "computed")).toHaveLength(1);
expect(results.filter((r) => r.source === "waited")).toHaveLength(49);
});
npx tsc --noEmit
npx vitest run
Expected result: Test Files 3 passed and Tests 15 passed. I also ran the adapter against a local Redis cluster: 30 parallel calls, one computation, 29 times “waited”.
Step 7: Deploy and measure
export CDK_DEFAULT_REGION=eu-central-1
npx cdk deploy -c cacheName=bookshop-cache -c cacheHost=<your-cache-endpoint>
# 50 concurrent requests for one genre
API=<ApiUrl from the CDK output>
seq 50 | xargs -P 50 -I{} curl -s "$API/picks/crime" | grep -o '"source":"[a-z]*"' | sort | uniq -c
On the first run against an empty cache you will see roughly one computed, many waited and possibly a few 503 responses if the model call takes longer than three seconds. Run the script again and you only get hit. It gets interesting after ten minutes: you then see one computed and otherwise stale, with no waiting at all. For continuous measurement, use CloudWatch Logs Insights:
fields @timestamp, @message
| parse @message '"source":"*"' as source
| filter ispresent(source)
| stats count(*) as requests by source, bin(5m)
Common mistakes with a cache lock
| Mistake | Consequence | Fix |
|---|---|---|
Releasing the lock with DEL
|
Someone else’s lock is deleted, two requests compute again | Compare the token: Lua script or DELIFEQ
|
| Lock without a lifetime | After a crash the key stays locked forever | Always set PX, longer than the slowest computation |
| Lambda timeout shorter than the lock | The function dies holding the lock, others wait in vain | Timeout greater than lockMs plus a buffer |
| Waiting requests call Bedrock themselves after the timeout | The stampede arrives late after all | Answer with 503 and Retry-After after maxWaitMs
|
| Same TTL for every key | Many entries expire at the same moment | Jitter of about 10 percent |
| Caching errors | An empty model answer sticks around for an hour | Only store valid values, throw on empty answers |
Cost and security
Every request now costs one or two extra Valkey commands. With ElastiCache Serverless you pay for stored data and processed ECPUs. Compared to a saved model call that is a small amount, but check it for your load with the AWS Pricing Calculator. You can play through how timeout and memory affect your Lambda costs with my Lambda cost calculator.
On the security side: the public endpoint of ElastiCache Serverless requires IAM authentication and TLS. Give the function elasticache:Connect for exactly this cache and this user, nothing more. Only values that are the same for all users belong in the cache. A personal pick with a name or order history has no business under the key pick:v1:crime. And the fixed genre list protects you from someone triggering any number of model calls with random paths.
My take
Most important first: this lock is an efficiency lock, not a correctness lock. If it gets lost now and then, for example during a failover in the cache, two requests compute the same pick. That costs one extra call, nothing more. This is why the simple pattern with SET NX on a single cache is enough here. I would not use Redlock or fencing tokens, which the Valkey documentation discusses for strict mutual exclusion. If money, stock or double bookings are at stake, a cache is the wrong place for the lock anyway. For those cases I prefer optimistic locking in DynamoDB.
I would not add the pattern everywhere either. For a query that takes 20 milliseconds and costs nothing, a stampede is barely measurable. As soon as a language model, a slow partner API or an expensive report sits behind the cache, however, the lock is part of my standard toolkit. My favorite is the combination of lock and stale value. It is easy to understand, and users notice nothing of the recomputation. I add XFetch when a key is really hot and the computation takes several seconds.
What I see in many codebases is the opposite: a carefully built cache with a fixed TTL of exactly one hour for everything. That works fine until somebody sends a newsletter.
FAQ
What is the difference between a cache stampede and a thundering herd?
Both describe many requests starting at the same time. In a cache stampede the trigger is an expired or missing cache entry, and everyone recomputes the same value. Thundering herd is the broader term, for example for many clients reconnecting at once after an outage.
Do I need Redlock for a cache lock?
Usually not. If the lock gets lost, a value is computed twice, which only costs a little money. Redlock is meant for cases where two simultaneous holders cause real damage, and even there the algorithm is disputed.
Does this also work with DynamoDB instead of Valkey?
Yes. The lock then becomes a conditional PutItem with attribute_not_exists or an expired timestamp. Check the timestamp yourself in the condition, because DynamoDB TTL deletes expired items only with a delay.
How long should the lock last?
Longer than the slowest realistic computation and shorter than the Lambda timeout. For a model call of a few seconds I use 15 to 20 seconds and a timeout of 25 seconds.
Your next step
Find the cache entry in your system that is read most often and takes the longest to compute. Log for a week how often it is recomputed more than once after expiring. If the number is greater than one, you know where to start. Which expensive computation sits behind your cache, and have you ever experienced a stampede? Let me know in the comments.
Continue the series
Previous part: Semantic cache for Bedrock with ElastiCache and Lambda · Next part: Bulkhead pattern with Lambda: isolate third-party systems · All parts of the series “Serverless patterns in practice”
All code from this article, ready to run: smirowstanitzok/sascha-ms-examples
Sources
- Wikipedia: Cache stampede
- Vattani, Chierichetti, Lowenstein: Optimal Probabilistic Cache Stampede Prevention (VLDB 2015)
- Valkey: SET
- Amazon ElastiCache: supported engine versions
- Valkey: DELIFEQ
- AWS Lambda: Understanding concurrency
- Valkey: Distributed locks
- Amazon Bedrock: Converse API
- Valkey GLIDE on GitHub
This post was created with AI assistance.
👉 Next step: Curious what your own Lambda functions cost? Try my Lambda cost calculator – it runs right in your browser.
📬 Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).
Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en

Top comments (0)