Cloudflare Workers caching: store values, not in-flight promises

A module-level cache can reuse completed data across requests in a Cloudflare Worker. Sharing a pending operation is a different decision: the work behind that promise may belong to the request that started it. Our rule for Teasa's isolate caches is to share completed values and let concurrent misses perform their own reads.

This article explains that tradeoff, provides a small JavaScript implementation, and tests the case that ordinary cache-hit tests miss: request A's read never completes, but request B still needs an answer.

The tempting promise cache

Deduplicating concurrent work is useful in many JavaScript applications. A typical implementation puts the promise into a map immediately:

const pending = new Map();

function loadConfig(key, read) {
  if (!pending.has(key)) {
    pending.set(key, read());
  }
  return pending.get(key);
}

Request B now depends on the operation request A started. This example also retains rejected promises forever and has no eviction. Adding cleanup fixes those two defects, but does not change which request owns the underlying I/O.

Cloudflare documents a related runtime error: Cannot perform I/O on behalf of a different request. Its example shares a Response across invocations; the response body belongs to the invocation that created it. See Cloudflare's explanation of request-scoped I/O.

The issue is not the existence of a promise. Promise.all for independent operations inside one request is normal. A completed JSON object is also different from a live response body. The question is whether a later invocation needs request-owned work or resources from an earlier one.

Separate the read from the reusable value

The settled-value approach has three steps:

1. Return an unexpired value if one exists.
2. On a miss, let this invocation perform and finish its own read.
3. Store the resulting data after that read succeeds.

Two simultaneous misses therefore mean two reads. That is deliberate. Neither caller waits for an in-flight operation owned by the other.

Event	Shared pending promise	Completed-value cache
A starts a read	Store A's promise	Keep the map empty
B arrives before A finishes	B waits for A	B starts its own read
B's read completes	B has no independent read	Store B's completed value
C arrives while that value is fresh	Depends on A's outcome	C uses the cached value

We use this pattern in Teasa's shared cache utility. The version below is a small standalone example for public data, with a TTL and an entry limit. It does not require Teasa packages.

Code example

Save this module as settled-cache.mjs. The test file is included below.

// A small public-data cache for one isolate. Concurrent misses read independently.
export class SettledCache {
  #entries = new Map();

  constructor({ ttlMs = 30_000, limit = 100, clock = Date.now } = {}) {
    if (!Number.isFinite(ttlMs) || ttlMs <= 0 || !Number.isInteger(limit) || limit < 1) {
      throw new RangeError("Use a positive TTL and a positive integer limit.");
    }
    this.ttlMs = ttlMs;
    this.limit = limit;
    this.clock = clock;
  }

  async load(key, read) {
    const entry = this.#entries.get(key);
    const age = entry ? this.clock() - entry.at : -1;
    if (entry && age >= 0 && age < this.ttlMs) {
      this.#entries.delete(key);
      this.#entries.set(key, entry); // Touch the least-recently-used order.
      return entry.value;
    }
    this.#entries.delete(key);

    // Do not put this promise, a Response, or a stream in the shared map.
    const value = await read();
    this.#entries.delete(key);
    this.#entries.set(key, { value, at: this.clock() });
    while (this.#entries.size > this.limit) {
      this.#entries.delete(this.#entries.keys().next().value);
    }
    return value;
  }
}

For example, a Worker can read a small public configuration file from its static assets binding, finish parsing it, and cache the resulting data:

import { SettledCache } from "./settled-cache.mjs";

const configCache = new SettledCache({ ttlMs: 30_000, limit: 8 });

export default {
  async fetch(request, env) {
    const config = await configCache.load("public-config:v1", async () => {
      const response = await env.ASSETS.fetch(
        new URL("/public-config.json", request.url)
      );
      if (!response.ok) throw new Error("Public configuration unavailable");
      return response.json();
    });
    return Response.json(config);
  }
};

Here public-config.json is a file you ship with your own Worker assets. The cache's await read() waits for JSON parsing to finish before storing anything. Each caller constructs its own Response; the shared value is the parsed public data.

Treat stored objects as immutable. The example returns the same object reference on a hit, so a caller that mutates it changes what later callers see.

Test independence before testing speed

The first test below leaves A's read pending, starts B's read for the same key, and expects B to finish independently. A short test watchdog turns accidental promise sharing into a failure instead of hanging the test runner. The test releases A during cleanup.

The remaining tests exercise behavior callers depend on:

• A failed read is retried on the next call. Errors never reach the insertion step.
• A cached false is a hit. Checking the value's truthiness would incorrectly trigger another read.
• The TTL expires exactly at its boundary. A backward clock jump also causes a fresh read.
• Reading an entry updates its position before least-recently-used eviction.
• If two reads finish out of order, the last completion wins, even when it contains older data.

These are deterministic tests of the cache algorithm. The Worker runtime's I/O ownership rule is documented by Cloudflare; testing that rule itself requires separate Worker invocations, not two function calls in Node.

Save the following as settled-cache.test.mjs beside the module, then run node --test settled-cache.test.mjs.

import test from "node:test";
import assert from "node:assert/strict";
import { SettledCache } from "./settled-cache.mjs";

test("a pending request does not hold another request's cache miss", async () => {
  const cache = new SettledCache();
  let finishFirst;
  const first = cache.load("public-config", () => new Promise(resolve => { finishFirst = resolve; }));
  const second = cache.load("public-config", async () => "second read");
  let timer;
  try {
    assert.equal(await Promise.race([
      second,
      new Promise((_, reject) => { timer = setTimeout(() => reject(new Error("shared pending read")), 200); })
    ]), "second read");
    assert.equal(await cache.load("public-config", () => { throw new Error("should hit cache"); }), "second read");
  } finally {
    clearTimeout(timer);
    finishFirst("first read");
    await first;
  }
});

test("errors are retried, and false values are real cache hits", async () => {
  const cache = new SettledCache();
  await assert.rejects(cache.load("flag", async () => { throw new Error("unavailable"); }));
  assert.equal(await cache.load("flag", async () => false), false);
  assert.equal(await cache.load("flag", () => { throw new Error("should hit cache"); }), false);
});

test("TTL expires at the boundary and a backward clock forces a fresh read", async () => {
  let now = 100;
  const cache = new SettledCache({ ttlMs: 10, clock: () => now });
  await cache.load("a", async () => 1);
  now = 109;
  assert.equal(await cache.load("a", async () => 2), 1);
  now = 110;
  assert.equal(await cache.load("a", async () => 2), 2);
  now = 109;
  assert.equal(await cache.load("a", async () => 3), 3);
});

test("reads touch LRU order before a new entry evicts the oldest", async () => {
  const cache = new SettledCache({ limit: 2 });
  await cache.load("a", async () => 1);
  await cache.load("b", async () => 2);
  await cache.load("a", async () => 99);
  await cache.load("c", async () => 3);
  assert.equal(await cache.load("a", async () => 99), 1);
  assert.equal(await cache.load("b", async () => 22), 22);
});

test("simultaneous misses use last completion, not newest data version", async () => {
  const cache = new SettledCache();
  let release;
  const older = cache.load("config", () => new Promise(resolve => { release = resolve; }));
  await cache.load("config", async () => "new version");
  release("old version");
  await older;
  assert.equal(await cache.load("config", async () => "unused"), "old version");
});

Decide what this cache is allowed to forget

An isolate cache is an optimization. A request can reach another isolate with an empty map, and an isolate can disappear. Keep authoritative state in durable storage and make a cold read correct.

Use a count limit when values are small and similarly sized. A hundred large documents can still consume too much memory: use a byte budget for variable-size payloads. TTL limits how long a value is reused; it does not itself bound how many keys the map holds.

The example caches shared public data. Do not reuse the same key for private results belonging to different users or tenants. Authorization still belongs on the request path, and any private cache needs an explicit isolation and invalidation design.

Concurrent misses also introduce a consistency choice. An older read can finish last and overwrite a newer result. For immutable assets, use versioned keys. For mutable data requiring ordering, compare versions or put coordination in a system that owns that state. A short TTL alone does not guarantee freshness.

Finally, independent reads do not solve upstream overload. Keep each read's timeout and failure handling at the caller, and decide whether the saved coordination complexity justifies duplicate reads during bursts. If you need global deduplication or strict consistency, an isolate-local map is the wrong coordination boundary.

Where waitUntil fits

ctx.waitUntil() can extend work beyond a response, within Cloudflare's documented lifetime. It does not transfer request ownership or turn a module-level map into durable shared state. Use it for appropriate background work; do not use it as a substitute for defining which invocation owns a read. See Cloudflare's execution context documentation.

For a small public-data cache, the resulting contract is straightforward: each miss owns its read, completed values may be reused, and a cold cache must still work.