The Cache Belongs to the Function

A few years ago, the question about caching in modern web frameworks was whether it should be on by default. That question is largely settled. Frameworks that defaulted to caching every fetch and rendering every page statically have walked the defaults back; frameworks that didn't, didn't have to. The argument that caching should be opt-in, and that the developer should be the one who decides where it pays, has won. Anyone arguing it today is arguing against a position the industry has already conceded.

What is not settled is where the caching primitive lives. The directive that marks a function as cacheable can be implemented in two structurally different ways, and the difference is not yet obvious to most of the people writing code that uses it.

The first design treats "use cache" as a marker on a function. The function carries its own caching contract. Wherever the function runs — on a server, on the edge, in a worker, in a browser — the directive means the same thing. The cache is a property of the function.

The second design treats "use cache" as a marker on a region of a rendering tree. The function exists, the directive is on it, but the cache machinery underneath is part of the framework's rendering pipeline. The cached output is a streaming shell that the framework stitches into a partially prerendered response, with dynamic holes carved out of it by <Suspense> boundaries. The cache is a property of how the page is built.

Both designs are coherent. Both are reasonable answers to a real engineering problem. They are not the same answer. This article is for the first one.

What was settled, and what wasn't

A short reset on the territory is worth the paragraph.

The case against default caching used to be obvious to anyone who had shipped a production application: you spend more time disabling the cache than enabling it. Routes get marked dynamic. Fetches get cache: 'no-store'. Layout segments get tagged force-dynamic. The defaults were calibrated for a population of pages where staleness is cheap and slowness is expensive, and most production applications are not that population. Every site that mattered ended up annotating its way out of the default.

The frameworks that shipped this model heard the criticism and inverted the defaults. In Next.js 15, fetch is no longer cached, segments are no longer static. The dynamicIO mode introduced "use cache" as the primitive a developer reaches for when caching actually pays. Inside that mode, uncached is the baseline; cache only what you mark. This is the design the critics asked for. They got it.

So when this article talks about what a caching primitive should look like, it is not arguing against caching by default. The default is gone. The argument that survives is about what the primitive in its place is made of.

Two places the cache can live

A caching primitive has to live somewhere. The two structural choices are with the function and with the renderer.

A cache that lives with the function is portable. It travels wherever the function travels. The directive is a contract between a function and a runtime; any runtime that understands the directive can honor it; any runtime that does not, ignores it and runs the function. The cache key is the function's inputs. The cache value is the function's output. Nothing about the surrounding system needs to be present for the cache to work, because the cache is the function's contract with its host.

A cache that lives with the renderer is part of a rendering pipeline. This is the design Next.js's Cache Components ships under dynamicIO. It works because the pipeline is in the loop — the cached value is a stream of bytes representing a region of the rendered tree, the dynamic holes are <Suspense> boundaries, the streaming response stitches the cached shell back together with the live data. The directive is a marker on a region of the tree the renderer cares about, and the cache is the part of the renderer that remembers what that region produced. Take the function out of the renderer and the cache disappears, because there is nothing to cache.

The first design is small. The second is integrated. Each one has a thing it is good at and a thing it cannot do.

The first cannot stitch shells around dynamic holes. It does not know about Suspense. It does not produce streaming responses with prerendered prefixes. If you want partial prerendering, the second design is the one you want.

The second cannot run outside the renderer. It cannot ship as a primitive in a library. It cannot run on the edge before the framework boots. It cannot run in the browser. It cannot dedupe a database lookup that happens during a worker job that has nothing to do with rendering. If you want a caching primitive you can reach for in any program, the first design is the one you want.

The two designs are not interchangeable. A cache that requires the framework to be present is a feature of the framework. A cache that requires only a function is a feature of the program. This article is for the second one, and the rest of it is the structural case for that choice — four properties the cache acquires when it belongs to the function and gives up when it belongs to the renderer.

Atomic, not ambient

A function-level cache is atomic. One function, one cache, one set of inputs, one output. The developer can assert — locally, by inspection — that the output is a function of the inputs and that nothing else in the world matters. The function's inputs are the developer's parameters; there is nowhere else for hidden state to come from.

Render-coupled caches give some of this up. A region of a rendering tree closes over the request, the user, the route parameters, the surrounding component state — and the cache machinery has to chase those captures and decide what is safe to serialize. The result is a more powerful cache, but the unit of reasoning has moved. The function is no longer what the developer reasons about. The region is.

The hard part of caching is not the syntax. It is the honesty — marking a function only when its output really is a function of its inputs, and not closing over state the key cannot see. A smaller unit makes that honesty checkable. A larger unit makes it a research project.

Caches that compose

A function-level cache composes. Two cached functions written by two different people, in two different libraries, called from a third function that caches nothing of its own, all behave the way the source reads. The outer call is uncached. The inner calls each consult their own caches. Each layer's decision belongs to that layer.

async function getOrder(id: string) {
  "use cache"
  const order = await db.orders.find(id)
  const customer = await getCustomer(order.customerId)   // also "use cache"
  const products = await Promise.all(
    order.lineItems.map(item => getProduct(item.productId)) // also "use cache"
  )
  return { order, customer, products }
}

Three caches stacked, no coordination. The outer cache stores the assembled order. The inner caches store the customer and each product. They have different TTLs, different tags, different lifetimes. None of them know about the others. A request that asks for an order seen recently hits one cache. A request for a fresh order whose customer is well-known hits two. A request for an entirely new order misses everything and fills in three caches at once. Every path through the system is correct, because every cache was a local decision.

Render-coupled caches compose under the renderer's rules. The shell's lifetime, the hole boundary, the streaming order — these are properties of the pipeline that two cached pieces inherit when they share a tree. The function-level cache carries no surrounding model. The diff is a one-line claim about the function. The blast radius is the function.

A function is a function

A directive that marks a single function does not care what runtime is reading it. The contract is between a function and its caller. The caller might be a server rendering RSC, an SSR pipeline streaming HTML, a worker, an edge runtime, a browser tab running the same code on the client side of an isomorphic boundary. The directive's meaning does not change. The function says: my output is my inputs. The runtime, whichever runtime that is, says: I will hold it.

This is the property the render-coupled cache cannot have, by construction. It works because the renderer is in the loop. Take the same code out of that loop — run it before the framework boots, in a worker job, in a browser tab that does not go through the request lifecycle — and the cache disappears, because the cache is the renderer remembering, and the renderer is not there.

A function-level cache survives the move. A library can ship cacheable utilities and rely on whatever runtime hosts them to honor the directive. There is no if (server) branch, no if (browser) branch, no separate cache wiring per environment. The same function, in any host that understands the directive, has the same contract. A host that does not understand it leaves the function alone.

This is what it means for a caching primitive to be portable. Not that the framework runs in many places — that is a deployment concern, and a different one. The render-coupled cache is a property of the host. The function-level cache is a property of the source. The function carries its own contract.

Locality buys you removability

A function-level cache is removable in a one-line diff. Delete the directive. Ship. The function reverts to first principles — it runs every time it is called — and you can investigate the staleness offline, where it is cheap to be wrong.

Render-coupled caches are removable too, when the unit being uncached is the unit the renderer marks. The harder cases are the surrounding ones: a cached region whose contents vary in ways the renderer's closure analysis did not capture, a tag-based revalidation that turned out to invalidate too much, a cacheLife profile that turned out to be wrong for one specific function in one specific context. The diff is still small; the diagnosis is not, because the failure isn't in the function — it's in the relationship between the function and the renderer.

The same property holds for code review. A "use cache" directive shows up in a diff. A reviewer asks: is this function actually a function of its inputs? Is the TTL right? When the unit being marked is a function, those questions have function-shaped answers. When the unit being marked is a region, the questions also have to ask about Suspense boundaries, about what the renderer captures, about how the streaming response composes. More variables, more places to be subtly wrong.

Scope is also a per-function decision

The strongest underdeveloped property of the function-level cache is scope. The clearest way to see it is to look at a page that needs three caches.

A product page calls getProduct(id) from three different components in the same render. They should see the same value; the database lookup should run once. This is request-scoped dedup.

The same page calls getProductCatalog(), the company's full catalog — refreshed nightly, shared across every request, identical for every user. This is a long-lived in-memory cache.

The same page calls getInventoryStatus(sku), which has to be synchronized across every server in the fleet, because two requests landing on different machines cannot disagree about whether an item is in stock. This is a shared store.

In current Next.js, those are three primitives. React.cache for the first. "use cache" for the second. A custom cache provider, or an external store reached through a server function, for the third. Each has its own API, its own keying, its own invalidation model. A developer who picks the wrong one rewrites the function when they discover the choice was wrong.

In a function-level design, all three are options on the same directive. "use cache: request" for the first. "use cache" for the second. "use cache: shared" for the third. The function shape does not change. The directive carries the answer to which scope it belongs to. Picking the wrong one is a one-line fix.

This is a real, structural advantage, and one of the places where the function-level design has not yet fully shipped in the largest framework that uses the syntax.

The shape of the contract

The directive is a contract between the developer and the runtime, and the mandatory part of the contract has only three lines.

The developer says: this function's output is determined by its inputs.

The runtime says: I will hold the output and serve it again the next time the inputs match.

Both parties say: when this is no longer true, the directive comes off.

That is the surface that has to be there. Everything else — a TTL, a tag, a named profile, a choice of storage, a choice of scope — is an option the developer attaches to the directive when the function calls for it. Tags are useful when there is something to invalidate by group. TTLs are useful when freshness has a known half-life. Named profiles are useful when several functions share the same caching shape and the shape is worth naming once.

None of these options are wrong. They are all part of the directive's optional surface, all developer-attached, all visible at the call site. The asymmetry that matters is between options the developer wrote down and options the runtime applied silently. A developer adding ttl=60 or tags=todos to a directive is making a decision visible in the source. A framework deciding the same thing on the developer's behalf is making the same decision invisible. Only the first kind is in the diff.

The same argument applies, structurally, to every directive in this family. "use client" is a marker that asserts a piece of code crosses a runtime boundary; the value of the marker is that you can read the program and see where the boundaries are. I have argued elsewhere that the directive should be allowed at finer granularity than a file — see The "use client" Tax — but the underlying point is the same. A directive is the developer telling the runtime something the runtime could not have inferred. That contract scales because it is small.

Two kinds of software

Underneath the function-vs-renderer choice is a more general one about what kind of software you are writing.

A framework is a packaged product. It optimizes for the page — for the user-visible artifact at the end of the rendering pipeline. Coupling caching to rendering is the breakthrough that makes partial prerendering work: a static shell streamed first, dynamic holes filled in afterward, no full server roundtrip on a navigation. That is a real win, and it is the win the render-coupled cache exists to deliver. A framework architect choosing the render-coupled design is making a coherent product decision.

A runtime is a primitive. It optimizes for the cache — for the contract a developer can hold in their head and reach for in any program. The function-level cache is not better than the render-coupled cache for the page. It is better for the cache. It composes outside a render tree. It runs in any environment. It survives library packaging. It does not require a mode flag to be turned on. A runtime architect choosing the function-level design is making a coherent primitive decision.

Both choices are defensible. They produce different software. A developer who wants the page wants the render-coupled cache; a developer who wants a caching primitive they can reach for unconditionally wants the function-level one. The directive looks the same in both. The systems underneath it do not.

What the ecosystem pays

The decision about where a caching primitive lives looks, inside one application, like a trade-off between two designs. From above the application — at the level of the JavaScript ecosystem the application depends on — it is something else.

A library author shipping a function cannot assume the consumer is in dynamicIO mode. They cannot assume the consumer is using a framework at all. So a library that wants to ship a cacheable utility — a database client that should dedupe identical queries, a markdown renderer that should not re-parse the same input twice, an API client that should pool requests — has one option under the render-coupled design: do not provide the cache. The library exposes raw functions; the consumer wires them into their framework's caching themselves; everyone reinvents the same wrappers in slightly different shapes, and the bugs all live in the wiring.

Under a function-level design, the library author writes "use cache" at the top of the function and ships. Any consumer whose runtime understands the directive gets the cache. Any consumer whose runtime does not, gets the raw function. The library does not have to know. The consumer does not have to wrap.

This is a pattern. Every time a framework absorbs a capability that could have been a primitive — caching, server functions, partial hydration, request-scoped state, routing — the ecosystem pays. The capability becomes available only inside that framework. Libraries that want it pick the framework as a hard dependency, ship their own version, or expose it as a configuration surface for the consumer to wire up. None of these are good for the developers a level removed from the framework. Each one moves complexity out of the framework and into a thousand small repositories that did not need to invent it.

The render-coupled cache is not the only place this happens. It is one place where the trade is unusually clear. The capability — memoizing a function on its inputs — has a canonical shape. The shape is small. It does not need a renderer to be useful. Putting the renderer in the loop trades that universality for an integration the framework can use to power partial prerendering. That trade is fine for the framework. It is paid by everyone else.

I have made a parallel version of this argument about a different misplaced primitive: a framework exposing a capability as a library API where a directive would have been smaller (RSC as a serializer, not a model). Misplaced primitives look different in the small. From far enough away they look the same.

The smaller point

Where a primitive lives determines what the developer can do with it. A cache that lives in the renderer can do things a function cache cannot — partial prerendering, streamed shells, suspense-aware regions. A cache that lives with the function can do things a render-coupled cache cannot — travel between environments, compose without coordination, ship in a library that does not know what framework will host it.

"use cache" is the same five letters in either design. The choice the developer is making by writing the directive is not really a choice about caching. It is a choice about which of those two things the caching primitive should be.

A cache that lives in the framework belongs to the framework. A cache that lives in the function belongs to the developer. Only one of those travels.