DEV Community: Ahmed Mahmoud

npm Supply-Chain Hardening in 2026: Lifecycle Scripts, minimumReleaseAge, and the Postinstall I Stopped Trusting

Ahmed Mahmoud — Mon, 14 Sep 2026 06:00:09 +0000

Headline: The npm attacks that actually landed over the past two years executed at install time, through lifecycle scripts, within hours of the malicious version being published. Turning dependency lifecycle scripts off by default, refusing versions younger than a few days, and removing long-lived publish tokens from CI blocks most of that path without changing a line of application code.

Key takeaways

npm lifecycle scripts (preinstall, install, postinstall, prepare) are shell commands declared inside a dependency's own package.json and executed by your package manager, with your user's permissions, during npm install — before any of your code runs.
pnpm 10 stopped running dependency lifecycle scripts by default; a package that genuinely needs a build step must be listed in onlyBuiltDependencies. npm has no such default, so I set ignore-scripts=true in .npmrc and allow-list explicitly.
minimumReleaseAge is a cooldown setting — in pnpm, in Renovate, and as Dependabot's cooldown config — that refuses any version published less than N minutes ago. Compromised versions are usually detected and pulled within hours to a few days, so a cooldown absorbs exactly the window in which you would have installed one.
npm provenance attestations prove which repository, commit, and CI workflow produced a tarball. They do not prove the code is benign: a compromised workflow emits perfectly valid provenance.
Trusted publishing over OIDC removes the long-lived npm token from CI. That token is the credential the self-replicating npm worm of 2025 harvested from postinstall scripts in order to publish itself into the maintainer's other packages.

What actually runs on my machine when I type npm install?

npm install executes arbitrary code from your dependency tree before any of your own code runs. A lifecycle script is a command declared in a package's scripts field that the package manager invokes automatically at install time — it inherits your shell environment, your ~/.npmrc, your SSH keys, and whatever cloud credentials sit on disk. There is no sandbox around it by default.

The part I under-weighted for years is that this applies to the whole transitive tree, not just the dependencies I chose. Five direct dependencies routinely resolve to several hundred packages, and any one of them can declare a postinstall.

The first useful step is knowing how many packages in your tree claim that right:

# every package in the installed tree that declares an install-time script
npm query ":attr(scripts, [postinstall])" | jq -r '.[].name'

On pnpm 10 the equivalent is pnpm approve-builds, which lists the packages whose build scripts were blocked and asks which ones to permit. The first time I ran either command on a mature repo, the list was shorter than I feared and less legitimate than I hoped — a handful of native modules that really do compile, and several pure-JavaScript packages running analytics pings.

How do I stop dependency lifecycle scripts from running?

Set ignore-scripts=true in the project's .npmrc for npm and Yarn, or stay on pnpm 10's default of not running dependency scripts, then allow-list the few packages that genuinely need a build step.

# .npmrc — checked into the repo, applies to every install here
ignore-scripts=true

The honest cost: npm's ignore-scripts is not selective. It also disables your own package's scripts, so a repo relying on prepare to install Husky hooks will silently stop doing that. I move those into an explicit npm run setup step that the README and the CI job both call, and run npm rebuild sharp better-sqlite3 for native modules that need compilation.

pnpm's version is narrower and better, because the allow-list is per package rather than all-or-nothing:

{
  "pnpm": {
    "onlyBuiltDependencies": ["esbuild", "sharp", "better-sqlite3"]
  }
}

The heuristic for that list: native addons and binary downloaders (esbuild, playwright) have a real reason to run at install. A pure-JavaScript utility asking for a postinstall does not, and that request is itself the signal.

What is minimumReleaseAge, and why does a release cooldown stop worm-style attacks?

minimumReleaseAge is a package-manager setting that refuses to install any version published more recently than a given number of minutes. pnpm reads it from pnpm-workspace.yaml, Renovate has supported it as a config option for years, and Dependabot exposes the same idea as a cooldown block in dependabot.yml.

# pnpm-workspace.yaml
minimumReleaseAge: 4320          # minutes; 3 days
minimumReleaseAgeExclude:
  - "@my-scope/*"                # our own packages publish and deploy same-day

// renovate.json
{
  "minimumReleaseAge": "3 days",
  "internalChecksFilter": "strict"
}

A cooldown is disproportionately effective because a malicious npm version has a short shelf life. Registry maintainers, scanners, and other developers find these within hours to a couple of days, and the version gets unpublished or deprecated. The cooldown does not make you smarter than the attacker; it makes you later than the people who find them, which is the same outcome for a fraction of the effort.

The trade-off is real: you are deliberately three days behind on everything, including genuine security patches. I keep a documented override — a one-line config exclusion plus a PR that says why — so consciously pulling a same-day CVE fix stays possible and stays visible.

Does npm provenance prove a package is safe?

No. npm provenance is a signed attestation, generated by npm publish --provenance running in a supported CI provider, that links a published tarball to the source repository, commit, and workflow that built it. It proves origin, not intent.

# verify registry signatures and attestations for the installed tree
npm audit signatures

What provenance buys me: a stolen laptop token can no longer publish a tarball that claims to come from our repo and carries a valid attestation, and I can diff a published artifact against the commit it names. What it does not cover: a maintainer who merges a malicious commit, a compromised build workflow, or the several hundred transitive packages that publish no attestation at all.

The complementary control is trusted publishing, where npm exchanges a short-lived OIDC token from the CI run instead of reading a long-lived secret. Token theft is what turned the 2025 npm worm from an incident into a spreading one: its postinstall payload scraped credentials and used them to publish itself into whatever else that maintainer owned. A publish flow with no long-lived token in the environment has nothing to scrape.

What should CI change to reduce the blast radius of a bad install?

Assume one install will eventually execute hostile code, and make that install a boring target.

Use npm ci, never npm install, in CI. npm ci installs strictly from package-lock.json and fails when the lockfile and package.json disagree, removing silent transitive drift between the version a human reviewed and the version CI resolved.

Pin GitHub Actions to a full commit SHA, not a tag. Git tags are mutable, and the 2025 tj-actions/changed-files compromise worked by repointing existing tags at a malicious commit, so everyone pinned to @v4 picked it up automatically.

- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

Keep secrets out of the job that installs dependencies. The install step needs no publish token, no cloud key, no deploy credential. Splitting install/build from publish/deploy into separate jobs, with permissions: contents: read on the first, means an install-time payload runs in a job holding nothing worth stealing.

Commit the lockfile and review its diff. The integrity hash pins bytes, so the lockfile diff is the one place a surprise transitive bump is actually visible to a reviewer.

Which hardening step buys the most safety per unit of pain?

Control	What it stops	What it costs
`ignore-scripts` / `onlyBuiltDependencies`	Install-time code execution from any package in the tree	Native modules break until allow-listed; npm's flag also disables your own scripts
`minimumReleaseAge` cooldown	Freshly published compromised versions, the dominant attack shape	You run N days behind on everything, security patches included
`npm ci` + committed lockfile	Unreviewed transitive drift between review and deploy	Effectively none
Actions pinned to commit SHA	Mutable-tag hijack of your CI pipeline	Bump churn; let Renovate update the SHAs
Trusted publishing (OIDC)	Token theft, and therefore worm-style self-replication	One-time setup; publishing must happen from CI
Provenance + `npm audit signatures`	Tarballs not built from the source they claim	Does not stop malicious source; ecosystem coverage is partial

If I could ship only one of these to a team tomorrow, it would be the cooldown: no allow-list maintenance, no developer behaviour change. Disabling lifecycle scripts is strictly stronger but has a first-week cost while you discover which packages actually needed them.

FAQ

Q: Will ignore-scripts=true break my project?

A: It can, because npm's ignore-scripts also disables your own package's lifecycle scripts such as prepare, which is how Husky installs git hooks. Move those into an explicit npm run setup step and run npm rebuild <package> for native modules that need compiling.

Q: Does pnpm block install scripts by default?

A: Yes. pnpm 10 does not run dependency lifecycle scripts unless the package is listed in onlyBuiltDependencies, and pnpm approve-builds shows which packages were blocked so you can permit them individually.

Q: How long should a release-age cooldown be?

A: Three to seven days covers the detection window for most publicly disclosed npm compromises. Keep a documented override path so an urgent security patch can still be pulled the same day.

Q: Isn't npm audit already covering this?

A: No. npm audit reports known CVEs from published advisories, and a malicious version has no advisory at the moment you would install it. npm audit signatures is a different check that verifies registry signatures and provenance attestations; the two are complementary.

Q: Does the lockfile's integrity hash protect me from a malicious package?

A: The integrity field is a subresource hash guaranteeing you received the same bytes as when the lockfile was written. It protects against tampering between the registry and you; it says nothing about whether those bytes were malicious when published.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Hybrid Search in Postgres with pgvector: Field Notes on HNSW, tsvector, and Why Pure Vector Search Missed Exact Matches

Ahmed Mahmoud — Sun, 06 Sep 2026 15:27:33 +0000

Headline: pgvector is a Postgres extension that adds vector column types and approximate-nearest-neighbour indexes, and used alone it is a mediocre search engine. Every retrieval bug I shipped in a RAG feature was fixed by fusing pgvector similarity with Postgres full-text search through Reciprocal Rank Fusion — not by buying a vector database and not by swapping the embedding model.

My first retrieval pipeline was one ORDER BY embedding <=> $1 LIMIT 5 and a prompt. It demoed well, then failed on the queries people actually type: exact error codes, function names, invoice numbers, product SKUs. Cosine similarity is perfectly happy to return five paragraphs that are about billing when the user typed a literal invoice ID that appears verbatim in exactly one row.

These are the notes from moving that pipeline to hybrid search on Postgres 17 with pgvector 0.8, kept in the same database as the application data.

Key takeaways

pgvector 0.8.0 adds four vector types (vector, halfvec, bit, sparsevec) and two ANN index types (HNSW and IVFFlat) to Postgres. It does not add keyword matching, reranking, or chunking.
Pure vector search fails on exact identifiers. An embedding encodes meaning, so a literal token like ERR_MODULE_NOT_FOUND holds no privileged position in the vector space, while Postgres full-text search matches it exactly.
Reciprocal Rank Fusion combines both retrieval arms without normalising scores. RRF scores each row as the sum of 1 / (k + rank) across arms, with k = 60 as the common default.
HNSW is the default index and IVFFlat is the exception. HNSW builds on an empty table; IVFFlat must be created after the table holds representative rows.
A WHERE clause on an HNSW query can return fewer rows than the LIMIT asks for. The hnsw.iterative_scan setting added in pgvector 0.8.0 is the fix.

What does pgvector actually add to Postgres?

pgvector is a Postgres extension that adds vector column types, distance operators, and approximate-nearest-neighbour indexes to an ordinary Postgres database. You enable it with CREATE EXTENSION vector; and you get four storage types: vector (4-byte floats), halfvec (2-byte floats, added in pgvector 0.7.0), bit (binary quantisation), and sparsevec.

The distance operators are worth memorising, because choosing the wrong one silently degrades ranking instead of throwing: <=> is cosine distance, <-> is L2 distance, <#> is negative inner product, and <+> is L1 distance. OpenAI's text-embedding-3-small and most hosted models return normalised vectors, so I use <=> with a vector_cosine_ops index and stop thinking about it.

The index operator class must match the query operator. An HNSW index built with vector_l2_ops is simply not used by a query ordering on <=>, and Postgres falls back to a sequential scan without complaining. My first "pgvector is slow" investigation was exactly that mismatch, visible in one EXPLAIN ANALYZE.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
  id          bigserial PRIMARY KEY,
  document_id bigint NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
  body        text NOT NULL,
  embedding   vector(1536) NOT NULL,
  tsv         tsvector GENERATED ALWAYS AS (to_tsvector('english', body)) STORED
);

CREATE INDEX chunks_embedding_hnsw
  ON chunks USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

CREATE INDEX chunks_tsv_gin ON chunks USING gin (tsv);

What pgvector does not give you: tokenisation, keyword matching, reranking, chunking, or query rewriting. It is a similarity index, not a search product.

Why did pure vector search miss exact matches?

Pure vector search misses exact matches because an embedding encodes meaning rather than literal tokens. When a support agent searches for ERR_MODULE_NOT_FOUND, the embedding of that string lands near "module", "import", and "error" in general — near enough to rank a generic troubleshooting page above the one paragraph containing the code verbatim.

Three query shapes broke consistently for me: exact error codes, product SKUs and order numbers, and rare proper nouns such as a customer's company name. All three share a property: the useful signal is a low-frequency token, and averaging it into a 1536-dimension chunk embedding dilutes it into noise.

Chunk size makes it worse. A 1,500-token chunk produces one vector representing the average of everything in it, so a single decisive sentence barely moves the vector. Cutting chunks to roughly 200–400 tokens with a small overlap improved my retrieval more than any embedding-model change I tried.

Postgres full-text search has the opposite failure mode. websearch_to_tsquery('english', 'how do I stop being billed') will not match a document titled "Subscription termination", because no shared stem exists. That complementary failure is the entire argument for hybrid search.

How do I combine full-text and vector search in one SQL query?

Run both searches as separate CTEs, rank each independently, then fuse the ranks with Reciprocal Rank Fusion in a single query. RRF ignores raw scores and uses only position, which is why it needs no tuning: cosine distance lives on roughly 0 to 2, while ts_rank_cd is unbounded, and weighting them directly turns into a magic-constant hunt.

Reciprocal Rank Fusion assigns each row a score of 1 / (k + rank) in every arm it appears in, then sums those scores. k = 60 is the value from the original RRF paper and the one I have never needed to change.

WITH semantic AS (
  SELECT id, RANK() OVER (ORDER BY embedding <=> $1::vector) AS rank
  FROM chunks
  ORDER BY embedding <=> $1::vector
  LIMIT 50
),
keyword AS (
  SELECT c.id, RANK() OVER (ORDER BY ts_rank_cd(c.tsv, q) DESC) AS rank
  FROM chunks c, websearch_to_tsquery('english', $2) q
  WHERE c.tsv @@ q
  ORDER BY ts_rank_cd(c.tsv, q) DESC
  LIMIT 50
)
SELECT c.id, c.body,
       COALESCE(1.0 / (60 + s.rank), 0.0)
     + COALESCE(1.0 / (60 + k.rank), 0.0) AS score
FROM chunks c
LEFT JOIN semantic s ON s.id = c.id
LEFT JOIN keyword  k ON k.id = c.id
WHERE s.id IS NOT NULL OR k.id IS NOT NULL
ORDER BY score DESC
LIMIT 10;

Two details matter. The LEFT JOIN plus COALESCE pattern lets a row win by being strong in one arm alone — that is how the exact SKU match survives even though its embedding ranks nowhere. And websearch_to_tsquery is the right parser for user input: it accepts quoted phrases and -exclusions and never throws a syntax error on hostile input the way to_tsquery does.

I retrieve 50 per arm and return 10. Retrieving 5 per arm produces two disjoint lists and fusion has nothing to fuse.

HNSW or IVFFlat: which pgvector index should I build?

Build HNSW unless index build time or memory is the binding constraint.

Property	HNSW	IVFFlat
Build on empty table	Yes	No — needs representative rows
Build parameters	`m`, `ef_construction`	`lists`
Query-time knob	`hnsw.ef_search` (default 40)	`ivfflat.probes` (default 1)
Build time	Slower	Faster
Index size / memory	Larger	Smaller
Recall per unit latency	Better	Lower at equal latency
Heavy inserts	Handled incrementally	Degrades; needs rebuilds

The knob that actually changes results is hnsw.ef_search, which controls how many candidates the graph traversal keeps. Set it per transaction so a background reindexing job and a user-facing query can use different values.

BEGIN;
SET LOCAL hnsw.ef_search = 100;
-- hybrid query here
COMMIT;

Why does adding a WHERE filter return fewer rows than my LIMIT?

An HNSW query with a WHERE clause can return fewer rows than the LIMIT requests because the index traversal collects ef_search candidates first and the filter is applied afterwards. Ask for 10 chunks belonging to one tenant that owns 0.1% of the table, and most of the 40 default candidates get discarded.

This bit me on a multi-tenant knowledge base, and it is the worst class of bug: no error, no slow query, just quietly thinner context arriving at the model.

Enable iterative scans. pgvector 0.8.0 added hnsw.iterative_scan, which keeps scanning until enough rows survive the filter. Use relaxed_order for throughput, strict_order when exact distance order matters, and cap work with hnsw.max_scan_tuples.
Use a partial index when the filter is a small fixed set of values, e.g. WHERE deleted_at IS NULL.
Raise ef_search as a blunt instrument when the filter is mildly selective.

SET LOCAL hnsw.iterative_scan = 'relaxed_order';
SET LOCAL hnsw.max_scan_tuples = 20000;

What broke in production?

The dimension mismatch is the first error everyone meets: ERROR: expected 1536 dimensions, not 768. A vector(1536) column is a hard constraint, so switching embedding models is a migration, not a config change. Embeddings from different models are not comparable at all — a model swap means a new column, a full backfill, and a cutover, never a partially re-embedded table.

The 2000-dimension index limit surprised me more. pgvector indexes vector columns up to 2,000 dimensions, so text-embedding-3-large at 3,072 dimensions cannot be indexed as a plain vector. Two honest options: store it as halfvec(3072) and index with halfvec_cosine_ops, which pgvector indexes up to 4,000 dimensions and which also halves storage; or request fewer dimensions from the API using the dimensions parameter.

Index builds were slower than expected on a few hundred thousand rows, because the default maintenance_work_mem is far too small for a graph index. Raising maintenance_work_mem and max_parallel_maintenance_workers turned an overnight build into a deploy-window build.

Row width is the quiet one. A vector(1536) value occupies roughly 6 KB, well past the point where Postgres moves the column out of line into TOAST storage, so every row fetch becomes an extra read. Keeping the chunk table narrow and joining to document metadata was worth more than any query rewrite.

Finally, the latency I chased in SQL was not in SQL. Embedding the user's query is a network round trip on every search, and it dominated my P95 long before Postgres did.

FAQ

Q: Do I need a dedicated vector database instead of pgvector?

A: Not for application-scale retrieval in the low millions of chunks. Keeping vectors in Postgres means one backup story, one connection pool, transactional consistency between documents and embeddings, and the ability to join retrieval results against permissions in the same query.

Q: Which distance operator should I use with OpenAI embeddings?

A: Cosine distance, the <=> operator, with an HNSW index created using vector_cosine_ops. The operator class must match the ORDER BY operator or Postgres silently falls back to a sequential scan.

Q: Does hybrid search require two embeddings per chunk?

A: No. It needs one vector column for semantic similarity and one tsvector column for lexical matching. Write the tsvector as a GENERATED ALWAYS AS ... STORED column so it can never drift from the body text.

Q: How many results should I retrieve before passing them to the model?

A: Around 50 candidates per arm, fused with RRF, then the top 5 to 10 fused chunks to the model. Feeding more usually lowers answer quality, because irrelevant context competes with relevant context inside the prompt.

Q: Should I add a reranking step on top of hybrid search?

A: Only after hybrid search is in place and you can measure that the right chunk is retrieved but ranked too low. Reranking cannot recover a chunk that retrieval never returned, so fixing recall first is the higher-leverage move.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Web Workers in the Next.js App Router: Field Notes on import.meta.url, DataCloneError, and What a Worker Cannot Fix

Ahmed Mahmoud — Sat, 05 Sep 2026 06:00:11 +0000

Headline: A Web Worker is a background JavaScript thread with no DOM access, and it only improves responsiveness when the bottleneck is your own CPU-bound code — not React rendering. I moved three features off the main thread in a Next.js App Router app, and every bug I hit came from serialization, bundler paths, or React StrictMode spawning a second worker I never terminated.

I had a client-side CSV importer that froze the tab for several seconds on large files. My first instinct was to memoize harder. That was wrong: the freeze was one long task inside my own parsing code, and no amount of useMemo moves work off the thread it already runs on. A Web Worker does.

A Web Worker is a separate JavaScript thread the browser runs alongside the main thread, with its own global scope, no DOM access, and communication only through message passing. I have since moved three features into workers in a Next.js 16 App Router app — CSV parsing, client-side image resizing, and a fuzzy search index. These are the notes I wish I had on day one.

Key takeaways

A Web Worker only helps when the long task is your own CPU-bound JavaScript. If the browser's Performance panel attributes the long task to React rendering, commit, or style recalculation, a worker moves nothing, because rendering must stay on the main thread.
Always construct workers with new Worker(new URL('./parse.worker.ts', import.meta.url)). Both webpack 5 and Turbopack detect that exact expression and emit the worker as its own hashed chunk; a plain string path resolves to a 404 in a production build.
postMessage serializes with the structured clone algorithm, which drops functions, class prototypes, DOM nodes, and getters. Sending any of those throws DataCloneError, and sending a class instance silently produces a plain object with no methods on the other side.
Comlink turns postMessage into awaited method calls using a JavaScript Proxy, so the worker exposes an object and the page calls await api.parse(file). It costs roughly 1 KB gzipped and removes all request-ID bookkeeping.
React StrictMode runs effects twice in development, so an uncleaned worker leaks a second thread. Return () => worker.terminate() from the effect; without it a hot-reloading page accumulates workers until the tab is slower than before.

What does a Web Worker actually fix, and what does it not?

A Web Worker fixes main-thread blocking caused by long-running synchronous JavaScript that does not touch the DOM. Parsing, compressing, diffing, hashing, tokenizing, building a search index, decoding a large payload, and running a WebAssembly module all move cleanly.

A Web Worker does not fix slow React rendering. React's reconciliation and commit phases must run on the main thread because they write to the DOM. If an Interaction to Next Paint (INP) regression comes from rendering a 5,000-row table, the fix is virtualization or fewer components, not a worker. I wasted an afternoon before I accepted that.

The distinction is visible in a Performance trace: expand the long task and read the flame chart. If the widest frames carry your own function names, a worker will help. If they are React internals or Recalculate Style, it will not.

Workers also have no access to window, document, or localStorage. They do get fetch, IndexedDB, WebAssembly, crypto.subtle, OffscreenCanvas, and timers. The rough rule I use now: if the code would run unchanged in Node.js, it will run in a worker.

How do I create a Web Worker in the Next.js App Router?

Create the worker inside a useEffect in a Client Component, using new URL(..., import.meta.url), and terminate it in the cleanup function. The Worker constructor does not exist in Node.js, so constructing one during server rendering or at module scope throws ReferenceError: Worker is not defined at build time.

'use client';

import { useEffect, useRef } from 'react';

export function CsvImporter() {
  const workerRef = useRef<Worker | null>(null);

  useEffect(() => {
    const worker = new Worker(
      new URL('../workers/parse.worker.ts', import.meta.url),
      { type: 'module' }
    );
    worker.onmessage = (e: MessageEvent<{ rows: number }>) => {
      console.log('parsed', e.data.rows);
    };
    workerRef.current = worker;
    return () => worker.terminate();
  }, []);

  // ...
}

Three details matter. The new URL('./x.worker.ts', import.meta.url) expression must appear literally inside the Worker constructor — hoisting it into a variable defeats the bundler's static analysis and the file is never emitted. The { type: 'module' } option is what lets the worker use import statements. And the worker file belongs outside app/ — I keep mine in src/workers/ — so the router never tries to interpret it as a route.

The worker file itself is an ordinary module that listens on its own global scope:

// src/workers/parse.worker.ts
self.onmessage = (e: MessageEvent<{ text: string }>) => {
  const rows = e.data.text.split('\n').length;
  self.postMessage({ rows });
};

export {};

The trailing export {} is not decorative. It makes the file a module so TypeScript scopes self to the worker context instead of colliding with the DOM lib's global declarations.

Why does my worker throw DataCloneError?

postMessage serializes its argument with the structured clone algorithm, which copies plain data but refuses functions, Symbols, DOM nodes, and anything holding a closure. Passing one of those throws DataCloneError: Failed to execute 'postMessage'.

The quieter failure is class instances. Structured clone copies own enumerable properties and discards the prototype, so a Decimal or a parser instance arrives as a plain object with no methods, and the first method call blows up far from the postMessage line that caused it. I now send only JSON-shaped data across the boundary and rehydrate on the receiving side.

Transferable objects are the escape hatch for large payloads. An ArrayBuffer, MessagePort, ImageBitmap, OffscreenCanvas, or ReadableStream can be transferred instead of copied, which hands ownership to the other thread in constant time and leaves the original detached:

const buffer = await file.arrayBuffer();
worker.postMessage({ buffer }, [buffer]);
// buffer.byteLength is now 0 on this thread — ownership moved

Forgetting that second argument is the difference between moving a pointer and copying every byte. For the image-resize worker this was the single change that made the interaction feel instant, because a multi-megabyte copy was happening on the main thread before the worker ever started.

When should I use Comlink instead of raw postMessage?

Comlink is a roughly 1 KB library from the Chrome team that wraps postMessage in a JavaScript Proxy so the worker looks like an awaitable object. Reach for it as soon as the worker has more than one operation, because hand-rolled routing means inventing request IDs, a pending-promise map, and a discriminated union of message types — then maintaining all three.

// src/workers/search.worker.ts
import * as Comlink from 'comlink';

const api = {
  async buildIndex(docs: Doc[]) { /* ... */ },
  async query(term: string) { return search(term); },
};

export type SearchApi = typeof api;
Comlink.expose(api);

import * as Comlink from 'comlink';
import type { SearchApi } from '../workers/search.worker';

const worker = new Worker(
  new URL('../workers/search.worker.ts', import.meta.url),
  { type: 'module' }
);
const api = Comlink.wrap<SearchApi>(worker);
const results = await api.query('next.js');

Two Comlink rules I learned by breaking them. Callbacks must be wrapped as Comlink.proxy(cb), because a bare function cannot be cloned. And transferables need Comlink.transfer(value, [buffer]) — Comlink will not infer a transfer list, so the zero-copy win silently disappears if you skip it.

Use import type for SearchApi so the worker module is never pulled into the main bundle, while the main thread still gets full autocomplete with every return type wrapped in a Promise.

Should I use a Web Worker or scheduler.yield()?

Use scheduler.yield() when the work must touch the DOM or is only moderately long; use a Web Worker when the work is pure computation measured in hundreds of milliseconds. scheduler.yield() is a Scheduling API method that returns a Promise and lets the browser service pending input before the same function continues — it splits one long task into several short ones on the same thread rather than moving the work.

for (const [i, item] of items.entries()) {
  render(item); // touches the DOM, must stay on the main thread
  if (i % 50 === 0) await scheduler.yield();
}

Aspect	Web Worker	scheduler.yield()
Thread	Separate thread; main thread stays free	Same thread; task split into chunks
DOM access	None	Full
Setup cost	Spawn a thread and parse a second bundle	One `await`
Data cost	Structured clone unless transferred	Zero — same memory
Best for	Parsing, hashing, indexing, WebAssembly	Long loops that build or mutate UI

The setup cost is real: spawning a worker means the browser creates a thread and parses a second bundle. For work measured in a couple of milliseconds, the round trip is pure overhead. I only move something into a worker once I can see it as a long task — over 50 ms — in a trace.

What broke for me the first time?

StrictMode spawned two workers. React StrictMode mounts, unmounts, and remounts every component in development. My effect created a worker and returned nothing, so each hot reload left an orphaned thread holding its index in memory. Returning () => worker.terminate() fixed it, and it is also the correct production behavior when a user navigates away mid-parse.

A hoisted URL produced a 404 in production. Development worked; the deployed build requested a worker path that did not exist and got an HTML error page back. The cause was refactoring new URL(...) into a shared constant, which defeats the bundler's static detection. Keep the expression inline inside the constructor.

Errors vanished. An exception thrown inside a raw worker does not reject anything on the main thread; it fires an error event on the worker object. I now always attach worker.onerror, and with Comlink I wrap calls in try/catch — remembering that only the message and stack survive the boundary, so a custom error class arrives as a generic object unless you register a Comlink.transferHandlers entry.

SharedArrayBuffer was not an option. Sharing memory between threads without copying requires cross-origin isolation: Cross-Origin-Opener-Policy: same-origin plus Cross-Origin-Embedder-Policy: require-corp. Those headers broke embedded third-party widgets on the same page, so I stayed with transferables. If your app embeds anything cross-origin, budget real time before assuming SharedArrayBuffer is available.

One worker was enough. I built a pool sized by navigator.hardwareConcurrency before measuring, then found a single worker already removed every long task from the trace. A pool is worth it when you have genuinely parallel independent chunks; it is not a default.

FAQ

Q: Do Web Workers work with Server Components in the Next.js App Router?
A: Not directly. Worker is a browser API, so the file that constructs it needs the 'use client' directive, and construction must happen inside useEffect or an event handler so it never runs during server rendering. A Server Component can freely render the Client Component that owns the worker.

Q: Does moving work to a Web Worker improve INP?
A: Only when the interaction is blocked by CPU-bound JavaScript you control. Interaction to Next Paint measures the delay from an interaction to the next frame, so moving a 400 ms parse off the main thread helps directly, while a slow React commit is unaffected because rendering cannot leave the main thread.

Q: Does Turbopack support new Worker(new URL(...))?
A: Yes. Turbopack, the default bundler for next dev and next build in Next.js 16, detects the new URL('./file', import.meta.url) pattern inside a Worker constructor and emits the worker as its own chunk, the same way webpack 5 does.

Q: What is the difference between a Web Worker and a Service Worker?
A: A Web Worker is a background compute thread owned by one page and terminated with it. A Service Worker is a network proxy between the page and the network that persists across page loads and powers offline caching and push notifications; it is not a place to run heavy computation for the current page.

Q: Can I share state between the main thread and a worker without copying?
A: Yes, with SharedArrayBuffer, but the page must be cross-origin isolated via COOP and COEP response headers. Without those headers, use transferable objects, which move ownership of an ArrayBuffer in constant time instead of sharing it.

Closing

The mental model that finally made workers easy: a worker is a tiny server that happens to run in the same tab. You send it a request, it answers with data, and everything crossing the gap must survive serialization. Once I stopped sending rich objects across the boundary and started sending buffers and plain JSON, the confusing failures went away.

The discipline is measurement first. Open a trace, find the long task, read the flame chart, and only then reach for a thread. Web Workers fix one specific problem very well, and they add real complexity to everything else.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Turborepo in a Real Monorepo: Field Notes on Cache Misses, NEXT_PUBLIC Poisoning, and the tasks Key That Replaced pipeline

Ahmed Mahmoud — Thu, 03 Sep 2026 06:00:08 +0000

Headline: Turborepo is a task runner that hashes each script's inputs, caches its outputs, and replays them instead of re-running the work. The cache is only as correct as the inputs, outputs, and env you declare in turbo.json — every wrong answer I got out of it was a declaration I never wrote.

I moved a two-app, six-package monorepo onto Turborepo 2 this year: a Next.js front end, a NestJS API, and shared UI, config, and type packages. The first week felt like a straight win, because CI stopped rebuilding packages nobody had touched. The second week I shipped a production build that was still pointing at the staging API, and the deploy log said cache hit, replaying logs. These are the notes I wish I had before that deploy.

Key takeaways

Turborepo is a memoization layer over package.json scripts. It does not install dependencies, link workspaces, or compile anything — pnpm and tsc still do that work.
Turborepo 2.0 renamed the top-level pipeline key in turbo.json to tasks. Run npx @turbo/codemod@latest migrate instead of editing the file by hand.
An undeclared environment variable is the most dangerous cache input. Next.js inlines NEXT_PUBLIC_* values into the client bundle at build time, so a cache hit can physically replay another environment's URLs.
Turborepo 2.0 defaults to strict environment mode, where a task only sees variables listed in env or globalEnv. That default converts a silent wrong-value bug into a loud undefined one.
The --affected flag is only as good as your git history. A shallow CI clone has no base commit to diff against, so the filter quietly degrades into building everything.

What does Turborepo actually do, and what does it not replace?

Turborepo is a task runner for JavaScript monorepos that builds a dependency graph of your scripts, hashes the inputs of each task, and restores cached outputs when the hash matches a previous run. That is the entire product. It does not install packages, does not resolve workspace links, and does not bundle a single byte.

The division of labour is worth stating plainly, because it decides where to debug:

pnpm, npm, or yarn workspaces install dependencies and symlink internal packages. Turborepo reads that workspace definition; it does not create it.
tsc, tsup, the Next.js compiler, and the Nest CLI do the actual compiling. Turborepo runs them as ordinary scripts.
Turborepo decides what runs, in what order, across how many cores — and whether it needs to run at all.

The useful mental model is memoization. turbo run build is a memoized call to pnpm build in every package, keyed on a hash of the source files, the dependency graph, the resolved lockfile entries, and the declared environment variables. Memoization is only correct when the function is deterministic with respect to its declared arguments. Every Turborepo bug I have hit was an argument I forgot to declare.

Why does turbo.json use tasks instead of pipeline now?

Turborepo 2.0 renamed the top-level pipeline key to tasks, because the old name implied a sequential pipeline when the value is really a set of task definitions whose order comes entirely from dependsOn. A configuration file still using pipeline fails on Turborepo 2 with a schema error, and npx @turbo/codemod@latest migrate renames it along with the other 2.x migrations.

{
  "$schema": "https://turbo.build/schema.json",
  "ui": "tui",
  "globalEnv": ["NODE_ENV"],
  "tasks": {
    "build": {
      "dependsOn": ["^build"],
      "inputs": ["$TURBO_DEFAULT$", ".env*"],
      "outputs": [".next/**", "!.next/cache/**", "dist/**"],
      "env": ["NEXT_PUBLIC_*", "DATABASE_URL"]
    },
    "test": { "dependsOn": ["build"], "outputs": ["coverage/**"] },
    "dev": { "cache": false, "persistent": true }
  }
}

Two pieces of that syntax carry most of the meaning. "dependsOn": ["^build"] with the caret means "build every package this package depends on first", walking up the dependency graph. "dependsOn": ["build"] without the caret means "run the build task of this same package first". Mixing those two up produces a graph that reads correctly and schedules wrong.

persistent: true marks a task that never exits, such as a dev server. Turborepo 2 refuses to let any other task declare a dependency on a persistent task, which is the right error to raise: a task waiting on something that never finishes is a deadlock, not a build order.

Why is my build cache always missing?

A Turborepo task misses the cache when something inside its hash changed, and it caches nothing at all when its outputs array is empty. Those are two different failures with one symptom — a full rebuild every time — and turbo run build --dry=json tells them apart without running anything, because it prints each task's resolved inputs, environment variables, and final hash.

For a run that already happened, turbo run build --summarize writes a JSON file into .turbo/runs/ containing the hash breakdown per task. Diffing two of those files across a miss is the fastest way I know to find the single input that moved.

Two shapes are worth recognising on sight:

A hash that never repeats usually means a generated artifact is being counted as source. Turborepo's default inputs are the git-tracked files in the package, so a committed dist/ folder or a build-stamped file makes every run unique. Gitignore it.
A hash that never changes when it should means a real input is invisible. .env.local is the classic case: it is gitignored, so it is not in the default inputs, so editing it invalidates nothing.

The fix for the second case is the $TURBO_DEFAULT$ sentinel, which means "the default inputs, plus what I list next". Writing "inputs": [".env*"] on its own replaces the defaults entirely and silently stops tracking your source files, which is a far worse bug than the one you set out to fix.

Why did a cached build ship the wrong NEXT_PUBLIC value?

Next.js inlines every NEXT_PUBLIC_* variable into the client bundle at build time as a string literal, so the compiled .next output physically contains the values from whichever environment produced it. If those variables are not listed in the task's env array, Turborepo's hash cannot see them, the build looks identical to a previous one, and the cache restores a bundle pointing at the wrong API. Nothing throws. The build is green.

"build": {
  "dependsOn": ["^build"],
  "env": ["NEXT_PUBLIC_*", "NEXTAUTH_URL"],
  "passThroughEnv": ["AWS_SECRET_ACCESS_KEY"],
  "outputs": [".next/**", "!.next/cache/**"]
}

Turborepo 2.0 changed the default environment mode to strict, and that default is the real fix. In strict mode a task's process receives only the variables named in env and globalEnv, plus a small built-in allowlist. An undeclared variable is therefore undefined at build time rather than quietly present, which turns an invisible wrong-value bug into an obvious failure you catch locally.

Two distinctions I now apply mechanically. env is for variables that change the artifact, so they belong in the hash. passThroughEnv is for variables a task needs at runtime but that provably cannot change its output, such as an upload credential used after the bundle is written; those reach the process without joining the hash. globalEnv is for variables that affect every task in the repo, which in practice is a very short list.

Should internal packages export TypeScript source or compiled output?

Export raw TypeScript source from an internal package when every consumer is a bundler, and compile to JavaScript when any consumer is a Node process or an npm publish target. Turborepo's documentation calls these Just-in-Time Packages and Compiled Packages, and the choice changes how much there is to cache in the first place.

A Just-in-Time package has no build step at all:

{
  "name": "@repo/ui",
  "exports": { ".": "./src/index.ts" }
}

The consuming Next.js app then needs transpilePackages: ['@repo/ui'] in next.config.ts so its compiler handles the untranspiled source. The upside is that there is no build task, so there is no cache entry to invalidate and no stale dist/ to debug. The downside is that every consuming app compiles the shared code again, and a NestJS service started with node dist/main.js cannot import a .ts file at all.

A Compiled package pays a build step to remove those limits:

{
  "name": "@repo/core",
  "exports": { ".": { "types": "./dist/index.d.ts", "default": "./dist/index.js" } },
  "scripts": { "build": "tsup src/index.ts --format esm --dts" }
}

My rule after living with both: UI components and type-only packages stay Just-in-Time, and anything imported by a Node runtime or published outside the repo gets compiled. Splitting on that line kept the build graph shallow, because most packages have no build task and ^build has almost nothing to wait for.

How do I build only what changed in CI?

turbo run build --affected restricts a run to the packages touched since the base branch, and it is the flag Turborepo 2.1 added to replace the older --filter=...[origin/main] syntax. It compares against main by default, and the TURBO_SCM_BASE and TURBO_SCM_HEAD variables override both ends of the comparison, which is what pull-request builds need.

The gotcha that cost me an afternoon is that --affected is a git operation. GitHub Actions' actions/checkout defaults to fetch-depth: 1, so the runner holds exactly one commit and has no base to diff against — and the run degrades into building everything with no error explaining why. Setting fetch-depth: 0 in the checkout step makes the flag behave.

Remote caching is what makes any of this matter on ephemeral runners. Without it, every CI job starts with an empty cache and the local hits you enjoy on your laptop help nobody else. turbo login followed by turbo link wires up Vercel's remote cache, and setting TURBO_API, TURBO_TOKEN, and TURBO_TEAM points the same client at a self-hosted one.

For Docker, turbo prune <app> --docker writes an out/json directory containing only the lockfile and the package.json files of that app's subgraph, plus an out/full directory with the source. Copying out/json and installing before copying out/full means the install layer only invalidates when a dependency actually changes, rather than on every source edit.

FAQ

Q: Does Turborepo replace pnpm workspaces?
A: No. Turborepo runs and caches tasks, while the package manager still installs dependencies and links internal packages. Turborepo reads the workspace definition from pnpm-workspace.yaml or the workspaces field in the root package.json.

Q: How is Turborepo different from Nx?
A: Both build a task graph and cache task outputs locally and remotely. Turborepo is configuration-only, wrapping the package.json scripts you already have in one turbo.json, while Nx adds plugins, generators, executors, and inferred targets. If your scripts already work, Turborepo is the smaller migration; if you want scaffolding and per-framework integrations, Nx does more.

Q: Can a Turborepo cache hit be wrong?
A: Yes. A cache hit is exactly as honest as the declared inputs and env. If a file or variable changes the output but is not part of the hash, Turborepo will confidently replay a stale artifact, so verify a suspicious hit with turbo run build --dry=json.

Q: Why does my dev task need persistent: true?
A: A dev server never exits, and persistent: true tells Turborepo not to wait for it. Turborepo 2 also rejects any task that declares dependsOn on a persistent task, because that dependency could never be satisfied.

Q: Is remote caching worth it for a solo developer?
A: For local work, no — the local cache under node_modules/.cache/turbo already covers it. For CI, yes, because CI containers are ephemeral and start with an empty cache on every run, which is exactly where a full rebuild costs the most.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Temporal in Production: Field Notes on Replacing JavaScript Date, the Same-Day Bug, and What Doesn't Cross the RSC Boundary

Ahmed Mahmoud — Wed, 02 Sep 2026 06:00:12 +0000

Headline: Temporal is the TC39 API that replaces JavaScript's Date object with immutable, time-zone-aware types. The migration is mechanical almost everywhere, except at two boundaries: a Temporal object cannot be passed as a prop from a Server Component to a Client Component, and it has to become a string again before it reaches a database driver.

I have written the same date bug at least four times in my career. A user in Cairo files a report at 22:30 on the 22nd, the serverless function runs in UTC, and the dashboard files it under the 21st. Every time, the fix was a patch on top of Date — a helper, a library, an offset subtracted somewhere hopeful. This year I stopped patching and moved a Next.js app's date handling onto Temporal instead.

Key takeaways

Temporal replaces Date, not Intl. Temporal owns arithmetic, comparison, and time zones; Intl.DateTimeFormat still does the human-facing formatting, and every Temporal object exposes toLocaleString().
Choose the Temporal type by what the value means. A birthday is a Temporal.PlainDate, an audit timestamp is a Temporal.Instant, and "3pm in Cairo" is a Temporal.ZonedDateTime.
Temporal objects are class instances, so React cannot serialize them across the RSC boundary. Call .toString() in the Server Component and .from() in the Client Component.
Every Temporal object is immutable. .add(), .subtract(), and .with() return new objects, and === never compares two Temporal values correctly — use .equals() or the static compare().
Install the temporal-polyfill package today. Native support across browsers and Node is still uneven, and the polyfill lets you write the final API now and delete one import later.

What does Temporal actually replace?

Temporal replaces the JavaScript Date object — the mutable, millisecond-based, single-time-zone type that has shipped essentially unchanged since 1995. Date has three defects that Temporal removes outright.

First, Date is mutable: d.setDate(d.getDate() + 1) edits an object that other code may still be holding. Second, Date knows exactly two time zones — UTC and whatever the host machine reports — so a function running in UTC and a browser running in Africa/Cairo disagree about what "today" means. Third, Date parsing is inconsistent by specification:

new Date('2026-08-22');           // UTC midnight — the date-only form is parsed as UTC
new Date('2026-08-22T00:00:00');  // local midnight — a date-time with no offset is local

Those two lines can differ by hours, and nothing in the code says so. Temporal turns the distinction into a type: Temporal.PlainDate.from('2026-08-22') is a calendar date with no instant attached, and Temporal.Instant.from('2026-08-22T00:00:00Z') is an exact point on the timeline. You cannot accidentally use one where you meant the other.

Which Temporal type should I use for each field?

Pick the Temporal type from what the value means to the business, not from the database column it happens to live in.

What the value means	Temporal type	String form
Birthday, invoice due date	`Temporal.PlainDate`	`2026-08-22`
Store opening hour	`Temporal.PlainTime`	`09:00:00`
"Meeting at 3pm in Cairo"	`Temporal.ZonedDateTime`	`2026-08-22T15:00:00+03:00[Africa/Cairo]`
`created_at`, audit log entry	`Temporal.Instant`	`2026-08-22T12:00:00Z`
Session length, cache TTL	`Temporal.Duration`	`PT2H30M`
Card expiry	`Temporal.PlainYearMonth`	`2026-08`

Temporal.Now is the entry point for the current moment: Temporal.Now.instant(), Temporal.Now.zonedDateTimeISO(zone), and Temporal.Now.plainDateISO(zone). That last signature is the API doing its job — asking for today's date forces you to answer "today according to whom?", which is exactly the question new Date() lets you skip.

Why did my "is it the same day?" check break for users in another time zone?

A same-day comparison breaks because two instants only fall on the same calendar day relative to a specific time zone, and Date.prototype.toDateString() silently picks the host machine's zone. On a serverless function running in UTC, an event at 22:30 in Cairo has already rolled over to tomorrow.

// Wrong: the server's time zone decides what a "day" is
const sameDay = a.toDateString() === b.toDateString();

// Right: name the zone the question is being asked in
const zone = 'Africa/Cairo';
const sameDay = a.toZonedDateTimeISO(zone).toPlainDate()
  .equals(b.toZonedDateTimeISO(zone).toPlainDate());

The same explicitness shows up in arithmetic across a daylight-saving transition, where Temporal.ZonedDateTime separates calendar units from exact units:

const zdt = Temporal.ZonedDateTime.from('2026-03-28T09:00[Europe/Berlin]');

zdt.add({ days: 1 });   // 2026-03-29T09:00 — same wall clock, 23 real hours later
zdt.add({ hours: 24 }); // 2026-03-29T10:00 — 24 exact hours, different wall clock

Both answers are correct; they answer different questions. "Same time tomorrow" is { days: 1 }. "Twenty-four hours from now" is { hours: 24 }. Date could not express the difference at all.

Can I pass a Temporal object from a Server Component to a Client Component?

No. React rejects class instances at the Server-to-Client boundary with the error Only plain objects, and a few built-ins, can be passed to Client Components from Server Components. Every Temporal type is a class, so a Temporal value has to be serialized to a string before it becomes a prop.

// app/invoices/page.tsx — Server Component
import { Temporal } from 'temporal-polyfill';

export default async function Page() {
  const invoice = await getInvoice();
  const due = Temporal.PlainDate.from(invoice.dueDate);

  // return <DueBadge due={due} />;        ✗ Temporal.PlainDate is a class instance
  return <DueBadge due={due.toString()} />; // ✓ "2026-08-22"
}

'use client';
import { Temporal } from 'temporal-polyfill';

export function DueBadge({ due }: { due: string }) {
  const date = Temporal.PlainDate.from(due);
  const overdue = Temporal.PlainDate.compare(date, Temporal.Now.plainDateISO()) < 0;

  return <span data-overdue={overdue}>{date.toLocaleString('en-GB')}</span>;
}

This one surprised me, because React's serializer does support Date — it is one of the few built-ins on the allowlist. Swapping Date for Temporal therefore breaks props that used to work without a word of warning. The consolation is that the string form carries more information than what it replaced: PlainDate.toString() emits 2026-08-22, and ZonedDateTime.toString() emits 2026-08-22T15:00:00+03:00[Africa/Cairo], which round-trips through from() with the zone intact. An epoch number in a JSON payload can never do that.

How do I store Temporal values in Postgres and read them back?

Store an instant in timestamptz, a calendar date in date, and the user's IANA time zone in its own text column whenever the wall-clock intent matters. Postgres has no column type that carries a time zone despite the name — timestamptz normalizes to UTC on write.

// node-postgres and Drizzle hand back a JS Date for timestamptz — convert at the edge
const createdAt = Temporal.Instant.fromEpochMilliseconds(row.created_at.getTime());

// Rebuild the user's wall clock from instant + stored zone
const local = createdAt.toZonedDateTimeISO(row.time_zone); // 'Africa/Cairo'

One precision detail caught me out: Temporal.Instant keeps nanoseconds, Postgres timestamptz keeps microseconds, and JS Date keeps milliseconds. A value that travels Temporal → Date → Postgres → Temporal is not always the value you started with. If exact round-trips matter to a test, round before writing with instant.round({ smallestUnit: 'microsecond' }) so the truncation is a decision you made rather than one the driver made for you.

For anything scheduled in the future — "send this reminder at 9am local, every week" — store a PlainDateTime plus the IANA zone rather than an instant. A stored instant freezes today's UTC offset, so the reminder silently shifts by an hour the next time that zone's DST rules change.

Should I delete date-fns, Day.js, or Luxon now?

Not in one commit. Temporal covers what those libraries exist to cover, but the honest comparison is narrower than "Temporal wins".

date-fns — replaceable for arithmetic and comparison. Its tree-shaken functions all operate on Date, so mixed code paths need adapters for the length of the migration.
Day.js and Moment — replaceable outright. Both wrap Date and inherit its time-zone model; Day.js needs its timezone plugin to do what Temporal.ZonedDateTime does natively.
Luxon — the closest in spirit, since Luxon and the Temporal proposal share an author and an immutable, zone-aware model. Migration is mostly renaming.
Intl.DateTimeFormat — keep it. Temporal deliberately does not format for humans; it hands off to Intl.

The migration order that worked for me was to convert at the edges first: parse every inbound string into a Temporal type at the API or form boundary, format back to a string at the render or database boundary, and let the middle of the app stop touching Date entirely. A Zod schema is a good place to put that parse — z.string().transform((s) => Temporal.PlainDate.from(s)) gives you a validated Temporal value and a real error message when the string is malformed.

On shipping it: Temporal is a TC39 Stage 3 proposal and browsers have started enabling it, with Firefox shipping it unflagged first and Safari following, while Chromium and Node are still catching up. Because coverage is uneven, install temporal-polyfill (the compact implementation) or @js-temporal/polyfill (the reference one), import it from one shared module, and look at your own bundle analyzer before assuming the weight is free.

FAQ

Q: Is Temporal a drop-in replacement for Date?
A: No. Temporal adds a separate global namespace and does not change Date at all. Existing code keeps working, and you convert between the two with Temporal.Instant.fromEpochMilliseconds(date.getTime()) and new Date(instant.epochMilliseconds).

Q: Why can't I compare two Temporal values with === or <?
A: Temporal values are objects, so === compares references rather than calendar values. Use a.equals(b) for equality and the static comparator Temporal.PlainDate.compare(a, b), which returns -1, 0, or 1 and can be passed straight to Array.prototype.sort.

Q: Does Temporal handle daylight saving time correctly?
A: Temporal.ZonedDateTime does, using the IANA time zone database. When a local time is ambiguous or does not exist because of a DST shift, from() accepts a disambiguation option of 'compatible', 'earlier', 'later', or 'reject', so the behaviour is a decision instead of an accident.

Q: Can I use Temporal in Node.js and inside Server Components?
A: Yes, through a polyfill. Import temporal-polyfill in server code exactly as you would in the browser — Temporal works fine inside Server Components, Route Handlers, and Server Actions, and only the props boundary needs strings.

Q: Should I store an instant or a wall-clock time in the database?
A: Store an instant in timestamptz for things that already happened, such as created_at. Store a PlainDateTime plus an IANA zone for things scheduled in the future, so a change to that zone's DST rules moves the event along with the user's clock.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Parallel Routes and Intercepting Routes in the Next.js App Router: Field Notes on the Photo-Modal Pattern

Ahmed Mahmoud — Tue, 01 Sep 2026 06:00:24 +0000

Headline: Parallel routes let a Next.js App Router layout render more than one page at the same URL through named @slot folders, and intercepting routes — the (.), (..), (..)(..), and (...) folder conventions — let a link swap in a different component for that slot without changing the URL. Together they're how an Instagram-style photo modal stays shareable: the same URL renders a modal on a soft navigation and a full page on a hard refresh, and default.tsx is the seam between the two.

Key takeaways

A parallel route is a named slot — a folder prefixed with @, like @modal — that Next.js renders as a prop into the nearest layout.tsx, alongside the implicit children slot. Slot folders don't add a segment to the URL.
Intercepting route conventions — (.), (..), (..)(..), (...) — match a route relative to the file system, not the URL, so a link clicked inside the feed can render /photo/[id] as a modal without any folder named (.)photo ever appearing in the URL.
default.tsx is what Next.js renders for a slot when the current URL doesn't match anything inside that slot. Skip it and a hard navigation to a route that doesn't fill every slot 404s.
A hard refresh or a shared link to /photo/123 is not supposed to show the modal — it's supposed to render app/photo/[id]/page.tsx as an ordinary full page. That fallback is the entire point of the pattern, not a bug to route around.
Each slot is its own subtree with its own loading.tsx and error.tsx, so a modal can suspend and stream independently of the page rendering behind it.

What problem do parallel routes actually solve?

A parallel route renders more than one page in the same layout at the same time, each addressed by a named slot instead of a URL segment. I first reached for this on a photo grid: clicking a thumbnail should open a lightbox without leaving the grid, but the lightbox also needed its own shareable URL so a link to one photo would open directly on a full page. A client-side modal driven by a boolean gets the first half for free and can't do the second half at all — there's no URL for a piece of state that lives in useState.

The folder convention is a name prefixed with @. Next.js passes each slot's matched content into the nearest layout as a prop named after the folder, alongside the implicit children slot every layout already receives.

app/
  layout.tsx
  page.tsx
  @modal/
    default.tsx
    (.)photo/
      [id]/
        page.tsx
  photo/
    [id]/
      page.tsx

// app/layout.tsx
export default function RootLayout({
  children,
  modal,
}: {
  children: React.ReactNode;
  modal: React.ReactNode;
}) {
  return (
    <html lang="en">
      <body>
        {children}
        {modal}
      </body>
    </html>
  );
}

Nothing here is modal-specific yet — a layout with two slots is just a layout that renders two independent subtrees. I've since used the same mechanism for a dashboard with @team and @analytics panes that navigate independently: clicking a link inside @analytics re-renders only that slot, and @team keeps whatever it was showing.

How does the (.) convention decide what gets intercepted?

The dot-segment conventions match a target route relative to where the intercepting file sits in the file system, not relative to the current URL:

Convention	Matches
`(.)`	A route at the same folder level
`(..)`	A route one folder level above
`(..)(..)`	A route two folder levels above
`(...)`	A route from the app root

app/@modal/(.)photo/[id]/page.tsx intercepts a client-side navigation to /photo/[id] when that navigation is triggered from a route sitting at the same level as the @modal folder. Follow a <Link href="/photo/123"> from that page and Next.js renders the intercepting component into the @modal slot instead of swapping out the whole tree. Open /photo/123 as a fresh page load — refresh, paste the URL, click a link from an external site — and Next.js resolves app/photo/[id]/page.tsx the ordinary way. The interception only fires on a client-side transition; it was never meant to change what a full page load returns.

Why does default.tsx exist, and what happens if I skip it?

The first time I skipped it, everything worked in dev until I hit refresh on a route with the modal open and got a 404. A slot without a matching segment for the current URL needs something to render, and on a full page load there's no prior client state to fall back to — Next.js has to render the slot from scratch. default.tsx is that fallback.

// app/@modal/default.tsx
export default function Default() {
  return null;
}

Client-side navigation is more forgiving: if a slot has no default.tsx and the new URL doesn't match anything inside it, Next.js keeps rendering whatever that slot last showed rather than unmounting it. That's genuinely useful for the dashboard-tabs case — switching tabs in @analytics shouldn't reset @team — but it means a missing default.tsx can hide itself in every manual click-through test and only surface on a hard reload, which is exactly the path a shared link takes.

Why does refreshing the modal route show the full page instead of the modal?

Because it's supposed to. That was the part I fought before I read the pattern correctly: I wanted the modal to somehow survive a refresh, and the actual design is that it shouldn't. app/photo/[id]/page.tsx is a complete, independent page — same content, no modal chrome, reachable without a single line of client JavaScript executing first. That's what makes the URL genuinely shareable and indexable: a search engine or a link preview bot that doesn't run your client-side router gets the real page, not an empty shell waiting for JavaScript to open a dialog.

How do I close the modal and get back to where I came from?

The modal component is a Client Component that calls router.back() from next/navigation, which reverses the soft navigation that opened it and lets the intercepted slot fall back to its default.tsx.

'use client';
import { useRouter } from 'next/navigation';

export function Modal({ children }: { children: React.ReactNode }) {
  const router = useRouter();
  return (
    <div className="modal-overlay" onClick={() => router.back()}>
      <div className="modal-content" onClick={(e) => e.stopPropagation()}>
        {children}
      </div>
    </div>
  );
}

The gap I found in production: a visitor who opens /photo/123 directly in a new tab from a shared link has no history entry to go back to, so router.back() either does nothing or leaves the app. I ended up rendering an explicit close link with a real href back to the gallery alongside the overlay handler, so closing the modal never depends on history existing.

Client-state modal vs. intercepting-route modal — what's the actual trade-off?

	Boolean-state modal	Intercepting route modal
Own URL	No	Yes
Shareable / shows on refresh	No	Yes — renders as a full page
Needs client JS to render content at all	Yes	No, on direct navigation
Setup cost	One `useState`	A slot, an intercepting folder, a fallback route, `default.tsx`

For a dialog that's genuinely disposable UI — a confirmation prompt, a settings panel — the boolean is still the right tool. Reach for the routed version specifically when the content behind the modal deserves its own URL.

FAQ

Q: Do parallel route slot folders like @modal show up in the URL?
A: No. The @ prefix marks a folder as a slot rather than a route segment, so it's invisible to the URL and only affects which prop of the layout receives its content.

Q: What happens if someone navigates straight to /photo/123 instead of clicking a thumbnail in the feed?
A: Next.js renders app/photo/[id]/page.tsx as a normal full page. The interception only happens on a client-side transition from a matching route, by design.

Q: Can a parallel route slot have its own loading.tsx?
A: Yes. Each slot is an independent subtree and can define its own loading.tsx, error.tsx, and not-found.tsx, so a modal can suspend on its own data fetch without blocking the page behind it.

Q: Why doesn't router.back() close the modal for someone who opened the link directly?
A: There's no history entry to go back to when a route is the first thing loaded in a tab. Pair the close handler with an explicit link to a real fallback route instead of relying on history alone.

Q: Can I combine (..)(..) with a parallel route slot defined several levels up the tree?
A: Yes — the dot count is independent of where the @slot folder lives; it only describes how many folder levels above the intercepting file's own location the target route sits.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Observability in the Next.js App Router: Field Notes on instrumentation.ts, onRequestError, and the Trace That Leaked Across Requests

Ahmed Mahmoud — Mon, 31 Aug 2026 06:00:08 +0000

Headline: Next.js's instrumentation.ts file and its onRequestError hook give the App Router real distributed tracing and centralized error reporting, but Vercel Fluid Compute reusing one warm instance across concurrent requests means any request state stored outside OpenTelemetry's own context propagation — a module-level variable, a naive "current request" global — leaks between unrelated users' traces.

Key takeaways

register(), exported from instrumentation.ts, runs once when a server instance boots, not once per request — that's exactly where a tracing SDK belongs, and exactly why request-scoped state can't live there.
@vercel/otel's registerOTel() wraps the Node OpenTelemetry SDK with sane defaults for exporters and span processors, cutting most of the boilerplate a manual setup needs.
onRequestError, stable since Next.js 15, catches errors from Server Components, Route Handlers, Server Actions, and Middleware before Next.js renders its own error response — a component-level try/catch only ever sees its own subtree.
Fluid Compute reuses one function instance across concurrent requests, so a module-scope "current trace" variable becomes shared mutable state. Only AsyncLocalStorage-based context — what the OpenTelemetry SDK already uses — keeps data scoped to a single request.
A trace, a structured log, and a metric answer different questions about the same request. Exporting only one of them leaves gaps the other two would have closed.

What does instrumentation.ts actually run, and when?

instrumentation.ts is a file at the project root — or inside src/ — that Next.js loads before any other application module. Its register() export runs exactly once, when a new server instance starts up. It does not run per request, per route, or per render. Since Next.js 15 this is stable behavior with no experimental.instrumentationHook flag required, which trips people who copy setup snippets from Next.js 13 or 14 tutorials.

That single-run-per-instance timing is the whole reason this file exists: it's the correct place to initialize a tracing SDK, open a long-lived connection, or read one-time configuration, and the wrong place to store anything specific to the request currently in flight.

// instrumentation.ts
import { registerOTel } from '@vercel/otel';

export function register() {
  // Runs once when this server instance boots. Never per-request.
  registerOTel({ serviceName: 'my-app' });
}

How do I wire up OpenTelemetry without hand-rolling the Node SDK?

@vercel/otel is a package that wraps the standard @opentelemetry/sdk-node setup — exporter selection, batch span processor configuration, and resource attributes like service name — behind a single registerOTel() call. Without it, wiring up the Node OpenTelemetry SDK by hand means picking a span processor, a context manager, and an exporter, and getting the instrumentation registration order right before any other module loads.

Set OTEL_EXPORTER_OTLP_ENDPOINT to point spans at a collector — Honeycomb, Axiom, Datadog, or a self-hosted OpenTelemetry Collector all accept OTLP. On Vercel specifically, once @vercel/otel is registered, the platform's own Observability tab picks up the same trace data automatically, which covers a lot of day-to-day debugging without standing up a separate backend.

The full Node OpenTelemetry SDK needs the Node.js runtime. That's rarely a real constraint now — Fluid Compute makes Node.js the practical default runtime on Vercel anyway, and the older edge runtime was never a good fit for a stateful SDK like this.

Why did spans leak across requests that had nothing to do with each other?

I had a small helper that stashed the current user's ID in a module-level variable, to avoid threading it through five layers of function calls just to attach it to a log line. It worked in local development, where next dev rarely has more than one request in flight against the same process. In production, under real concurrent traffic, it produced log lines and trace attributes with the wrong user's ID attached — request A's logs carrying request B's identity.

The cause is Fluid Compute's core behavior: it reuses one warm function instance to serve many concurrent requests instead of spinning up a fresh instance per request. That's good for cold-start latency, and it's exactly why a plain module-level variable is dangerous — it's shared mutable state across every request currently running on that instance, and two requests interleaving on the same instance will stomp on each other's value.

// Do not do this on Fluid Compute — or any environment that
// reuses one process across concurrent requests.
let currentUserId: string | undefined;

export function setUser(id: string) {
  currentUserId = id; // shared across every request on this instance
}

The fix is AsyncLocalStorage, a Node.js primitive that scopes a value to the current async call chain rather than to the module. It's the same mechanism OpenTelemetry's own context manager is built on, and the same reason Next.js's own headers() and cookies() functions can be request-scoped without you passing a request object everywhere.

import { AsyncLocalStorage } from 'node:async_hooks';

type RequestContext = { userId: string; requestId: string };
const requestContext = new AsyncLocalStorage<RequestContext>();

export function withRequestContext<T>(ctx: RequestContext, fn: () => T): T {
  return requestContext.run(ctx, fn);
}

export function currentUserId() {
  // Scoped to this async call chain, not shared across concurrent requests.
  return requestContext.getStore()?.userId;
}

What does onRequestError catch that a component-level try/catch can't?

onRequestError is an optional export from instrumentation.ts, stable since Next.js 15. Next.js calls it whenever it catches an unhandled error while rendering a Server Component, executing a Route Handler, running a Server Action, or inside Middleware — before it renders its own error boundary or returns a 500 response to the client.

It receives the error, a request object with path, method, and headers, and a context object with routerKind, routePath, and routeType (render, route, action, or middleware). That last piece is what a scattered set of try/catch blocks never gives you for free: an error boundary inside one component only knows about its own subtree, not which route it belongs to or whether it happened during a render or a Server Action.

// instrumentation.ts
export async function onRequestError(
  error: unknown,
  request: { path: string; method: string; headers: Record<string, string> },
  context: {
    routerKind: 'Pages Router' | 'App Router';
    routePath: string;
    routeType: 'render' | 'route' | 'action' | 'middleware';
  },
) {
  await reportToErrorTracker(error, {
    route: context.routePath,
    kind: context.routeType,
    path: request.path,
  });
}

I moved every ad-hoc console.error I had scattered across route handlers into this one hook. Nothing else in the app decides how errors get reported anymore.

How do I connect a trace to the log line that explains what happened?

A span ID by itself doesn't help whoever is reading a log at 2 a.m. — the log has to carry the same trace ID as the request that produced it. Vercel's Runtime Logs capture console.log output automatically, but they don't correlate a log line to a trace unless you attach the IDs yourself.

import { trace, context } from '@opentelemetry/api';
import pino from 'pino';

const baseLogger = pino();

export function requestLogger() {
  const spanContext = trace.getSpan(context.active())?.spanContext();
  return baseLogger.child({
    trace_id: spanContext?.traceId,
    span_id: spanContext?.spanId,
  });
}

trace.getSpan(context.active()) reads the span attached to the currently active OpenTelemetry context — the same context that AsyncLocalStorage propagates — so this works correctly under concurrent requests for the same reason the fix above does.

Traces vs structured logs vs metrics — what should I actually export?

Each signal answers a different question, and none of them substitutes for the others.

Signal	Answers	Use it for
Trace	Where did the time go inside this one request?	A waterfall across a database call, an external API, and a render — spotting the one slow span
Structured log	What happened, with what data, in this one request?	Specific error payloads, business events, anything you'd grep for by request ID
Metric	How often, across every request?	Dashboards and alerting thresholds — error rate, p95 latency, request volume

I export all three now, and the trace ID is the thing that ties a spike on a metric dashboard back to the specific log line and the specific span that explains it.

FAQ

Q: Do I need the experimental.instrumentationHook flag for instrumentation.ts in Next.js 16?
A: No. The instrumentation.ts file and its register() export have been stable since Next.js 15; the experimental flag from Next.js 13 and 14 no longer exists.

Q: Does instrumentation.ts run on the Edge runtime?
A: Only a subset of it. The full Node OpenTelemetry SDK needs the Node.js runtime, which is the practical default on Vercel now that Fluid Compute makes Node.js the standard choice.

Q: What's the difference between onRequestError and a React error.tsx boundary?
A: An error.tsx boundary catches errors within its own component subtree and renders fallback UI for the user. onRequestError is a framework-level hook that fires for errors across Server Components, Route Handlers, Server Actions, and Middleware regardless of any particular boundary — it's for reporting, not for rendering.

Q: Can I use a module-level variable instead of AsyncLocalStorage to pass request context around?
A: Only if the instance running your code never serves more than one request at a time. Fluid Compute reuses instances across concurrent requests, so a module-level variable becomes shared mutable state and will leak data between unrelated requests.

Q: Do I still need a separate observability vendor if I'm on Vercel?
A: Vercel's Observability tab shows traces automatically once @vercel/otel is registered, which covers a lot of routine debugging. Longer retention, cross-service correlation, or custom alerting usually still means exporting via OTLP to a dedicated backend.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Database Connections in Serverless: Field Notes on Pool Math, PgBouncer Transaction Mode, and the Singleton HMR Kept Recreating

Ahmed Mahmoud — Fri, 28 Aug 2026 06:00:12 +0000

Headline: A database connection pool in a serverless deployment is per-instance, not per-application. The real ceiling is concurrent instances multiplied by pool size, which is why the fix is almost never a bigger database — it is a connection pooler in transaction mode plus a driver configured to survive one.

I hit FATAL: sorry, too many clients already on a Next.js app whose traffic would not have troubled a single Postgres box in 2005. The database was idle. The application code was fine. What was not fine was my mental model: I had configured one pool with a sensible max and assumed that number described the deployment. It described one function instance.

Key takeaways

A connection pool is a cache of open TCP connections living inside one Node.js process. Total connections opened by a serverless app are roughly concurrent instances multiplied by each pool's max, not the max you configured.
PostgreSQL ships with max_connections = 100 by default and reserves three for superusers, and every connection is a separate backend process with its own memory.
PgBouncer in transaction mode multiplexes many clients onto few server connections, but it discards session state: prepared statements, LISTEN/NOTIFY, session-level SET, and advisory locks held between transactions all break.
Drizzle with postgres.js needs prepare: false behind a transaction-mode pooler; Prisma needs ?pgbouncer=true plus a separate directUrl for migrations.
Vercel Fluid Compute reuses one instance across concurrent requests, so max: 1 now serializes handlers instead of protecting the database.

Why does my serverless app run out of Postgres connections?

Because each function instance runs its own Node.js process holding its own pool, so the connections you actually open are concurrent instances multiplied by each pool's max. Nothing about a pool is shared between processes. Ten instances configured with max: 10 is one hundred connections, not ten.

That product collides with a hard server-side limit. PostgreSQL defaults to max_connections = 100, holds three back for superuser access, and spawns a separate backend process per connection. The failure is load-shaped: a traffic spike opens more instances, each opens its own pool, and Postgres answers with FATAL: sorry, too many clients already while database CPU stays flat.

What did Fluid Compute change about connection pooling?

Fluid Compute reuses a single function instance across concurrent requests, so one module-scope pool is shared by several in-flight requests inside the same process. The classic advice — set max: 1, because an instance only ever handles one request — is actively harmful under that model: a pool of one serializes concurrent handlers behind a single connection.

// lib/db.ts
import { Pool } from 'pg';

// Module scope: created once per instance, reused by every request
// that instance serves, including concurrent ones under Fluid Compute.
export const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 5,                      // per instance, not per deployment
  idleTimeoutMillis: 10_000,   // hand connections back to the pooler
  connectionTimeoutMillis: 5_000,
});

Which PgBouncer pooling mode should I use?

Transaction mode, for ordinary application traffic. PgBouncer is a lightweight proxy that multiplexes many client connections onto a small set of real server connections, and pool_mode decides how long a client keeps one of them.

Mode	Server connection held for	What it breaks	Use it for
`session`	The whole client session	Nothing	Migrations, `LISTEN/NOTIFY`, long-lived workers
`transaction`	One transaction	Prepared statements, session `SET`, advisory locks outside a transaction, `WITH HOLD` cursors	Serverless application traffic
`statement`	One statement	Everything above, plus multi-statement transactions	Rare; sharded setups without transactions

Supabase now fronts Postgres with Supavisor rather than PgBouncer, and Neon and Amazon RDS Proxy ship their own implementations, but the mode semantics are identical everywhere.

Why did my prepared statements break behind the pooler?

Because a protocol-level prepared statement is session state on one server connection, and transaction mode may hand you a different connection for the next transaction. The symptom is a pair of errors alternating under load: prepared statement "s1" already exists and prepared statement "s1" does not exist.

PgBouncer 1.21 and later can track prepared statements in transaction mode when max_prepared_statements is above zero — confirm your provider enables it before depending on it. Otherwise disable them in the driver. node-postgres only uses named prepared statements when you explicitly name a query; postgres.js prepares by default and needs prepare: false; Prisma needs ?pgbouncer=true.

// postgres.js + Drizzle behind a transaction-mode pooler
import postgres from 'postgres';
import { drizzle } from 'drizzle-orm/postgres-js';

const client = postgres(process.env.DATABASE_URL!, {
  prepare: false, // named statements do not survive transaction mode
  max: 5,
});

export const db = drizzle(client);

How do I stop dev HMR from opening a new pool on every save?

Cache the pool on globalThis in development. Next.js hot module replacement re-evaluates changed modules, so new Pool() at module scope runs again on every save while the previous pool keeps its sockets open. Twenty saves is twenty live pools, and eventually the local database refuses connections.

// lib/db.ts
import { Pool } from 'pg';

const globalForDb = globalThis as unknown as { pool?: Pool };

export const pool =
  globalForDb.pool ??
  new Pool({ connectionString: process.env.DATABASE_URL, max: 5 });

if (process.env.NODE_ENV !== 'production') globalForDb.pool = pool;

Which connection string should migrations use?

The direct, session-mode connection string — never the transaction-mode pooler. Migrations take advisory locks and run DDL that must stay held across statements, and a transaction-mode pooler can route the next statement to a different backend where that lock does not exist.

datasource db {
  provider  = "postgresql"
  url       = env("DATABASE_URL")         // pooled, transaction mode
  directUrl = env("DIRECT_DATABASE_URL")  // direct, session mode
}

Drizzle Kit takes the same split by pointing its config at the direct URL while the runtime client uses the pooled one. The rule generalises: LISTEN/NOTIFY, advisory locks used as a distributed mutex, and anything that sets a session variable belongs on the direct connection.

When is an HTTP database driver the better choice?

When your handler runs one self-contained statement and you would rather not manage a TCP pool at all. Neon's @neondatabase/serverless package exposes neon(), which sends a single SQL statement over HTTP — no pool to size, no connection to leak. The cost is that HTTP is stateless: no interactive transactions, no session settings. The same package ships a WebSocket-backed Pool for cases that need real transactions.

My rule is boring: HTTP driver for read handlers running one query, TCP pool through a transaction-mode pooler for multi-statement transactions, direct connection for migrations and workers.

FAQ

Q: How large should max be in a serverless connection pool?
A: Small — start around three to five per instance, then watch the server-side connection count at peak concurrency. The number that matters is instances × max.

Q: Do I still need PgBouncer if I use Prisma?
A: Yes. Prisma's client-side pool is per-instance like any other and does not coordinate across processes. Add ?pgbouncer=true so Prisma stops relying on named prepared statements.

Q: Does transaction-mode pooling break database transactions?
A: No. It pins one server connection for the full duration of a transaction. It breaks state that lives between transactions.

Q: Why does my local database run out of connections when production does not?
A: Almost always HMR creating a new pool on every file save. Cache the pool on globalThis in development.

Q: Can I use LISTEN/NOTIFY from a serverless function?
A: Not through a transaction-mode pooler — the listening session is not preserved. Use a direct session connection on a long-lived process, or a real queue.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Environment Variables in Next.js: Field Notes on NEXT_PUBLIC_ Inlining, the Secret That Almost Shipped, and Failing the Build with Zod

Ahmed Mahmoud — Tue, 25 Aug 2026 06:00:11 +0000

Headline: A NEXT_PUBLIC_ environment variable is not read at runtime — Next.js replaces every process.env.NEXT_PUBLIC_* expression with a string literal at build time. That single fact explains the value that refuses to update without a rebuild, the dynamic lookup that returns undefined in the browser, and why one build artifact cannot serve two environments.

An environment variable in Next.js is one name with two different lives. On the server it lives in process.env, a real Node.js object read at request time. In the browser it does not exist at all: any variable prefixed with NEXT_PUBLIC_ is copied into the JavaScript bundle as a literal during next build, and everything else is stripped. I have watched the same three failures come out of that split on several projects this year — a stale API URL that survived a redeploy, a secret that nearly rode a prop into the HTML, and a missing variable that surfaced as a runtime crash instead of a failed build. These notes are the checklist I now run before the first deploy.

Key takeaways

NEXT_PUBLIC_ variables are inlined into the client bundle at build time. Changing the value in your host's dashboard does nothing until the next build.
Inlining is static text replacement. process.env[name] with a dynamic key and const { NEXT_PUBLIC_X } = process.env both return undefined in the browser.
Server-only variables are stripped from client bundles, so process.env.SECRET in a Client Component is undefined, not a leak. Secrets leak through props serialized into the RSC payload and through renaming a variable to NEXT_PUBLIC_.
The server-only package turns "a Client Component imported my secrets module" into a build error instead of a silent risk.
Validate environment variables with a Zod schema at module load and import that module in next.config.ts, so a missing variable fails next build instead of the first production request.

Why did changing my NEXT_PUBLIC_ variable do nothing?

Because the value the browser sees was compiled into the JavaScript at the last build. During next build, the bundler performs a find-and-replace: every static process.env.NEXT_PUBLIC_API_URL expression becomes the string the variable held on the build machine at that moment. The deployed bundle contains the literal URL, not a lookup. Editing the variable in a dashboard, a Docker -e flag, or .env.production changes what the next build will see — the running one is frozen.

The replacement is textual, which produces a failure mode that looks like a bug in Next.js and is not:

// Replaced at build time with a string literal — works in the browser
const url = process.env.NEXT_PUBLIC_API_URL;

// undefined in the browser: inlining is static text replacement,
// and no process.env object exists at runtime to index into
const name = 'NEXT_PUBLIC_API_URL';
const url2 = process.env[name];

// Also undefined after bundling: destructuring is not a static reference
const { NEXT_PUBLIC_API_URL } = process.env;

Server code has none of these restrictions. Inside a Server Component, a route handler, or a Server Action, process.env is the live Node.js object, read at request time, dynamic keys and all.

How does a server secret actually reach the browser?

Not through process.env — Next.js strips unprefixed variables from client bundles, so process.env.STRIPE_SECRET_KEY in a Client Component evaluates to undefined. The leaks I have actually seen take two other roads. The first is serialization: a Server Component reads a secret and passes it as a prop to a Client Component, and the value is embedded in the RSC payload inside the HTML response, visible in View Source. The second is the "fix" reflex: a developer sees undefined in the browser, renames API_SECRET to NEXT_PUBLIC_API_SECRET, the error disappears, and the secret is now a string literal in a public JavaScript file.

The cheap defence is the server-only package — an empty module whose import fails the build if it ends up in the client graph:

// lib/secrets.ts
import 'server-only';

export const stripeSecret = process.env.STRIPE_SECRET_KEY!;
export const webhookSecret = process.env.STRIPE_WEBHOOK_SECRET!;

Any module that touches secrets gets that import at the top. If a Client Component ever imports it, directly or through a chain, the build fails with a readable error instead of shipping. React's experimental taint API (experimental_taintUniqueValue) covers the prop-serialization road as well, but server-only is stable today and catches the common case.

How do I fail the build when a variable is missing?

Parse the environment through a Zod schema in a module that runs during the build, and import every variable through it. A missing or malformed variable then stops next build with a named error instead of surfacing as undefined in whatever code happened to read it first.

// src/env.ts
import { z } from 'zod';

const schema = z.object({
  DATABASE_URL: z.string().url(),
  STRIPE_SECRET_KEY: z.string().min(1),
  NEXT_PUBLIC_API_URL: z.string().url(),
});

export const env = schema.parse({
  DATABASE_URL: process.env.DATABASE_URL,
  STRIPE_SECRET_KEY: process.env.STRIPE_SECRET_KEY,
  // Client vars must be referenced literally so the bundler can inline them
  NEXT_PUBLIC_API_URL: process.env.NEXT_PUBLIC_API_URL,
});

Two details earn their place. The keys are listed explicitly instead of passing process.env wholesale, because client-referenced variables must appear as literal member expressions for the bundler to inline them. And importing this module from next.config.ts (import './src/env';) forces the schema to execute at build time even if no route touches it. The @t3-oss/env-nextjs package wraps the same idea with a server/client split.

Which .env file wins, and which ones do I commit?

Next.js loads .env files itself — no dotenv package needed — with a fixed precedence: an already-set shell variable beats every file, and more specific files beat general ones.

File	Loaded when	Commit it?
`.env`	Every environment	Yes — shared defaults
`.env.local`	Every environment except `NODE_ENV=test`	No — machine secrets
`.env.development` / `.env.production`	When `NODE_ENV` matches	Yes — per-env defaults
`.env.development.local` / `.env.production.local`	When `NODE_ENV` matches; beats other files	No

NODE_ENV itself is not free-form: Next.js recognises exactly development, production, and test. next dev forces the first; next build and next start force the second. A staging environment is therefore NODE_ENV=production plus your own variable such as APP_ENV=staging. One more sharp edge: .env.local is deliberately ignored when NODE_ENV=test, so test runs stay reproducible across machines.

Can one build artifact serve both staging and production?

For server-side variables, yes — they are read at request time, so the same Docker image can boot with different DATABASE_URL values. For NEXT_PUBLIC_ variables, no — their values are already inside the JavaScript. On Vercel this stays invisible because every environment gets its own build with its own variables. A output: 'standalone' Docker deployment that promotes the same image across environments exposes it immediately.

Three honest ways out, in the order I try them: keep configuration server-side and let client code call same-origin paths (a rewrite that proxies /api removes most reasons a browser needs an absolute URL); read the value in a Server Component at request time and pass it down as a prop; or accept one build per environment. What does not work is editing the variable and redeploying the same artifact — and it fails silently, because the old value keeps being served with no error anywhere.

FAQ

Q: Are my server-only variables exposed to the browser?

A: No. Next.js strips variables without the NEXT_PUBLIC_ prefix from client bundles, so reading one in a Client Component returns undefined. Exposure happens when a secret is passed as a prop into a Client Component or renamed to a NEXT_PUBLIC_ variable.

Q: Do I need the dotenv package in a Next.js project?

A: No. Next.js loads .env, .env.local, and the NODE_ENV-specific variants itself, in a documented order. Adding dotenv on top usually just creates a second, conflicting load order.

Q: Can I set NODE_ENV to staging?

A: No. Next.js recognises only development, production, and test, and the CLI commands set it for you. Model staging as NODE_ENV=production plus your own variable, for example APP_ENV=staging.

Q: Why is process.env[name] undefined in the browser?

A: Because inlining is a static build-time replacement of literal process.env.NEXT_PUBLIC_* expressions. There is no process.env object in the browser to index with a dynamic key, so only the literal form works.

Q: How do I get type-safe environment variables in TypeScript?

A: Export a Zod-validated env object from one module and import it everywhere instead of touching process.env directly. Augmenting the ProcessEnv type gives autocomplete but no runtime guarantee; the schema gives both.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Webhooks in the Next.js App Router: Field Notes on Raw Bodies, Signature Verification, and Returning 200 Before the Work

Ahmed Mahmoud — Mon, 24 Aug 2026 06:00:08 +0000

Headline: A webhook signature is an HMAC computed over the raw request bytes, so a handler that parses the body before verifying has already destroyed the evidence. In the Next.js App Router, await req.text() inside a route handler returns those raw bytes — verify first, acknowledge fast, and do the real work idempotently after the response.

A webhook is an HTTP POST a provider sends to my server when something happens on their side — a payment settles, a repository receives a push, a subscription cancels. I have wired webhooks into several Next.js App Router apps this year, and every one of them broke in the same three places first: verifying a signature against a body I had already parsed, doing too much work before responding, and processing the same event twice. These notes are the checklist I now start from.

Key takeaways

Webhook signatures — Stripe's Stripe-Signature, GitHub's X-Hub-Signature-256 — are HMACs over the raw request bytes. Verify against await req.text(), never a re-serialized JSON.parse result.
App Router route handlers do not pre-parse request bodies, so the Pages Router bodyParser: false escape hatch is unnecessary — req.text() is already the raw payload.
Return a 2xx quickly. Providers treat a slow response as a failed delivery and retry it, so a slow handler races its own retries.
Delivery is at-least-once and unordered: dedupe by event ID with a database unique constraint, and fetch the current object from the provider's API instead of trusting payload state.
Compare signatures with crypto.timingSafeEqual and enforce a timestamp tolerance so a captured request cannot be replayed later.

Why does my signature check fail on a genuine request?

Because the signature was computed over the exact bytes the provider sent, and the handler is verifying different bytes. A webhook signature is a keyed hash — an HMAC — of the raw request body, produced with a shared signing secret. If I call JSON.parse on the body and re-serialize it to verify, key order, whitespace, and unicode escaping can all change, and the HMAC no longer matches even though the request is genuine. The failure is silent and total: every event gets a 400, the provider retries, and the retry queue fills while the code looks correct.

The App Router makes the correct version easy. A route handler receives the standard web Request object, and Next.js does not pre-parse it — await req.text() returns the payload byte-for-byte. The Pages Router needed export const config = { api: { bodyParser: false } } for the same access; that configuration does nothing in the App Router.

// app/api/webhooks/stripe/route.ts
import Stripe from 'stripe';

const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!);

export async function POST(req: Request) {
  const rawBody = await req.text();               // raw bytes — parse AFTER verification
  const signature = req.headers.get('stripe-signature');

  let event: Stripe.Event;
  try {
    event = stripe.webhooks.constructEvent(
      rawBody,
      signature!,
      process.env.STRIPE_WEBHOOK_SECRET!
    );
  } catch {
    return new Response('invalid signature', { status: 400 });
  }

  return new Response('ok', { status: 200 });
}

When the provider ships a verification helper, I use it. stripe.webhooks.constructEvent checks both the HMAC and the signed timestamp in one call and throws on either failure.

How do I verify a webhook signature without an SDK helper?

Compute the HMAC yourself and compare in constant time. Two rules are non-negotiable. First, compare with crypto.timingSafeEqual, because an ordinary string comparison returns early at the first differing character and leaks timing information an attacker can use to probe signatures byte by byte. Second, enforce a timestamp tolerance — most providers include a signed timestamp, and rejecting anything older than about five minutes stops a captured request from being replayed later.

import { createHmac, timingSafeEqual } from 'node:crypto';

export function verifySignature(rawBody: string, header: string, secret: string) {
  const expected = createHmac('sha256', secret).update(rawBody).digest('hex');
  const received = header.replace(/^sha256=/, '');   // GitHub prefixes the hex digest
  const a = Buffer.from(expected, 'hex');
  const b = Buffer.from(received, 'hex');
  return a.length === b.length && timingSafeEqual(a, b);
}

One structural note: a webhook endpoint must be a route handler, not a Server Action. A Server Action is an RPC mechanism for my own application's frontend, addressed by framework-generated identifiers. A webhook needs a stable public POST URL with byte-level body access, and app/api/webhooks/<provider>/route.ts is exactly that.

Why should the handler return 200 before doing the real work?

Because the provider treats a slow response as a failed delivery. Delivery timeouts are measured in seconds, and a handler that exceeds one gets marked failed and retried — so the slow handler ends up running concurrently with its own retry. Fulfillment logic that takes ten seconds guarantees every real event arrives at least twice.

The shape I use now is verify, record, acknowledge, then work. On Vercel, waitUntil from @vercel/functions keeps the function instance alive after the response has been sent.

import { waitUntil } from '@vercel/functions';

export async function POST(req: Request) {
  const event = await verifyAndParse(req);
  if (!event) return new Response('invalid signature', { status: 400 });

  const fresh = await recordEventId(event.id);   // unique constraint = dedupe
  if (fresh) waitUntil(processEvent(event));     // runs after the response is sent

  return new Response('ok', { status: 200 });    // acknowledge in milliseconds
}

Where the work runs	Ack speed	On a crash	Reach for it when
Inline, before the response	Slow — bounded by the work	Provider retry re-delivers the event	Trivial work: set a flag, update one row
`waitUntil`, after the response	Fast	Work is lost; event already acknowledged	Losable side effects: cache warming, notifications
Queue or job row, separate worker	Fast	Job survives and is retried	Money, entitlements, anything you cannot lose

The queue row is the only option I trust for money. waitUntil work that dies in a crash was already acknowledged with a 200, and the provider will never resend it.

How do I handle retries and out-of-order events?

By assuming at-least-once delivery and no ordering, because both are documented provider behavior, not edge cases.

Deduplication: every provider event carries a stable ID — evt_… on a Stripe event, the X-GitHub-Delivery header on a GitHub delivery. I insert that ID into a table with a unique constraint before processing; a constraint violation means the event was already handled, so the handler returns 200 and stops. A SELECT-then-INSERT check is a race under concurrent retries — the constraint is the lock.

Ordering: I do not build state by applying payloads in arrival order. The event is a notification that something changed, not the change itself. For anything stateful I fetch the current object from the provider's API before writing, so a stale payload cannot overwrite newer state.

How do I test webhooks locally?

A provider cannot reach localhost, so local testing needs a bridge. Three options cover my use: a provider CLI that forwards events — stripe listen --forward-to localhost:3000/api/webhooks/stripe prints a temporary signing secret for the session; a tunnel such as cloudflared or ngrok plus the provider dashboard's manual redelivery button; and signed fixtures in tests.

The fixtures are the ones that pay rent. I capture one real payload, compute its HMAC with a test secret, and assert three things: the verifier accepts the valid pair, rejects a mutated body, and rejects an expired timestamp. That test catches the raw-body regression — someone adding a JSON middleware or moving the parse above the verify — before it ships and silently 400s every event.

FAQ

Q: Can a Server Action be a webhook endpoint?

A: No. A Server Action is invoked through framework-generated identifiers and is designed for your own application's components. A webhook provider needs a stable public POST URL with raw-body access, which is a route handler.

Q: Do I still need bodyParser: false in the App Router?

A: No. That option configures Pages Router API routes. An App Router route handler leaves the body untouched until you call req.text(), req.json(), or req.formData().

Q: What should I return for event types I do not handle?

A: Return 200. A non-2xx response tells the provider the delivery failed, so it retries events you will never process, and some providers disable an endpoint that keeps failing. Reserve 400 for signature failures.

Q: What happens to events sent while my deployment was down?

A: Providers retry failed deliveries with backoff — Stripe retries for days — so short downtime is usually absorbed. For critical state I also run a periodic reconciliation job against the provider's API.

Q: How do I rotate a webhook secret without downtime?

A: Verify incoming signatures against both the old and the new secret during the rotation window. Providers like Stripe allow an old signing secret to stay active for an overlap period for exactly this reason.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

next/image in Next.js 16: Field Notes on LCP, the sizes Prop That Doubles Bandwidth, and What Optimization Costs

Ahmed Mahmoud — Sat, 22 Aug 2026 13:36:14 +0000

Headline: Using next/image is not the same as having fast images. The component lazy-loads every image it renders unless you pass priority, and a wrong sizes prop makes a 400-pixel-wide card download the 3840-pixel candidate on a high-DPR screen.

next/image is the Next.js built-in component that generates a responsive srcset, converts source files to modern formats on demand, and reserves layout space before a byte arrives. I have used it on every Next.js project I have built, and I still spent an afternoon this month working out why a page full of <Image> tags had a worse Largest Contentful Paint than the plain <img> markup it replaced. The layout-shift half is automatic. The bandwidth and latency half is not.

Key takeaways

next/image lazy-loads every image by default, including the one that is your Largest Contentful Paint element. Only the priority prop opts an image out of lazy loading and adds a preload hint.
The sizes prop tells the browser how wide the image will render before CSS is applied. An <Image fill> with no sizes is treated as 100vw, which selects the largest candidate in the srcset.
Next.js 16 removed the images.domains option, so images.remotePatterns is the only way to allow a remote host.
Next.js 16 restricts the quality prop to values listed in images.qualities, which defaults to [75].
An image transformation is a unique combination of source image, width, quality and output format, so a wide deviceSizes array multiplies both cost and cache misses.

Why is my LCP still slow when I already use next/image?

Because next/image lazy-loads every image by default, including the one that is your Largest Contentful Paint element. Largest Contentful Paint measures when the biggest visible element finishes rendering. A lazy image is not requested until the browser has run layout and decided the image is near the viewport, so the preload scanner — which normally starts image downloads while the HTML is still being parsed — never sees it.

The fix is one prop, applied to exactly one image per route:

import Image from 'next/image';
import hero from '@/public/hero.jpg';

export default function Hero() {
  return (
    <Image
      src={hero}        // static import: width, height and blurDataURL come for free
      alt=""
      priority          // no lazy loading, fetchpriority="high", preload hint in head
      sizes="100vw"
      className="w-full h-auto"
    />
  );
}

The priority prop does three things: it removes loading="lazy", sets fetchpriority="high", and emits a preload link in the document head. Marking six images priority is the same as marking none, because six high-priority requests then compete for the same connection.

Two related traps cost me time. First, placeholder="blur" inlines a base64 data URI into the HTML, so a heavy blurDataURL grows the document on the critical path. Second, an image rendered by a client component that only mounts after hydration cannot be preloaded at all, whatever you pass to priority — the markup does not exist when the preload scanner runs.

What does the sizes prop actually do, and when does it double my bandwidth?

The sizes attribute tells the browser how wide the image will be rendered, so it can pick a srcset candidate before stylesheets are applied. Choose the candidate list badly and the browser downloads the biggest file you offered it.

Without sizes, Next.js emits a fixed 1x/2x srcset built from the width you passed. With sizes, it emits a full candidate list drawn from images.deviceSizes (default 640, 750, 828, 1080, 1200, 1920, 2048, 3840) and images.imageSizes (default 16, 32, 48, 64, 96, 128, 256, 384). The fill prop with no sizes is treated as 100vw, so on a 1920-pixel display at device pixel ratio 2 the browser asks for the 3840-pixel file — for a card that renders at 400 pixels.

// A product card image that is never wider than 400 CSS pixels.
<Image
  src={product.image}
  alt={product.name}
  fill
  sizes="(max-width: 640px) 100vw, (max-width: 1024px) 50vw, 400px"
/>

The rule I now apply in review: an <Image fill> with no sizes prop is a defect, not a style preference. Verify it in the Chrome DevTools Network panel by comparing the transferred size of each image against the box it renders into.

Should I use fill or explicit width and height?

Use explicit width and height whenever you know the intrinsic dimensions, because Next.js turns them into a CSS aspect-ratio that reserves space and keeps Cumulative Layout Shift at zero. A static import supplies both dimensions and a generated blurDataURL at build time, so it is the cheapest correct option for any asset in your repository.

Reach for fill only when the rendered box is decided by CSS and the source aspect ratio varies — user-uploaded avatars, CMS hero images, a masonry grid. fill absolutely positions the image, so the parent needs position: relative and a non-zero height, and cropping is your job through object-fit. Every fill image also needs a sizes prop.

What changed for images in Next.js 16?

Next.js 16 tightened image configuration in three ways that break existing config files. The images.domains option was removed in favour of images.remotePatterns. The quality prop is now restricted to values listed in images.qualities, which defaults to [75]. And images.localPatterns lets you restrict which local paths the optimizer will accept, which matters because /_next/image is a public endpoint that anyone can call with arbitrary parameters.

// next.config.ts
import type { NextConfig } from 'next';

const nextConfig: NextConfig = {
  images: {
    // images.domains was removed in Next.js 16. remotePatterns is the only form.
    remotePatterns: [
      { protocol: 'https', hostname: 'cdn.example.com', pathname: '/products/**' },
    ],
    // Only these quality values are accepted. Anything else is rejected.
    qualities: [50, 75],
    localPatterns: [{ pathname: '/assets/**', search: '' }],
    formats: ['image/avif', 'image/webp'],
    // Set this explicitly. A short TTL recomputes transformations you already paid for.
    minimumCacheTTL: 60 * 60 * 24 * 31,
  },
};

export default nextConfig;

SVG is still refused by the optimizer unless you set dangerouslyAllowSVG: true, and that default is correct: an SVG is an executable document, and optimizing one from an untrusted host turns your own origin into the delivery vehicle for its scripts.

Should images.formats list AVIF or WebP first?

List AVIF first when bandwidth is the constraint, and WebP first when first-request latency is. The images.formats array is ordered by preference, and the optimizer picks the first entry the requesting browser accepts.

Format	File size	Encode cost	Reach for it when
AVIF	Smaller than WebP at equal visual quality	Noticeably slower to encode	Images are cached and served many times; bytes dominate
WebP	Larger than AVIF, far smaller than JPEG	Fast	Long-tail images with few hits, where every request is a cache miss
JPEG/PNG source	Largest	None	Fallback only, for clients that accept neither

The trade-off is uneven across your site. On a marketing page with five hero images that every visitor loads, AVIF encoding happens once and the smaller bytes win forever. On a catalogue with fifty thousand product photos where most are viewed once, the slower encode is on the critical path of a real user every time.

How do I keep image transformation cost bounded?

An image transformation is a unique combination of source image, requested width, quality and output format, and each unique combination is computed and billed once before it is cached. That definition is the whole cost model: everything that multiplies the number of distinct combinations multiplies your bill.

Trim deviceSizes. Eight default widths times two formats is sixteen possible transformations per source image. If your layout has three real breakpoints, list three widths.
Use one quality value. Next.js 16 already forces you to declare them; declare one.
Raise minimumCacheTTL. Next.js honours an upstream Cache-Control max-age when it is longer than minimumCacheTTL, so a CMS that sends no-store quietly defeats the cache.
Set unoptimized on assets that are already optimized — sprite sheets, small PNG icons, anything your build pipeline has already squeezed.
Watch for cache-busting query strings. A source URL with a changing token is a new source image every time, and therefore a new transformation every time.

When should I bypass next/image entirely?

Three cases justify leaving the component behind. For art direction — a different crop on mobile than desktop — call getImageProps() (stable since Next.js 15, previously unstable_getImgProps) to obtain the generated srcset and feed it into your own <picture> element or a CSS background. For a static export (output: 'export') there is no server to run the optimizer, so you must set images.unoptimized: true or supply a custom loader. And if you already pay for an image CDN such as Cloudinary or imgix, point images.loaderFile at it rather than optimizing twice.

Self-hosting has one more requirement worth stating plainly: the built-in optimizer needs the sharp package installed. The pure-JavaScript fallback was removed in earlier releases, so a self-hosted deployment without sharp will fail to optimize rather than silently degrade.

FAQ

Q: Should I add priority to every above-the-fold image?

A: No. Add priority to the single image most likely to be the Largest Contentful Paint element. Multiple high-priority images compete for bandwidth and delay the one that actually determines the metric.

Q: Why does my optimized image look soft on a Retina screen?

A: Next.js never upscales beyond the intrinsic size of the source file. If the browser requests a 1600-pixel candidate and the source is 800 pixels wide, you get 800 pixels rendered into a 1600-pixel box. Replace the source asset; no configuration fixes it.

Q: Do I need sharp when self-hosting Next.js?

A: Yes. The built-in image optimizer requires the sharp package outside of Vercel, where the platform provides its own optimization layer.

Q: Does next/image work with output: 'export'?

A: Not with the default loader, because a static export has no server. Set images.unoptimized: true to emit plain <img> tags, or configure images.loaderFile to point at an external image CDN.

Q: Is AVIF always the better choice?

A: No. AVIF produces smaller files than WebP at comparable quality but takes measurably longer to encode, so on images with low cache-hit rates the encode time lands on a real user's first request.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.

Rate Limiting in the Next.js App Router: Field Notes on Middleware, Redis, and the Server Action That Looks Like Every Other POST

Ahmed Mahmoud — Wed, 19 Aug 2026 06:00:14 +0000

Headline: An in-memory rate limiter is wrong on serverless because every function instance keeps its own counter, so a limit of 10 requests per minute becomes 10 requests per minute per instance. The second trap is that every Next.js Server Action POSTs to the URL of the page that called it, so path-based limiting in middleware cannot tell one action from another.

A rate limiter is a counter with a deadline: allow N requests per identity per time window and reject the rest with HTTP 429. I have written that in ten lines before and been happy with it. Moving the same idea into a Next.js 16 App Router project on Vercel broke it in three places I did not expect — where the counter lives, who counts as an identity, and how you attach a limit to a Server Action that has no URL of its own.

Key takeaways

An in-memory Map rate limiter is per-instance on serverless. With ten warm function instances, a 10-requests-per-minute limit permits 100 requests per minute.
Next.js middleware is the cheapest place to shed abusive traffic because it runs before the route's own function boots, but it also sees RSC prefetch requests that carry the RSC: 1 header and that the user never intentionally made.
NextRequest.ip was removed in Next.js 15. On Vercel, read the client address with ipAddress(request) from @vercel/functions instead of trusting a raw x-forwarded-for header.
Every Next.js Server Action POSTs to the current page URL and carries a build-generated Next-Action header, so the only reliable place to limit a specific action is inside the action body.
Reject with status 429 and a Retry-After header, and decide fail-open versus fail-closed per route before your Redis has its first outage.

Why does an in-memory rate limiter break on serverless?

A module-scope Map is private to one function instance, and a serverless platform runs many instances at once. Each instance therefore enforces the full limit on its own, so the effective limit is your configured limit multiplied by the number of warm instances. That number is not something you control or can observe from inside the request.

This is the code I have shipped and regretted:

// Do not ship this to a serverless runtime.
const hits = new Map<string, { count: number; reset: number }>();

export function limit(key: string, max = 10, windowMs = 60_000) {
  const now = Date.now();
  const entry = hits.get(key);
  if (!entry || entry.reset < now) {
    hits.set(key, { count: 1, reset: now + windowMs });
    return { ok: true };
  }
  entry.count += 1;
  return { ok: entry.count <= max };
}

Vercel Fluid Compute makes this harder to notice rather than easier. Fluid Compute reuses a single function instance across concurrent requests instead of spawning one per request, so the Map survives far longer than it did under classic serverless. In local development and in a quiet preview deployment the limiter looks correct. It only comes apart under traffic spread across enough instances to matter, and nothing in the logs announces it.

The Map also never shrinks. On a long-lived instance every unique key you have ever seen stays resident until the instance is recycled, which is a slow memory leak wearing a rate limiter costume.

The fix is not a cleverer Map. The counter has to live in a store that every instance shares and that supports an atomic increment: Redis, or any datastore with a compare-and-set primitive.

Should the limit run in middleware or in the route handler?

Both, for different jobs. Middleware runs before Next.js resolves the route, so a request rejected there never boots the route's function and never touches your database. That makes middleware the correct place for a coarse, identity-agnostic abuse limit. The route handler knows the authenticated user, the parsed body, and the business meaning of the call, which makes it the correct place for a per-user quota.

// middleware.ts
import { NextResponse, type NextRequest } from 'next/server';
import { ipAddress } from '@vercel/functions';
import { ratelimit } from '@/lib/ratelimit';

export const config = {
  matcher: ['/api/:path*', '/login', '/signup'],
};

export async function middleware(request: NextRequest) {
  // RSC prefetches fire on link hover. The user did not ask for these.
  if (request.headers.get('rsc') === '1') return NextResponse.next();

  const ip = ipAddress(request) ?? '127.0.0.1';
  const { success, limit, remaining, reset } = await ratelimit.limit(`ip:${ip}`);

  if (success) return NextResponse.next();

  return NextResponse.json(
    { error: 'rate_limited', message: 'Too many requests.' },
    {
      status: 429,
      headers: {
        'Retry-After': String(Math.ceil((reset - Date.now()) / 1000)),
        'X-RateLimit-Limit': String(limit),
        'X-RateLimit-Remaining': String(remaining),
        'X-RateLimit-Reset': String(reset),
      },
    },
  );
}

The rsc header check is the line I added after watching my own quota drain while I did nothing but move a mouse. The Next.js router prefetches linked routes on hover and on viewport entry, and those requests are real HTTP requests that hit middleware with an RSC: 1 header. Counting them means a user who scrolls a navigation-heavy page is rate limited before they click anything.

Since Next.js 15.5 you can also opt middleware into the Node.js runtime with export const config = { runtime: 'nodejs' }, which lets you use a normal Redis client there. I still prefer an HTTP-based store in middleware, because middleware sits on the latency path of every matched request.

Placement	Sees	Cost of a rejection	Use it for
Middleware	URL, headers, cookies, IP	Lowest — route function never boots	IP-level abuse and brute-force shielding
Route handler	Everything, including session and body	Function has already started	Per-user quotas, per-endpoint cost control
Server Action body	Session, typed arguments	Function has already started	Form submissions and mutations

Which algorithm should I actually use?

Pick the cheapest algorithm whose failure mode you can live with.

Fixed window keeps one counter per window, so a check is a single INCR. Its flaw is the boundary: ten requests at 11:59:59 and ten more at 12:00:00 pass a "ten per minute" limit while delivering twenty requests in one second.
Sliding window log stores a timestamp per request and is exact, but its memory grows with your traffic, which is the wrong direction for a defence against traffic.
Sliding window counter weights the previous window by how much of it still overlaps the current one. Bounded memory, no boundary burst, and my default.
Token bucket gives each identity a capacity and a refill rate, so a quiet client may spend saved tokens at once. Right for APIs whose clients legitimately batch.

The @upstash/ratelimit package ships all four as fixedWindow, slidingWindow, tokenBucket, and cachedFixedWindow. If you write your own against Redis, the thing to get right is atomicity. INCR followed by a separate EXPIRE is two round trips, and if the process dies between them you have created a key with no expiry, which locks that identity out permanently. Do it in one script:

-- One round trip. The TTL is set only on the first hit of a window.
local current = redis.call('INCR', KEYS[1])
if current == 1 then
  redis.call('PEXPIRE', KEYS[1], ARGV[1])
end
return current

How do I identify the client when everything sits behind a proxy?

A rate limit is only as good as its key. NextRequest.ip and NextRequest.geo were removed in Next.js 15, so reading request.ip is now a type error rather than a subtle wrong answer. On Vercel the replacement is ipAddress(request) from @vercel/functions.

Do not key on a raw x-forwarded-for header unless you are certain a trusted proxy overwrites it. x-forwarded-for is an ordinary request header, so on any origin reachable directly, an attacker sets it to a new value per request and gets an unlimited number of fresh limit buckets. That is a complete bypass, not a partial failure.

IPv6 needs its own rule. A single residential subscriber is routinely assigned an entire /64 prefix, so keying on the full 128-bit address hands one attacker 2^64 distinct identities.

function identityKey(request: NextRequest, userId?: string) {
  // A stable account beats a network address whenever you have one.
  if (userId) return `user:${userId}`;

  const ip = ipAddress(request) ?? '0.0.0.0';
  if (ip.includes(':')) {
    // One subscriber can own a whole /64. Key the prefix, not the address.
    return `ip6:${ip.split(':').slice(0, 4).join(':')}`;
  }
  return `ip4:${ip}`;
}

How do I rate limit a Server Action when every action is a POST to the same URL?

A Server Action is a function marked with the 'use server' directive that the client invokes over the network. The invocation is an HTTP POST to the URL of the page the action was called from, with a Next-Action header containing a build-generated identifier for that specific action. There is no dedicated route path, which is exactly what breaks the obvious approach.

In middleware, a POST from a contact form on /contact and a POST from a delete-account button on /contact are the same URL and the same method. You can read request.headers.get('next-action') to at least distinguish action POSTs from ordinary document requests, but that identifier is a hash that changes when the build changes, so branch on its presence and never on its value.

The reliable place is inside the action:

'use server';

import { headers } from 'next/headers';
import { auth } from '@/lib/auth';
import { ratelimit } from '@/lib/ratelimit';

export async function sendMessage(_prev: State, formData: FormData): Promise<State> {
  const session = await auth();
  const key = session?.user.id ?? (await headers()).get('x-forwarded-for') ?? 'anon';

  const { success, reset } = await ratelimit.limit(`action:sendMessage:${key}`);
  if (!success) {
    const seconds = Math.ceil((reset - Date.now()) / 1000);
    // Return a value. A thrown error reaches production as an opaque digest.
    return { ok: false, error: `Too many messages. Try again in ${seconds}s.` };
  }

  // ...the real work
  return { ok: true };
}

Returning a value rather than throwing matters. An uncaught error inside a Server Action is redacted in production and surfaces to the client as a generic message with a digest, so the user is told something went wrong instead of being told to wait forty seconds. Rate limiting is a normal outcome, not an exception.

What should a 429 response actually contain?

HTTP 429 Too Many Requests is the correct status, and Retry-After is the header that makes a limiter usable by anyone other than a human staring at a browser. Its value is either a number of seconds or an HTTP date.

Alongside it, emit the limit state. The X-RateLimit-Limit / X-RateLimit-Remaining / X-RateLimit-Reset convention is not a standard but it is what most SDKs already parse. The IETF draft draft-ietf-httpapi-ratelimit-headers defines RateLimit and RateLimit-Policy as structured fields; adopt it only if your consumers understand it.

Two mistakes I have made and seen: returning 403 for a rate limit, which tells the client to stop forever rather than to retry, and returning 500, which pollutes your error rate with your own defences working correctly.

Then decide what happens when the store is unreachable, because it will be. Failing open on a login endpoint turns a Redis outage into an open brute-force window. Failing closed on a public read endpoint turns a Redis outage into a full outage of your site. I choose per route: fail closed on authentication and on anything that spends money, fail open on reads.

FAQ

Q: Can I rate limit in Next.js without Redis?
A: Only if your app runs as a single long-lived process, such as one container instance. On any serverless or autoscaled deployment the counter must live in a store shared across instances, because in-process state is multiplied by your instance count.

Q: Does Next.js middleware run on RSC prefetch requests?
A: Yes. Router prefetches are real HTTP requests that match your middleware matcher and carry the RSC: 1 header. Exclude them from user-facing quotas or they will consume a visitor's budget before the visitor clicks anything.

Q: How do I get the client IP in Next.js 15 and Next.js 16?
A: NextRequest.ip was removed in Next.js 15. On Vercel, call ipAddress(request) from @vercel/functions. Elsewhere, read the forwarded header your own trusted proxy sets and confirm that the proxy overwrites rather than appends it.

Q: Should a rate limiter fail open or fail closed when Redis is down?
A: Decide per route. Fail closed on login, signup, password reset, and payment endpoints, where failing open creates a security window. Fail open on public reads, where failing closed converts a dependency outage into a site outage.

Q: Is Vercel BotID a replacement for rate limiting?
A: No. Vercel BotID is bot detection, which answers whether a caller is automated. A rate limit answers how often any caller, human or not, may perform an expensive operation. They defend different things and compose well together.

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.