DEV Community: uninterrupted

Achieving Instant Navigations in Next.js 16.3

uninterrupted — Wed, 07 Oct 2026 07:09:06 +0000

Server Components can reduce client-side JavaScript, keep sensitive logic on the server, and prevent data-fetching waterfalls. But they also introduced a frustrating user experience in many dynamic applications: after clicking a link, the user sometimes had to wait for the server before seeing the next screen.

Next.js 16.3 addresses this problem with Instant Navigations, an opt-in set of rendering, prefetching, development, and testing tools. The goal is not to make every data request finish immediately. It is to make sure the application always has meaningful UI ready when navigation begins.

This is particularly useful for e-commerce platforms, dashboards, and account areas. These applications cannot prerender every personalized state at build time, but they should still respond to clicks without a visible pause.

In this guide, I will explain how Instant Navigations work, how to choose between streaming and caching, and how to protect navigation performance with Playwright tests.

Why Dynamic Next.js Navigations Can Feel Slow

In a traditional server-driven navigation, the sequence often looks like this:

The user clicks a link.
The browser requests the target route from the server.
The server reads data and renders the route.
The browser finally displays the next UI.

The architecture can be efficient, but the delay between the first and fourth steps is visible. A client-driven SPA usually feels more responsive because it already has enough code in the browser to display the next screen or its loading state immediately.

Before Next.js 16.3, teams had two common ways to reduce this delay:

prerender the full page at build time, which does not work for every dynamic or personalized route
fully prefetch each destination with <Link prefetch={true}>, which can generate unnecessary requests and increase server load

A loading.tsx file could provide an immediate loading state, but it was easy to forget and applied at the route-segment level rather than around the exact data dependency.

Instant Navigations provide a more precise model. Instead of asking whether the whole route is static or dynamic, you decide which parts can be ready before the click and which parts should arrive later.

Enabling Instant Navigations in Next.js 16.3

Instant Navigations are opt-in in Next.js 16.3. Enable Cache Components and Partial Prefetching in next.config.ts:

import type { NextConfig } from 'next'

const nextConfig: NextConfig = {
  cacheComponents: true,
  partialPrefetching: true,
}

export default nextConfig

cacheComponents enables the explicit caching model built around 'use cache', cacheLife, and cacheTag. It also allows routes to combine prerendered UI with dynamic content behind Suspense boundaries.

partialPrefetching changes how Next.js prepares routes on the client. Instead of treating prefetching as an all-or-nothing decision, Next.js can prefetch only the reusable UI needed to start a navigation.

Enabling cacheComponents is a migration step, not just a performance flag. During production builds, Next.js attempts to prerender as much of a route as possible and can report an error when runtime or uncached data is accessed outside a Suspense boundary. Each case must be streamed behind Suspense, cached with 'use cache' when appropriate, or explicitly allowed to block with export const instant = false. Existing applications may also need to remove incompatible route segment configuration such as dynamic, revalidate, or fetchCache.

These behaviors are planned to become defaults in a future major version. In 16.3, enabling them explicitly gives teams time to review their asynchronous boundaries and migration requirements.

The Core Model: Stream, Cache, or Block

When a route depends on asynchronous data, Next.js 16.3 asks you to make the intended behavior explicit. There are three useful choices.

1. Stream dynamic content with Suspense

Use <Suspense> when the data must stay fresh or depends on the active request. The fallback becomes part of the route shell, so it can appear immediately while the server renders the final content.

This is a good fit for:

inventory and delivery estimates
cart state
account-specific data
live analytics
personalized recommendations

The navigation is instant because the user sees the fallback without waiting for the dynamic operation. The final data still arrives from the server later.

2. Cache reusable content with `'use cache'`

Use 'use cache' when the result can be safely reused. Cached UI can be included in the prefetched response and displayed immediately during navigation.

This works well for:

product descriptions
category content
CMS blocks
public pricing shared by a region
configuration that changes infrequently

The cache boundary should follow the business meaning of the data. Content that is safe to share globally should not be mixed with customer-specific prices or session state.

3. Allow the route to block

Not every route needs to navigate instantly. If displaying a loading state would be misleading and the route should wait for complete server output, mark that decision explicitly:

export const instant = false

This can be appropriate for a simple content page where the team prefers to show the complete article at once, or for a flow where a partial state would create confusion.

The important change is that blocking becomes an intentional route decision rather than an accidental result of one unresolved data dependency.

How Partial Prefetching Changes the Network Cost

The previous prefetching model could issue a request for every visible link. Consider an account sidebar with twenty links to /orders/[id]. Even though all links share the same route structure, the browser could prefetch each destination separately.

With Partial Prefetching, Next.js can generate one reusable shell for /orders/[id], store it in the browser, and use it for multiple links during the session. This is closer to route-level code splitting in a SPA: the browser receives enough UI to react immediately, but it does not need the complete data for every possible destination.

If a particular link deserves a deeper prefetch, you can still opt in:

<Link href={`/products/${product.slug}`} prefetch={true}>
  {product.title}
</Link>

Next.js then prerenders as much of that destination as it safely can. Synchronous values from the URL and content marked with 'use cache' can be included, while request-bound work remains behind its Suspense boundary.

This avoids a common tradeoff in dynamic applications: either prefetch nothing and make the user wait, or prefetch complete pages and make the server do work that may never be used.

A Practical E-Commerce Example

A product page usually combines data with very different freshness requirements:

product copy and media are broadly reusable
inventory changes often
customer pricing may depend on identity, region, or sales channel
the cart belongs to the current session

Making the whole page dynamic means one slow operation can block navigation. Caching the whole page is more dangerous because it can expose stale or private data.

A better boundary looks like this:

import { Suspense } from 'react'
import { cacheLife, cacheTag } from 'next/cache'

async function ProductSummary({ slug }: { slug: string }) {
  'use cache'

  cacheTag(`product:${slug}`)
  cacheLife('hours')

  const product = await getProduct(slug)

  return (
    <section>
      <h1>{product.title}</h1>
      <p>{product.description}</p>
    </section>
  )
}

async function Inventory({ slug }: { slug: string }) {
  const inventory = await getLiveInventory(slug)

  return <p>{inventory.available} items available</p>
}

export default async function ProductPage({
  params,
}: PageProps<'/products/[slug]'>) {
  const { slug } = await params

  return (
    <main>
      <ProductSummary slug={slug} />

      <Suspense fallback={<p>Checking availability...</p>}>
        <Inventory slug={slug} />
      </Suspense>
    </main>
  )
}

The product summary can be cached and reused. Live inventory remains dynamic, but it no longer prevents the route from showing useful content. During navigation, the browser can display the product UI or its prepared shell first and fill in availability when the server responds.

For a real storefront, customer-specific pricing should follow the same principle as inventory: isolate it in a private or request-time boundary instead of adding cookies or session access to the entire page.

Finding Slow Routes with Instant Insights

Performance problems are difficult to fix when they only appear as a vague feeling that the application is slow. Next.js 16.3 adds Instant Insights to the development tools to make the problem visible.

When you encounter a non-instant navigation during development, Instant Insights identifies the route and points to the asynchronous work that blocks its shell. In most cases, the fix is one of the following:

move request-specific work below a Suspense boundary
mark reusable work with 'use cache'
move a blocking dependency out of a shared layout
explicitly accept blocking with export const instant = false

The Navigation Inspector complements this by pausing navigation at the prefetched shell. This lets you inspect exactly what the user can see before dynamic content arrives. It is useful because production prefetching is normally disabled in development, where loading behavior can otherwise be difficult to reproduce accurately.

Testing Instant Navigations with Playwright

A route that feels instant today can become blocking after a small refactor. A new cookies() call in a shared component or a moved Suspense boundary may be enough to change the result.

Next.js 16.3 includes the instant() helper in @next/playwright. It temporarily holds back dynamic content, allowing the test to assert which UI is available without waiting for the network:

import { expect, test } from '@playwright/test'
import { instant } from '@next/playwright'

test('product navigation shows useful UI instantly', async ({ page }) => {
  await page.goto('/products/shoes')

  await instant(page, async () => {
    await page.click('a[href="/products/hats"]')
    await page.waitForURL((url) => url.pathname === '/products/hats')

    await expect(page.locator('h1')).toContainText('Baseball Cap')
    await expect(page.getByText('Checking availability...')).toBeVisible()
  })

  await expect(page.getByText(/items available/)).toBeVisible()
})

The test describes a user-facing contract:

the product title must be available immediately
the inventory fallback must be visible while live data is pending
the final inventory must appear after the dynamic work completes

This is more useful than testing an arbitrary duration such as 100 milliseconds. Network and CI timing can vary, but the boundary between prefetched UI and network-dependent UI is deterministic.

The v0 team used this property to improve real routes with a coding agent. The agent wrote a failing instant() test, moved or cached the blocking work, rebuilt the application, and repeated the process until the test passed. The resulting 16 tests were kept in CI to prevent future regressions.

Where Teams Usually Get It Wrong

Mistake 1: Treating “instant” as “fully loaded”

Instant Navigations do not make databases or APIs respond with zero latency. They guarantee that useful UI can respond to the click without waiting for those systems.

The distinction matters. A complete page may still stream in over time, but the application should never look unresponsive.

Mistake 2: Placing Suspense too high in the tree

Wrapping an entire page in one Suspense boundary may technically make navigation instant, but the user will see a generic full-page skeleton. Place boundaries close to the data that actually needs to wait so stable navigation, headings, and controls remain visible.

Mistake 3: Reading dynamic data in a shared layout

A session or request-dependent operation near the root can block many routes at once. Keep shared layouts focused on reusable UI and move dynamic reads into the components that need them.

Mistake 4: Enabling full prefetching on every link

prefetch={true} is still useful, but it should represent a deliberate choice to prepare more than the route shell. Applying it everywhere can recreate the server and network cost that Partial Prefetching is designed to avoid.

Mistake 5: Using meaningless loading states

An instant blank rectangle is not a good user experience. The shell should preserve the layout and provide enough context for the user to understand where the navigation is going.

For a product page, that may include the title, gallery frame, price area, and an inventory placeholder. For an account page, it may include the navigation, page heading, and form structure.

A Practical Adoption Strategy

For an existing Next.js application, avoid changing every route at once. Start with the journeys where a delay has the largest user or business impact.

Enable cacheComponents and partialPrefetching in a branch.
Use Instant Insights to identify blocking navigations.
Choose one important journey, such as category to product or cart to checkout.
Write a failing test with instant() that describes the minimum UI required after the click.
Stream request-bound data and cache only content that is safe to reuse.
Inspect the shell with Navigation Inspector.
Keep the test in CI after the route passes.

This process turns perceived performance into an architectural requirement that can be reviewed and tested. It also creates a clear stopping point: the work is complete when the intended shell is visible without network-dependent content.

Final Thoughts

The most important part of Instant Navigations is not another prefetch option. It is the clearer boundary between UI that must be ready for a click and data that is allowed to arrive later.

For dynamic applications, this removes the need to choose between a server-driven architecture and SPA-like responsiveness. Server Components can continue to handle rendering and sensitive data, while Suspense, Cache Components, and reusable route shells keep navigation responsive.

The best result comes from combining three decisions:

stream data that must remain dynamic
cache content that is genuinely reusable
block only when waiting is an intentional product choice

Once those decisions are protected by instant() tests, navigation performance stops being something that can disappear unnoticed during the next refactor.

Implementing Postgres Row-Level Security in Next.js: The Drizzle Multi-Tenant Pattern

uninterrupted — Wed, 16 Sep 2026 07:00:00 +0000

The first time we turned on Row-Level Security, nothing happened. Policies were in place, ENABLE ROW LEVEL SECURITY had run, and every query still returned every tenant's rows. Took a minute to remember: we were connected as postgres, the superuser, and superusers ignore RLS entirely. That one gotcha basically shaped the rest of this setup.

The problem we were actually trying to solve is boring and everyone's had it: a route handler forgets WHERE organization_id = ?, code review doesn't catch it because the query still "works," and Tenant A quietly gets to see Tenant B's data. Nobody notices until someone does.

Our answer was to stop trusting application code to remember the filter and push isolation into Postgres itself with Row-Level Security. The pitch is simple: if a query forgets tenant scoping, it should get zero rows back, not someone else's data. Here's the setup we landed on — two DB roles, a SET LOCAL session variable, a Next.js wrapper so route handlers don't have to think about any of this — plus the four things that actually bit us building it.

Two roles, two connections

Back to that first surprise: superusers and table owners bypass RLS by default. The docs say it plainly — superuser or BYPASSRLS, policies don't apply, full stop. Table owners get the same pass unless you explicitly add FORCE ROW LEVEL SECURITY. If your app connects as postgres, which is what basically every local Postgres setup defaults to, your carefully written policies are just decoration.

So we run two roles now:

Role	RLS	Used by
`postgres` (superuser)	Bypassed	Auth library, admin routes, migrations
`app_user` (non-superuser)	Enforced	All tenant-scoped queries

Each gets its own pool, wrapped as two Drizzle instances:

// db.ts
export function getDbAdmin(): PostgresJsDatabase<Schema> {
  if (!_dbAdmin) {
    const client = postgres({
      ...adminConnectionConfig, // DATABASE_URL, superuser
      max: 2,
      connection: { application_name: "app-admin" },
    });
    _dbAdmin = drizzle(client, { schema });
  }
  return _dbAdmin;
}

export function getDb(): PostgresJsDatabase<Schema> {
  if (!_db) {
    const client = postgres({
      ...appConnectionConfig, // APP_DATABASE_URL, app_user
      max: 4,
      connection: { application_name: "app-tenant" },
    });
    _db = drizzle(client, { schema });
  }
  return _db;
}

Worth flagging two things here. First, the lazy init isn't stylistic — Next.js runs module scope at build time, and if you open a connection pool at import time, next build needs a live database to succeed. That bit us once in CI. Wrapping construction in a getter defers the actual connection until something queries it.

We also wrap both in a Proxy so call sites can still just import db and use it like a normal object:

export const db = new Proxy({} as PostgresJsDatabase<Schema>, {
  get(_, prop) {
    return Reflect.get(getDb(), prop);
  },
});

I won't pretend this is elegant. It works, and it means nobody importing db has to know it's lazy underneath. Fine trade.

Second thing: we set a distinct application_name per pool. Costs nothing, and the first time something's connection-starved you'll be very glad pg_stat_activity tells you exactly which pool is choking instead of guessing.

The policy itself

Every tenant table gets the same shape:

ALTER TABLE projects ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON projects
  USING (organization_id = current_setting('app.organization_id', true)::uuid)
  WITH CHECK (organization_id = current_setting('app.organization_id', true)::uuid);

Two things matter more than they look like they should.

That true as the second argument to current_setting means "don't error if the setting's unset, just give me NULL." NULL compared to anything is NULL, and Postgres treats a NULL policy result as "exclude this row" — same as false. So a query that runs without tenant context isn't a crash and isn't a leak, it's just an empty table. Someone forgets to set context, they get a confusing empty result, they go debug it. Nobody's data went anywhere in the meantime. That's the property I actually care about here.

And don't skip WITH CHECK. It's easy to write the USING clause, ship it, and feel done — but USING only governs reads. Without WITH CHECK, nothing stops a tenant from INSERTing or UPDATEing a row with someone else's organization_id in it. We missed this on the first pass of one migration and caught it in review, which is exactly the kind of thing you don't want to catch in review.

Getting context into the transaction: SET LOCAL

Policies check current_setting('app.organization_id'), so something has to set it per request. The obvious first attempt — plain SET app.organization_id = '...' — is wrong, and it's wrong in a way that won't show up until you're under load. SET is session-scoped, connections are pooled, and a session-scoped setting sticks around for whoever grabs that connection next. That's tenant bleed waiting to happen the moment your pool gets reused across requests, which it will.

SET LOCAL is the fix — scoped to the current transaction, reverts automatically on commit or rollback. Open transaction, SET LOCAL, run your queries, done, connection goes back to the pool clean.

In Next.js we wrapped this into one function every tenant route goes through:

export function withAuth(handler: AuthenticatedHandler) {
  return async (request: NextRequest, ctx?: RouteContext) => {
    const result = await authenticate(); // session lookup
    if (!result.success) {
      return NextResponse.json({ error: result.error }, { status: result.status });
    }

    const { user, organizationId } = result;

    return db.transaction(async (tx) => {
      await setOrgContext(tx, organizationId);
      return handler(request, { user, organizationId, tx: castTx(tx) }, ctx);
    });
  };
}

Route handlers never see any of this. They just get a tx that's already scoped:

export const GET = withAuth(async (request, { tx }) => {
  const rows = await tx.select().from(projects); // only this org's rows, guaranteed
  return NextResponse.json({ data: rows });
});

No WHERE organizationId = ? anywhere in that query, and it's still safe. We keep the explicit filters in most places anyway — belt and suspenders, and it documents intent for the next person reading the code — but the point stands: if you forgot it, RLS still has you.

Server components got a matching helper since they're not going through route handlers at all:

export async function withOrgDb<T>(
  organizationId: string,
  callback: (tx: DbInstance) => Promise<T>,
): Promise<T> {
  return getDb().transaction(async (tx) => {
    await setOrgContext(tx, organizationId);
    return callback(castTx(tx));
  });
}

Gotcha #1: you can't parameterize SET

setOrgContext looks like this:

const UUID_RE =
  /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;

export async function setOrgContext(tx: DrizzleTx, organizationId: string) {
  if (!UUID_RE.test(organizationId)) {
    throw new Error("Invalid organization ID format");
  }
  await tx.execute(
    sql`SET LOCAL app.organization_id = '${sql.raw(organizationId)}'`,
  );
}

That sql.raw() should bother you a little — it bothered me too, first time I wrote it. Here's the thing though: SET is a utility command, not a regular statement, and utility commands in Postgres just don't take $1-style parameters. You cannot write SET LOCAL app.organization_id = $1. It'll error. You've got two real options — set_config('app.organization_id', $1, true), which is a normal function call and does accept a parameter, or raw string interpolation gated behind strict validation. We went with the second, gated on a UUID regex: the value either looks exactly like a UUID or the function throws before it ever touches SQL. I still don't love it stylistically, but there's no injection surface left once that regex has to match.

Point either way: you're going to hit this wall the first time you try to parameterize a SET, and it's not a bug in your code, it's just how the command works.

Gotcha #2: Drizzle's transaction type doesn't match its client type

Drizzle's transaction callback hands you a PostgresJsTransaction, and that type is not structurally the same as PostgresJsDatabase — even though every method you'd actually call (select, insert, update, delete) is identical between them. If AuthContext.tx gets typed as the transaction class, every shared data function downstream now needs that same internal type, which means importing a class Drizzle never really meant for you to reach into.

We just centralized the cast once and moved on:

// Extract the tx type without importing internal Drizzle classes
type DrizzleTx = Parameters<Parameters<DbInstance["transaction"]>[0]>[0];

export function castTx(tx: DrizzleTx): DbInstance {
  return tx as unknown as DbInstance;
}

Now shared data-layer code just takes a DbInstance and doesn't care if it's the root client, the admin client, or a live transaction. One cast, one comment explaining why, easy to delete later if Drizzle ever unifies the types properly.

Gotcha #3: don't let a stream hold your transaction

withAuth holds a transaction — and the pooled connection under it — for as long as the request runs. For a JSON endpoint that returns in 40ms, fine. For a Server-Sent Events endpoint that stays open for twenty minutes, that's a problem: with a pool of 4, three people leaving a dashboard tab open exhausts it and everyone else starts timing out.

We learned this the hard way and split off a separate wrapper for streaming routes — it authenticates, but deliberately never opens a transaction:

export const GET = withAuthStream(async (request, { organizationId }) => {
  // Short transaction for the initial state read
  const job = await withOrgDb(organizationId, (tx) =>
    tx.select().from(jobs).where(eq(jobs.id, jobId)),
  );

  const stream = new ReadableStream({
    /* ...on each event, use withOrgDb() again for any query... */
  });
  return new Response(stream, {
    headers: { "Content-Type": "text/event-stream" },
  });
});

The rule we now follow: RLS context only lives as long as a transaction, so keep the transaction as short as the query needs — never as long as the request.

Gotcha #4: knowing which connection you're on

Two Drizzle instances means the failure mode shifted from "forgot a WHERE clause" to "grabbed the wrong client." We ended up needing a cheat sheet:

Scenario	Connection
Tenant API route	`tx` from `withAuth`
SSE route	`withOrgDb()` per query
Server component, tenant data	`withOrgDb()`
Server component, auth tables only	`dbAdmin`
Admin route	`dbAdmin`
Auth library internals	`dbAdmin`
Migrations / background workers	superuser

The reassuring part: the mistake in the safe direction fails loudly. Query tenant tables through db with no context set, and you get zero rows — a visibly broken feature in dev, not a security incident. Using dbAdmin where you meant tenant scoping is the dangerous direction, and that's the one worth actually watching for in review. At least it's greppable.

Postgres LISTEN/NOTIFY in Production: Safe Reconnects & SSE

uninterrupted — Fri, 14 Aug 2026 07:24:09 +0000

If your app needs to push live updates to connected clients — a job progress bar, a balance that changes when a background worker finishes, a "this record was deleted elsewhere" toast — the reflex is to reach for Redis Pub/Sub or a message broker. But if you already run PostgreSQL, it ships with a pub/sub primitive built in: LISTEN / NOTIFY. No extra infrastructure, no extra failure domain.

What is Postgres LISTEN/NOTIFY? It's a built-in pub/sub mechanism: one session runs pg_notify(channel, payload) inside a transaction, and any other session that has issued LISTEN <channel> on a persistent connection receives the payload once that transaction commits. There's no persistence and no delivery guarantee — it's a live signal, not a queue — so it fits best as a "something changed, go re-check" nudge rather than a system of record.

This post walks through the implementation we actually run in production: a single shared LISTEN connection multiplexing many logical channels, wired into per-client Server-Sent Events (SSE) streams. Along the way: the reconnect logic and the race conditions that come with it.

How Postgres LISTEN/NOTIFY works

Postgres gives you two SQL-level building blocks:

SELECT pg_notify('job_status_123', '{"status":"done"}');

and, on a persistent connection:

LISTEN job_status_123;

Any session that has issued LISTEN on a channel receives the payload once the transaction that ran pg_notify commits, not the instant the call executes. Multiple identical notifications on the same channel/payload fired within one transaction get collapsed into one delivery. Several constraints shape everything that follows (see the Postgres NOTIFY reference for the authoritative spec):

Payload is capped at 8000 bytes. Fine for status pings, not for shipping large blobs.
Channel names are Postgres identifiers, truncated at 63 characters. Patterns like job_status_<uuid> need to fit under that limit or get silently truncated and collide.
No persistence, no delivery guarantee. If nobody is listening when pg_notify fires, the message is gone. This is a live signal, not a queue: any client relying on it needs a way to independently reconcile state, e.g. re-fetch from the DB.
The notification queue itself is bounded (8GB by default, per the Postgres async messaging docs). If a listening session sits idle inside an open transaction, it stops draining the queue for everyone, and pg_notify starts erroring once the queue fills. Keep LISTEN sessions out of long-running transactions.

Because LISTEN requires a session-scoped, long-lived connection, and most apps talk to Postgres through a connection pool, you can't just LISTEN from your normal query pool. Pooled connections get recycled and your subscription vanishes with them. You need one connection dedicated to listening, held open for the life of the process.

This also rules out transaction-mode poolers like pgbouncer or RDS Proxy for the LISTEN connection specifically. They can hand your session's underlying socket to a different client between transactions, which silently kills the subscription. The dedicated LISTEN connection needs to bypass the pooler and go straight to Postgres. If you run multiple app instances, each one holds its own LISTEN connection and each receives every NOTIFY independently, which is usually what you want: every instance can push updates to its own locally-connected clients.

One connection, many channels

Rather than open a Postgres connection per subscriber, a single physical LISTEN connection can multiplex an arbitrary number of logical channels. In-process, that's just a map from channel name to a set of callbacks:

class NotificationHub {
  private client: ReturnType<typeof postgres> | null = null;
  private readonly listeners = new Map<Channel, Set<MessageCallback>>();
  private readonly unlistenFns = new Map<Channel, () => Promise<void>>();
  ...

The connection itself is configured to never idle out and to identify itself for observability:

this.client = postgres({
  ...getAppConnectionConfig(),
  max: 1,
  idle_timeout: 0,
  connect_timeout: 10,
  connection: { application_name: "app-listen" },
  onclose: () => {
    /* ... */
  },
});

This pool is deliberately capped at one connection: it exists purely to host a dedicated LISTEN session, not to serve queries. Mixing it into a general-purpose pool would mean losing subscriptions every time the pool recycles a connection.

Handling reconnects: capped exponential backoff

A long-lived TCP connection to a database will eventually drop: network blips, database failover, load balancer idle timeouts. The interesting design question isn't "will it disconnect" but "what happens to every registered channel when it does."

onclose: () => {
  if (!this.closed) {
    dbLogger.warn("LISTEN connection closed unexpectedly");
    this.setState("disconnected");
    this.scheduleReconnect();
  }
},

Reconnect uses a capped exponential backoff so a flapping database doesn't get hammered with reconnect attempts:

const RECONNECT_DELAYS = [1_000, 2_000, 4_000, 8_000, 15_000] as const;

private scheduleReconnect(): void {
  if (this.closed || this.reconnectTimer) return;
  const delayIndex = Math.min(this.reconnectAttempt, RECONNECT_DELAYS.length - 1);
  const delay = RECONNECT_DELAYS[delayIndex];
  this.reconnectAttempt++;
  this.reconnectTimer = setTimeout(() => {
    this.reconnectTimer = null;
    void this.connect();
  }, delay);
}

reconnectAttempt resets to 0 on a successful connect(), so backoff only escalates across consecutive failures. A single flaky reconnect doesn't leave future reconnects artificially delayed.

On reconnect, every channel that had at least one active subscriber gets re-LISTENed from scratch:

const channels = Array.from(this.listeners.keys());
const failedChannels: Channel[] = [];
for (const channel of channels) {
  try {
    await this.listenToChannel(channel);
  } catch (err) {
    failedChannels.push(channel);
  }
}

if (failedChannels.length > 0) {
  this.setState("disconnected");
  this.scheduleReconnect();
  return;
}

Note the all-or-nothing treatment: if even one channel fails to re-LISTEN, the whole reconnect is considered failed and retried. A hub that's "half connected," silently missing notifications on some channels, is worse than one that's visibly disconnected and retrying. At least then the failure isn't invisible to callers.

The unsubscribe race, solved by ordering, not locking

Multiple parts of an app can subscribe to the same channel concurrently (two browser tabs watching the same job, for instance). The tricky moment is when the last subscriber for a channel unsubscribes at the same time a new subscriber for that same channel is arriving:

unsubscribe: async () => {
  for (const channel of channels) {
    const callbacks = this.listeners.get(channel);
    if (!callbacks) continue;
    callbacks.delete(onMessage);

    if (callbacks.size === 0) {
      this.listeners.delete(channel);
      const unlisten = this.unlistenFns.get(channel);
      // Delete before awaiting so a concurrent subscribe() for this
      // channel calls listenToChannel() again. PostgreSQL silently
      // ignores a duplicate LISTEN on the same connection, so the race
      // is harmless — the hub will have an active LISTEN either way.
      this.unlistenFns.delete(channel);
      if (unlisten) {
        try {
          await this.timeoutUnlisten(unlisten);
        } catch { /* timed out or connection gone — safe to ignore */ }
      }
    }
  }
},

The naive version would await UNLISTEN first, then clean up bookkeeping. But between "decide to unlisten" and "UNLISTEN finishes," a new subscriber could arrive, see the channel already has a bookkeeping entry, and skip re-LISTENing. That leaves the hub with no active LISTEN for a channel it thinks it's subscribed to.

By deleting the map entry before awaiting UNLISTEN, a concurrent subscribe() call always sees "no entry" and issues its own LISTEN unconditionally. Postgres treats a duplicate LISTEN on the same session as a no-op, so the worst case is two harmless LISTEN calls in flight instead of a silently dropped subscription. When you can't easily lock across an async boundary, look for an operation idempotent enough to make the race safe by construction.

Callers should never worry about connection state

The public subscribe() API registers callbacks unconditionally, regardless of whether the underlying Postgres connection is currently up. It only rejects if an opportunistic LISTEN call fails outright while attempting an already-connected session, and that case is recovered by the reconnect logic on its next attempt:

async subscribe(channels: Channel[], onMessage: MessageCallback): Promise<Subscription> {
  // Register callbacks regardless of connection state
  for (const channel of channels) {
    let callbacks = this.listeners.get(channel);
    if (!callbacks) {
      callbacks = new Set();
      this.listeners.set(channel, callbacks);
    }
    callbacks.add(onMessage);
  }

  if (this.state === "connected" && this.client) {
    for (const channel of channels) {
      await this.listenToChannel(channel); // no-op if already LISTENing
    }
  }

  if (this.state === "disconnected" && !this.connecting && !this.reconnectTimer) {
    void this.connect();
  }

  return { unsubscribe: async () => { /* ... */ } };
}

Callbacks are registered in memory first; the actual LISTEN happens opportunistically if connected, or gets picked up automatically on the next successful reconnect. Every consumer of the hub stays decoupled from the connection's health. They just subscribe, and messages show up when the pipe is open.

Wiring it into Server-Sent Events

The hub by itself is transport-agnostic; a thin SSE layer turns it into per-client streams with keep-alives and cleanup on disconnect:

const stream = new ReadableStream({
  start(controller) {
    let subscription: Subscription | null = null;

    const onAbort = () => {
      void cleanup();
    };
    options.request.signal.addEventListener("abort", onAbort);

    keepAliveInterval = setInterval(
      () => safeEnqueue(`: ping\n\n`),
      keepAliveMs,
    );

    const init = async () => {
      await options.onStart(safeEnqueue, () => {
        void cleanup();
      });
      if (!options.subscribe || isClosed) return;

      const { channels, onMessage } = options.subscribe;

      unsubscribeState = onHubStateChange((state) => {
        safeEnqueue(
          `data: ${JSON.stringify({ type: "info", liveUpdates: state === "connected" })}\n\n`,
        );
      });

      subscription = await subscribe(channels, (ch, message) => {
        if (!isClosed)
          onMessage(ch, message, safeEnqueue, () => {
            void cleanup();
          });
      });

      if (options.onAfterSubscribe) {
        await options.onAfterSubscribe(safeEnqueue, () => {
          void cleanup();
        });
      }
    };

    init().catch(() => {
      void cleanup();
    });
  },
  cancel() {
    return cleanupFn?.();
  },
});

Two details worth calling out.

The hub's connection state is surfaced to the client. When onHubStateChange reports "disconnected", an {"type":"info","liveUpdates":false} frame goes out over SSE so the UI can show "reconnecting..." instead of silently going stale. Since NOTIFY gives no delivery guarantee, telling the client when it might be missing updates is almost as important as delivering the updates themselves.

There's also a read-then-subscribe race window. onStart typically does an initial DB read to send current state, and only afterwards does the code call subscribe() to start receiving live updates. If a NOTIFY fires in the gap between that read and the LISTEN actually being established, it's lost for good: the classic "check-then-watch" race in any pub/sub system. The onAfterSubscribe hook closes that window. After the subscription is confirmed active, it re-checks state one more time, so any update that landed in the gap gets picked up on the second read instead of being silently missed. The client-side handler needs to tolerate receiving the same update twice (once from the catch-up read, once from a live NOTIFY that arrives moments later). Treat updates as idempotent, not as a strictly-once stream.

When this pattern fits

Postgres LISTEN/NOTIFY works well when:

Update volume is modest (dashboards, job status, per-user notifications), not a high-throughput event bus.
Clients treat NOTIFY as a nudge to re-sync, not as the sole source of truth. Always have a path to reconstruct state via a normal query if a notification is missed.
You want one fewer piece of infrastructure to run and monitor.

It's the wrong tool when you need guaranteed delivery, durable queues, replay, or very high fan-out. That's what a real message broker is for. But for "tell connected clients something changed," Postgres has had a message bus built in the whole time.

Faster Geospatial Animations with PMTiles: Multi-Layer MVT Time Series in OpenLayers

uninterrupted — Wed, 08 Jul 2026 06:00:00 +0000

Most people try to animate time series by stacking separate tile sources or flipping CSS styles mid-frame. That causes flicker and a surprising amount of wasted CPU/GPU work. The core takeaway is simple: bake your timesteps into a single vector-tile archive (PMTiles) with per-timestep named layers, load it once, and drive animation by re-styling already-decoded tiles.

I tested this with OpenLayers and noticed the difference right away: playback became smoother and far less CPU-bound once tiles were pre-baked and cached. The hard part is server-side — how you generate, package, and serve tiles matters more than which client library you pick.

When you animate a map by switching between separate tile sources, each frame causes one of two expensive operations: a network fetch of new tiles, or client-side re-decoding and re-styling. Baking timesteps into one archive avoids that by turning each frame into a cheap style re-evaluation on already-decoded features.

The architecture: single archive, dynamic layers

MVT (Mapbox Vector Tile) is a compact protobuf-based vector tile format that encodes geometry and attributes per z/x/y tile. PMTiles packages MVT tiles into a single file for efficient distribution and partial HTTP range retrieval.

The key workflow:

Server side — each timestep is a separate GeoJSON file. tippecanoe joins them into one PMTiles archive, giving each timestep its own named MVT layer (timestep_00, timestep_01, …).
Client side — one VectorTileSource loads the archive once. One VectorTileLayer uses a style function that reads a mutable currentLayerName variable. Advancing a frame means: update the variable, call layer.changed(). OpenLayers re-runs the style function on the already-decoded tile features and repaints — no network request, no re-decode.

Complete working example

!animation.gif

This section contains everything needed to reproduce the demo from scratch: 11 per-timestep GeoJSON files, a tile generation script, and the client HTML.

Step 1 — Per-timestep GeoJSON files

Create 11 files, one per timestep. Each contains a single large polygon with a value property. The temporal identity comes from the file name (and the MVT layer name tippecanoe will assign), not from a property on the feature.

for i in $(seq 0 10); do
  pad=$(printf "%02d" $i)
  value=$((i * 10))
  cat > "square_t${pad}.geojson" << EOF
{
  "type": "FeatureCollection",
  "features": [{
    "type": "Feature",
    "geometry": {
      "type": "Polygon",
      "coordinates": [[[-60,-50],[60,-50],[60,50],[-60,50],[-60,-50]]]
    },
    "properties": { "value": ${value} }
  }]
}
EOF
done

This produces square_t00.geojson (value=0) through square_t10.geojson (value=100).

Step 2 — Generate PMTiles with named layers (`generate-tiles.sh`)

#!/usr/bin/env bash
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

tippecanoe \
  --named-layer=timestep_00:"$SCRIPT_DIR/square_t00.geojson" \
  --named-layer=timestep_01:"$SCRIPT_DIR/square_t01.geojson" \
  --named-layer=timestep_02:"$SCRIPT_DIR/square_t02.geojson" \
  --named-layer=timestep_03:"$SCRIPT_DIR/square_t03.geojson" \
  --named-layer=timestep_04:"$SCRIPT_DIR/square_t04.geojson" \
  --named-layer=timestep_05:"$SCRIPT_DIR/square_t05.geojson" \
  --named-layer=timestep_06:"$SCRIPT_DIR/square_t06.geojson" \
  --named-layer=timestep_07:"$SCRIPT_DIR/square_t07.geojson" \
  --named-layer=timestep_08:"$SCRIPT_DIR/square_t08.geojson" \
  --named-layer=timestep_09:"$SCRIPT_DIR/square_t09.geojson" \
  --named-layer=timestep_10:"$SCRIPT_DIR/square_t10.geojson" \
  -o "$SCRIPT_DIR/square.pmtiles" \
  -z 8 -Z 0 \
  --no-simplification \
  --no-tile-size-limit \
  --no-feature-limit \
  -f

Run with: bash generate-tiles.sh

Step 3 — Start the server

npx serve

Step 4 — Client animation (`index.html`)

The full self-contained HTML file. No build step, no npm install — all dependencies come from esm.sh, which rewrites bare module specifiers (like pbf imported inside ol/format/MVT) to absolute CDN URLs that the browser can resolve.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>MVT Animation</title>
  <link rel="stylesheet" href="<https://cdn.jsdelivr.net/npm/ol@10.9.0/ol.css>" />
  <style>
    * { box-sizing: border-box; margin: 0; padding: 0; }
    body { font-family: monospace; background: #111; color: #eee;
           display: flex; flex-direction: column; height: 100vh; }
    #map  { flex: 1; }
    #hud  { padding: 12px 20px; background: #1a1a1a; border-top: 1px solid #333;
            display: flex; align-items: center; gap: 24px; }
    #hud label { font-size: 13px; color: #aaa; }
    #hud span  { font-size: 16px; font-weight: bold; }
    #timestep-val { color: #7af; }
    #value-val    { color: #fa7; }
    #color-swatch { width: 28px; height: 28px; border-radius: 4px;
                    border: 1px solid #555; display: inline-block; }
    #progress-bar { flex: 1; height: 8px; background: #333; border-radius: 4px; overflow: hidden; }
    #progress-fill { height: 100%; background: linear-gradient(to right, #00f, #f00); border-radius: 4px; }
    #controls { display: flex; gap: 8px; }
    button { background: #333; color: #eee; border: 1px solid #555;
             padding: 5px 14px; border-radius: 4px; cursor: pointer; font-size: 13px; }
    button:hover { background: #444; }
    #loading { position: absolute; inset: 0; display: flex; align-items: center;
               justify-content: center; background: rgba(0,0,0,0.55); color: #fff;
               font-size: 15px; pointer-events: none; z-index: 999; }
    #loading.hidden { display: none; }
  </style>
</head>
<body>
  <div id="map"><div id="loading">Loading tiles…</div></div>
  <div id="hud">
    <div><label>Timestep </label><span id="timestep-val"></span></div>
    <div><label>Value </label><span id="value-val"></span></div>
    <div style="display:flex;align-items:center;gap:8px;">
      <label>Color </label><span id="color-swatch"></span>
    </div>
    <div id="progress-bar"><div id="progress-fill" style="width:0%"></div></div>
    <div id="controls">
      <button id="btn-playpause">Pause</button>
      <button id="btn-reset">Reset</button>
    </div>
  </div>

  <script type="module">
    import Map              from '<https://esm.sh/ol@10.9.0/Map>';
    import View             from '<https://esm.sh/ol@10.9.0/View>';
    import TileLayer        from '<https://esm.sh/ol@10.9.0/layer/Tile>';
    import OSM              from '<https://esm.sh/ol@10.9.0/source/OSM>';
    import VectorTileLayer  from '<https://esm.sh/ol@10.9.0/layer/VectorTile>';
    import VectorTileSource from '<https://esm.sh/ol@10.9.0/source/VectorTile>';
    import MVT              from '<https://esm.sh/ol@10.9.0/format/MVT>';
    import Style            from '<https://esm.sh/ol@10.9.0/style/Style>';
    import Fill             from '<https://esm.sh/ol@10.9.0/style/Fill>';
    import Stroke           from '<https://esm.sh/ol@10.9.0/style/Stroke>';
    import { fromLonLat }   from '<https://esm.sh/ol@10.9.0/proj>';
    import { PMTiles }      from '<https://esm.sh/pmtiles@4.4.1>';

    const LAYER_NAMES = [
      'timestep_00', 'timestep_01', 'timestep_02', 'timestep_03',
      'timestep_04', 'timestep_05', 'timestep_06', 'timestep_07',
      'timestep_08', 'timestep_09', 'timestep_10',
    ];
    const VALUES = LAYER_NAMES.map((_, i) => i * 10);

    const STYLES = LAYER_NAMES.map((_, i) => new Style({
      fill:   new Fill({ color: `rgb(${Math.round(255*i/10)},0,${Math.round(255*(1-i/10))})` }),
      stroke: new Stroke({ color: '#fff', width: 2 }),
    }));

    const archive = new PMTiles('square.pmtiles');

    const source = new VectorTileSource({
      format: new MVT(),
      url: 'square.pmtiles',
      maxZoom: 8,
    });

    let currentLayerIndex = 0;

    const squareLayer = new VectorTileLayer({
      source,
      style: (feature) => {
        return feature.get('layer') === LAYER_NAMES[currentLayerIndex]
          ? STYLES[currentLayerIndex] : null;
      },
    });

    const map = new Map({
      target: 'map',
      layers: [ new TileLayer({ source: new OSM() }), squareLayer ],
      view: new View({ center: fromLonLat([0, 0]), zoom: 2 }),
    });

    const FRAME_MS = 500;

    function animate(frames, onFrame = () => {}) {
      let i = 0;
      let intervalId = setInterval(() => {
        currentLayerIndex = (currentLayerIndex + 1) % frames.length;
        squareLayer.changed();
        onFrame(frames[i]);
        i = (i + 1) % frames.length;
      }, FRAME_MS);

      return {
        pause()  { clearInterval(intervalId); },
        resume() { intervalId = setInterval(() => {
          currentLayerIndex = (currentLayerIndex + 1) % frames.length;
          squareLayer.changed();
          onFrame(frames[i]);
          i = (i + 1) % frames.length;
        }, FRAME_MS); },
        reset()  { i = 0; },
      };
    }

    const timestepEl   = document.getElementById('timestep-val');
    const valueEl      = document.getElementById('value-val');
    const swatchEl     = document.getElementById('color-swatch');
    const progressFill = document.getElementById('progress-fill');

    function onFrame(layerName) {
      const i = LAYER_NAMES.indexOf(layerName);
      timestepEl.textContent    = layerName;
      valueEl.textContent       = VALUES[i];
      swatchEl.style.background = STYLES[i].getFill().getColor();
      progressFill.style.width  = `${(i / (LAYER_NAMES.length - 1)) * 100}%`;
    }

    onFrame(LAYER_NAMES[0]);

    const loadingEl = document.getElementById('loading');
    let player = null;
    let started = false;

    function startWhenReady() {
      if (started) return;
      started = true;
      loadingEl.classList.add('hidden');
      player = animate(LAYER_NAMES, onFrame);
    }

    source.once('tileloadend', startWhenReady);
    map.once('rendercomplete', startWhenReady);
    setTimeout(startWhenReady, 10000);

    let playing = true;
    document.getElementById('btn-playpause').addEventListener('click', (e) => {
      if (!player) return;
      playing = !playing;
      e.target.textContent = playing ? 'Pause' : 'Play';
      playing ? player.resume() : player.pause();
    });
    document.getElementById('btn-reset').addEventListener('click', () => {
      if (!player) return;
      player.reset();
      if (!playing) {
        playing = true;
        document.getElementById('btn-playpause').textContent = 'Pause';
        player.resume();
      }
    });
  </script>
</body>
</html>

Running the full setup

# 1. Install tippecanoe (macOS)
brew install tippecanoe

# 2. Create the 11 per-timestep GeoJSON files
for i in $(seq 0 10); do
  pad=$(printf "%02d" $i); value=$((i * 10))
  echo '{"type":"FeatureCollection","features":[{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[-60,-50],[60,-50],[60,50],[-60,50],[-60,-50]]]},"properties":{"value":'$value'}}]}' \
    > square_t${pad}.geojson
done

# 3. Generate the PMTiles archive
bash generate-tiles.sh

# 4. Start the server
npx serve

# 5. Open the demo
open <http://localhost:3000/index.html>

Conclusion: paint time, not flip it

Don't stack separate tile sources for time-series animation. Bake timesteps into a single PMTiles archive as named MVT layers, load it once, and drive playback by updating a closure variable and calling layer.changed(). Serve the archive with npx serve. Start animation only after the first tile arrives.

The result: smooth playback from first load, no flicker on hard refresh, no blank map with cache disabled.

Next.js 16 Caching for E-Commerce: Cache Components, use cache, revalidateTag, and Fresh Product Data

uninterrupted — Tue, 02 Jun 2026 22:00:00 +0000

Caching in e-commerce is never just about speed. A fast storefront is useful only if it still shows the right price, the correct stock level, and the right experience for the current customer.

That is why caching in a Next.js storefront can be deceptively hard. Some data should be shared broadly and kept warm for SEO and performance. Some data should be refreshed often. Some should never be shared between users at all.

Next.js 16 gives teams a much clearer toolbox for solving this problem with Cache Components, use cache, tag-based invalidation, and explicit cache lifetime controls. Used properly, these features let you keep pages fast without drifting into stale commerce data.

In this guide, I will walk through a practical way to think about caching in a modern storefront and show how to combine use cache, cacheLife, and revalidateTag for real e-commerce use cases.

Why Caching Is Harder in E-Commerce Than in a Typical Content Site

On a standard marketing site, most content changes infrequently. If a page is cached for a few minutes or even a few hours, the business impact is usually negligible.

Commerce systems work differently.

The same product page may contain:

stable product descriptions and category copy
semi-dynamic data such as price, availability, shipping estimates, or promotion labels
private data such as cart state, recently viewed items, or customer-specific pricing

Treating all of that data the same way leads to one of two bad outcomes:

You cache too aggressively and serve stale prices or availability.
You disable caching everywhere and lose the performance benefits that help SEO and conversion.

The better approach is to split your data by volatility and audience.

The Three Cache Boundaries That Matter Most

For most commerce projects, the cleanest mental model is to divide data into three groups.

1. Stable catalog content

This is the part of the page that usually changes only when content editors or merchandisers update the catalog.

Examples:

product title and description
brand information
CMS blocks, FAQs, and long-form copy
category landing page copy
SEO metadata

This is the best candidate for shared caching because it improves performance for everyone and is usually safe to invalidate on demand when content changes.

2. Shared but fast-changing commerce data

This data is still shared between many users, but it changes more often and has stronger operational impact.

Examples:

current price for a product and region
inventory status
campaign badges
delivery promises
low-stock signals

This layer often benefits from a shorter cache lifetime and explicit invalidation.

3. User-specific or session-specific data

This is the data that should not be globally cached.

Examples:

cart contents
logged-in customer state
B2B contract pricing
personalized recommendations
account-specific discounts
checkout calculations tied to the active session

This is where teams most often make expensive mistakes. If the output depends on the current user, you need to keep that boundary private.

What Next.js 16 Changes in Practice

The major advantage of Next.js 16 is not just that it caches. It is that caching becomes more explicit.

Instead of treating an entire route as fully static or fully dynamic, you can choose more precise boundaries:

cache a component or function with use cache
assign cache tags to data and invalidate them later
control lifetime with cacheLife
keep per-user data private where needed
stream dynamic parts separately instead of forcing the entire page into request-time rendering

For e-commerce storefronts, that means you can serve a fast shell and still keep critical parts fresh.

A Practical Storefront Strategy

Let us look at a product page. In a typical headless stack, product content might come from Medusa or another commerce backend, while editorial content may come from a CMS.

The important point is not the backend itself. The important point is how you cache each layer.

Cache the stable product content

If product descriptions and media do not change every minute, cache them explicitly and tag them so they can be invalidated when merchandisers update the catalog.

import { cacheTag } from 'next/cache'

async function getProduct(slug: string) {
  'use cache'

  cacheTag(`product:${slug}`)

  const response = await fetch(
    `${process.env.COMMERCE_API_URL}/products/${slug}`
  )

  if (!response.ok) {
    throw new Error('Failed to load product')
  }

  return response.json()
}

This is a good fit for data that should be broadly reusable and revalidated only when a product actually changes.

Use a shorter cache boundary for price and stock

Price and stock need more care. In some stores, they can still be shared safely for a short period. In others, especially B2B or highly promotional environments, they may need user-level or request-level handling.

For shared price or inventory data, a short-lived cache can be a sensible compromise.

import { cacheLife, cacheTag } from 'next/cache'

async function getOffer(productId: string, regionId: string) {
  'use cache'

  cacheTag(`offer:${productId}:${regionId}`)
  cacheLife({ expire: 60 })

  const response = await fetch(
    `${process.env.COMMERCE_API_URL}/offers/${productId}?region=${regionId}`
  )

  if (!response.ok) {
    throw new Error('Failed to load offer')
  }

  return response.json()
}

Here the offer can be shared, but only briefly. That is often enough to reduce backend pressure while still keeping the storefront operationally safe.

Keep customer-specific data out of the shared cache

If pricing, recommendations, or entitlements depend on the active customer, do not treat them as generic shared data.

This applies especially to:

B2B price lists
customer group discounts
partner-only catalog visibility
personalized recommendation widgets

At that point, a private boundary or request-time fetching becomes more appropriate than a shared cache.

The principle is simple: if another customer must never see the same output, that data should not live in a shared cache.

If you do want to cache user-specific data briefly to avoid repeating the same work during a session, keep that cache private instead of shared.

import { cacheLife } from 'next/cache'

async function getRecommendations(productId: string, customerId: string) {
  'use cache: private'

  cacheLife({ expire: 60 })

  const response = await fetch(
    `${process.env.RECOMMENDATION_API_URL}/products/${productId}/recommendations?customer=${customerId}`
  )

  if (!response.ok) {
    throw new Error('Failed to load recommendations')
  }

  return response.json()
}

That keeps the result bounded to the current user context instead of letting it leak into global storefront output. It is also worth noting that 'use cache: private' does not store its cache entries on the server. The function still runs during server rendering, but the cached result is kept only in the browser’s memory and does not survive a full page reload.

Why This Matters for Medusa and Other Headless Commerce Backends

This pattern becomes especially valuable in headless commerce because the storefront is often responsible for combining multiple systems:

commerce backend
CMS
search service
recommendation engine
analytics and experiment layers

If all of those systems are fetched at request time for every page view, the frontend becomes slower and more fragile than it needs to be.

If all of them are aggressively cached together, the storefront becomes fast but operationally unsafe.

The right answer is to cache by business meaning, not by convenience.

For example:

CMS content can usually be cached until editors publish changes.
catalog copy can usually be cached and invalidated via tags.
regional shared pricing can often be cached briefly.
logged-in B2B pricing should stay private.
cart and checkout state should stay request-scoped.

That is the difference between a caching strategy that helps the business and one that only looks good in benchmarks.

Using `revalidateTag` to Keep Data Fresh

One of the most useful patterns in Next.js 16 is tag-based invalidation.

Instead of revalidating whole pages blindly, you can invalidate the exact data domain that changed. That is particularly useful when updates come from webhooks triggered by your CMS, PIM, or commerce backend.

For example, if a product is updated in Medusa or an editorial team changes CMS content linked to that product, you can invalidate the relevant tag immediately.

import { revalidateTag } from 'next/cache'

export async function POST(request: Request) {
  const payload = await request.json()

  revalidateTag(`product:${payload.slug}`, 'max')
  revalidateTag(`offer:${payload.id}:${payload.regionId}`, 'max')

  return Response.json({ ok: true })
}

In practice, this lets you keep product content cached most of the time while still reacting quickly to actual changes.

That is a much better fit for commerce than short global revalidation windows everywhere.

One operational note matters here: never expose a public revalidation endpoint without request validation. If invalidation is triggered by a CMS, Medusa, or ERP webhook, verify the source with a shared secret or signature check before calling revalidateTag.

Where Teams Usually Get It Wrong

There are a few caching mistakes that repeatedly appear in storefront projects.

Mistake 1: Caching personalized pricing globally

This is the most dangerous one. If the result depends on customer identity, customer group, contract terms, or session data, do not let it leak into a shared cache boundary.

Mistake 2: Making the whole page dynamic because one widget is dynamic

A product page does not need to become fully request-time just because the cart badge or recommendation box is personalized.

Keep the stable content cached and isolate the dynamic parts instead.

Mistake 3: Using `no-store` too broadly

no-store is sometimes necessary, but if it becomes the default reaction to uncertainty, you throw away much of the value of the App Router architecture.

Use it for truly request-bound data, not as a shortcut for unclear cache design.

Mistake 4: Forgetting business dimensions in cache keys

Prices are rarely just “product price.” They are often tied to region, currency, channel, campaign, or customer group.

If those dimensions are not reflected in the data boundary, the cache becomes unsafe.

Mistake 5: Mixing editorial freshness with operational freshness

A product description may be safe to cache until a content change occurs. Inventory and promotions may not be. These two layers should not be forced into the same lifetime.

A Good Default Model for Product Pages

If you need a practical starting point, this model works well for many storefronts:

Data type	Suggested approach
Product description, SEO copy, editorial blocks	`use cache` with tags and on-demand invalidation
Shared regional price and stock	short cache lifetime plus tags
Recommendations	private or isolated dynamic boundary
Cart state	request-time or session-specific logic
Checkout totals and payment readiness	request-time only
B2B contract pricing	private boundary, never generic shared output

This is not a framework rule. It is a business-safe baseline.

Why This Works Well With Streaming and PPR Thinking

If you have already explored Partial Prerendering in Next.js 16, this caching model should feel familiar.

The idea is the same:

keep the stable shell fast
isolate dynamic or high-volatility parts
avoid blocking the entire page for data that does not deserve that privilege

For commerce, that means a product page can still load quickly for users and search engines while the truly dynamic fragments remain fresh.

That is a much more scalable model than forcing every route into either “fully static” or “fully dynamic.”

A Note on Webhooks and Operational Ownership

Caching only works well if invalidation is treated as part of the architecture, not as an afterthought.

If your team updates product data in Medusa, content in a CMS, or prices in an ERP, your storefront should know how those changes trigger cache invalidation.

Typical patterns include:

Medusa webhooks invalidating product and offer tags
CMS publish events invalidating page or content tags
ERP sync jobs invalidating pricing-related tags
stock update events invalidating inventory-related tags only

This is where many storefronts move from “fast in development” to “reliable in production.”

Final Thoughts

The real value of caching in Next.js 16 is not that it makes everything static. It is that it helps you decide what should be shared, what should be refreshed, and what should remain private.

For e-commerce teams, that distinction matters more than raw speed numbers. The fastest storefront is not the one that caches the most. It is the one that caches the right things.

If you separate stable catalog content, shared operational data, and user-specific state into different cache boundaries, you can keep your storefront fast without risking stale or incorrect business data.

Asset-Based Data Orchestration: Lessons from Building a Multi-State Social Data Platform

uninterrupted — Thu, 26 Mar 2026 07:21:25 +0000

Building reliable data platforms rarely fails because of scale alone.
More often, reliability collapses under heterogeneity: multiple data providers, inconsistent schemas, partial updates, and unclear ownership.

While building a multi-state social data platform ingesting resource data from dozens of organizations, we discovered that reliability is not a property of pipelines. It is a property of data artifacts and their relationships.

1. Why Reliability Becomes a Systems Problem at Scale

The accompanying essay frames trust as something earned through consistent system behavior under real-world pressure. What that framing leaves implicit, but what became unavoidable in practice, is that reliability stops being a property of individual components very early on. Once multiple organizations, jurisdictions, and publishing surfaces are involved, reliability becomes an emergent property of the entire system.

For us, this meant that no single pipeline, connector, or database could be made "reliable enough" in isolation. Failures were rarely total. They were partial, localized, and often silent. The engineering challenge was not preventing all failure, but designing the system so that failures were isolated, detectable, and explainable.

That requirement drove nearly every architectural decision that followed.

2. System Overview

Technically, the platform ingests social service resource data from independent organizations operating across multiple U.S. states. Each organization exposes data via a different source system (for example iCarol, WellSky, VisionLink, RTM), with varying schemas, update cadences, and quality guarantees.

At a high level, the system consists of:

Writers - per-tenant ingestion and transformation projects that:
- Fetch raw data from source systems via connector adapters
- Persist raw data into Snowflake source schemas
- Normalize and standardize data via DBT
- Apply enhancements such as geocoding or translations
Readers - publisher processes that:
- React to completed writer runs, either bulk or incremental
- Publish curated artifacts to OpenSearch, MongoDB, and optionally Postgres

Snowflake acts as the system of record for intermediate and normalized datasets. Dagster coordinates the execution and materialization of data assets. DBT is used explicitly for set-based transformations, not orchestration.

Scale is not extreme in raw volume, but complexity is high: dozens of tenants, hundreds of tables per tenant, and frequent partial updates.

High-Level Architecture

The following diagram illustrates the writer → Snowflake → DBT → asset orchestration → readers pattern.

The platform is designed around asset lineage and normalization contracts, not pipelines.

3. Source Heterogeneity as the Dominant Constraint

The hardest constraint was not throughput or storage. It was heterogeneity.

Each data provider differed along several axes simultaneously:

Schema shape: even when nominally "the same" entities existed, fields varied
Semantics: identical fields often meant subtly different things
Update cadence: some sources updated continuously, others weekly or ad hoc
Quality guarantees: missing fields, stale records, or partial exports were common

Early on, we underestimated how strongly these differences would dominate system design. Attempting to treat ingestion as a uniform, pipeline-shaped process led to brittle assumptions and cross-tenant coupling.

The system only became manageable once heterogeneity was treated as fundamental, not incidental.

4. Normalization (HSDS, SDOH or Equivalent) as an Architectural Contract

Normalization into an HSDS-like model was not implemented as a downstream convenience. It became an architectural contract.

All downstream consumers, internal and external, implicitly rely on the guarantees of the normalized model: stable fields, predictable relationships, and documented semantics. That meant normalization could not be "best effort" or delayed until the end of a pipeline.

In practice:

Raw source data is written verbatim into Snowflake source schemas
DBT ELT projects transform this raw data into standardized intermediate models
DBT STAGE projects apply tenant-specific adaptations while preserving the contract

This separation made it explicit where interpretation happens. If a field is wrong in the normalized model, the question becomes which contract was violated, not "what broke in the pipeline".

5. What We Got Wrong Initially

Several early assumptions did not survive contact with reality.

Pipeline-first thinking

We initially modeled work as long-running jobs. This obscured which intermediate datasets were durable, reusable, or safe to depend on. Debugging often meant rerunning more than necessary.

Manual validation

Data quality checks lived outside the orchestration layer. Engineers and analysts manually inspected outputs, which worked at small scale but failed under concurrency and time pressure.

Shared failure domains

Multiple tenants often shared execution paths. A failure in one tenant’s ingestion could block or delay others, even when their data was unrelated.

None of these issues were catastrophic individually. Together, they made reliability depend on human attention.

6. Transitioning to Asset-Based Data Orchestration with Dagster

The shift to asset-based orchestration was driven less by tooling preference and more by a change in mental model.

Instead of asking "what jobs should run?", we started asking:

What data artifacts must exist?
What do they depend on?
How fresh do they need to be?
What constitutes success or failure for this artifact?

Dagster assets provided a way to encode those questions directly.

A simplified example from a writer project shows how DBT models are treated as assets rather than opaque steps:

# writer-xyz/assets.py (excerpt)
from dagster_dbt import load_assets_from_dbt_project

dbt_elt_assets = load_assets_from_dbt_project(
    project_dir="dbt_elt",
    profiles_dir="dbt_elt",
)

This does not explain how DBT runs. It declares that the resulting tables are first-class assets with lineage and state.

Once assets replaced jobs as the primary abstraction, freshness, lineage, and partial recomputation became explicit rather than implicit.

7. Partitioning for Failure Isolation

Partitioning was critical for isolating failures.

We partitioned primarily along tenant and state boundaries, not time. This reflected operational reality: data issues almost always affected a single organization or region.

In Dagster terms, this meant:

Separate writer projects per tenant
Independent schedules and sensors
Asset materializations scoped to a tenant’s data domain

A failure in one writer no longer blocked publishing for others. More importantly, remediation could be targeted and auditable.

8. Data Quality Embedded in the Asset Graph

Data validation moved into the asset graph itself.

Instead of post-hoc checks, validations became explicit dependencies. If a validation asset failed, downstream assets simply did not materialize.

An example pattern used across writers:

@asset
def validate_staging_tables(staging_tables):
    assert staging_tables.count_missing_ids() == 0

This is intentionally simple. The key point is not the check itself, but that failure is structural. The system records that an expected artifact does not exist, rather than silently publishing bad data.

This shifted failure detection earlier and reduced the blast radius of errors.

9. Operational Outcomes

Day-to-day operations changed in several concrete ways:

On-call work shifted from rerunning pipelines to inspecting asset lineage
Partial backfills became routine rather than exceptional
Publishing delays were easier to attribute to specific upstream causes
New tenants could be added without increasing shared operational risk
None of this eliminated operational effort. It made that effort more focused and less reactive.

10. Open Trade-offs and Unresolved Questions

Some challenges remain unresolved:

Cross-tenant schema evolution still requires coordination and discipline
Observability across Snowflake, DBT, Dagster, and downstream stores is fragmented
Cost attribution at the asset level is still coarse-grained
Human review remains necessary for certain semantic validations

With more time, we would invest earlier in unified observability and more formal schema versioning.

11. Why These Lessons Matter Beyond This Platform

These lessons are not unique to this system.

Any platform operating in civic tech, govtech, or environmental data shares similar constraints: multiple data producers, uneven quality, and real-world consequences for failure.

The core takeaway is not "use asset-based orchestration", but treat data artifacts as obligations. Once that shift happens, many architectural decisions become clearer.

Reliability stops being something you hope for and becomes something you can reason about.

Final Thoughts

The biggest lesson from this platform was not about any particular tool. It was about how we model the system itself.

Once data artifacts became the core abstraction, many reliability problems became easier to reason about. Failures became visible, dependencies became explicit, and operational work shifted from firefighting pipelines to managing data contracts.

For data platforms operating across heterogeneous sources, this shift can be the difference between a system that merely runs - and one that can be trusted.

FAQ

What is asset-based data orchestration?

Asset-based data orchestration treats data artifacts (tables, datasets, models) as the primary units of orchestration instead of pipelines or jobs. Systems like Dagster allow teams to define dependencies between assets, enabling better lineage tracking, partial recomputation, and failure isolation.

Why is asset-based orchestration useful for complex data platforms?

In large systems with many data producers and consumers, failures are rarely binary. Asset-based orchestration makes dependencies explicit, allowing teams to:

isolate failures
recompute only affected datasets
track lineage across transformations
enforce data quality checks before publishing

How does Dagster differ from traditional pipeline orchestrators?

Traditional orchestrators schedule jobs or workflows. Dagster emphasizes data assets and lineage. This enables better visibility into:

what datasets exist
what produced them
what depends on them
whether they meet freshness or quality requirements

When should teams adopt asset-based orchestration?

Asset-based orchestration becomes particularly valuable when systems have:

many data sources
heterogeneous schemas
multiple downstream consumers
partial or incremental updates
strict reliability requirements

These conditions are common in multi-tenant data platforms and civic tech systems.

Does asset-based orchestration replace tools like DBT?

No. Asset-based orchestration complements transformation tools like DBT. In many architectures:

DBT performs set-based transformations
Dagster manages asset lineage, dependencies, scheduling, and validation

This separation keeps orchestration and transformation responsibilities clear.

Next.js 16 Partial Prerendering (PPR): The Best of Static and Dynamic Rendering

uninterrupted — Thu, 05 Mar 2026 08:21:46 +0000

Next.js has long been a leader in giving developers flexible, high-performance rendering strategies - Static Site Generation (SSG), Server-Side Rendering (SSR), Incremental Static Regeneration (ISR), and Client-Side Rendering (CSR) all play roles depending on your use case. With Next.js 14, Partial Prerendering (PPR) was introduced as an experimental feature, starting from Next.js 16, PPR has become a stable and becomes a foundational part of this landscape by letting you blend static pre-rendering and dynamic behavior in the same route, improving perceived performance and SEO without sacrificing real-time data needs.

What Is Partial Prerendering (PPR)?

Partial Prerendering is a rendering strategy that lets Next.js deliver a static HTML “shell” immediately, then stream in dynamic content as it becomes available - all in the same server response. Instead of choosing static or dynamic at the page level, PPR combines both on a per-component basis.

The static shell (layout, logo, product titles, navigation) is pre-generated at build time or cached ahead of requests.
Dynamic parts (cart state, recommendations, session data) load later and are streamed to the client using React’s <Suspense> boundaries.
Users see meaningful UI immediately, with dynamic content filling in seamlessly.

This leads to faster Time to First Byte (TTFB) and a smoother perceived experience, while still allowing data personalization and real-time updates.

How Partial Prerendering Works in Next.js 16

Cache Components: The Heart of PPR

In Next.js 16, PPR isn’t just an experimental flag - it’s integrated into the Cache Components system. Cache Components lets you opt-in to cache certain components or data instead of rendering them on every request, making pre-rendering more predictable and performant.

cacheComponents: true in next.config.ts enables PPR and component-level caching.
The new "use cache" directive lets you mark data-fetching functions or components as cacheable.
Anything not cacheable or depending on request-specific data can be wrapped in a <Suspense> fallback.

This approach gives you fine-grained control: static UI is reused across requests, while dynamic parts rehydrate or stream in only when needed.

Streaming and Suspense: The Engine Under the Hood

React’s <Suspense> plays a key role in PPR:

Wrap dynamic components in <Suspense> with a fallback UI (like a skeleton loader).
During rendering, Suspense tells Next.js where to halt pre-rendering and stream dynamic content later.
The server sends the pre-rendered shell immediately and then streams dynamic sections in parallel as they’re ready.

This strategy avoids blocks in page delivery and reduces the “white screen” effect often seen in traditional SSR or CSR approaches.

Why PPR Matters for E-commerce

For e-commerce platforms, performance directly impacts sales and SEO - factors also emphasized in the U11D articles about rendering strategies. A slow page can hurt conversions and search rankings. Partial Prerendering improves this by:

Faster initial loads: Users see the page skeleton instantly, improving engagement and performance metrics.
SEO advantages: Static content is available for crawler indexing immediately.
Dynamic personalization: Cart contents, recommendations, user-specific prices or availability can appear without blocking the initial render.
More flexible than SSG/SSR alone: You’re not limited to fully static or completely dynamic pages - the best of both lives together.

In e-commerce apps where product detail pages might be static for most users but show personalized recommendations or inventory status, PPR is a compelling hybrid.

How to Enable and Use PPR in Next.js 16

Enable Cache Components:

In your next.config.ts:

import type { NextConfig } from 'next';

const nextConfig: NextConfig = {
  cacheComponents: true, // enables Partial Prerendering
};

export default nextConfig;

Wrap dynamic UI:

Use React’s Suspense for dynamic content:

import { Suspense } from 'react';

export default function ProductPage() {
  return (
    <>
      <Header />
      <Suspense fallback={<div>Loading product...</div>}>
        <ProductDetails />
      </Suspense>
    </>
  );
}

Use use cache for predictable dynamic data:

Inside dynamic data functions, prefix with "use cache" to mark them cacheable:
```
'use cache';
const product = await getProduct(id);
```
This tells Next.js that caching is safe for that data.

How PPR Fits with Other Rendering Strategies

Partial Prerendering doesn’t replace SSR, SSG, ISR, or CSR - but coordinates with them. While U11D’s articles give a great overview of traditional strategies like SSG, SSR, and ISR, PPR adds a hybrid strategy that sits between:

SSG/ISR: Good for full static pages or pages that regenerate occasionally.
SSR: Ideal for real-time personalized content.
CSR: Great for highly interactive client-heavy UI.
PPR (Next.js 16): Combines static cached shells with streaming dynamic content.

Think of PPR as a component-level SSG + dynamic hybrid - fast to load like SSG, flexible like SSR.

Wrapping Up

Next.js 16 elevates Partial Prerendering from an experimental concept to a practical, integrated performance strategy through Cache Components and React Suspense. It’s especially powerful for complex, dynamic sites like e-commerce stores, where you want the benefits of static pre-rendering and dynamic personalization without sacrificing UX or SEO.

By serving instantly usable HTML and streaming dynamic parts in parallel, PPR bridges the traditional divide between static and dynamic rendering - helping your app feel faster while staying robust and scalable.

If you’re building with Next.js 16, definitely explore PPR alongside other strategies like SSR, ISR, and CSR to find the most performance-optimized combination for your routes.

Using Proxy (before Middleware) in Next.js: a modern layer

uninterrupted — Wed, 26 Nov 2025 07:00:00 +0000

Next.js 16 introduces a new file-convention: proxy.js / proxy.ts (often “Proxy”) which supersedes the older middleware.js. This feature allows you to intercept HTTP requests early (before routing/rendering) and run custom logic—redirects, rewrites, header manipulation, or forwarding (proxying) to another service.

In e-commerce (or any frontend/back-end scenario) this gives you a powerful tool: you can make your Next.js app act as a Backend-for-Frontend (BFF) layer, hiding upstream APIs, consolidating micro-services, enforcing auth, caching, rewriting URLs, etc. This aligns well with many of the architectures discussed in your referenced articles about Next.js for commerce (SSG/ISR/SSR strategies etc).

What is Proxy and when to use it

The Proxy file (proxy.js/.ts) in Next.js is a special server-side file (located in the project root or under src) that runs before any route handling or page rendering occurs. In fact, Proxy (and Middleware) executes before every incoming request — not just those for pages or APIs — including requests for assets like images, icons, or JavaScript files. Because of this, Next.js provides the matcher configuration, allowing you to specify which routes the Proxy should apply to and avoid intercepting unnecessary requests.

It allows you to inspect the incoming request, then respond, redirect, rewrite, or modify headers based on your application’s needs.

Relation to Middleware & Rewrites

In previous versions, middleware.js (App Router) was used for this pattern; Proxy replaces it.
While rewrites() in next.config.js can act like a proxy by mapping URL paths to different destinations, the Proxy feature goes further — it gives you direct access to the request object itself, including headers, cookies, and other request data.

When to use Proxy

Proxy works best in scenarios that require lightweight, real-time adjustments to incoming requests — such as performing quick redirects based on request details, dynamically rewriting routes for A/B testing or experimental features, or modifying request and response headers across specific routes or the entire application.

However, it’s not well-suited for tasks involving heavy data fetching or complex session management, as those are better handled by dedicated API routes or backend services.

Relation to Backend-for-Frontend (BFF) pattern

In the BFF pattern, the frontend has a dedicated backend layer (the “for the frontend”) that aggregates, transforms, or proxies requests to multiple microservices. Next.js supports this pattern via Route Handlers, Proxy and server actions.

Setting up Proxy: conventions & example

File convention

Create a proxy.ts (or proxy.js) file in the project root or inside src/ alongside your top‐level app/ or pages/folder.
Only one proxy file per project is supported—though you can modularize logic within it by importing helpers.
Export a function named proxy (or default) that receives a NextRequest and returns a NextResponse.
Optionally export a config object with a matcher property to specify which routes this proxy applies to.

Basic example

// proxy.ts
import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';

export function proxy(request: NextRequest) {
  // simple redirect
  return NextResponse.redirect(new URL('/home', request.url));
}

export const config = {
  matcher: '/about/:path*',
}

This intercepts /about/* and sends users to /home.

Proxying to different pathname

// proxy.ts
import { NextResponse } from 'next/server';

export function proxy(request: Request) {
  if (request.nextUrl.pathname === '/v1/docs') {
    request.nextUrl.pathname = '/v2/docs'
    return NextResponse.redirect(request.nextUrl);
  }
}

This way /v1/docs in your Next.js app becomes a proxy to your v2 version.

Important flags & matching

matcher can be a string, array of strings, or objects with source, has, missing, regexp for fine control.
You should exclude internal Next.js paths such as /_next/, static assets etc, to avoid intercepting everything accidentally.
Be aware of runtime: The Proxy is executed by default in the Edge Runtime environment; performance matters.

Use cases & how it fits e-commerce / frontend/back-end scenarios

Use case 1: Authentication / session gating before forwarding

Before forwarding to upstream API, you can inspect cookies/headers in Proxy, validate session, add or strip auth headers, or reject unauthorized requests early. This centralises “frontend backend” logic.

For example:

if (!request.cookies.get('sessionToken')) {
  return NextResponse.redirect('/login')
}

Use case 2: A/B testing, feature flags, routing to different backends

E-commerce often runs experiments: you might want to route certain requests to a “new checkout backend” or “legacy order API” based on a feature flag. Proxy allows dynamic routing:

if (featureFlag.isEnabled('newCheckout') && pathname.startsWith('/checkout')) {
  return NextResponse.rewrite(new URL('/new-checkout' + rest, request.url))
}

This is exactly where Proxy shines, beyond simple rewrites. The docs mention this scenario.

Use case 3: Multi-zone / micro-frontend routing

If you break your site into zones (e.g., /shop, /blog, /dashboard) and each is a separate Next.js app or separate backend service, you can use Proxy to route to the correct service based on path or condition. The docs mention this under Multi-Zones: “Proxy can also be used when there is a need for a dynamic decision when routing.”

How this complements SSG/ISR/SSR strategies

In your e-commerce context (as your referenced articles explore SSG/ISR/SSR), Proxy supports architecture by giving you control over how client-requests hit your backend:

If you statically generate product pages (SSG/ISR) but still need dynamic APIs behind the scenes (inventory, pricing), Proxy sits in front of those APIs.
If you use SSR for checkout or account pages, you might fetch via your BFF rather than directly from client; Proxy gives you that layer.
This allows you to decouple the frontend build (Next.js) from the backend services, while still providing unified domain, consistent cookies/sessions, and reduction of client-side complexity.

Summary

The Proxy file convention in Next.js is a modern, flexible tool for intercepting HTTP requests early in your application lifecycle. It’s especially valuable in setups where your frontend is part of a larger ecosystem (microservices, e-commerce, feature-flagged experiments) and you need a lightweight backend-for-frontend layer without building a full custom server.

For e-commerce platforms built with Next.js—where you may have static product pages, dynamic inventory/pricing APIs, customer sessions, checkout flows—the Proxy gives you an elegant place to manage domain-consolidation, routing, auth, A/B experimentation and API facade logic.

However, you should use it wisely: keep logic lean, use it for the right cases (redirects, rewrites, header manipulation, request forwarding) rather than heavy data aggregation. Combined with Next.js’s rendering strategies (SSG, ISR, SSR) and BFF pattern, Proxy becomes an important architectural building block.

Automated Tax Compliance for Global E-Commerce Growth

uninterrupted — Mon, 17 Nov 2025 08:06:29 +0000

Choosing an e-commerce platform today means choosing your capabilities for the next 5-10 years. Most businesses focus on features and design, overlooking a critical question: can this platform handle tax compliance as we grow, regulations change, and new sales channels emerge?

Three Compliance Challenges That Break Legacy Platforms

Real-time performance matters more than you think. Every millisecond at checkout affects conversion. Legacy tax calculation systems add 300-500ms latency per request — enough to impact sales. Worse, they often fail silently during high-traffic periods, forcing you to choose between lost sales or compliance risk.

Headless architecture becomes critical as your business evolves. Selling through mobile apps, marketplaces, social commerce, and IoT devices means your tax system needs API-first design. Shopify's monolithic architecture forces you into their frontend. WooCommerce wasn't built for headless operations. Magento's complexity makes omnichannel integration expensive.

Audit readiness isn't optional. Tax authorities increasingly audit e-commerce businesses, and "we did our best" isn't a defense. You need complete transaction history, documentation of every tax decision, and proof that calculations were accurate at transaction time. Most platforms treat this as an afterthought, storing minimal data that won't satisfy auditors.

The Performance-Compliance Tradeoff (That Shouldn't Exist)

Avalara's real-time API calculates taxes in under 100ms, handling peak traffic without degradation. This isn't just about speed — it's about reliability. During Black Friday traffic spikes, tax calculations don't become a bottleneck.

Combined with Medusa's modern architecture, you get:

Sub-second checkout experiences with real-time tax accuracy
Async tax calculation options for ultra-high-volume scenarios
Automatic failover and retry logic for maximum reliability
Complete transaction logging for audit trails

The Technical Foundation Matters

Medusa's API-first, headless design means tax compliance works consistently across every channel. Selling through your website, mobile app, and Amazon? Same tax logic, same accuracy, same audit trail — without custom integration for each channel.

This architecture also enables sophisticated business rules:

Customer-specific tax exemptions applied automatically across all channels
Dynamic tax calculation based on real-time inventory location
Split fulfillment from multiple warehouses with correct origin-based taxation
International expansion without rebuilding your tax infrastructure

Building Audit-Ready From Day One

When tax authorities audit your business, they want answers to specific questions:

How did you determine tax rates for each transaction?
Where are exemption certificates for tax-exempt sales?
How do you handle returns and refunds?
What's your documentation for nexus determination?

The Medusa-Avalara integration maintains complete records automatically:

Every tax calculation with jurisdiction breakdown
Customer exemption certificates with validation status
Transaction lifecycle (calculated → committed → voided/refunded)
Address validation records proving due diligence

You're not preparing for audits — you're always ready. Export complete documentation on demand, proving compliance for every transaction.

The Platform Investment Decision

Legacy platforms force tradeoffs: Shopify is easy but inflexible. Magento is powerful but expensive and complex. WooCommerce is customizable but scales poorly.

Medusa eliminates these tradeoffs: open-source flexibility, modern architecture, enterprise capabilities without enterprise costs. When paired with Avalara's tax automation, you get infrastructure that grows with your business.

Scale Without Technical Debt

Most e-commerce businesses outgrow their first platform. They've built workarounds for limitations, accumulated technical debt, and face expensive migration projects. This is avoidable.

Starting with proper infrastructure—headless architecture, real-time tax automation, complete compliance documentation—means your platform grows with you instead of holding you back.

Tax compliance isn't exciting, but it's foundational. Get it right from the start, and it becomes invisible infrastructure. Get it wrong, and it becomes an expensive barrier to growth.

Need help with Avalara integration?

Looking to implement automated tax compliance with Avalara in your Medusa store? u11d specializes in e-commerce integrations and tax automation solutions. We can help you set up, customize, and optimize your Avalara integration for maximum compliance and efficiency. Our team has extensive experience with both Medusa e-commerce platforms and Avalara tax systems, ensuring seamless integration that meets your specific business requirements.

A Practical Guide to Scaling Medusa with Kubernetes Autoscalers

uninterrupted — Wed, 29 Oct 2025 09:56:36 +0000

As your Medusa.js e-commerce platform grows, performance and reliability depend on how well it scales under load. Kubernetes provides native tools like the Horizontal Pod Autoscaler (HPA) and KEDA to automatically adjust resources based on real-time demand.
In this guide, you’ll learn how to configure Medusa for horizontal scaling in Kubernetes, using Prometheus, cAdvisor, and HPA - ensuring your store remains responsive even during peak traffic periods.

Prerequisites for Medusa Autoscaling on Kubernetes

Before implementing autoscaling, ensure that your monitoring and metric systems are in place.

Required Tools: cAdvisor, Prometheus, and KEDA

To make HPA and KEDA work efficiently, you’ll need:

cAdvisor – collects container-level CPU and memory metrics.
Prometheus – scrapes, stores, and visualizes time-series metrics.
Prometheus Adapter or KEDA – exposes those metrics to the HPA.

This setup is essential to achieve a reliable, responsive scaling mechanism that monitors pod and container utilization with fine-grained metric resolution (ideally 1 second or less).

Setting Up Metrics for HPA

Configuring cAdvisor for container-level metrics

cAdvisor (Container Advisor) is a running daemon that provides per-container resource usage data. It collects and exports information about all containers running on a host.

Key configuration tips:

Avoid collecting all metrics to reduce resource consumption.
Adjust collection intervals according to your scaling sensitivity.

  - --allow_dynamic_housekeeping=false
  - --housekeeping_interval=1s
  - --max_housekeeping_interval=2s

These flags ensure you get up-to-date metrics while keeping overhead low.

Prometheus Configuration for Scraping cAdvisor Data

Prometheus collects metrics from cAdvisor and stores them for HPA or KEDA to consume. Below is a sample configuration to scrape and relabel container metrics efficiently:

scrape_configs:
  - job_name: cadvisor
    scrape_interval: 1s
    static_configs:
      - targets: ["cadvisor.cadvisor.svc.cluster.local:8080"]
    metric_relabel_configs:
      - source_labels: [container_label_io_kubernetes_pod_namespace]
        target_label: namespace
      - source_labels: [container_label_io_kubernetes_pod_name]
        target_label: pod
      - source_labels: [container_label_io_kubernetes_container_name]
        target_label: container
      - source_labels: [container_label_io_kubernetes_pod_node_name]
        target_label: node
      - regex: container_label_.*
        action: labeldrop

Tip: Use a 1-second scrape interval for real-time scaling accuracy, but monitor your Prometheus load — frequent scrapes can impact cluster performance.

Using Prometheus Adapter to Expose Custom Metrics

Prometheus Adapter acts as a bridge between Prometheus and the Kubernetes HPA. It translates Prometheus metrics into Kubernetes-readable custom metrics that HPA can act upon.

Example configuration for CPU and memory metrics:

resource:
  cpu:
    containerQuery: |
      sum by (<<.GroupBy>>) (
        rate(container_cpu_usage_seconds_total{container!="",<<.LabelMatchers>>}[5s])
      )
    nodeQuery: |
      sum  by (<<.GroupBy>>) (
        rate(node_cpu_seconds_total{mode!="idle",mode!="iowait",mode!="steal",<<.LabelMatchers>>}[3m])
      )
    resources:
      overrides:
        node:
          resource: node
        namespace:
          resource: namespace
        pod:
          resource: pod
    containerLabel: container
  memory:
    containerQuery: |
      sum by (<<.GroupBy>>) (
        avg_over_time(container_memory_working_set_bytes{container!="",<<.LabelMatchers>>}[5s])
      )
    nodeQuery: |
      sum by (<<.GroupBy>>) (
        avg_over_time(node_memory_MemTotal_bytes{<<.LabelMatchers>>}[3m])
        -
        avg_over_time(node_memory_MemAvailable_bytes{<<.LabelMatchers>>}[3m])
      )
    resources:
      overrides:
        node:
          resource: node
        namespace:
          resource: namespace
        pod:
          resource: pod
    containerLabel: container

This configuration exposes per-pod resource metrics that HPA can use to make scaling decisions.

Scaling Medusa with KEDA

How KEDA Integrates with Kubernetes HPA

KEDA (Kubernetes Event-Driven Autoscaler) enhances Kubernetes autoscaling by allowing scaling based on external events - for example, message queue depth, API requests, or Prometheus metrics.

KEDA works alongside HPA to provide fine-grained, event-driven scaling for your Medusa backend.

Below is a sample configuration for scaling Medusa using KEDA with Prometheus as the metric source:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: medusa-backend
spec:
  scaleTargetRef:
    name: medusa-backend
  pollingInterval: 1
  cooldownPeriod: 60
  minReplicaCount: 3
  maxReplicaCount: 10
  triggers:
    - type: prometheus
      metricType: AverageValue
      metadata:
        serverAddress: <http://prometheus-server.default.svc.cluster.local:80>
        metricName: avg_cpu_usage
        threshold: "0.5"
        query: |
          sum by (pod) (rate(container_cpu_usage_seconds_total{pod=~"medusa-backend-.*", container!=""}[5s]))

This YAML defines a ScaledObject that dynamically adjusts Medusa’s replica count based on CPU usage metrics scraped by Prometheus.

Fine-Tuning Kubernetes HPA

kube-controller-manager Parameters for Faster Scaling

The kube-controller-manager is a core Kubernetes component that controls how quickly HPA reacts to metric changes.

By tuning its parameters, you can make scaling nearly instantaneous - for example, achieving a 5-second response time to CPU utilization spikes.

Ensure you’ve optimized:

--horizontal-pod-autoscaler-sync-period=5s

Lowering this interval makes the autoscaler evaluate conditions more frequently, but note that some managed Kubernetes distributions may restrict access to these flags.

Visualizing Scaling with Grafana

Grafana dashboards can help you track CPU utilization, pod counts, and scaling behavior in real time.
Below is an example visualization showing how HPA scales pods in and out based on CPU load.

This visualization also highlights the importance of minimizing container startup and readiness probe times, which directly affect how quickly new replicas become active.

Best Practices and Common Pitfalls

Optimize Startup Times and Readiness Probes

Ensure Medusa starts and becomes ready as quickly as possible.
Use readiness probes to signal when pods can receive traffic.
Long startup times can delay scaling and degrade performance.

Selecting the Right Scaling Metrics

CPU utilization is common, but not always optimal.
Consider custom business metrics (e.g., requests per second, queue depth).

Dealing with HPA Sync Delays

The default HPA sync period is 15 seconds, which might be too slow for bursty workloads.
Decreasing it improves responsiveness but can increase API traffic and controller load.

Conclusion

Implementing HPA and KEDA for your Medusa.js e-commerce platform ensures efficient scaling, better performance under load, and optimal resource utilization.

Key takeaways:

Always configure readiness probes and optimize startup times.
Choose scaling metrics carefully - CPU isn’t always the best indicator.
Monitor your autoscaler’s responsiveness using Grafana and Prometheus.
Test your setup under realistic load scenarios to validate scaling behavior.

With the right configuration, Kubernetes can make your Medusa deployment both resilient and self-scaling, ensuring you’re always ready for traffic spikes.

FAQ: Scaling Medusa in Kubernetes

Q1: What is the best way to scale Medusa on Kubernetes?

A1: Use Kubernetes HPA with metrics from Prometheus and cAdvisor, or event-driven scaling through KEDA for more flexibility.

Q2: Does Medusa support autoscaling natively?

A2: Medusa itself doesn’t manage scaling, but it runs well in Kubernetes environments that use HPA or KEDA.

Q3: Which metrics should I track for autoscaling?

A3: Start with CPU and memory usage; consider adding metrics like request rate, queue size, or API latency for advanced control.

Q4: How can I monitor Medusa’s scaling performance?

A4: Integrate Prometheus with Grafana dashboards to visualize pod utilization and replica changes in real time.

Q5: What’s the difference between HPA and KEDA?

A5: HPA relies on internal Kubernetes metrics, while KEDA extends it to external sources like Prometheus queries or event systems.

Missing files in your Packer built image? You might be skipping graceful shutdowns

uninterrupted — Thu, 23 Oct 2025 11:04:49 +0000

Are your files and folders missing after you run the newly created image? This is a common issue when building images with Packer. It is a widely used tool for creating machine images in a repeatable and automated way. When using the QEMU builder to generate local or CI-friendly images, it's common to upload files with the file provisioner and configure systems using shell provisioner. One often overlooked but critical part of this workflow is how the virtual machine shuts down after provisioning. Without a proper shutdown, the resulting image can be unstable, incomplete, or even unbootable.

Let’s see why graceful shutdowns matter.

After provisioning completes, Packer snapshots the virtual disk to produce the final image. If the guest operating system is not properly shut down before this snapshot, you risk file system corruption, lost configuration changes, services failing to start, or persistent files missing entirely - especially those written during the final steps of provisioning. These issues often go unnoticed until the image is deployed in staging or production environments, where the consequences are far more disruptive.

By default, the shutdown_command is set to an empty string (""). This means Packer does not attempt a clean shutdown unless you explicitly configure one. Instead, the virtual machine is forcefully terminated, which is equivalent to abruptly cutting power on a physical machine - a common cause of data loss and image inconsistency.

To prevent this, you must define a shutdown_command that instructs the OS to shut down safely after provisioning. This ensures that services have time to stop, file operations complete, and the system reaches a stable state before the image is captured.

Correct usage of `shutdown_command`

A clean shutdown requires a properly formed command with the necessary privileges, and the exact command you use depends on the operating system inside the virtual machine. Most Linux distributions support the shutdown command, but the syntax, required privileges, and available tools can vary slightly.

Here’s a reliable example in HCL2 for a Debian/Ubuntu based image:

...
shutdown_command = "echo 'packer' | sudo --stdin shutdown --poweroff now"
...

This command assumes the use of a user with password-based sudo access. It triggers a privileged system shutdown, allowing services to stop cleanly and file buffers to flush before power-off. The --poweroff flag ensures the machine actually powers off, which is essential for clean termination when using virtual hardware emulation. Adjust the password to match your base image setup.

If you're using a different base image - such as CentOS, RHEL, or Alpine - the shutdown binary might be located elsewhere, require different flags, or use an alternative like poweroff or halt. You may also need to configure sudoers to avoid prompts or password issues during provisioning.

Always test the shutdown command interactively inside a VM before inserting it into your template. It should work silently and reliably in the automated provisioning context.

Conclusion

To sum up, graceful shutdowns are a necessary safeguard when building machine images with the QEMU builder in Packer. Skipping this step or relying on Packer’s default behavior (which can forcibly terminate the VM) introduces the risk of unstable and corrupted images. By using a proper shutdown_command, you ensure the system shuts down in a controlled way, preserving the integrity of the file system and producing a clean, bootable image.

This practice is simple to implement and greatly improves reliability in both development and production environments. If you're managing CI pipelines or building images for downstream automation, it's a foundational step that prevents subtle and costly failures later in the lifecycle.

Headless CMS in E-Commerce: How to Integrate Medusa for Scalable, Content-Rich Online Stores

uninterrupted — Wed, 01 Oct 2025 08:03:00 +0000

Modern e-commerce isn’t just about selling products; it’s about delivering dynamic, content-driven experiences across multiple channels. To achieve this, businesses lean on Content Management Systems (CMS) for structured, scalable content workflows. With the emergence of headless commerce engines like Medusa.js and modern framework like Next.js, integrating a CMS has become more streamlined and customizable than ever.

Defining CMS in the E-Commerce Stack

A Content Management System (CMS) is a tool that lets you create, edit, and organize website content without touching code.

In e-commerce terms:

It separates content (product descriptions, blog posts, banners) from your storefront (the part customers see and shop in).
Editors and marketers can add or update pages in minutes without breaking anything.
Developers don’t have to be “content gatekeepers” — the CMS is the safe workspace for all non-technical updates.

Think of it as Google Docs for your website content, but integrated into your online store.

Medusa.js: Architecture and Integration

Medusa.js is an open-source, Node.js-based headless commerce engine. Unlike traditional monolithic platforms (e.g. Magento, Shopify), Medusa.js separates the commerce backend from the frontend, exposing commerce logic through APIs. This architecture supports integration with any frontend framework (e.g. Next.js) or CMS.

Why Pair Medusa.js with a CMS?

Medusa.js powers the transactional side of your storefront. A CMS adds that storytelling layer, and storefront (e.g. Next.js) is the perfect glue that pulls both content and commerce into a seamless experience.

Examples:

Blog posts — CMS stores articles, Next.js fetches them alongside Medusa product data for a fully integrated shopping-and-reading experience.
Landing pages — Launch seasonal campaigns in the CMS; Next.js dynamically renders them with real-time Medusa product data.
Rich product descriptions — Pull long-form text, videos, and extra images from the CMS while keeping inventory and pricing from Medusa.
SEO improvements — Use Next.js server-side rendering (SSR) or static site generation (SSG) to ensure Google sees your content instantly.

Key Decision Factors When Choosing a CMS

When selecting a CMS to pair with Medusa, consider the following trade-offs:

Cloud (SaaS) vs. Self-Hosted
- Cloud (SaaS): Ideal when your scale is small to medium or when you lack in-house DevOps expertise. Fast to set up, minimal maintenance, and predictable subscription costs. However, you trade off some control, and expenses can grow significantly as traffic, content volume, or user count increases.
- Self-Hosted: Full control and deep customization, with the potential for lower raw hosting costs in the long term. But infrastructure setup, deployments, scaling, security, and ongoing maintenance require time and expertise — which translates directly into operational cost.
Developer Friendliness vs. Ease of Use for Editors
- Some CMSs (like Payload) are highly flexible and API-driven, offering developers full control but requiring training for non-technical editors.
- Others (like Strapi) strike a balance, with strong developer features and a user-friendly admin panel.
- Platforms like Sanity and Contentful lean heavily toward editor experience, prioritizing usability and real-time collaboration.
Community and Plugins
- A strong community speeds up problem-solving and provides a library of ready-to-use plugins.
- For example, Strapi’s plugin ecosystem includes tools for image optimization, role-based permissions, analytics, and more.

CMS Selection Matrix for Medusa.js Integration

CMS	Type	Key Features	Best For
Strapi	Open-source (self-hosted or cloud)	Customizable, large OSS community, plugin ecosystem	Teams wanting balance between dev control and editor usability
Payload	Open-source (self-hosted or cloud)	Developer-centric, clean editor, powerful APIs	Dev-heavy teams building tailored workflows
Sanity	Headless SaaS	Real-time collaboration, content portability	Content-heavy teams with remote collaboration needs
Contentful	Headless SaaS	Enterprise-ready, robust APIs, scalable	Large organizations needing stability and global delivery

Example Implementation: Medusa + Strapi + Next.js

Medusa.js: Manages catalog, checkout, orders, and authentication workflows.
Strapi: Controls homepage banners, blog content, landing pages, and SEO metadata.
Next.js: Consumes APIs from both Medusa and Strapi to render the frontend.

This decoupled structure allows content managers and developers to work in parallel, ensuring rapid deployment and updates across both commerce and content layers.

Conclusion

Combining a headless CMS with Medusa.js and modern storefront framework enables highly customizable, scalable, and performant e-commerce platforms. This architecture decouples content and commerce, allowing independent optimization and rapid iteration. As the industry moves toward open, API-driven systems, Medusa.js and compatible CMS platforms provide a robust foundation for next-generation digital commerce.

DEV Community: uninterrupted

Achieving Instant Navigations in Next.js 16.3

Why Dynamic Next.js Navigations Can Feel Slow

Enabling Instant Navigations in Next.js 16.3

The Core Model: Stream, Cache, or Block

1. Stream dynamic content with Suspense

2. Cache reusable content with 'use cache'

3. Allow the route to block

How Partial Prefetching Changes the Network Cost

A Practical E-Commerce Example

Finding Slow Routes with Instant Insights

Testing Instant Navigations with Playwright

Where Teams Usually Get It Wrong

Mistake 1: Treating “instant” as “fully loaded”

Mistake 2: Placing Suspense too high in the tree

Mistake 3: Reading dynamic data in a shared layout

Mistake 4: Enabling full prefetching on every link

Mistake 5: Using meaningless loading states

A Practical Adoption Strategy

Final Thoughts

Implementing Postgres Row-Level Security in Next.js: The Drizzle Multi-Tenant Pattern

Two roles, two connections

The policy itself

Getting context into the transaction: SET LOCAL

Gotcha #1: you can't parameterize SET

Gotcha #2: Drizzle's transaction type doesn't match its client type

Gotcha #3: don't let a stream hold your transaction

Gotcha #4: knowing which connection you're on

Postgres LISTEN/NOTIFY in Production: Safe Reconnects & SSE

How Postgres LISTEN/NOTIFY works

One connection, many channels

Handling reconnects: capped exponential backoff

The unsubscribe race, solved by ordering, not locking

Callers should never worry about connection state

Wiring it into Server-Sent Events

When this pattern fits

Faster Geospatial Animations with PMTiles: Multi-Layer MVT Time Series in OpenLayers

The architecture: single archive, dynamic layers

Complete working example

Step 1 — Per-timestep GeoJSON files

Step 2 — Generate PMTiles with named layers (generate-tiles.sh)

Step 3 — Start the server

Step 4 — Client animation (index.html)

Running the full setup

Conclusion: paint time, not flip it

Next.js 16 Caching for E-Commerce: Cache Components, use cache, revalidateTag, and Fresh Product Data

Why Caching Is Harder in E-Commerce Than in a Typical Content Site

The Three Cache Boundaries That Matter Most

1. Stable catalog content

2. Shared but fast-changing commerce data

3. User-specific or session-specific data

What Next.js 16 Changes in Practice

A Practical Storefront Strategy

Cache the stable product content

Use a shorter cache boundary for price and stock

Keep customer-specific data out of the shared cache

Why This Matters for Medusa and Other Headless Commerce Backends

Using revalidateTag to Keep Data Fresh

Where Teams Usually Get It Wrong

Mistake 1: Caching personalized pricing globally

Mistake 2: Making the whole page dynamic because one widget is dynamic

Mistake 3: Using no-store too broadly

Mistake 4: Forgetting business dimensions in cache keys

Mistake 5: Mixing editorial freshness with operational freshness

A Good Default Model for Product Pages

Why This Works Well With Streaming and PPR Thinking

A Note on Webhooks and Operational Ownership

Final Thoughts

Asset-Based Data Orchestration: Lessons from Building a Multi-State Social Data Platform

1. Why Reliability Becomes a Systems Problem at Scale

2. System Overview

High-Level Architecture

3. Source Heterogeneity as the Dominant Constraint

4. Normalization (HSDS, SDOH or Equivalent) as an Architectural Contract

5. What We Got Wrong Initially

6. Transitioning to Asset-Based Data Orchestration with Dagster

7. Partitioning for Failure Isolation

8. Data Quality Embedded in the Asset Graph

9. Operational Outcomes

10. Open Trade-offs and Unresolved Questions

2. Cache reusable content with `'use cache'`

Step 2 — Generate PMTiles with named layers (`generate-tiles.sh`)

Step 4 — Client animation (`index.html`)

Using `revalidateTag` to Keep Data Fresh

Mistake 3: Using `no-store` too broadly

Correct usage of `shutdown_command`