DEV Community: uninterrupted

Faster Geospatial Animations with PMTiles: Multi-Layer MVT Time Series in OpenLayers

uninterrupted — Wed, 08 Jul 2026 06:00:00 +0000

Most people try to animate time series by stacking separate tile sources or flipping CSS styles mid-frame. That causes flicker and a surprising amount of wasted CPU/GPU work. The core takeaway is simple: bake your timesteps into a single vector-tile archive (PMTiles) with per-timestep named layers, load it once, and drive animation by re-styling already-decoded tiles.

I tested this with OpenLayers and noticed the difference right away: playback became smoother and far less CPU-bound once tiles were pre-baked and cached. The hard part is server-side — how you generate, package, and serve tiles matters more than which client library you pick.

When you animate a map by switching between separate tile sources, each frame causes one of two expensive operations: a network fetch of new tiles, or client-side re-decoding and re-styling. Baking timesteps into one archive avoids that by turning each frame into a cheap style re-evaluation on already-decoded features.

The architecture: single archive, dynamic layers

MVT (Mapbox Vector Tile) is a compact protobuf-based vector tile format that encodes geometry and attributes per z/x/y tile. PMTiles packages MVT tiles into a single file for efficient distribution and partial HTTP range retrieval.

The key workflow:

Server side — each timestep is a separate GeoJSON file. tippecanoe joins them into one PMTiles archive, giving each timestep its own named MVT layer (timestep_00, timestep_01, …).
Client side — one VectorTileSource loads the archive once. One VectorTileLayer uses a style function that reads a mutable currentLayerName variable. Advancing a frame means: update the variable, call layer.changed(). OpenLayers re-runs the style function on the already-decoded tile features and repaints — no network request, no re-decode.

Complete working example

!animation.gif

This section contains everything needed to reproduce the demo from scratch: 11 per-timestep GeoJSON files, a tile generation script, and the client HTML.

Step 1 — Per-timestep GeoJSON files

Create 11 files, one per timestep. Each contains a single large polygon with a value property. The temporal identity comes from the file name (and the MVT layer name tippecanoe will assign), not from a property on the feature.

for i in $(seq 0 10); do
  pad=$(printf "%02d" $i)
  value=$((i * 10))
  cat > "square_t${pad}.geojson" << EOF
{
  "type": "FeatureCollection",
  "features": [{
    "type": "Feature",
    "geometry": {
      "type": "Polygon",
      "coordinates": [[[-60,-50],[60,-50],[60,50],[-60,50],[-60,-50]]]
    },
    "properties": { "value": ${value} }
  }]
}
EOF
done

This produces square_t00.geojson (value=0) through square_t10.geojson (value=100).

Step 2 — Generate PMTiles with named layers (`generate-tiles.sh`)

#!/usr/bin/env bash
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

tippecanoe \
  --named-layer=timestep_00:"$SCRIPT_DIR/square_t00.geojson" \
  --named-layer=timestep_01:"$SCRIPT_DIR/square_t01.geojson" \
  --named-layer=timestep_02:"$SCRIPT_DIR/square_t02.geojson" \
  --named-layer=timestep_03:"$SCRIPT_DIR/square_t03.geojson" \
  --named-layer=timestep_04:"$SCRIPT_DIR/square_t04.geojson" \
  --named-layer=timestep_05:"$SCRIPT_DIR/square_t05.geojson" \
  --named-layer=timestep_06:"$SCRIPT_DIR/square_t06.geojson" \
  --named-layer=timestep_07:"$SCRIPT_DIR/square_t07.geojson" \
  --named-layer=timestep_08:"$SCRIPT_DIR/square_t08.geojson" \
  --named-layer=timestep_09:"$SCRIPT_DIR/square_t09.geojson" \
  --named-layer=timestep_10:"$SCRIPT_DIR/square_t10.geojson" \
  -o "$SCRIPT_DIR/square.pmtiles" \
  -z 8 -Z 0 \
  --no-simplification \
  --no-tile-size-limit \
  --no-feature-limit \
  -f

Run with: bash generate-tiles.sh

Step 3 — Start the server

npx serve

Step 4 — Client animation (`index.html`)

The full self-contained HTML file. No build step, no npm install — all dependencies come from esm.sh, which rewrites bare module specifiers (like pbf imported inside ol/format/MVT) to absolute CDN URLs that the browser can resolve.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>MVT Animation</title>
  <link rel="stylesheet" href="<https://cdn.jsdelivr.net/npm/ol@10.9.0/ol.css>" />
  <style>
    * { box-sizing: border-box; margin: 0; padding: 0; }
    body { font-family: monospace; background: #111; color: #eee;
           display: flex; flex-direction: column; height: 100vh; }
    #map  { flex: 1; }
    #hud  { padding: 12px 20px; background: #1a1a1a; border-top: 1px solid #333;
            display: flex; align-items: center; gap: 24px; }
    #hud label { font-size: 13px; color: #aaa; }
    #hud span  { font-size: 16px; font-weight: bold; }
    #timestep-val { color: #7af; }
    #value-val    { color: #fa7; }
    #color-swatch { width: 28px; height: 28px; border-radius: 4px;
                    border: 1px solid #555; display: inline-block; }
    #progress-bar { flex: 1; height: 8px; background: #333; border-radius: 4px; overflow: hidden; }
    #progress-fill { height: 100%; background: linear-gradient(to right, #00f, #f00); border-radius: 4px; }
    #controls { display: flex; gap: 8px; }
    button { background: #333; color: #eee; border: 1px solid #555;
             padding: 5px 14px; border-radius: 4px; cursor: pointer; font-size: 13px; }
    button:hover { background: #444; }
    #loading { position: absolute; inset: 0; display: flex; align-items: center;
               justify-content: center; background: rgba(0,0,0,0.55); color: #fff;
               font-size: 15px; pointer-events: none; z-index: 999; }
    #loading.hidden { display: none; }
  </style>
</head>
<body>
  <div id="map"><div id="loading">Loading tiles…</div></div>
  <div id="hud">
    <div><label>Timestep </label><span id="timestep-val"></span></div>
    <div><label>Value </label><span id="value-val"></span></div>
    <div style="display:flex;align-items:center;gap:8px;">
      <label>Color </label><span id="color-swatch"></span>
    </div>
    <div id="progress-bar"><div id="progress-fill" style="width:0%"></div></div>
    <div id="controls">
      <button id="btn-playpause">Pause</button>
      <button id="btn-reset">Reset</button>
    </div>
  </div>

  <script type="module">
    import Map              from '<https://esm.sh/ol@10.9.0/Map>';
    import View             from '<https://esm.sh/ol@10.9.0/View>';
    import TileLayer        from '<https://esm.sh/ol@10.9.0/layer/Tile>';
    import OSM              from '<https://esm.sh/ol@10.9.0/source/OSM>';
    import VectorTileLayer  from '<https://esm.sh/ol@10.9.0/layer/VectorTile>';
    import VectorTileSource from '<https://esm.sh/ol@10.9.0/source/VectorTile>';
    import MVT              from '<https://esm.sh/ol@10.9.0/format/MVT>';
    import Style            from '<https://esm.sh/ol@10.9.0/style/Style>';
    import Fill             from '<https://esm.sh/ol@10.9.0/style/Fill>';
    import Stroke           from '<https://esm.sh/ol@10.9.0/style/Stroke>';
    import { fromLonLat }   from '<https://esm.sh/ol@10.9.0/proj>';
    import { PMTiles }      from '<https://esm.sh/pmtiles@4.4.1>';

    const LAYER_NAMES = [
      'timestep_00', 'timestep_01', 'timestep_02', 'timestep_03',
      'timestep_04', 'timestep_05', 'timestep_06', 'timestep_07',
      'timestep_08', 'timestep_09', 'timestep_10',
    ];
    const VALUES = LAYER_NAMES.map((_, i) => i * 10);

    const STYLES = LAYER_NAMES.map((_, i) => new Style({
      fill:   new Fill({ color: `rgb(${Math.round(255*i/10)},0,${Math.round(255*(1-i/10))})` }),
      stroke: new Stroke({ color: '#fff', width: 2 }),
    }));

    const archive = new PMTiles('square.pmtiles');

    const source = new VectorTileSource({
      format: new MVT(),
      url: 'square.pmtiles',
      maxZoom: 8,
    });

    let currentLayerIndex = 0;

    const squareLayer = new VectorTileLayer({
      source,
      style: (feature) => {
        return feature.get('layer') === LAYER_NAMES[currentLayerIndex]
          ? STYLES[currentLayerIndex] : null;
      },
    });

    const map = new Map({
      target: 'map',
      layers: [ new TileLayer({ source: new OSM() }), squareLayer ],
      view: new View({ center: fromLonLat([0, 0]), zoom: 2 }),
    });

    const FRAME_MS = 500;

    function animate(frames, onFrame = () => {}) {
      let i = 0;
      let intervalId = setInterval(() => {
        currentLayerIndex = (currentLayerIndex + 1) % frames.length;
        squareLayer.changed();
        onFrame(frames[i]);
        i = (i + 1) % frames.length;
      }, FRAME_MS);

      return {
        pause()  { clearInterval(intervalId); },
        resume() { intervalId = setInterval(() => {
          currentLayerIndex = (currentLayerIndex + 1) % frames.length;
          squareLayer.changed();
          onFrame(frames[i]);
          i = (i + 1) % frames.length;
        }, FRAME_MS); },
        reset()  { i = 0; },
      };
    }

    const timestepEl   = document.getElementById('timestep-val');
    const valueEl      = document.getElementById('value-val');
    const swatchEl     = document.getElementById('color-swatch');
    const progressFill = document.getElementById('progress-fill');

    function onFrame(layerName) {
      const i = LAYER_NAMES.indexOf(layerName);
      timestepEl.textContent    = layerName;
      valueEl.textContent       = VALUES[i];
      swatchEl.style.background = STYLES[i].getFill().getColor();
      progressFill.style.width  = `${(i / (LAYER_NAMES.length - 1)) * 100}%`;
    }

    onFrame(LAYER_NAMES[0]);

    const loadingEl = document.getElementById('loading');
    let player = null;
    let started = false;

    function startWhenReady() {
      if (started) return;
      started = true;
      loadingEl.classList.add('hidden');
      player = animate(LAYER_NAMES, onFrame);
    }

    source.once('tileloadend', startWhenReady);
    map.once('rendercomplete', startWhenReady);
    setTimeout(startWhenReady, 10000);

    let playing = true;
    document.getElementById('btn-playpause').addEventListener('click', (e) => {
      if (!player) return;
      playing = !playing;
      e.target.textContent = playing ? 'Pause' : 'Play';
      playing ? player.resume() : player.pause();
    });
    document.getElementById('btn-reset').addEventListener('click', () => {
      if (!player) return;
      player.reset();
      if (!playing) {
        playing = true;
        document.getElementById('btn-playpause').textContent = 'Pause';
        player.resume();
      }
    });
  </script>
</body>
</html>

Running the full setup

# 1. Install tippecanoe (macOS)
brew install tippecanoe

# 2. Create the 11 per-timestep GeoJSON files
for i in $(seq 0 10); do
  pad=$(printf "%02d" $i); value=$((i * 10))
  echo '{"type":"FeatureCollection","features":[{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[-60,-50],[60,-50],[60,50],[-60,50],[-60,-50]]]},"properties":{"value":'$value'}}]}' \
    > square_t${pad}.geojson
done

# 3. Generate the PMTiles archive
bash generate-tiles.sh

# 4. Start the server
npx serve

# 5. Open the demo
open <http://localhost:3000/index.html>

Conclusion: paint time, not flip it

Don't stack separate tile sources for time-series animation. Bake timesteps into a single PMTiles archive as named MVT layers, load it once, and drive playback by updating a closure variable and calling layer.changed(). Serve the archive with npx serve. Start animation only after the first tile arrives.

The result: smooth playback from first load, no flicker on hard refresh, no blank map with cache disabled.

Next.js 16 Caching for E-Commerce: Cache Components, use cache, revalidateTag, and Fresh Product Data

uninterrupted — Tue, 02 Jun 2026 22:00:00 +0000

Caching in e-commerce is never just about speed. A fast storefront is useful only if it still shows the right price, the correct stock level, and the right experience for the current customer.

That is why caching in a Next.js storefront can be deceptively hard. Some data should be shared broadly and kept warm for SEO and performance. Some data should be refreshed often. Some should never be shared between users at all.

Next.js 16 gives teams a much clearer toolbox for solving this problem with Cache Components, use cache, tag-based invalidation, and explicit cache lifetime controls. Used properly, these features let you keep pages fast without drifting into stale commerce data.

In this guide, I will walk through a practical way to think about caching in a modern storefront and show how to combine use cache, cacheLife, and revalidateTag for real e-commerce use cases.

Why Caching Is Harder in E-Commerce Than in a Typical Content Site

On a standard marketing site, most content changes infrequently. If a page is cached for a few minutes or even a few hours, the business impact is usually negligible.

Commerce systems work differently.

The same product page may contain:

stable product descriptions and category copy
semi-dynamic data such as price, availability, shipping estimates, or promotion labels
private data such as cart state, recently viewed items, or customer-specific pricing

Treating all of that data the same way leads to one of two bad outcomes:

You cache too aggressively and serve stale prices or availability.
You disable caching everywhere and lose the performance benefits that help SEO and conversion.

The better approach is to split your data by volatility and audience.

The Three Cache Boundaries That Matter Most

For most commerce projects, the cleanest mental model is to divide data into three groups.

1. Stable catalog content

This is the part of the page that usually changes only when content editors or merchandisers update the catalog.

Examples:

product title and description
brand information
CMS blocks, FAQs, and long-form copy
category landing page copy
SEO metadata

This is the best candidate for shared caching because it improves performance for everyone and is usually safe to invalidate on demand when content changes.

2. Shared but fast-changing commerce data

This data is still shared between many users, but it changes more often and has stronger operational impact.

Examples:

current price for a product and region
inventory status
campaign badges
delivery promises
low-stock signals

This layer often benefits from a shorter cache lifetime and explicit invalidation.

3. User-specific or session-specific data

This is the data that should not be globally cached.

Examples:

cart contents
logged-in customer state
B2B contract pricing
personalized recommendations
account-specific discounts
checkout calculations tied to the active session

This is where teams most often make expensive mistakes. If the output depends on the current user, you need to keep that boundary private.

What Next.js 16 Changes in Practice

The major advantage of Next.js 16 is not just that it caches. It is that caching becomes more explicit.

Instead of treating an entire route as fully static or fully dynamic, you can choose more precise boundaries:

cache a component or function with use cache
assign cache tags to data and invalidate them later
control lifetime with cacheLife
keep per-user data private where needed
stream dynamic parts separately instead of forcing the entire page into request-time rendering

For e-commerce storefronts, that means you can serve a fast shell and still keep critical parts fresh.

A Practical Storefront Strategy

Let us look at a product page. In a typical headless stack, product content might come from Medusa or another commerce backend, while editorial content may come from a CMS.

The important point is not the backend itself. The important point is how you cache each layer.

Cache the stable product content

If product descriptions and media do not change every minute, cache them explicitly and tag them so they can be invalidated when merchandisers update the catalog.

import { cacheTag } from 'next/cache'

async function getProduct(slug: string) {
  'use cache'

  cacheTag(`product:${slug}`)

  const response = await fetch(
    `${process.env.COMMERCE_API_URL}/products/${slug}`
  )

  if (!response.ok) {
    throw new Error('Failed to load product')
  }

  return response.json()
}

This is a good fit for data that should be broadly reusable and revalidated only when a product actually changes.

Use a shorter cache boundary for price and stock

Price and stock need more care. In some stores, they can still be shared safely for a short period. In others, especially B2B or highly promotional environments, they may need user-level or request-level handling.

For shared price or inventory data, a short-lived cache can be a sensible compromise.

import { cacheLife, cacheTag } from 'next/cache'

async function getOffer(productId: string, regionId: string) {
  'use cache'

  cacheTag(`offer:${productId}:${regionId}`)
  cacheLife({ expire: 60 })

  const response = await fetch(
    `${process.env.COMMERCE_API_URL}/offers/${productId}?region=${regionId}`
  )

  if (!response.ok) {
    throw new Error('Failed to load offer')
  }

  return response.json()
}

Here the offer can be shared, but only briefly. That is often enough to reduce backend pressure while still keeping the storefront operationally safe.

Keep customer-specific data out of the shared cache

If pricing, recommendations, or entitlements depend on the active customer, do not treat them as generic shared data.

This applies especially to:

B2B price lists
customer group discounts
partner-only catalog visibility
personalized recommendation widgets

At that point, a private boundary or request-time fetching becomes more appropriate than a shared cache.

The principle is simple: if another customer must never see the same output, that data should not live in a shared cache.

If you do want to cache user-specific data briefly to avoid repeating the same work during a session, keep that cache private instead of shared.

import { cacheLife } from 'next/cache'

async function getRecommendations(productId: string, customerId: string) {
  'use cache: private'

  cacheLife({ expire: 60 })

  const response = await fetch(
    `${process.env.RECOMMENDATION_API_URL}/products/${productId}/recommendations?customer=${customerId}`
  )

  if (!response.ok) {
    throw new Error('Failed to load recommendations')
  }

  return response.json()
}

That keeps the result bounded to the current user context instead of letting it leak into global storefront output. It is also worth noting that 'use cache: private' does not store its cache entries on the server. The function still runs during server rendering, but the cached result is kept only in the browser’s memory and does not survive a full page reload.

Why This Matters for Medusa and Other Headless Commerce Backends

This pattern becomes especially valuable in headless commerce because the storefront is often responsible for combining multiple systems:

commerce backend
CMS
search service
recommendation engine
analytics and experiment layers

If all of those systems are fetched at request time for every page view, the frontend becomes slower and more fragile than it needs to be.

If all of them are aggressively cached together, the storefront becomes fast but operationally unsafe.

The right answer is to cache by business meaning, not by convenience.

For example:

CMS content can usually be cached until editors publish changes.
catalog copy can usually be cached and invalidated via tags.
regional shared pricing can often be cached briefly.
logged-in B2B pricing should stay private.
cart and checkout state should stay request-scoped.

That is the difference between a caching strategy that helps the business and one that only looks good in benchmarks.

Using `revalidateTag` to Keep Data Fresh

One of the most useful patterns in Next.js 16 is tag-based invalidation.

Instead of revalidating whole pages blindly, you can invalidate the exact data domain that changed. That is particularly useful when updates come from webhooks triggered by your CMS, PIM, or commerce backend.

For example, if a product is updated in Medusa or an editorial team changes CMS content linked to that product, you can invalidate the relevant tag immediately.

import { revalidateTag } from 'next/cache'

export async function POST(request: Request) {
  const payload = await request.json()

  revalidateTag(`product:${payload.slug}`, 'max')
  revalidateTag(`offer:${payload.id}:${payload.regionId}`, 'max')

  return Response.json({ ok: true })
}

In practice, this lets you keep product content cached most of the time while still reacting quickly to actual changes.

That is a much better fit for commerce than short global revalidation windows everywhere.

One operational note matters here: never expose a public revalidation endpoint without request validation. If invalidation is triggered by a CMS, Medusa, or ERP webhook, verify the source with a shared secret or signature check before calling revalidateTag.

Where Teams Usually Get It Wrong

There are a few caching mistakes that repeatedly appear in storefront projects.

Mistake 1: Caching personalized pricing globally

This is the most dangerous one. If the result depends on customer identity, customer group, contract terms, or session data, do not let it leak into a shared cache boundary.

Mistake 2: Making the whole page dynamic because one widget is dynamic

A product page does not need to become fully request-time just because the cart badge or recommendation box is personalized.

Keep the stable content cached and isolate the dynamic parts instead.

Mistake 3: Using `no-store` too broadly

no-store is sometimes necessary, but if it becomes the default reaction to uncertainty, you throw away much of the value of the App Router architecture.

Use it for truly request-bound data, not as a shortcut for unclear cache design.

Mistake 4: Forgetting business dimensions in cache keys

Prices are rarely just “product price.” They are often tied to region, currency, channel, campaign, or customer group.

If those dimensions are not reflected in the data boundary, the cache becomes unsafe.

Mistake 5: Mixing editorial freshness with operational freshness

A product description may be safe to cache until a content change occurs. Inventory and promotions may not be. These two layers should not be forced into the same lifetime.

A Good Default Model for Product Pages

If you need a practical starting point, this model works well for many storefronts:

Data type	Suggested approach
Product description, SEO copy, editorial blocks	`use cache` with tags and on-demand invalidation
Shared regional price and stock	short cache lifetime plus tags
Recommendations	private or isolated dynamic boundary
Cart state	request-time or session-specific logic
Checkout totals and payment readiness	request-time only
B2B contract pricing	private boundary, never generic shared output

This is not a framework rule. It is a business-safe baseline.

Why This Works Well With Streaming and PPR Thinking

If you have already explored Partial Prerendering in Next.js 16, this caching model should feel familiar.

The idea is the same:

keep the stable shell fast
isolate dynamic or high-volatility parts
avoid blocking the entire page for data that does not deserve that privilege

For commerce, that means a product page can still load quickly for users and search engines while the truly dynamic fragments remain fresh.

That is a much more scalable model than forcing every route into either “fully static” or “fully dynamic.”

A Note on Webhooks and Operational Ownership

Caching only works well if invalidation is treated as part of the architecture, not as an afterthought.

If your team updates product data in Medusa, content in a CMS, or prices in an ERP, your storefront should know how those changes trigger cache invalidation.

Typical patterns include:

Medusa webhooks invalidating product and offer tags
CMS publish events invalidating page or content tags
ERP sync jobs invalidating pricing-related tags
stock update events invalidating inventory-related tags only

This is where many storefronts move from “fast in development” to “reliable in production.”

Final Thoughts

The real value of caching in Next.js 16 is not that it makes everything static. It is that it helps you decide what should be shared, what should be refreshed, and what should remain private.

For e-commerce teams, that distinction matters more than raw speed numbers. The fastest storefront is not the one that caches the most. It is the one that caches the right things.

If you separate stable catalog content, shared operational data, and user-specific state into different cache boundaries, you can keep your storefront fast without risking stale or incorrect business data.

Asset-Based Data Orchestration: Lessons from Building a Multi-State Social Data Platform

uninterrupted — Thu, 26 Mar 2026 07:21:25 +0000

Building reliable data platforms rarely fails because of scale alone.
More often, reliability collapses under heterogeneity: multiple data providers, inconsistent schemas, partial updates, and unclear ownership.

While building a multi-state social data platform ingesting resource data from dozens of organizations, we discovered that reliability is not a property of pipelines. It is a property of data artifacts and their relationships.

1. Why Reliability Becomes a Systems Problem at Scale

The accompanying essay frames trust as something earned through consistent system behavior under real-world pressure. What that framing leaves implicit, but what became unavoidable in practice, is that reliability stops being a property of individual components very early on. Once multiple organizations, jurisdictions, and publishing surfaces are involved, reliability becomes an emergent property of the entire system.

For us, this meant that no single pipeline, connector, or database could be made "reliable enough" in isolation. Failures were rarely total. They were partial, localized, and often silent. The engineering challenge was not preventing all failure, but designing the system so that failures were isolated, detectable, and explainable.

That requirement drove nearly every architectural decision that followed.

2. System Overview

Technically, the platform ingests social service resource data from independent organizations operating across multiple U.S. states. Each organization exposes data via a different source system (for example iCarol, WellSky, VisionLink, RTM), with varying schemas, update cadences, and quality guarantees.

At a high level, the system consists of:

Writers - per-tenant ingestion and transformation projects that:
- Fetch raw data from source systems via connector adapters
- Persist raw data into Snowflake source schemas
- Normalize and standardize data via DBT
- Apply enhancements such as geocoding or translations
Readers - publisher processes that:
- React to completed writer runs, either bulk or incremental
- Publish curated artifacts to OpenSearch, MongoDB, and optionally Postgres

Snowflake acts as the system of record for intermediate and normalized datasets. Dagster coordinates the execution and materialization of data assets. DBT is used explicitly for set-based transformations, not orchestration.

Scale is not extreme in raw volume, but complexity is high: dozens of tenants, hundreds of tables per tenant, and frequent partial updates.

High-Level Architecture

The following diagram illustrates the writer → Snowflake → DBT → asset orchestration → readers pattern.

The platform is designed around asset lineage and normalization contracts, not pipelines.

3. Source Heterogeneity as the Dominant Constraint

The hardest constraint was not throughput or storage. It was heterogeneity.

Each data provider differed along several axes simultaneously:

Schema shape: even when nominally "the same" entities existed, fields varied
Semantics: identical fields often meant subtly different things
Update cadence: some sources updated continuously, others weekly or ad hoc
Quality guarantees: missing fields, stale records, or partial exports were common

Early on, we underestimated how strongly these differences would dominate system design. Attempting to treat ingestion as a uniform, pipeline-shaped process led to brittle assumptions and cross-tenant coupling.

The system only became manageable once heterogeneity was treated as fundamental, not incidental.

4. Normalization (HSDS, SDOH or Equivalent) as an Architectural Contract

Normalization into an HSDS-like model was not implemented as a downstream convenience. It became an architectural contract.

All downstream consumers, internal and external, implicitly rely on the guarantees of the normalized model: stable fields, predictable relationships, and documented semantics. That meant normalization could not be "best effort" or delayed until the end of a pipeline.

In practice:

Raw source data is written verbatim into Snowflake source schemas
DBT ELT projects transform this raw data into standardized intermediate models
DBT STAGE projects apply tenant-specific adaptations while preserving the contract

This separation made it explicit where interpretation happens. If a field is wrong in the normalized model, the question becomes which contract was violated, not "what broke in the pipeline".

5. What We Got Wrong Initially

Several early assumptions did not survive contact with reality.

Pipeline-first thinking

We initially modeled work as long-running jobs. This obscured which intermediate datasets were durable, reusable, or safe to depend on. Debugging often meant rerunning more than necessary.

Manual validation

Data quality checks lived outside the orchestration layer. Engineers and analysts manually inspected outputs, which worked at small scale but failed under concurrency and time pressure.

Shared failure domains

Multiple tenants often shared execution paths. A failure in one tenant’s ingestion could block or delay others, even when their data was unrelated.

None of these issues were catastrophic individually. Together, they made reliability depend on human attention.

6. Transitioning to Asset-Based Data Orchestration with Dagster

The shift to asset-based orchestration was driven less by tooling preference and more by a change in mental model.

Instead of asking "what jobs should run?", we started asking:

What data artifacts must exist?
What do they depend on?
How fresh do they need to be?
What constitutes success or failure for this artifact?

Dagster assets provided a way to encode those questions directly.

A simplified example from a writer project shows how DBT models are treated as assets rather than opaque steps:

# writer-xyz/assets.py (excerpt)
from dagster_dbt import load_assets_from_dbt_project

dbt_elt_assets = load_assets_from_dbt_project(
    project_dir="dbt_elt",
    profiles_dir="dbt_elt",
)

This does not explain how DBT runs. It declares that the resulting tables are first-class assets with lineage and state.

Once assets replaced jobs as the primary abstraction, freshness, lineage, and partial recomputation became explicit rather than implicit.

7. Partitioning for Failure Isolation

Partitioning was critical for isolating failures.

We partitioned primarily along tenant and state boundaries, not time. This reflected operational reality: data issues almost always affected a single organization or region.

In Dagster terms, this meant:

Separate writer projects per tenant
Independent schedules and sensors
Asset materializations scoped to a tenant’s data domain

A failure in one writer no longer blocked publishing for others. More importantly, remediation could be targeted and auditable.

8. Data Quality Embedded in the Asset Graph

Data validation moved into the asset graph itself.

Instead of post-hoc checks, validations became explicit dependencies. If a validation asset failed, downstream assets simply did not materialize.

An example pattern used across writers:

@asset
def validate_staging_tables(staging_tables):
    assert staging_tables.count_missing_ids() == 0

This is intentionally simple. The key point is not the check itself, but that failure is structural. The system records that an expected artifact does not exist, rather than silently publishing bad data.

This shifted failure detection earlier and reduced the blast radius of errors.

9. Operational Outcomes

Day-to-day operations changed in several concrete ways:

On-call work shifted from rerunning pipelines to inspecting asset lineage
Partial backfills became routine rather than exceptional
Publishing delays were easier to attribute to specific upstream causes
New tenants could be added without increasing shared operational risk
None of this eliminated operational effort. It made that effort more focused and less reactive.

10. Open Trade-offs and Unresolved Questions

Some challenges remain unresolved:

Cross-tenant schema evolution still requires coordination and discipline
Observability across Snowflake, DBT, Dagster, and downstream stores is fragmented
Cost attribution at the asset level is still coarse-grained
Human review remains necessary for certain semantic validations

With more time, we would invest earlier in unified observability and more formal schema versioning.

11. Why These Lessons Matter Beyond This Platform

These lessons are not unique to this system.

Any platform operating in civic tech, govtech, or environmental data shares similar constraints: multiple data producers, uneven quality, and real-world consequences for failure.

The core takeaway is not "use asset-based orchestration", but treat data artifacts as obligations. Once that shift happens, many architectural decisions become clearer.

Reliability stops being something you hope for and becomes something you can reason about.

Final Thoughts

The biggest lesson from this platform was not about any particular tool. It was about how we model the system itself.

Once data artifacts became the core abstraction, many reliability problems became easier to reason about. Failures became visible, dependencies became explicit, and operational work shifted from firefighting pipelines to managing data contracts.

For data platforms operating across heterogeneous sources, this shift can be the difference between a system that merely runs - and one that can be trusted.

FAQ

What is asset-based data orchestration?

Asset-based data orchestration treats data artifacts (tables, datasets, models) as the primary units of orchestration instead of pipelines or jobs. Systems like Dagster allow teams to define dependencies between assets, enabling better lineage tracking, partial recomputation, and failure isolation.

Why is asset-based orchestration useful for complex data platforms?

In large systems with many data producers and consumers, failures are rarely binary. Asset-based orchestration makes dependencies explicit, allowing teams to:

isolate failures
recompute only affected datasets
track lineage across transformations
enforce data quality checks before publishing

How does Dagster differ from traditional pipeline orchestrators?

Traditional orchestrators schedule jobs or workflows. Dagster emphasizes data assets and lineage. This enables better visibility into:

what datasets exist
what produced them
what depends on them
whether they meet freshness or quality requirements

When should teams adopt asset-based orchestration?

Asset-based orchestration becomes particularly valuable when systems have:

many data sources
heterogeneous schemas
multiple downstream consumers
partial or incremental updates
strict reliability requirements

These conditions are common in multi-tenant data platforms and civic tech systems.

Does asset-based orchestration replace tools like DBT?

No. Asset-based orchestration complements transformation tools like DBT. In many architectures:

DBT performs set-based transformations
Dagster manages asset lineage, dependencies, scheduling, and validation

This separation keeps orchestration and transformation responsibilities clear.

Next.js 16 Partial Prerendering (PPR): The Best of Static and Dynamic Rendering

uninterrupted — Thu, 05 Mar 2026 08:21:46 +0000

Next.js has long been a leader in giving developers flexible, high-performance rendering strategies - Static Site Generation (SSG), Server-Side Rendering (SSR), Incremental Static Regeneration (ISR), and Client-Side Rendering (CSR) all play roles depending on your use case. With Next.js 14, Partial Prerendering (PPR) was introduced as an experimental feature, starting from Next.js 16, PPR has become a stable and becomes a foundational part of this landscape by letting you blend static pre-rendering and dynamic behavior in the same route, improving perceived performance and SEO without sacrificing real-time data needs.

What Is Partial Prerendering (PPR)?

Partial Prerendering is a rendering strategy that lets Next.js deliver a static HTML “shell” immediately, then stream in dynamic content as it becomes available - all in the same server response. Instead of choosing static or dynamic at the page level, PPR combines both on a per-component basis.

The static shell (layout, logo, product titles, navigation) is pre-generated at build time or cached ahead of requests.
Dynamic parts (cart state, recommendations, session data) load later and are streamed to the client using React’s <Suspense> boundaries.
Users see meaningful UI immediately, with dynamic content filling in seamlessly.

This leads to faster Time to First Byte (TTFB) and a smoother perceived experience, while still allowing data personalization and real-time updates.

How Partial Prerendering Works in Next.js 16

Cache Components: The Heart of PPR

In Next.js 16, PPR isn’t just an experimental flag - it’s integrated into the Cache Components system. Cache Components lets you opt-in to cache certain components or data instead of rendering them on every request, making pre-rendering more predictable and performant.

cacheComponents: true in next.config.ts enables PPR and component-level caching.
The new "use cache" directive lets you mark data-fetching functions or components as cacheable.
Anything not cacheable or depending on request-specific data can be wrapped in a <Suspense> fallback.

This approach gives you fine-grained control: static UI is reused across requests, while dynamic parts rehydrate or stream in only when needed.

Streaming and Suspense: The Engine Under the Hood

React’s <Suspense> plays a key role in PPR:

Wrap dynamic components in <Suspense> with a fallback UI (like a skeleton loader).
During rendering, Suspense tells Next.js where to halt pre-rendering and stream dynamic content later.
The server sends the pre-rendered shell immediately and then streams dynamic sections in parallel as they’re ready.

This strategy avoids blocks in page delivery and reduces the “white screen” effect often seen in traditional SSR or CSR approaches.

Why PPR Matters for E-commerce

For e-commerce platforms, performance directly impacts sales and SEO - factors also emphasized in the U11D articles about rendering strategies. A slow page can hurt conversions and search rankings. Partial Prerendering improves this by:

Faster initial loads: Users see the page skeleton instantly, improving engagement and performance metrics.
SEO advantages: Static content is available for crawler indexing immediately.
Dynamic personalization: Cart contents, recommendations, user-specific prices or availability can appear without blocking the initial render.
More flexible than SSG/SSR alone: You’re not limited to fully static or completely dynamic pages - the best of both lives together.

In e-commerce apps where product detail pages might be static for most users but show personalized recommendations or inventory status, PPR is a compelling hybrid.

How to Enable and Use PPR in Next.js 16

Enable Cache Components:

In your next.config.ts:

import type { NextConfig } from 'next';

const nextConfig: NextConfig = {
  cacheComponents: true, // enables Partial Prerendering
};

export default nextConfig;

Wrap dynamic UI:

Use React’s Suspense for dynamic content:

import { Suspense } from 'react';

export default function ProductPage() {
  return (
    <>
      <Header />
      <Suspense fallback={<div>Loading product...</div>}>
        <ProductDetails />
      </Suspense>
    </>
  );
}

Use use cache for predictable dynamic data:

Inside dynamic data functions, prefix with "use cache" to mark them cacheable:
```
'use cache';
const product = await getProduct(id);
```
This tells Next.js that caching is safe for that data.

How PPR Fits with Other Rendering Strategies

Partial Prerendering doesn’t replace SSR, SSG, ISR, or CSR - but coordinates with them. While U11D’s articles give a great overview of traditional strategies like SSG, SSR, and ISR, PPR adds a hybrid strategy that sits between:

SSG/ISR: Good for full static pages or pages that regenerate occasionally.
SSR: Ideal for real-time personalized content.
CSR: Great for highly interactive client-heavy UI.
PPR (Next.js 16): Combines static cached shells with streaming dynamic content.

Think of PPR as a component-level SSG + dynamic hybrid - fast to load like SSG, flexible like SSR.

Wrapping Up

Next.js 16 elevates Partial Prerendering from an experimental concept to a practical, integrated performance strategy through Cache Components and React Suspense. It’s especially powerful for complex, dynamic sites like e-commerce stores, where you want the benefits of static pre-rendering and dynamic personalization without sacrificing UX or SEO.

By serving instantly usable HTML and streaming dynamic parts in parallel, PPR bridges the traditional divide between static and dynamic rendering - helping your app feel faster while staying robust and scalable.

If you’re building with Next.js 16, definitely explore PPR alongside other strategies like SSR, ISR, and CSR to find the most performance-optimized combination for your routes.

Using Proxy (before Middleware) in Next.js: a modern layer

uninterrupted — Wed, 26 Nov 2025 07:00:00 +0000

Next.js 16 introduces a new file-convention: proxy.js / proxy.ts (often “Proxy”) which supersedes the older middleware.js. This feature allows you to intercept HTTP requests early (before routing/rendering) and run custom logic—redirects, rewrites, header manipulation, or forwarding (proxying) to another service.

In e-commerce (or any frontend/back-end scenario) this gives you a powerful tool: you can make your Next.js app act as a Backend-for-Frontend (BFF) layer, hiding upstream APIs, consolidating micro-services, enforcing auth, caching, rewriting URLs, etc. This aligns well with many of the architectures discussed in your referenced articles about Next.js for commerce (SSG/ISR/SSR strategies etc).

What is Proxy and when to use it

The Proxy file (proxy.js/.ts) in Next.js is a special server-side file (located in the project root or under src) that runs before any route handling or page rendering occurs. In fact, Proxy (and Middleware) executes before every incoming request — not just those for pages or APIs — including requests for assets like images, icons, or JavaScript files. Because of this, Next.js provides the matcher configuration, allowing you to specify which routes the Proxy should apply to and avoid intercepting unnecessary requests.

It allows you to inspect the incoming request, then respond, redirect, rewrite, or modify headers based on your application’s needs.

Relation to Middleware & Rewrites

In previous versions, middleware.js (App Router) was used for this pattern; Proxy replaces it.
While rewrites() in next.config.js can act like a proxy by mapping URL paths to different destinations, the Proxy feature goes further — it gives you direct access to the request object itself, including headers, cookies, and other request data.

When to use Proxy

Proxy works best in scenarios that require lightweight, real-time adjustments to incoming requests — such as performing quick redirects based on request details, dynamically rewriting routes for A/B testing or experimental features, or modifying request and response headers across specific routes or the entire application.

However, it’s not well-suited for tasks involving heavy data fetching or complex session management, as those are better handled by dedicated API routes or backend services.

Relation to Backend-for-Frontend (BFF) pattern

In the BFF pattern, the frontend has a dedicated backend layer (the “for the frontend”) that aggregates, transforms, or proxies requests to multiple microservices. Next.js supports this pattern via Route Handlers, Proxy and server actions.

Setting up Proxy: conventions & example

File convention

Create a proxy.ts (or proxy.js) file in the project root or inside src/ alongside your top‐level app/ or pages/folder.
Only one proxy file per project is supported—though you can modularize logic within it by importing helpers.
Export a function named proxy (or default) that receives a NextRequest and returns a NextResponse.
Optionally export a config object with a matcher property to specify which routes this proxy applies to.

Basic example

// proxy.ts
import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';

export function proxy(request: NextRequest) {
  // simple redirect
  return NextResponse.redirect(new URL('/home', request.url));
}

export const config = {
  matcher: '/about/:path*',
}

This intercepts /about/* and sends users to /home.

Proxying to different pathname

// proxy.ts
import { NextResponse } from 'next/server';

export function proxy(request: Request) {
  if (request.nextUrl.pathname === '/v1/docs') {
    request.nextUrl.pathname = '/v2/docs'
    return NextResponse.redirect(request.nextUrl);
  }
}

This way /v1/docs in your Next.js app becomes a proxy to your v2 version.

Important flags & matching

matcher can be a string, array of strings, or objects with source, has, missing, regexp for fine control.
You should exclude internal Next.js paths such as /_next/, static assets etc, to avoid intercepting everything accidentally.
Be aware of runtime: The Proxy is executed by default in the Edge Runtime environment; performance matters.

Use cases & how it fits e-commerce / frontend/back-end scenarios

Use case 1: Authentication / session gating before forwarding

Before forwarding to upstream API, you can inspect cookies/headers in Proxy, validate session, add or strip auth headers, or reject unauthorized requests early. This centralises “frontend backend” logic.

For example:

if (!request.cookies.get('sessionToken')) {
  return NextResponse.redirect('/login')
}

Use case 2: A/B testing, feature flags, routing to different backends

E-commerce often runs experiments: you might want to route certain requests to a “new checkout backend” or “legacy order API” based on a feature flag. Proxy allows dynamic routing:

if (featureFlag.isEnabled('newCheckout') && pathname.startsWith('/checkout')) {
  return NextResponse.rewrite(new URL('/new-checkout' + rest, request.url))
}

This is exactly where Proxy shines, beyond simple rewrites. The docs mention this scenario.

Use case 3: Multi-zone / micro-frontend routing

If you break your site into zones (e.g., /shop, /blog, /dashboard) and each is a separate Next.js app or separate backend service, you can use Proxy to route to the correct service based on path or condition. The docs mention this under Multi-Zones: “Proxy can also be used when there is a need for a dynamic decision when routing.”

How this complements SSG/ISR/SSR strategies

In your e-commerce context (as your referenced articles explore SSG/ISR/SSR), Proxy supports architecture by giving you control over how client-requests hit your backend:

If you statically generate product pages (SSG/ISR) but still need dynamic APIs behind the scenes (inventory, pricing), Proxy sits in front of those APIs.
If you use SSR for checkout or account pages, you might fetch via your BFF rather than directly from client; Proxy gives you that layer.
This allows you to decouple the frontend build (Next.js) from the backend services, while still providing unified domain, consistent cookies/sessions, and reduction of client-side complexity.

Summary

The Proxy file convention in Next.js is a modern, flexible tool for intercepting HTTP requests early in your application lifecycle. It’s especially valuable in setups where your frontend is part of a larger ecosystem (microservices, e-commerce, feature-flagged experiments) and you need a lightweight backend-for-frontend layer without building a full custom server.

For e-commerce platforms built with Next.js—where you may have static product pages, dynamic inventory/pricing APIs, customer sessions, checkout flows—the Proxy gives you an elegant place to manage domain-consolidation, routing, auth, A/B experimentation and API facade logic.

However, you should use it wisely: keep logic lean, use it for the right cases (redirects, rewrites, header manipulation, request forwarding) rather than heavy data aggregation. Combined with Next.js’s rendering strategies (SSG, ISR, SSR) and BFF pattern, Proxy becomes an important architectural building block.

Automated Tax Compliance for Global E-Commerce Growth

uninterrupted — Mon, 17 Nov 2025 08:06:29 +0000

Choosing an e-commerce platform today means choosing your capabilities for the next 5-10 years. Most businesses focus on features and design, overlooking a critical question: can this platform handle tax compliance as we grow, regulations change, and new sales channels emerge?

Three Compliance Challenges That Break Legacy Platforms

Real-time performance matters more than you think. Every millisecond at checkout affects conversion. Legacy tax calculation systems add 300-500ms latency per request — enough to impact sales. Worse, they often fail silently during high-traffic periods, forcing you to choose between lost sales or compliance risk.

Headless architecture becomes critical as your business evolves. Selling through mobile apps, marketplaces, social commerce, and IoT devices means your tax system needs API-first design. Shopify's monolithic architecture forces you into their frontend. WooCommerce wasn't built for headless operations. Magento's complexity makes omnichannel integration expensive.

Audit readiness isn't optional. Tax authorities increasingly audit e-commerce businesses, and "we did our best" isn't a defense. You need complete transaction history, documentation of every tax decision, and proof that calculations were accurate at transaction time. Most platforms treat this as an afterthought, storing minimal data that won't satisfy auditors.

The Performance-Compliance Tradeoff (That Shouldn't Exist)

Avalara's real-time API calculates taxes in under 100ms, handling peak traffic without degradation. This isn't just about speed — it's about reliability. During Black Friday traffic spikes, tax calculations don't become a bottleneck.

Combined with Medusa's modern architecture, you get:

Sub-second checkout experiences with real-time tax accuracy
Async tax calculation options for ultra-high-volume scenarios
Automatic failover and retry logic for maximum reliability
Complete transaction logging for audit trails

The Technical Foundation Matters

Medusa's API-first, headless design means tax compliance works consistently across every channel. Selling through your website, mobile app, and Amazon? Same tax logic, same accuracy, same audit trail — without custom integration for each channel.

This architecture also enables sophisticated business rules:

Customer-specific tax exemptions applied automatically across all channels
Dynamic tax calculation based on real-time inventory location
Split fulfillment from multiple warehouses with correct origin-based taxation
International expansion without rebuilding your tax infrastructure

Building Audit-Ready From Day One

When tax authorities audit your business, they want answers to specific questions:

How did you determine tax rates for each transaction?
Where are exemption certificates for tax-exempt sales?
How do you handle returns and refunds?
What's your documentation for nexus determination?

The Medusa-Avalara integration maintains complete records automatically:

Every tax calculation with jurisdiction breakdown
Customer exemption certificates with validation status
Transaction lifecycle (calculated → committed → voided/refunded)
Address validation records proving due diligence

You're not preparing for audits — you're always ready. Export complete documentation on demand, proving compliance for every transaction.

The Platform Investment Decision

Legacy platforms force tradeoffs: Shopify is easy but inflexible. Magento is powerful but expensive and complex. WooCommerce is customizable but scales poorly.

Medusa eliminates these tradeoffs: open-source flexibility, modern architecture, enterprise capabilities without enterprise costs. When paired with Avalara's tax automation, you get infrastructure that grows with your business.

Scale Without Technical Debt

Most e-commerce businesses outgrow their first platform. They've built workarounds for limitations, accumulated technical debt, and face expensive migration projects. This is avoidable.

Starting with proper infrastructure—headless architecture, real-time tax automation, complete compliance documentation—means your platform grows with you instead of holding you back.

Tax compliance isn't exciting, but it's foundational. Get it right from the start, and it becomes invisible infrastructure. Get it wrong, and it becomes an expensive barrier to growth.

Need help with Avalara integration?

Looking to implement automated tax compliance with Avalara in your Medusa store? u11d specializes in e-commerce integrations and tax automation solutions. We can help you set up, customize, and optimize your Avalara integration for maximum compliance and efficiency. Our team has extensive experience with both Medusa e-commerce platforms and Avalara tax systems, ensuring seamless integration that meets your specific business requirements.

A Practical Guide to Scaling Medusa with Kubernetes Autoscalers

uninterrupted — Wed, 29 Oct 2025 09:56:36 +0000

As your Medusa.js e-commerce platform grows, performance and reliability depend on how well it scales under load. Kubernetes provides native tools like the Horizontal Pod Autoscaler (HPA) and KEDA to automatically adjust resources based on real-time demand.
In this guide, you’ll learn how to configure Medusa for horizontal scaling in Kubernetes, using Prometheus, cAdvisor, and HPA - ensuring your store remains responsive even during peak traffic periods.

Prerequisites for Medusa Autoscaling on Kubernetes

Before implementing autoscaling, ensure that your monitoring and metric systems are in place.

Required Tools: cAdvisor, Prometheus, and KEDA

To make HPA and KEDA work efficiently, you’ll need:

cAdvisor – collects container-level CPU and memory metrics.
Prometheus – scrapes, stores, and visualizes time-series metrics.
Prometheus Adapter or KEDA – exposes those metrics to the HPA.

This setup is essential to achieve a reliable, responsive scaling mechanism that monitors pod and container utilization with fine-grained metric resolution (ideally 1 second or less).

Setting Up Metrics for HPA

Configuring cAdvisor for container-level metrics

cAdvisor (Container Advisor) is a running daemon that provides per-container resource usage data. It collects and exports information about all containers running on a host.

Key configuration tips:

Avoid collecting all metrics to reduce resource consumption.
Adjust collection intervals according to your scaling sensitivity.

  - --allow_dynamic_housekeeping=false
  - --housekeeping_interval=1s
  - --max_housekeeping_interval=2s

These flags ensure you get up-to-date metrics while keeping overhead low.

Prometheus Configuration for Scraping cAdvisor Data

Prometheus collects metrics from cAdvisor and stores them for HPA or KEDA to consume. Below is a sample configuration to scrape and relabel container metrics efficiently:

scrape_configs:
  - job_name: cadvisor
    scrape_interval: 1s
    static_configs:
      - targets: ["cadvisor.cadvisor.svc.cluster.local:8080"]
    metric_relabel_configs:
      - source_labels: [container_label_io_kubernetes_pod_namespace]
        target_label: namespace
      - source_labels: [container_label_io_kubernetes_pod_name]
        target_label: pod
      - source_labels: [container_label_io_kubernetes_container_name]
        target_label: container
      - source_labels: [container_label_io_kubernetes_pod_node_name]
        target_label: node
      - regex: container_label_.*
        action: labeldrop

Tip: Use a 1-second scrape interval for real-time scaling accuracy, but monitor your Prometheus load — frequent scrapes can impact cluster performance.

Using Prometheus Adapter to Expose Custom Metrics

Prometheus Adapter acts as a bridge between Prometheus and the Kubernetes HPA. It translates Prometheus metrics into Kubernetes-readable custom metrics that HPA can act upon.

Example configuration for CPU and memory metrics:

resource:
  cpu:
    containerQuery: |
      sum by (<<.GroupBy>>) (
        rate(container_cpu_usage_seconds_total{container!="",<<.LabelMatchers>>}[5s])
      )
    nodeQuery: |
      sum  by (<<.GroupBy>>) (
        rate(node_cpu_seconds_total{mode!="idle",mode!="iowait",mode!="steal",<<.LabelMatchers>>}[3m])
      )
    resources:
      overrides:
        node:
          resource: node
        namespace:
          resource: namespace
        pod:
          resource: pod
    containerLabel: container
  memory:
    containerQuery: |
      sum by (<<.GroupBy>>) (
        avg_over_time(container_memory_working_set_bytes{container!="",<<.LabelMatchers>>}[5s])
      )
    nodeQuery: |
      sum by (<<.GroupBy>>) (
        avg_over_time(node_memory_MemTotal_bytes{<<.LabelMatchers>>}[3m])
        -
        avg_over_time(node_memory_MemAvailable_bytes{<<.LabelMatchers>>}[3m])
      )
    resources:
      overrides:
        node:
          resource: node
        namespace:
          resource: namespace
        pod:
          resource: pod
    containerLabel: container

This configuration exposes per-pod resource metrics that HPA can use to make scaling decisions.

Scaling Medusa with KEDA

How KEDA Integrates with Kubernetes HPA

KEDA (Kubernetes Event-Driven Autoscaler) enhances Kubernetes autoscaling by allowing scaling based on external events - for example, message queue depth, API requests, or Prometheus metrics.

KEDA works alongside HPA to provide fine-grained, event-driven scaling for your Medusa backend.

Below is a sample configuration for scaling Medusa using KEDA with Prometheus as the metric source:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: medusa-backend
spec:
  scaleTargetRef:
    name: medusa-backend
  pollingInterval: 1
  cooldownPeriod: 60
  minReplicaCount: 3
  maxReplicaCount: 10
  triggers:
    - type: prometheus
      metricType: AverageValue
      metadata:
        serverAddress: <http://prometheus-server.default.svc.cluster.local:80>
        metricName: avg_cpu_usage
        threshold: "0.5"
        query: |
          sum by (pod) (rate(container_cpu_usage_seconds_total{pod=~"medusa-backend-.*", container!=""}[5s]))

This YAML defines a ScaledObject that dynamically adjusts Medusa’s replica count based on CPU usage metrics scraped by Prometheus.

Fine-Tuning Kubernetes HPA

kube-controller-manager Parameters for Faster Scaling

The kube-controller-manager is a core Kubernetes component that controls how quickly HPA reacts to metric changes.

By tuning its parameters, you can make scaling nearly instantaneous - for example, achieving a 5-second response time to CPU utilization spikes.

Ensure you’ve optimized:

--horizontal-pod-autoscaler-sync-period=5s

Lowering this interval makes the autoscaler evaluate conditions more frequently, but note that some managed Kubernetes distributions may restrict access to these flags.

Visualizing Scaling with Grafana

Grafana dashboards can help you track CPU utilization, pod counts, and scaling behavior in real time.
Below is an example visualization showing how HPA scales pods in and out based on CPU load.

This visualization also highlights the importance of minimizing container startup and readiness probe times, which directly affect how quickly new replicas become active.

Best Practices and Common Pitfalls

Optimize Startup Times and Readiness Probes

Ensure Medusa starts and becomes ready as quickly as possible.
Use readiness probes to signal when pods can receive traffic.
Long startup times can delay scaling and degrade performance.

Selecting the Right Scaling Metrics

CPU utilization is common, but not always optimal.
Consider custom business metrics (e.g., requests per second, queue depth).

Dealing with HPA Sync Delays

The default HPA sync period is 15 seconds, which might be too slow for bursty workloads.
Decreasing it improves responsiveness but can increase API traffic and controller load.

Conclusion

Implementing HPA and KEDA for your Medusa.js e-commerce platform ensures efficient scaling, better performance under load, and optimal resource utilization.

Key takeaways:

Always configure readiness probes and optimize startup times.
Choose scaling metrics carefully - CPU isn’t always the best indicator.
Monitor your autoscaler’s responsiveness using Grafana and Prometheus.
Test your setup under realistic load scenarios to validate scaling behavior.

With the right configuration, Kubernetes can make your Medusa deployment both resilient and self-scaling, ensuring you’re always ready for traffic spikes.

FAQ: Scaling Medusa in Kubernetes

Q1: What is the best way to scale Medusa on Kubernetes?

A1: Use Kubernetes HPA with metrics from Prometheus and cAdvisor, or event-driven scaling through KEDA for more flexibility.

Q2: Does Medusa support autoscaling natively?

A2: Medusa itself doesn’t manage scaling, but it runs well in Kubernetes environments that use HPA or KEDA.

Q3: Which metrics should I track for autoscaling?

A3: Start with CPU and memory usage; consider adding metrics like request rate, queue size, or API latency for advanced control.

Q4: How can I monitor Medusa’s scaling performance?

A4: Integrate Prometheus with Grafana dashboards to visualize pod utilization and replica changes in real time.

Q5: What’s the difference between HPA and KEDA?

A5: HPA relies on internal Kubernetes metrics, while KEDA extends it to external sources like Prometheus queries or event systems.

Missing files in your Packer built image? You might be skipping graceful shutdowns

uninterrupted — Thu, 23 Oct 2025 11:04:49 +0000

Are your files and folders missing after you run the newly created image? This is a common issue when building images with Packer. It is a widely used tool for creating machine images in a repeatable and automated way. When using the QEMU builder to generate local or CI-friendly images, it's common to upload files with the file provisioner and configure systems using shell provisioner. One often overlooked but critical part of this workflow is how the virtual machine shuts down after provisioning. Without a proper shutdown, the resulting image can be unstable, incomplete, or even unbootable.

Let’s see why graceful shutdowns matter.

After provisioning completes, Packer snapshots the virtual disk to produce the final image. If the guest operating system is not properly shut down before this snapshot, you risk file system corruption, lost configuration changes, services failing to start, or persistent files missing entirely - especially those written during the final steps of provisioning. These issues often go unnoticed until the image is deployed in staging or production environments, where the consequences are far more disruptive.

By default, the shutdown_command is set to an empty string (""). This means Packer does not attempt a clean shutdown unless you explicitly configure one. Instead, the virtual machine is forcefully terminated, which is equivalent to abruptly cutting power on a physical machine - a common cause of data loss and image inconsistency.

To prevent this, you must define a shutdown_command that instructs the OS to shut down safely after provisioning. This ensures that services have time to stop, file operations complete, and the system reaches a stable state before the image is captured.

Correct usage of `shutdown_command`

A clean shutdown requires a properly formed command with the necessary privileges, and the exact command you use depends on the operating system inside the virtual machine. Most Linux distributions support the shutdown command, but the syntax, required privileges, and available tools can vary slightly.

Here’s a reliable example in HCL2 for a Debian/Ubuntu based image:

...
shutdown_command = "echo 'packer' | sudo --stdin shutdown --poweroff now"
...

This command assumes the use of a user with password-based sudo access. It triggers a privileged system shutdown, allowing services to stop cleanly and file buffers to flush before power-off. The --poweroff flag ensures the machine actually powers off, which is essential for clean termination when using virtual hardware emulation. Adjust the password to match your base image setup.

If you're using a different base image - such as CentOS, RHEL, or Alpine - the shutdown binary might be located elsewhere, require different flags, or use an alternative like poweroff or halt. You may also need to configure sudoers to avoid prompts or password issues during provisioning.

Always test the shutdown command interactively inside a VM before inserting it into your template. It should work silently and reliably in the automated provisioning context.

Conclusion

To sum up, graceful shutdowns are a necessary safeguard when building machine images with the QEMU builder in Packer. Skipping this step or relying on Packer’s default behavior (which can forcibly terminate the VM) introduces the risk of unstable and corrupted images. By using a proper shutdown_command, you ensure the system shuts down in a controlled way, preserving the integrity of the file system and producing a clean, bootable image.

This practice is simple to implement and greatly improves reliability in both development and production environments. If you're managing CI pipelines or building images for downstream automation, it's a foundational step that prevents subtle and costly failures later in the lifecycle.

Headless CMS in E-Commerce: How to Integrate Medusa for Scalable, Content-Rich Online Stores

uninterrupted — Wed, 01 Oct 2025 08:03:00 +0000

Modern e-commerce isn’t just about selling products; it’s about delivering dynamic, content-driven experiences across multiple channels. To achieve this, businesses lean on Content Management Systems (CMS) for structured, scalable content workflows. With the emergence of headless commerce engines like Medusa.js and modern framework like Next.js, integrating a CMS has become more streamlined and customizable than ever.

Defining CMS in the E-Commerce Stack

A Content Management System (CMS) is a tool that lets you create, edit, and organize website content without touching code.

In e-commerce terms:

It separates content (product descriptions, blog posts, banners) from your storefront (the part customers see and shop in).
Editors and marketers can add or update pages in minutes without breaking anything.
Developers don’t have to be “content gatekeepers” — the CMS is the safe workspace for all non-technical updates.

Think of it as Google Docs for your website content, but integrated into your online store.

Medusa.js: Architecture and Integration

Medusa.js is an open-source, Node.js-based headless commerce engine. Unlike traditional monolithic platforms (e.g. Magento, Shopify), Medusa.js separates the commerce backend from the frontend, exposing commerce logic through APIs. This architecture supports integration with any frontend framework (e.g. Next.js) or CMS.

Why Pair Medusa.js with a CMS?

Medusa.js powers the transactional side of your storefront. A CMS adds that storytelling layer, and storefront (e.g. Next.js) is the perfect glue that pulls both content and commerce into a seamless experience.

Examples:

Blog posts — CMS stores articles, Next.js fetches them alongside Medusa product data for a fully integrated shopping-and-reading experience.
Landing pages — Launch seasonal campaigns in the CMS; Next.js dynamically renders them with real-time Medusa product data.
Rich product descriptions — Pull long-form text, videos, and extra images from the CMS while keeping inventory and pricing from Medusa.
SEO improvements — Use Next.js server-side rendering (SSR) or static site generation (SSG) to ensure Google sees your content instantly.

Key Decision Factors When Choosing a CMS

When selecting a CMS to pair with Medusa, consider the following trade-offs:

Cloud (SaaS) vs. Self-Hosted
- Cloud (SaaS): Ideal when your scale is small to medium or when you lack in-house DevOps expertise. Fast to set up, minimal maintenance, and predictable subscription costs. However, you trade off some control, and expenses can grow significantly as traffic, content volume, or user count increases.
- Self-Hosted: Full control and deep customization, with the potential for lower raw hosting costs in the long term. But infrastructure setup, deployments, scaling, security, and ongoing maintenance require time and expertise — which translates directly into operational cost.
Developer Friendliness vs. Ease of Use for Editors
- Some CMSs (like Payload) are highly flexible and API-driven, offering developers full control but requiring training for non-technical editors.
- Others (like Strapi) strike a balance, with strong developer features and a user-friendly admin panel.
- Platforms like Sanity and Contentful lean heavily toward editor experience, prioritizing usability and real-time collaboration.
Community and Plugins
- A strong community speeds up problem-solving and provides a library of ready-to-use plugins.
- For example, Strapi’s plugin ecosystem includes tools for image optimization, role-based permissions, analytics, and more.

CMS Selection Matrix for Medusa.js Integration

CMS	Type	Key Features	Best For
Strapi	Open-source (self-hosted or cloud)	Customizable, large OSS community, plugin ecosystem	Teams wanting balance between dev control and editor usability
Payload	Open-source (self-hosted or cloud)	Developer-centric, clean editor, powerful APIs	Dev-heavy teams building tailored workflows
Sanity	Headless SaaS	Real-time collaboration, content portability	Content-heavy teams with remote collaboration needs
Contentful	Headless SaaS	Enterprise-ready, robust APIs, scalable	Large organizations needing stability and global delivery

Example Implementation: Medusa + Strapi + Next.js

Medusa.js: Manages catalog, checkout, orders, and authentication workflows.
Strapi: Controls homepage banners, blog content, landing pages, and SEO metadata.
Next.js: Consumes APIs from both Medusa and Strapi to render the frontend.

This decoupled structure allows content managers and developers to work in parallel, ensuring rapid deployment and updates across both commerce and content layers.

Conclusion

Combining a headless CMS with Medusa.js and modern storefront framework enables highly customizable, scalable, and performant e-commerce platforms. This architecture decouples content and commerce, allowing independent optimization and rapid iteration. As the industry moves toward open, API-driven systems, Medusa.js and compatible CMS platforms provide a robust foundation for next-generation digital commerce.

Kubernetes logs unavailable behind a proxy: Diagnosing API server communication issues

uninterrupted — Wed, 06 Aug 2025 13:20:14 +0000

In modern infrastructure setups, especially on-premises environments, the presence of outbound proxies is increasingly common. These proxies often enforce organizational access policies and provide observability into network traffic. While container orchestration platforms like Kubernetes can generally operate well under such constraints, some subtle and easily overlooked configuration issues can lead to unexpected behavior.

In this article, I’ll walk through a real-world issue we encountered where Kubernetes pod logs were not retrievable via the API, despite the cluster appearing healthy otherwise. We'll also dive deeper into how proxy behavior interacts with Kubernetes components.

The symptom: cluster looks healthy, but `kubectl logs` fails

We were running a Kubernetes cluster on bare metal within a datacenter. The environment was placed behind a corporate HTTP proxy.

On the surface, everything appeared operational:

kubectl get pods, kubectl get svc, and other API queries worked fine,
etcd connectivity was intact,
Node and pod statuses reported healthy.

However, when attempting to retrieve logs from running pods using kubectl logs <pod>, the command failed with a timeout or a generic connection error. This was observed across all pods and namespaces.

Investigating the root cause

Kubernetes retrieves pod logs through the kubelet on the node where the pod is running. The kube-apiserver acts as a reverse proxy for these requests and must establish a direct connection to the kubelet endpoint (typically via HTTPS on port 10250). This is where the problem manifested.

We first validated that logs were, in fact, being generated. By SSH-ing into the node where the pod was scheduled and using crictl logs <container_id>, we could view logs without issue. This pointed to a problem not with the container runtime or logging configuration, but with how the kube-apiserver was trying to reach kubelet.

Further inspection revealed that the kube-apiserver container was not bypassing the proxy for internal node-to-node communication. This wasn't a failure of the proxy settings - it was because the NO_PROXY environment variable had never been set in the first place. In environments operating behind a proxy, this omission can critically impair the cluster's internal operations. Without an explicit NO_PROXY configuration, even local traffic destined for internal services like the kubelet may be incorrectly routed through the proxy, leading to timeouts and unpredictable failures.

How HTTP proxies work in Kubernetes environments

To understand the root of the problem, it's important to grasp how HTTP proxies interact with Kubernetes networking.

When you set the environment variables HTTP_PROXY or HTTPS_PROXY, applications on the host — including kube-apiserver, container runtime clients, and even kubelet — will route outbound traffic through the proxy server, unless explicitly told not to via NO_PROXY.

In a Kubernetes cluster:

Most control plane components communicate over private IPs or node-local addresses,
The kube-apiserver connects to the kubelet to fetch logs, execute commands, or forward ports.

If the API server tries to connect to the kubelet via a private IP (e.g., 10.0.0.12) and this IP is not included in the NO_PROXYlist, the request is sent to the proxy. Since proxies typically can't route to internal IPs or ports like 10250 (kubelet), the request fails silently or times out.

This is further complicated on bare metal because:

Cloud provider integrations (e.g., in EKS, GKE) often auto-configure these exemptions,
Static pods don’t inherit host environment variables unless explicitly defined in their manifests.

The fix: update static pod environment variables

Because kube-apiserver was running as a static pod, its environment variables had to be defined explicitly in the corresponding manifest file, typically located at:

/etc/kubernetes/manifests/kube-apiserver.yml

We modified this file to include proper proxy settings, specifically ensuring the NO_PROXY variable covered:

All node IP addresses
The cluster CIDR
The service CIDR
Loopback addresses
Hostnames used by the control plane

Here is a simplified version of the environment variable section added to the static pod spec:

env:
  - name: HTTP_PROXY
    value: "http://proxy.example.com:3128"
  - name: HTTPS_PROXY
    value: "http://proxy.example.com:3128"
  - name: NO_PROXY
    value: "127.0.0.1,localhost,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12,.cluster.local"

After saving the file, the kubelet automatically reloaded the manifest and restarted the kube-apiserver. Once the API server came back online, kubectl logs began returning expected output from all pods.

Takeaways and best practices

This experience reinforced the importance of configuring NO_PROXY correctly when operating in a proxied environment, particularly on bare metal where defaults that “just work” in cloud environments may not apply.

Key points to remember:

The API server needs direct access to kubelets to retrieve logs, exec into pods, and perform port-forwarding
Failing to set NO_PROXY for internal traffic can cause silent and hard-to-debug failures
Static pods require environment variables to be set in the pod manifest, not in the host shell environment
Use tools like crictl, curl, or tcpdump on the control plane node to confirm where traffic is being routed
Explicitly list IP ranges and DNS suffixes used within the cluster in NO_PROXY

Layouts in Next.js: How to Structure Your E-commerce Platform the Right Way

uninterrupted — Mon, 04 Aug 2025 12:58:03 +0000

The Next.js 15 application router provides first-class support for layouts, making shared UI patterns seamless, efficient and composable. Here's a deeper dive into how layouts work, their benefits and best practices.

What are Layouts?

Layouts are React components that wrap pages or sub-layouts in app/, rendering children properties for nested content
The main layout (in app/layout.tsx) is mandatory - it must contain <html>...<body> tags and applies globally.
Additional layouts in subdirectories (e.g., app/categories/layout.tsx) only include routes in that segment.

Folder-based hierarchy and nesting

File system routing dictates layout behavior:

app/
  layout.tsx          ← root layout (all routes)
  page.tsx            ← homepage
  categories/
    layout.tsx        ← wraps categories routes
    page.tsx          ← /categories
    [slug]/
      page.tsx        ← /categories/[slug]
  profile/
    layout.tsx        ← wraps client profile route
    page.tsx          ← /client

Here root layout wraps every page, profile layout wraps only profile routes - nesting children accordingly. Wrapping a folder name in square brackets (e.g. [slug]) creates a dynamic route segment which is used to generate multiple pages from data. (e.g. specific category pages, dedicated product pages, etc.)

Benefits of using Layouts

1. Preservation of state
Since layouts persist during navigation, the state of shared components (e.g. sidebars) is not lost, reducing reassembly and improving UX. This makes an ideal place to initialize global contexts. Especially useful when some of the values come from cookies or headers - helping to eliminate prop drilling and improving the maintainability of your codebase.

2. Performance optimization
Layouts are server-side components by default and are cached. Client-side navigation keeps layouts live, minimizing re-rendering.

3. Modular UI Structure
You can define different layouts for different sections of the application - e.g., category area vs. client profile - by nesting or using route groups.

4. Clean code organization
Layouts enforce structure and collocated logic - styling, metadata, additional providers - all in route-specific files.

First steps: Basic concepts

Defining a root layout:

// app/layout.tsx
export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        <Header />
        <main>{children}</main>
        <Footer />
      </body>
    </html>
  )
}

This defines a globally shared UI and is required.

Adding nested layouts

// app/categories/layout.tsx
export default function CategoriesLayout({ children }: { children: React.ReactNode }) {
  return (
    <div className="flex">
      <SideNav />
      <div className="flex-grow">{children}</div>
    </div>
  )
}

Wraps categories/ routes, stacking inside the root layout.

Best practices

Main vs. nested: Only main layouts contain <html> and <body> tags. Subordinate layouts wrap inside them without duplicating these tags.
Clean structure: Store layout.tsx, page.tsx and relevant UI components together by route segment.
Selective nesting: Use nested layouts only where necessary to avoid unnecessary UI on pages.
Use of server components: Default layouts are server-side - ideal for retrieving data via parameters or shared UI logic.
Route groups: Create clear layout boundaries (e.g., categories/ vs. profile/) to isolate sections - allowing for multiple layouts at the root level if needed.

Summary

Layouts empower you to:

Function	Description
Shared UI	Headers, footers, navigations applied globally or segmented
Nested UIs	Each route folder can define its own wrap layout
Smooth state	Maintain React state in nav without full reloading
Organized code	UI and logic co-located per route for ease of maintenance

Next steps

Define a global shell in app/layout.tsx.
Add segment-specific layouts if necessary.
Organize code by route directory for modularity.
Use route groups if you separate contexts with different user interfaces.

These patterns make your Next.js application more maintainable, efficient and user-friendly.

Happy building!

Manage user cookie consent with Google Tag Manager: Adapting to CookieConsent v3

uninterrupted — Mon, 14 Jul 2025 07:00:00 +0000

Intro

With the release of CookieConsent v3, we've decided to create this article to help you understand and adapt to the new version. This article builds on concepts and areas discussed in our previous post. For a deeper dive and to see the full adaptation process, please read our previous step-by-step guide.

Adding a Cookie Consent Banner

First, let's look at the changes in the default configuration of Cookie Consent. Here's an example of the code you need to paste into Custom HTML in Tag Configuration:

<link rel="stylesheet" type="text/css"
  href="https://cdn.jsdelivr.net/gh/orestbida/cookieconsent@3.0.1/dist/cookieconsent.css" />
<script src="https://cdn.jsdelivr.net/gh/orestbida/cookieconsent@3.0.1/dist/cookieconsent.umd.js"></script>
<script>
  CookieConsent.run({
    // https://cookieconsent.orestbida.com/reference/configuration-reference.html#guioptions
    guiOptions: {
      consentModal: {
        layout: 'cloud inline',
        position: 'bottom left',
        equalWeightButtons: true,
        flipButtons: false
      },
      preferencesModal: {
        layout: 'box',
        equalWeightButtons: true,
        flipButtons: false
      }
    },

    onFirstConsent: function(cookie) {
      // callback triggered only once on the first accept/reject action
    },

    categories: {
      necessary: {
        enabled: true,  // this category is enabled by default
        readOnly: true  // this category cannot be disabled
      },
      analytics: {
        autoClear: {
          cookies: [
            {
              name: /^_ga/,   // regex: match all cookies starting with '_ga'
            },
            {
              name: '_gid',   // string: exact cookie name
            }
          ]
        },

        // https://cookieconsent.orestbida.com/reference/configuration-reference.html#category-services
        services: {
          ga: {
            label: 'Google Analytics',
            cookies: [
              {
                name: /^(_ga|_gid)/
              }
            ]
          }
        }
      },
      targeting: {}
    },

    language: {
      default: 'en',
      translations: {
        en: {
          consentModal: {
            title: 'We use cookies',
            description: 'Cookie modal description',
            acceptAllBtn: 'Accept all',
            acceptNecessaryBtn: 'Reject all',
            showPreferencesBtn: 'Manage Individual preferences'
          },
          preferencesModal: {
            title: 'Manage cookie preferences',
            acceptAllBtn: 'Accept all',
            acceptNecessaryBtn: 'Reject all',
            savePreferencesBtn: 'Accept current selection',
            closeIconLabel: 'Close modal',
            sections: [
              {
                title: "Cookie usage",
                description: "We use cookies to ensure the basic functionalities of the website and to enhance your online experience ..."
              },
              {
                title: "Strictly necessary cookies",
                description: "These cookies are essential for the proper functioning of my website. Without these cookies, the website would not work properly",
                linkedCategory: "necessary"
              },
              {
                title: "Performance and Analytics cookies",
                description: "These cookies allow the website to remember the choices you have made in the past",
                linkedCategory: "analytics",
                cookieTable: {
                    headers: {
                      name: "Name",
                      domain: "Service",
                      description: "Description",
                      expiration: "Expiration"
                    },
                    body: [
                      {
                        name: "_ga",
                        domain: "Google Analytics",
                        description: "Cookie set by <a href=\"#das\">Google Analytics</a>",
                        expiration: "Expires after 12 days"
                      },
                      {
                        name: "_gid",
                        domain: "Google Analytics",
                        description: "Cookie set by <a href=\"#das\">Google Analytics</a>",
                        expiration: "Session"
                      }
                    ]
                  }
                },
                {
                    title: 'Targeting and Advertising',
                    description: 'These cookies are used to make advertising messages more relevant to you and your interests. The intention is to display ads that are relevant and engaging for the individual user and thereby more valuable for publishers and third party advertisers.',
                    linkedCategory: 'targeting',
                },
                {
                  title: 'More information',
                  description: 'For any queries in relation to my policy on cookies and your choices, please <a href="#contact-page">contact us</a>'
                }
            ]
          }
        }
      }
    }
  });
</script>

Once you preview your application through the GTM panel, you should see a banner in the bottom left corner of the page:

Configuring GTM Variables

When reading a cookie, it's possible that you might receive a string instead of an object when trying to read it by name.

To prevent this, make sure to check the URI-decode cookie box when adding a 1st Party Cookie in Variable Configuration:

Consent Variables

The script we presented earlier for reading the selected field in the banner works correctly for selecting an entire category.

function() {
  var cookie = {{CC Cookie}}
  if (!cookie) { return 'denied'; }
  var json = JSON.parse(cookie)
  if (json["categories"].includes("analytics")) {
    return "granted";
  } else {
    return 'denied';
  }
}

In this article, we'll stick with this configuration. However, Cookie Consent also allows for the configuration of acceptance for individual services if they have been added to the configuration.

// https://cookieconsent.orestbida.com/reference/configuration-reference.html#category-services
services: {
  ga: {
    label: 'Google Analytics',
    cookies: [
      {
        name: /^(_ga|_gid)/
      }
    ]
  }
}

If you want to create variables for individual services, replace the condition in the script. For example json["categories"].includes("analytics") to json["services"]["analytics"].includes("ga").

💡 Tip: Note that even if there are no services in a given category, the services object will contain an array for that category, which will be empty.

Updating GTM Consents

Another update involves renaming fields in Consent Mode (Google tags). These fields now appear as follows:

💡 Tip: You can also utilize the functionality of services to assign the appropriate consent to individual fields.

Conclusion

Thank you for reading! For more detailed steps and additional information, make sure to check out our previous post. If you have any questions or need further assistance, feel free to reach out.

DEV Community: uninterrupted

Faster Geospatial Animations with PMTiles: Multi-Layer MVT Time Series in OpenLayers

The architecture: single archive, dynamic layers

Complete working example

Step 1 — Per-timestep GeoJSON files

Step 2 — Generate PMTiles with named layers (generate-tiles.sh)

Step 3 — Start the server

Step 4 — Client animation (index.html)

Running the full setup

Conclusion: paint time, not flip it

Next.js 16 Caching for E-Commerce: Cache Components, use cache, revalidateTag, and Fresh Product Data

Why Caching Is Harder in E-Commerce Than in a Typical Content Site

The Three Cache Boundaries That Matter Most

1. Stable catalog content

2. Shared but fast-changing commerce data

3. User-specific or session-specific data

What Next.js 16 Changes in Practice

A Practical Storefront Strategy

Cache the stable product content

Use a shorter cache boundary for price and stock

Keep customer-specific data out of the shared cache

Why This Matters for Medusa and Other Headless Commerce Backends

Using revalidateTag to Keep Data Fresh

Where Teams Usually Get It Wrong

Mistake 1: Caching personalized pricing globally

Mistake 2: Making the whole page dynamic because one widget is dynamic

Mistake 3: Using no-store too broadly

Mistake 4: Forgetting business dimensions in cache keys

Mistake 5: Mixing editorial freshness with operational freshness

A Good Default Model for Product Pages

Why This Works Well With Streaming and PPR Thinking

A Note on Webhooks and Operational Ownership

Final Thoughts

Asset-Based Data Orchestration: Lessons from Building a Multi-State Social Data Platform

1. Why Reliability Becomes a Systems Problem at Scale

2. System Overview

High-Level Architecture

3. Source Heterogeneity as the Dominant Constraint

4. Normalization (HSDS, SDOH or Equivalent) as an Architectural Contract

5. What We Got Wrong Initially

6. Transitioning to Asset-Based Data Orchestration with Dagster

7. Partitioning for Failure Isolation

8. Data Quality Embedded in the Asset Graph

9. Operational Outcomes

10. Open Trade-offs and Unresolved Questions

11. Why These Lessons Matter Beyond This Platform

Final Thoughts

FAQ

What is asset-based data orchestration?

Why is asset-based orchestration useful for complex data platforms?

How does Dagster differ from traditional pipeline orchestrators?

When should teams adopt asset-based orchestration?

Does asset-based orchestration replace tools like DBT?

Next.js 16 Partial Prerendering (PPR): The Best of Static and Dynamic Rendering

What Is Partial Prerendering (PPR)?

How Partial Prerendering Works in Next.js 16

Cache Components: The Heart of PPR

Streaming and Suspense: The Engine Under the Hood

Why PPR Matters for E-commerce

How to Enable and Use PPR in Next.js 16

How PPR Fits with Other Rendering Strategies

Wrapping Up

Using Proxy (before Middleware) in Next.js: a modern layer

What is Proxy and when to use it

Setting up Proxy: conventions & example

Use cases & how it fits e-commerce / frontend/back-end scenarios

Summary

Automated Tax Compliance for Global E-Commerce Growth

Three Compliance Challenges That Break Legacy Platforms

The Performance-Compliance Tradeoff (That Shouldn't Exist)

The Technical Foundation Matters

Building Audit-Ready From Day One

The Platform Investment Decision

Scale Without Technical Debt

Need help with Avalara integration?

A Practical Guide to Scaling Medusa with Kubernetes Autoscalers

Prerequisites for Medusa Autoscaling on Kubernetes

Required Tools: cAdvisor, Prometheus, and KEDA

Setting Up Metrics for HPA

Configuring cAdvisor for container-level metrics

Step 2 — Generate PMTiles with named layers (`generate-tiles.sh`)

Step 4 — Client animation (`index.html`)

Using `revalidateTag` to Keep Data Fresh

Mistake 3: Using `no-store` too broadly

Correct usage of `shutdown_command`

The symptom: cluster looks healthy, but `kubectl logs` fails