DEV Community: Muhammad Masad Ashraf

8 Error Recovery Patterns That Keep Shopify GraphQL Clients Alive

Muhammad Masad Ashraf — Mon, 29 Jun 2026 15:00:17 +0000

If you have shipped anything against the Shopify GraphQL API, you have probably been burned by this: the request returns 200 OK, your logging looks clean, and yet the order never got created.

That is the part of GraphQL that catches people off guard. The transport layer and the application layer fail separately. A clean HTTP status tells you almost nothing.

Below are the eight patterns I lean on to build clients that hold up under real traffic. Think of it less as theory and more as the checklist I run through before calling an integration "done."

First, stop trusting the status code

REST trained us to read the HTTP status and move on. 404 = missing, 500 = server died, body is an afterthought.

GraphQL flips that. A response can succeed at the HTTP level while quietly failing in the payload:

{
  "data": { "product": null },
  "errors": [
    {
      "message": "Throttled",
      "extensions": { "code": "THROTTLED" }
    }
  ]
}

Two takeaways:

data can hold partial results.
errors explains what broke.

Your client has to read both, every time.

Classify before you react

Different failures want different responses. Retrying a validation error is just burning rate limit for nothing.

Error type	Where it shows up	Retryable?	What to do
Network failure	Transport layer	Yes	Retry with backoff
Throttling (`THROTTLED` / 429)	Top-level `errors`	Yes	Wait, then retry
Server error (5xx)	HTTP status	Yes	Retry with backoff
User errors	`userErrors` field	No	Fix input, surface to user
Validation errors	Top-level `errors`	No	Fix the query
Auth errors	HTTP 401 / 403	No	Refresh token / re-auth

Rule of thumb: transient errors fix themselves with time, everything else needs a code or input change.

Pattern 1: Treat `userErrors` as a real failure

This is the one that bites everyone. Mutations report through two channels: top-level errors for system stuff, and userErrors inside the payload for business-logic stuff.

{
  "data": {
    "productUpdate": {
      "product": null,
      "userErrors": [
        { "field": ["title"], "message": "Title can't be blank" }
      ]
    }
  }
}

HTTP 200. Empty top-level errors. Still failed.

function handleMutation(response) {
  const userErrors = response.data?.productUpdate?.userErrors || [];
  if (userErrors.length > 0) {
    // Business logic failure — do NOT retry.
    return { success: false, errors: userErrors };
  }
  return { success: true, data: response.data.productUpdate.product };
}

Skip this check and you get silent data loss. Always treat a non-empty userErrors as a failure.

Pattern 2: Exponential backoff (with jitter)

Transient errors deserve a retry. Blind retries make everything worse — hammering a throttled API just deepens the throttle.

async function withRetry(fn, maxRetries = 5) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (!isRetryable(err) || attempt === maxRetries - 1) throw err;
      const delay = Math.min(1000 * 2 ** attempt, 30000);
      await sleep(delay + Math.random() * 200); // jitter
    }
  }
}

The Math.random() is not decoration. Without jitter, every client retries at the same instant and you get a thundering herd that re-crashes the API. The Math.min caps the delay so waits stay sane.

Pattern 3: Throttle before Shopify throttles you

Reactive retries are fine. Proactive throttling is better. Shopify hands you a query-cost budget on every response — read it:

{
  "extensions": {
    "cost": {
      "requestedQueryCost": 101,
      "throttleStatus": {
        "maximumAvailable": 1000,
        "currentlyAvailable": 899,
        "restoreRate": 50
      }
    }
  }
}

Track currentlyAvailable. When it dips low, slow yourself down instead of waiting for a THROTTLED slap.

Pattern 4: Circuit breakers for cascading failures

Sometimes the problem is upstream. Retrying a struggling Shopify just piles on load. A circuit breaker fails fast instead.

State	Behavior	Transition
Closed	Requests flow	Opens after failure threshold
Open	Fail fast, no calls	Half-open after cooldown
Half-open	One test request	Closes on success, reopens on failure

class CircuitBreaker {
  constructor(threshold = 5, cooldown = 30000) {
    this.failures = 0;
    this.threshold = threshold;
    this.cooldown = cooldown;
    this.state = "closed";
    this.openedAt = null;
  }

  async call(fn) {
    if (this.state === "open") {
      if (Date.now() - this.openedAt > this.cooldown) {
        this.state = "half-open";
      } else {
        throw new Error("Circuit open");
      }
    }
    try {
      const result = await fn();
      this.reset();
      return result;
    } catch (err) {
      this.recordFailure();
      throw err;
    }
  }

  recordFailure() {
    this.failures++;
    if (this.failures >= this.threshold) {
      this.state = "open";
      this.openedAt = Date.now();
    }
  }

  reset() {
    this.failures = 0;
    this.state = "closed";
  }
}

Pattern 5: Use partial data, don't discard it

GraphQL can return some good fields alongside some failed ones. Don't throw the whole response away — log the bad parts, use the good ones.

function processResponse(response) {
  const { data, errors } = response;
  if (errors?.length && data) {
    logErrors(errors);
    return { partial: true, data };
  }
  if (errors?.length) return { partial: false, errors };
  return { partial: false, data };
}

This matters most on bulk reads. One missing product should never nuke an entire catalog sync.

Pattern 6: Idempotency makes retries safe

Here is the trap: a mutation creates an order, the network times out before you get the response, your retry logic fires, and now you have two orders.

Idempotency keys fix this. Attach a unique key per operation, and the server returns the original result instead of running it again. Without this, every retry path you build is a duplicate-data risk.

Pattern 7: Dead letter queues for the ones that still fail

Some operations exhaust every retry. Don't drop them silently:

Operation fails after all retries.
Push the payload + error context to a dead letter queue.
Alert the team.
Fix the cause, then replay from the queue.

This is your safety net during outages. The difference between "we lost 40 minutes of orders" and "we replayed them once the API recovered."

Pattern 8: You can't fix what you can't see

Recovery without observability is guessing. Track these per operation:

Metric	What it tells you
Error rate by type	Which failures dominate
Retry count	How hard the client is working
Throttle frequency	Whether you're over budget
Circuit breaker trips	When things are degrading
Query cost trends	Where your budget goes

Log full context: query, variables, response, timing.

Putting it together

The patterns stack in a specific order:

Classify the error.
Apply idempotency keys before any retry.
Retry transient errors with backoff + jitter.
Throttle proactively using cost data.
Break the circuit when failures cascade.
Queue the unrecoverable ones.
Monitor all of it.

Build this in from day one. Retrofitting recovery into a live system costs far more than designing it up front — usually right when you can least afford it (hi, Black Friday).

I originally wrote a longer version of this with extra implementation notes on the Kolachi Tech blog: Error Recovery Patterns in Shopify GraphQL. What does your retry + idempotency setup look like? Curious how other people handle the dead-letter replay step.

How I Stopped Shopify's GraphQL API From Throttling My App (Parallel Query Patterns That Actually Work)

Muhammad Masad Ashraf — Thu, 25 Jun 2026 21:45:25 +0000

A dashboard I built was taking 4+ seconds to load. Products, orders, inventory, customer data — each query waited politely for the one before it to finish. Classic sequential bottleneck.

The fix was parallelism. But naive parallelism on Shopify's GraphQL API gets you throttled fast, because it doesn't count requests — it counts cost. So I had to learn how to go parallel and stay under the rate ceiling.

This post is everything I wish I'd known before I started. Code included.

Why sequential queries are the silent killer

Picture a dashboard loading three things:

Product data
Order history
Inventory counts

Sequential, you pay the sum of all three. Parallel, you pay only the slowest single request. At scale, that gap is the difference between a snappy app and one users abandon.

First, understand the cost model (this is the whole game)

Shopify doesn't limit you by request count. It uses a leaky-bucket cost system. Every query drains points; the bucket refills at a fixed rate.

Plan	Bucket Size (points)	Restore Rate (points/sec)
Standard	1,000	50
Advanced	2,000	100
Plus	10,000	500

Every parallel request you add drains the bucket faster. So before you parallelize anything, read the extensions field that comes back on every response:

const { data, extensions } = await response.json();

console.log(extensions.cost);
// {
//   requestedQueryCost: 12,
//   actualQueryCost: 8,
//   throttleStatus: {
//     maximumAvailable: 1000,
//     currentlyAvailable: 920,
//     restoreRate: 50
//   }
// }

That throttleStatus block is your fuel gauge. Tune everything else around it.

Pattern 1: Controlled concurrent batching

The naive version is just Promise.all:

const [products, orders, customers] = await Promise.all([
  fetchProducts(),
  fetchOrders(),
  fetchCustomers(),
]);

This works for a handful of independent queries. But fire 50 at once and you'll drain the bucket instantly. The fix is a concurrency limiter — cap how many run simultaneously:

import pLimit from "p-limit";

const limit = pLimit(5); // max 5 in flight at any time

const results = await Promise.all(
  productIds.map((id) => limit(() => fetchProduct(id)))
);

Five concurrent requests is a safe starting point. Watch your throttleStatus, then push it up or down.

Pattern 2: Aliasing — many queries, one request

If you already know the IDs, you don't even need multiple network calls. GraphQL lets you alias the same field repeatedly inside one query:

query {
  first: product(id: "gid://shopify/Product/1") { title }
  second: product(id: "gid://shopify/Product/2") { title }
  third: product(id: "gid://shopify/Product/3") { title }
}

One round trip, three products. The catch: the cost is the sum of all aliases, so don't pack so many in that a single query drains your bucket. Group enough to save round trips, not so many that you self-throttle.

Pattern 3: Parallel pagination via segmentation

Cursor-based pagination is inherently sequential — you can't know page 2's cursor until page 1 returns. So you can't naively parallelize it.

The trick is segmentation. Split the dataset by a known field (date range, ID range), then paginate each segment in parallel:

const segments = [
  { query: "created_at:>=2026-01-01 created_at:<2026-04-01" },
  { query: "created_at:>=2026-04-01 created_at:<2026-07-01" },
];

const limit = pLimit(2);
const pages = await Promise.all(
  segments.map((seg) => limit(() => paginateAll(seg.query)))
);

Each segment paginates independently. Just remember each one still burns points, so coordinate your concurrency.

Pattern 4: When to stop parallelizing and use Bulk Operations

Here's the counterintuitive part: sometimes parallelism is the wrong answer. If you need every product or every order, stop fighting rate limits and use the Bulk Operations API. It runs your query async on Shopify's servers and hands you a downloadable file. No throttling.

Quick decision table:

Scenario	Best approach
Few known records	Query aliasing
Real-time dashboard	Concurrent batching
Full catalog export	Bulk operations
Paginated lists	Parallel pagination

Pattern 5: Coalesce duplicate in-flight requests

Two components both asking for the same product? Don't fire two queries. Coalesce them — detect the identical in-flight request and have both callers await the same promise:

const inFlight = new Map();

function dedupedFetch(key, fetcher) {
  if (inFlight.has(key)) return inFlight.get(key);

  const promise = fetcher().finally(() => inFlight.delete(key));
  inFlight.set(key, promise);
  return promise;
}

Pair this with response caching and you eliminate a huge chunk of parallel load before it ever reaches Shopify.

Backpressure: mirror Shopify's bucket on your side

The cleanest way to never get throttled is to model the bucket locally. Run your own token bucket — acquire a token before each request, refill at Shopify's restore rate. No token, you wait. Your app physically cannot exceed the allowed rate.

For high-volume systems, push this into a real queue so spikes get absorbed and metered instead of slamming the API all at once.

Handle throttles gracefully (because they'll still happen)

Even with good planning, a traffic spike or an unexpectedly expensive query pushes you over. When Shopify returns THROTTLED, never just retry immediately. Use exponential backoff with jitter:

async function withBackoff(fn, attempt = 0) {
  try {
    return await fn();
  } catch (err) {
    if (!isThrottled(err) || attempt >= 5) throw err;
    const base = 2 ** attempt * 500;
    const jitter = Math.random() * 500;
    await sleep(base + jitter);
    return withBackoff(fn, attempt + 1);
  }
}

The jitter is the important bit — without it, all your retries fire at the same instant and you get a thundering herd.

Design for partial failure

In a parallel batch, one request can fail while the rest succeed. Don't let Promise.all reject the whole thing. Use Promise.allSettled and handle each outcome:

const results = await Promise.allSettled(tasks);

const succeeded = results.filter((r) => r.status === "fulfilled");
const failed = results.filter((r) => r.status === "rejected");

// retry only the failures

Degrade gracefully. A single timeout shouldn't nuke an entire batch.

Measure everything

You can't tune what you don't watch. The four metrics I track:

Metric	What it tells you
Throttle rate	How often you hit limits
Average query cost	Per-request efficiency
Concurrent request count	Active parallelism level
Retry frequency	System stability

High throttle rate → dial concurrency down. Consistently low utilization → push it up. Set alerts on throttle spikes so you catch problems before users do.

The payoff: parallelism lowers your bill too

This isn't only about speed. Efficient batching, coalescing, and caching all cut redundant calls. Fewer calls = lower API consumption = smaller bills and more headroom. The goal isn't just faster queries — it's faster queries that cost less.

TL;DR

Read the cost model first. It counts cost, not requests.
Batch with a concurrency limiter, not raw Promise.all.
Alias known IDs into single requests.
Segment to parallelize pagination.
Use Bulk Operations for full exports.
Coalesce + cache to kill duplicate calls.
Add backpressure, jittered backoff, and allSettled.
Monitor and tune from real data.

I write about Shopify performance and API architecture. The full, deeper version of this guide lives on my company blog. If you're scaling a Shopify app and hitting these walls, that's the kind of work we do at Kolachi Tech.

How I Build Fast Shopify API Clients (Batching, Caching, Backoff)

Muhammad Masad Ashraf — Tue, 23 Jun 2026 19:48:32 +0000

Slow API calls cost real money. They delay orders, stall inventory syncs, and frustrate users right when traffic peaks.

Here's the thing most teams miss: when a Shopify integration crawls under load, the bottleneck is almost never Shopify. It's the client layer you wrote around it.

This is how I think about building a client that stays fast whether you push 100 orders a day or 100,000.

The client is your performance ceiling

Shopify gives you the API. How you call it sets your speed.

Most clients start as a thin wrapper around fetch. Fine at small scale. Then volume grows and the cracks show up: 429s, timeouts, duplicate calls.

A solid client juggles three things at once:

Request rate (don't get throttled)
Data reuse (don't fetch what you already have)
Failure handling (don't crash the whole flow on one bad response)

Nail those three and optimization stops being a fire drill. It becomes a property of the system.

REST vs GraphQL

Your API choice shapes everything downstream.

REST is predictable but forces multiple round trips and over-fetches. GraphQL lets you grab exactly the fields you want in one call, at the cost of a query-cost budget you have to respect.

Factor	REST	GraphQL
Data shape	Fixed per endpoint	Custom per query
Over-fetching	Common	Rare
Round trips	Often multiple	Usually one
Rate limiting	Request-based (leaky bucket)	Cost-based (points)
Best for	Simple, stable reads	Complex, nested data

For most modern builds, GraphQL wins. Just budget your query cost carefully, a sloppy nested query drains your points fast.

Respect rate limits by design

Rate limits aren't obstacles. They're the rules of the road.

REST uses a leaky bucket. GraphQL uses a calculated cost model. Either way, hit the ceiling and you eat a 429.

A good client never blindly retries. It reads the rate-limit headers on every response and paces itself before the wall, not after. The cleanest version is a token bucket on your side that mirrors Shopify's:

class TokenBucket {
  constructor(capacity, refillPerSec) {
    this.capacity = capacity;
    this.tokens = capacity;
    this.refillPerSec = refillPerSec;
    this.last = Date.now();
  }
  async take(cost = 1) {
    this.refill();
    while (this.tokens < cost) {
      await new Promise(r => setTimeout(r, 100));
      this.refill();
    }
    this.tokens -= cost;
  }
  refill() {
    const now = Date.now();
    this.tokens = Math.min(
      this.capacity,
      this.tokens + ((now - this.last) / 1000) * this.refillPerSec
    );
    this.last = now;
  }
}

Batch to cut round trips

Every network call carries overhead. Fewer calls = faster results.

GraphQL batches naturally with aliases:

query {
  a: product(id: "gid://shopify/Product/1") { title }
  b: product(id: "gid://shopify/Product/2") { title }
  c: product(id: "gid://shopify/Product/3") { title }
}

Ten products in one call instead of ten. Latency drops, rate budget stretches.

For big datasets, reach for the Bulk Operations API. It runs server-side and hands you a downloadable file when ready.

Method	Best for	Speed profile
Single request	Real-time single record	Lowest latency
Batched query	A few related records	Fast, fewer calls
Bulk operation	Thousands of records	Highest throughput

Cache hard, invalidate smart

The fastest API call is the one you never make.

Product data, collections, and shop settings rarely change. Cache them. Set a TTL per data type, long for static, short for volatile stuff like inventory.

The hard part is invalidation. Stale data means wrong prices and overselling. Use webhooks to bust cache entries the moment upstream data changes.

A layered cache wins: in-memory for hot keys, Redis for distributed access.

Concurrency without self-throttling

Parallel requests speed you up. Too many at once get you throttled.

Use a concurrency limiter that caps parallel requests and queues the rest. Set the cap based on your rate budget, not your CPU count, your machine can fire hundreds, but Shopify will reject most.

Pair the concurrency pool with the token bucket: the pool controls parallel slots, the bucket controls overall pace.

Retry logic that heals itself

Failures happen. A naive client gives up or retries instantly (making it worse). A smart one backs off.

async function withRetry(fn, max = 5) {
  for (let attempt = 0; attempt < max; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (![429, 500, 502, 503, 504].includes(err.status)) throw err;
      const base = 2 ** attempt * 200;
      const jitter = Math.random() * 200;
      await new Promise(r => setTimeout(r, base + jitter));
    }
  }
  throw new Error("Max retries exceeded");
}

Retry only transient errors (429, 5xx, timeouts). Never retry a 400 or 422, the request itself is broken. Always cap retries, and pair them with idempotency keys so repeats never create duplicate orders.

Paginate the right way

Large result sets arrive in pages via cursors. Never loop through a huge dataset synchronously, push it into a background job.

Fetch a page, process it, request the next with the returned cursor, stop when hasNextPage is false.

Go async

Not every task needs an instant answer. When an order lands, accept it fast and queue the heavy lifting, inventory sync, CRM updates, notifications.

This keeps the client responsive during spikes and smooths your API usage instead of bursting.

Architect for resilience

A fast client on a fragile system still fails. Spread load across services. Add circuit breakers so one failing dependency doesn't drag everything down. Monitor latency, error rates, and throttle counts, you can't fix what you can't see.

Design for failure as the normal case. Degrade gracefully, don't collapse.

Quick audit checklist

Area	Action	Impact
API choice	Prefer GraphQL for nested data	Fewer round trips
Rate limits	Track headers, self-throttle	No 429 storms
Batching	Group reads, use bulk ops	Higher throughput
Caching	Layer memory + Redis	Skip redundant calls
Concurrency	Cap parallel requests	Stable under load
Retries	Backoff with jitter	Self-healing
Async	Queue heavy work	Responsive client
Monitoring	Track latency + errors	Fast diagnosis

Wrapping up

A high-performance client isn't one trick. It's a stack of good decisions: right API, respected limits, batched reads, smart caching, capped concurrency, self-healing retries, async processing, all wrapped in monitoring.

I wrote a fuller version of this on our blog with deeper links into each topic: Building High-Performance Shopify API Clients.

What does your retry/backoff setup look like? Curious how others handle the cost-budget side of GraphQL.

How Shopify's GraphQL Rate Limits Actually Work (and How to Stop Getting 429'd)

Muhammad Masad Ashraf — Mon, 22 Jun 2026 20:22:24 +0000

Originally published on the Kolachi Tech blog. Canonical link points back to the original.

If you treat Shopify's GraphQL Admin API like a REST endpoint and fire concurrent paginated requests, you will hit a wall of 429s fast. The reason is simple: Shopify doesn't rate-limit by request count. It limits by calculated query cost.

Once that clicks, the limits stop being a wall and become a budget. Here's the working model.

The core idea: cost, not count

REST charges the same for every call, whether you read one field or a thousand. Shopify scrapped that for GraphQL. The server runs static analysis on your query before execution, scores it by complexity, and deducts points from a bucket.

Cheap reads barely register. Heavy nested reads and writes cost real points. You can run many small queries or a few expensive ones, as long as you stay inside the budget.

How query cost is scored

The point values are fixed and deterministic, so you can predict cost before shipping.

Field type	Represents	Cost
Scalar / enum	A final value (`id`, `title`, `email`, boolean)	0 points
Object	A single related object (`order.customer`)	1 point
Connection	A paginated list (`products`, `orders`, `variants`)	2 points + 1 per returned item
Mutation	A write (create / update / delete)	10 points

Mutations cost more because they have side effects: DB writes, index updates, webhooks, emails.

Every response includes an extensions.cost object reporting both the requested cost (static analysis) and the actual cost (measured at runtime). The actual cost is often lower.

{
  "extensions": {
    "cost": {
      "requestedQueryCost": 52,
      "actualQueryCost": 12,
      "throttleStatus": {
        "maximumAvailable": 1000,
        "currentlyAvailable": 988,
        "restoreRate": 50
      }
    }
  }
}

Add the Shopify-GraphQL-Cost-Debug=1 header during development for a field-by-field breakdown.

Limits by plan

Plan tier	Bucket size	Restore rate
Standard (Basic, Shopify, Advanced)	1,000 points	50 points/sec
Shopify Plus	2,000 points	100 points/sec

Three things to burn into memory:

Every plan caps a single query at 1,000 points. Plus gives you a deeper bucket, not bigger queries. A query for 250 products × 100 variants each can blow past the ceiling without warning.
Restore is per second, not per minute. At 50 pts/sec you can run roughly one typical order query per second without throttling.
The bucket is scoped per app + store. One app's traffic never touches another app's budget on the same store.

The portability trap

A 1,500-point query works on Plus and fails instantly on a standard store. If your app spans multiple merchant tiers, cap max query cost at 1,000 points and rely on pagination, even when today's store runs Plus.

Read your usage before you get punished

Don't wait for a 429. Every response hands you throttleStatus. Check your next query's cost against currentlyAvailable before dispatching, and sleep if you're short.

A minimal local token-bucket guard in TypeScript:

type ThrottleStatus = {
  maximumAvailable: number;
  currentlyAvailable: number;
  restoreRate: number;
};

let bucket: ThrottleStatus = {
  maximumAvailable: 1000,
  currentlyAvailable: 1000,
  restoreRate: 50,
};

async function runQuery(query: string, estimatedCost: number) {
  // Refill estimate before dispatch
  if (bucket.currentlyAvailable < estimatedCost) {
    const deficit = estimatedCost - bucket.currentlyAvailable;
    const waitMs = Math.ceil((deficit / bucket.restoreRate) * 1000);
    await new Promise((r) => setTimeout(r, waitMs));
  }

  const res = await fetch(
    "https://your-store.myshopify.com/admin/api/2026-04/graphql.json",
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "X-Shopify-Access-Token": process.env.SHOPIFY_TOKEN!,
      },
      body: JSON.stringify({ query }),
    }
  );

  const json = await res.json();

  // Trust the server's reported state over your local estimate
  if (json.extensions?.cost?.throttleStatus) {
    bucket = json.extensions.cost.throttleStatus;
  }

  return json;
}

Mirror Shopify's bucket in Redis if you have multiple workers, so they coordinate instead of each discovering the limit by failing.

Patterns that make throttling disappear

Request only the fields you need. If you need a title, don't pull the description and all 50 images. This compounds across millions of calls.

Back off cleanly on 429. Exponential backoff with jitter so retries don't stampede:

async function withBackoff<T>(fn: () => Promise<T>, max = 5): Promise<T> {
  for (let attempt = 0; attempt < max; attempt++) {
    try {
      return await fn();
    } catch (err) {
      const base = 2 ** attempt * 100;
      const jitter = Math.random() * 100;
      await new Promise((r) => setTimeout(r, base + jitter));
    }
  }
  throw new Error("Max retries exceeded");
}

Prefer webhooks over polling. Polling burns points on data that hasn't changed. Subscribe to change events and poll only as a fallback.

Paginate deliberately. Connections cost 2 + 1 per item. Tune page size to your bucket: smaller pages cost less per call but add round trips.

Don't forget `userErrors`

The GraphQL API often returns 200 OK even when a mutation fails validation. The status code lies; the body tells the truth.

const { data } = await runQuery(mutation, 10);
const errors = data?.productUpdate?.userErrors ?? [];
if (errors.length) {
  // handle validation failures the HTTP status hid from you
  console.error(errors);
}

When to drop synchronous queries entirely

Single queries cap at 1,000 points. Some datasets just won't fit. That's what the Bulk Operations API is for.

Dataset	Approach
Under 1,000 records	Synchronous paginated queries
Over 1,000 records	Bulk Operations API
Deeply nested connections	Bulk Operations API
Real-time, small reads	Standard queries

Bulk runs async: submit a query, Shopify processes it in the background, you download a file. No max cost, no per-query rate limit. A bulk query can return 100k+ products without touching your bucket.

The architecture that survives production

Webhooks first, polling only as fallback. Kills most read pressure.
Local token bucket governs every outbound call.
Bulk operations for anything over 1,000 records; synchronous queries stay lean.
Observability across all of it, feeding throttle telemetry into a dashboard.

Get those four right and the Shopify API quota stops being a wall and becomes a budget you plan around.

Full write-up with more detail lives on the Kolachi Tech blog. If your team wants help building a throttle-resistant integration, that's our day job.

What patterns are you using to stay under the limit? Drop them in the comments.

Caching Shopify GraphQL: A Practical Guide for Developers

Muhammad Masad Ashraf — Fri, 19 Jun 2026 20:54:35 +0000

TL;DR: GraphQL can't be cached by URL like REST. Cache by query + variables, layer your caches (client → edge → app → persisted queries), match TTLs to data volatility, and invalidate via webhooks. Never cache carts or customer-specific pricing.

The Core Problem

REST caching is URL-based. One endpoint = one cache entry. Easy.

GraphQL uses a single endpoint for everything. The query body defines the response, so two requests to the same URL can return totally different data. URL-based caching is useless here.

Your cache key has to include the query + variables + user context.

// Naive (broken) approach
const key = endpoint; // same key for every query — wrong

// Correct approach
const key = hash(JSON.stringify({
  query: normalizedQuery,
  variables: sortedVariables,
  locale,
  buyerSegment
})); // correct

Factor	REST	GraphQL
Endpoints	Many	One
Cache key	URL	Query + variables
Granularity	Coarse	Field-level possible
Invalidation	Simpler	More complex

The 4 Cache Layers

Don't think "a cache." Think layers, each catching a different request type.

Client cache (Apollo / urql) — session reuse
Edge / CDN cache — public storefront pages
App cache (Redis / Memcached) — shared, semi-static data
Persisted queries — stable hash-based keys

All of these sit in front of the Shopify GraphQL API.

Layer	Location	Best for	Typical TTL
Client	Browser/app	Single session	Session length
Edge/CDN	Network edge	Public data	Minutes to hours
App	Your server	Shared data	Seconds to hours
Persisted query	Server	Stable identity	Long-lived

What to Cache (and What Will Burn You)

Cache hard: product details, collections, shop settings
Cache short: pricing, availability (a few seconds)
Never cache: carts, checkout, customer-specific pricing

Data	Volatility	Approach
Product details	Low	Cache hours, invalidate on update
Collections	Low	Cache hours
Inventory	High	Cache seconds or skip
Pricing	Med-high	Short TTL + invalidation
Cart/checkout	Very high	Don't cache
Customer data	High + private	Scope per user or skip

I once cached inventory too long and oversold during a launch. Learn from my pain.

Cache Invalidation: 3 Strategies

1. TTL (time-based) — simplest, but you're guessing the window.

await redis.set(key, payload, 'EX', 60); // expire in 60s

2. Event-based (webhooks) — most accurate. Product updates fire a webhook, you purge the entry.

// products/update webhook handler
app.post('/webhooks/products/update', verifyHmac, async (req, res) => {
  const productId = req.body.id;
  await redis.del(`product:${productId}:*`);
  res.sendStatus(200);
});

A dropped webhook means stale cache. Make your consumers reliable (retries, dead-letter queues).

3. Stale-while-revalidate — serve stale instantly, refresh in background.

Cache-Control: max-age=60, stale-while-revalidate=300

Method	Freshness	Complexity	Best for
TTL	Medium	Low	Predictable data
Event-based	High	Med-high	Inventory, pricing
SWR	High	Medium	Public pages

Smart Cache Keys (don't leak data!)

For B2B stores with tiered pricing, the buyer's company must be in the key or you'll serve Customer A's contract price to Customer B.

function buildKey({ query, variables, context }) {
  return hash(JSON.stringify({
    q: normalize(query),
    v: sortObjectKeys(variables), // sort for consistency
    locale: context.locale,
    currency: context.currency,
    buyer: context.companyId ?? 'anonymous'
  }));
}

Handling Personalized Data

Field-level splitting is the cleanest pattern:

Catalog data: cache once, globally
Cart + pricing: fetch fresh, per user, no cache

Keep your hit rate high where it counts, fetch fresh where it matters.

Measure It

Hit ratio = hits / (hits + misses) — maximize
Latency delta = before vs after — should drop
API calls avoided = cache hits — cost savings
Stale incidents = wrong price/stock reports — target zero

Common Mistakes

Caching personal data in a shared key
TTLs so long prices go stale
Ignoring webhook reliability
Caching mutation results

Wrap-Up

Layer your caches. Match TTLs to volatility. Invalidate via webhooks. Build keys that respect personalization. Measure, then tune.

Done right, caching turns a throttled, sluggish app into a fast, resilient one.

I wrote a longer, more detailed version with extra comparison tables and architecture notes here:
Caching Strategies for Shopify GraphQL

What's your go-to invalidation strategy? Drop it in the comments.

How I Cut Our Shopify API Usage in Half (Without Losing Features)

Muhammad Masad Ashraf — Thu, 18 Jun 2026 13:26:25 +0000

If you build on Shopify long enough, you hit the wall. Throttled requests. Stalled inventory syncs. Webhook backlogs piling up during a flash sale.

Shopify does not charge you per call in dollars. It charges you in rate limits. Blow through them and your app slows to a crawl. The real cost shows up as engineering time, infrastructure bills, and lost sales from stale data.

Here is what actually moved the needle for us.

First, know what you are paying for

Shopify uses two rate-limiting models depending on the API.

API	Model	How it works
REST Admin	Leaky bucket	Fixed requests per second
GraphQL Admin	Query cost	Points based on complexity
Storefront	Query cost	Higher limits, complexity-based

The GraphQL model is your friend here. You pay for what you ask for. Ask for less, pay less.

Move from REST to GraphQL

This was our biggest single win. REST makes you stitch together multiple calls to get related data. GraphQL gets it in one.

Fetching an order with its line items, customer, and shipping in REST:

// REST: 4 round trips
const order = await fetch(`/admin/api/2024-01/orders/${id}.json`);
const customer = await fetch(`/admin/api/2024-01/customers/${custId}.json`);
const lineItems = await fetch(`/admin/api/2024-01/orders/${id}/line_items.json`);
const shipping = await fetch(`/admin/api/2024-01/orders/${id}/shipping_address.json`);

The same thing in GraphQL:

# GraphQL: 1 request, only the fields you need
query GetOrder($id: ID!) {
  order(id: $id) {
    name
    customer { firstName lastName email }
    shippingAddress { address1 city zip }
    lineItems(first: 50) {
      edges { node { title quantity } }
    }
  }
}

One request. No over-fetching. Lower cost.

The trap: people migrate to GraphQL and then request every field anyway. Don't. Every field adds to your query cost. If you only need a title and price, do not pull variants, images, and metafields.

Stop polling. Use webhooks.

Polling is the silent budget killer. An app that checks for new orders every few seconds burns thousands of calls a day, and most return nothing new.

Webhooks flip it. Shopify pushes the event the moment it happens.

// Instead of polling every 5 seconds...
app.post('/webhooks/orders/create', verifyWebhook, (req, res) => {
  const order = req.body;
  queue.add('process-order', order); // hand off, respond fast
  res.sendStatus(200);
});

Two things to get right:

Respond fast. Acknowledge with a 200 immediately, then process async. Shopify retries if you are slow.
Expect duplicates. Shopify sometimes sends the same event twice. Use an idempotency key so you do not fire redundant follow-up calls.

async function handleEvent(event) {
  const key = event.id;
  if (await seen(key)) return; // already processed
  await markSeen(key);
  await process(event);
}

Use Bulk Operations for big data sets

Paginating through 50,000 products with standard queries shreds your rate limit. The Bulk Operations API runs the whole thing async and hands you one file.

mutation {
  bulkOperationRunQuery(
    query: """
    { products { edges { node { id title } } } }
    """
  ) {
    bulkOperation { id status }
    userErrors { field message }
  }
}

Approach	Calls	Rate limit hit
Standard pagination	Hundreds to thousands	High
Bulk Operations	One operation	Minimal

Poll the operation status, download the JSONL when done, process locally.

Cache the stuff that does not change

A surprising amount of API traffic fetches data that is stable for hours or days. Product descriptions. Collection structure. Store settings.

Data	Cache?	Why
Product descriptions	Yes	Rarely changes
Collection structure	Yes	Stable
Store settings	Yes	Infrequent changes
Live inventory	Carefully	Changes constantly
Order status	Short TTL	Updates often

Invalidate on the relevant webhook instead of guessing at TTLs.

Handle rate limits like an adult

Even a lean app hits limits sometimes. Read the headers Shopify returns and throttle before you crash.

async function callWithBackoff(fn, retries = 5) {
  for (let i = 0; i < retries; i++) {
    const res = await fn();
    if (res.status !== 429) return res;
    const wait = Math.pow(2, i) * 1000; // exponential backoff
    await new Promise(r => setTimeout(r, wait));
  }
  throw new Error('Rate limited after retries');
}

Centralize this in a middleware layer so every call gets the same treatment. Do not scatter retry logic across your codebase.

The checklist

Tactic	Impact	Effort
REST to GraphQL	High	Medium
Request only needed fields	High	Low
Bulk Operations API	High	Medium
Cache stable data	High	Medium
Webhooks over polling	Very High	Medium
Batch requests	Medium	Low
Smart rate limiting	Medium	Medium
Deduplicate calls	Medium	Low
Async processing	Medium	High
Monitor usage	High	Low

Start here

If you do nothing else: replace polling with webhooks, switch to GraphQL with trimmed queries, and cache stable data. Those three alone cut our usage roughly in half.

Then layer in batching, async jobs, and monitoring for the long game.

I wrote a longer version with deeper architecture notes on our blog: Reducing Shopify API Consumption Costs. Happy to answer questions in the comments.

What is the worst API throttle situation you have run into? Drop it below.

Shopify GraphQL Pagination: How to Handle Large Datasets Without Slowing Down Your App

Muhammad Masad Ashraf — Fri, 12 Jun 2026 18:53:55 +0000

When you build Shopify apps or integrations, pagination becomes important very quickly.

A small test store may have a few products and orders.

A real merchant store can have thousands of products, variants, orders, customers, inventory items, metafields, and fulfillment records.

You cannot fetch all of that data in one Shopify GraphQL request.

You need pagination.

More importantly, you need pagination that performs well.

Poor Shopify GraphQL pagination can create slow syncs, API throttling, timeout errors, duplicate processing, and incomplete exports.

This post explains how Shopify GraphQL pagination works and how to handle large Shopify datasets in a practical way.

What Shopify GraphQL Pagination Solves

Pagination lets your app retrieve data in smaller chunks.

Instead of asking Shopify for 50,000 products at once, your app asks for 100 or 250 products per request.

Shopify returns the data and gives your app information about the next page.

This protects your app from huge responses and protects Shopify from heavy requests.

It also gives your integration more control over retries, progress tracking, and background processing.

Shopify Uses Cursor-Based Pagination

Shopify GraphQL uses cursor-based pagination.

That means you do not request data using page numbers.

You request the next page using a cursor from the previous response.

A basic product pagination query looks like this:

query GetProducts($cursor: String) {
  products(first: 100, after: $cursor) {
    nodes {
      id
      title
      handle
      updatedAt
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

The first time you run this query, pass cursor as null.

Shopify returns the first 100 products and gives you an endCursor.

Use that endCursor as the after value in the next request.

Keep doing this until hasNextPage is false.

Why Cursors Work Better Than Page Numbers

Offset pagination usually works like this:

page=1
page=2
page=3

or:

offset=5000&limit=100

This approach becomes inefficient when datasets grow.

The system may need to skip many records before returning the next page.

It also becomes less reliable when data changes while your app is reading it.

Shopify data changes constantly.

Orders are created. Products are updated. Inventory changes. Customers register.

Cursor-based pagination handles this type of data better because it continues from a known position.

Page Size Is a Performance Decision

Shopify GraphQL connections can return up to 250 resources in one page.

That does not mean you should always use first: 250.

For simple queries, 250 may work well.

Example:

products(first: 250) {
  nodes {
    id
    title
  }
}

But if your query loads nested data, 250 can become too heavy.

Example:

products(first: 250) {
  nodes {
    id
    title
    variants(first: 50) {
      nodes {
        id
        sku
        inventoryQuantity
        metafields(first: 20) {
          nodes {
            key
            value
          }
        }
      }
    }
  }
}

This query may look convenient, but it can become expensive because it fetches products, variants, inventory, and metafields together.

For complex queries, use a smaller page size.

A practical starting point:

Query Type	Page Size
Basic products	100 to 250
Products with variants	50 to 100
Orders with line items	25 to 100
Metafield-heavy queries	25 to 100
Admin UI lists	20 to 50
Background sync jobs	100 to 250

The right number depends on query cost, response size, processing time, and failure rate.

Watch GraphQL Query Cost

GraphQL performance is not only about request count.

Shopify calculates query cost.

A simple query costs less.

A nested query costs more.

If you try to fetch everything in one request, you may hit throttling or timeout issues faster.

Instead of creating one huge query, split the workflow.

For example:

Fetch product IDs and basic product fields.
Save them locally.
Fetch variants in a separate job.
Fetch metafields only when needed.
Process everything through a queue.

This keeps your queries smaller and easier to retry.

A Reliable Pagination Loop

A production pagination loop should not be a basic while loop only.

It should track progress and recover safely.

A good flow looks like this:

1. Start with cursor = null
2. Request a page
3. Process records
4. Save endCursor
5. Check hasNextPage
6. Respect throttle status
7. Retry failed requests with backoff
8. Resume from saved cursor if the job stops

You should also make processing idempotent.

For example, if you sync products into your own database, use the Shopify product ID as a unique key and upsert the record.

Do not blindly insert records.

If your worker crashes and retries the same page, idempotent logic prevents duplicate data.

Use Filters Before Pagination

Do not paginate more data than you need.

If you only need recent orders, filter by date.

Example:

query GetRecentOrders($cursor: String) {
  orders(first: 100, after: $cursor, query: "created_at:>=2026-01-01") {
    nodes {
      id
      name
      createdAt
      displayFinancialStatus
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

This reduces the number of pages your app needs to process.

Filters are especially useful for incremental syncs.

Instead of syncing all products every time, sync records updated after the last successful sync.

When to Use Bulk Operations

Cursor pagination is great for many use cases.

Use it for:

Admin tables
Small exports
Search results
Incremental syncs
Recent orders
Focused data reads

But for very large exports, use Shopify Bulk Operations.

Bulk Operations work better for:

Full product catalog exports
Historical order exports
Large customer exports
ERP syncs
Large metafield audits
Reporting pipelines

With Bulk Operations, Shopify processes the job asynchronously and gives you a file to download when it is ready.

This avoids thousands of paginated API calls.

Use Webhooks for Real-Time Updates

Pagination should not replace webhooks.

Use webhooks when you need real-time updates.

Use pagination for backfills, audits, and reconciliation.

A better architecture looks like this:

Webhook receives new order
Queue stores the event
Worker fetches missing details with GraphQL
Scheduled job reconciles recent records
Bulk Operation handles full historical export

This approach keeps your app fast and reduces unnecessary API calls.

It also helps when webhook delivery is delayed or when your system needs to verify data consistency.

Common Mistakes

Here are mistakes I often see in Shopify GraphQL pagination workflows:

Mistake	Problem
Always using `first: 250`	Heavy queries may slow down
Fetching too many nested fields	Query cost increases
Not saving cursors	Failed jobs restart from zero
Ignoring throttling	API calls start failing
No retry logic	Temporary errors break syncs
No idempotency	Duplicate records appear
Using pagination for huge exports	Bulk Operations would perform better
No logs or alerts	Failures stay hidden

Most pagination issues happen because developers treat the job like a simple data loop.

For production apps, treat it like a data pipeline.

Final Thoughts

Shopify GraphQL pagination is one of the most important patterns for scalable Shopify apps.

Cursor-based pagination helps your app fetch large datasets in manageable chunks.

But good performance depends on more than cursors.

You need lean queries, smart page sizes, filters, saved cursors, retries, queues, webhook support, and Bulk Operations for very large datasets.

For small and medium workflows, cursor pagination is usually enough.

For large exports, Bulk Operations are the better option.

For real-time updates, use webhooks.

The strongest Shopify integrations combine all three.

Originally published on KolachiTech: https://kolachitech.com/shopify-graphql-pagination/

Shopify Bulk Operations API at Scale: A Practical Developer Guide

Muhammad Masad Ashraf — Thu, 11 Jun 2026 17:55:51 +0000

Every Shopify developer hits this wall eventually.

You need to export 200,000 orders. Or sync a 500K-product catalog. Or run a price update across every variant you carry. Standard GraphQL queries collapse under that pressure. Rate limits fire. Timeouts pile up. Your integration breaks.

The Shopify Bulk Operations API exists to fix this. It processes millions of records asynchronously, returns output as a downloadable JSONL file, and sidesteps the throttling that kills traditional query approaches.

Here is how it works, and how to scale it properly.

What Is the Shopify Bulk Operations API?

It is a subset of the Shopify GraphQL Admin API built for large-scale data retrieval and mutation. Instead of paginating through thousands of API calls, you submit one GraphQL operation and Shopify runs it server-side in the background.

When the job completes, Shopify generates a JSONL file available at a signed URL. Each line in the file is one resource object.

Two operation types exist:

Operation Type	What It Does	Common Use Case
`bulkOperationRunQuery`	Exports data asynchronously	Orders export, catalog dump, customer list
`bulkOperationRunMutation`	Applies mutations to a large dataset	Price updates, tag writes, metafield updates

Both follow the same lifecycle: create, poll, download.

The Operation Lifecycle

Step 1: Submit

Send a bulkOperationRunQuery mutation. Shopify queues the job and returns an operation ID with status CREATED.

mutation {
  bulkOperationRunQuery(
    query: """
    {
      products {
        edges {
          node {
            id
            title
            variants {
              edges {
                node {
                  id
                  price
                  sku
                }
              }
            }
          }
        }
      }
    }
    """
  ) {
    bulkOperation {
      id
      status
    }
    userErrors {
      field
      message
    }
  }
}

Step 2: Poll for Status

Query currentBulkOperation at intervals until status reaches COMPLETED or FAILED.

query {
  currentBulkOperation {
    id
    status
    errorCode
    createdAt
    completedAt
    objectCount
    fileSize
    url
    partialDataUrl
  }
}

Poll every 3 to 10 seconds for small jobs. For large datasets, back off to 30 to 60 seconds.

Step 3: Download and Parse

When COMPLETED, the url field contains a signed link valid for 7 days. Parse the JSONL line by line. Parent-child relationships are encoded via __parentId on child nodes.

Status reference:

Status	Meaning	Action
CREATED	Queued, not yet running	Keep polling
RUNNING	Actively processing	Keep polling
COMPLETED	Done	Download file
FAILED	Error encountered	Check `errorCode`, retry
CANCELED	Manually or auto-canceled	Resubmit if needed
CANCELING	Cancel in progress	Wait, then resubmit

Why This Beats Standard Pagination at Scale

Standard GraphQL pagination uses cursor-based after arguments. Fine for 10,000 records. For 1 million, you make thousands of API calls and burn through your query cost budget fast.

Bulk Operations bypass per-request rate limits almost entirely. Shopify does the work server-side. You wait, then download one file.

Factor	Standard Pagination	Bulk Operations API
Rate Limit Exposure	High	Very low
Max Records	Limited by throttle	Millions
Processing Model	Synchronous	Asynchronous
Error Recovery	Per-request	Job-level
Data Format	JSON in response	JSONL file download
Best For	Up to ~50K records	50K to millions

Key Constraints You Must Know

One operation per store at a time

Only one bulk operation runs per store at any moment. Submitting a second while the first runs fails immediately. In a multi-tenant app, build per-store operation locks into your job queue.

No nested mutations

Bulk mutations require a flat JSONL input file. Each line maps to one mutation call. Nested mutations inside a single bulk operation are not supported.

Stream the output file

JSONL output can reach several gigabytes. Never load it fully into memory. Use line-by-line buffered reads.

Check `partialDataUrl` on failure

When a job fails or gets canceled, Shopify may still produce a partialDataUrl containing whatever completed before failure. Always check this field. Process the partial data, then retry only the remaining records.

Bulk Mutations: The Real Power Move

Bulk mutations handle operations like price updates, tag management, and metafield writes across millions of records.

The flow:

1. Stage an upload via stagedUploadsCreate to get a signed PUT URL.

2. Upload your JSONL input file. Each line is a JSON object with variables for one mutation call.

{"input": {"id": "gid://shopify/ProductVariant/123456789", "price": "29.99"}}

3. Submit the bulk mutation referencing the staged upload URL in bulkOperationRunMutation.

Output results include a __lineNumber field so you can map every success and failure back to your input file precisely.

Scaling Patterns for Production Systems

Pattern 1: Webhook-Triggered Completion

Subscribe to BULK_OPERATIONS_FINISH instead of polling. Shopify pushes completion status and the download URL to your endpoint. Your system stays idle until Shopify calls you.

Make your webhook handler idempotent. Shopify can fire the same completion event more than once.

Pattern 2: Job Orchestration Layer

For apps managing bulk operations across hundreds or thousands of merchant stores, build a layer that:

Queues bulk operation submissions per store
Tracks the active operation ID per store in a database
Handles completions via webhooks
Retries failed jobs with exponential backoff
Logs partialDataUrl before discarding failed results

Pattern 3: Chunked Mutation Strategy

Even though bulk mutations support millions of records, split very large jobs into chunks of 50K to 100K records per operation. Smaller jobs complete faster, fail cheaper, and produce lighter output files.

Track your last successfully completed chunk in a persistent state store. On failure, resume from the checkpoint rather than reprocessing everything.

Pattern 4: JSONL Processing Pipeline

Build a streaming parser that:

Reads the output file line by line
Reconstructs parent-child relationships via __parentId
Writes records to your database or downstream system
Tracks failed lines separately for targeted retries

Use worker processes separate from your web layer for this step. Monitor memory consumption carefully on multi-gigabyte files.

Error Codes and Retry Logic

Error Code	Cause	Action
`ACCESS_DENIED`	Missing API scope	Update OAuth scopes
`INTERNAL_SERVER_ERROR`	Shopify-side failure	Retry with exponential backoff
`TIMEOUT`	Query too complex or dataset too large	Simplify query, chunk inputs
`TOO_MANY_FILE_STORAGE_REQUESTS`	Too many staged uploads in flight	Throttle upload submissions

For timeouts: simplify the query first. Remove fields your downstream system does not use. A leaner query fixes most timeouts without reducing dataset scope.

Performance Benchmarks

Dataset Size	Typical Completion Time	Approximate File Size
10,000 products	30 to 90 seconds	5 to 15 MB
100,000 orders	3 to 8 minutes	100 to 300 MB
500,000 customers	15 to 40 minutes	500 MB to 2 GB
1M+ line items	30 to 90 minutes	2 to 10 GB

These are estimates, not guarantees. Always build for the upper end of each range. Never assume a fixed completion time in production logic.

App Architecture Implications

Bulk operations shift your bottleneck. You stop hammering the API with thousands of requests. Instead, you make a few API calls, then process a large local file. The bottleneck moves from network throughput to local compute and storage I/O.

Plan accordingly:

Use worker processes separate from your web layer
Write processed results to a fast intermediate store (Redis, PostgreSQL) before your final destination
Use streaming HTTP clients that do not buffer the full response body
Alert on jobs stuck in RUNNING state beyond expected thresholds

Separate real-time event pipelines from batch bulk operation pipelines entirely. Use event-driven patterns for real-time updates. Reserve the bulk pipeline for scheduled or triggered large-batch jobs.

Wrapping Up

The Shopify Bulk Operations API is the right tool any time your data volume pushes past what paginated GraphQL can handle cleanly. The lifecycle is simple. The constraints are manageable. The architecture patterns are proven.

Build it with webhook-triggered completion, idempotent processing, chunked mutations, and streaming JSONL parsing, and you get a system that handles enterprise-scale data without breaking under load.

Originally published on KolachiTech: https://kolachitech.com/bulk-operations-api-at-scale/

KolachiTech is a Shopify-focused development agency specializing in API architecture, ERP integrations, and scalable app development. If you need a production-grade bulk operations system built for your store or app, get in touch.

Shopify GraphQL Query Cost Optimization: Stop Burning API Points

Muhammad Masad Ashraf — Tue, 09 Jun 2026 16:27:46 +0000

Every Shopify app developer hits the same wall eventually.

You build a feature, push it to production, and watch your API calls start failing at scale. The culprit is almost always the same: inefficient GraphQL queries burning through your rate limit bucket faster than it refills.

Shopify's GraphQL Admin API uses cost-based throttling, not request-per-second counting. A single poorly structured query can cost 1,000 points. A well-optimized one may cost under 50.

This post covers how the system works, where developers waste points, and what to do about it.

How the Leaky Bucket Works

Shopify assigns a calculated cost to every query before executing it. That cost gets deducted from your bucket instantly.

Parameter	Value
Bucket size (standard)	1,000 points
Bucket size (Shopify Plus)	2,000 points
Refill rate	50 points/second
Max single query cost	1,000 points
Throttle error	`THROTTLED`

Every API response includes cost metadata under extensions.cost:

{
  "extensions": {
    "cost": {
      "requestedQueryCost": 412,
      "actualQueryCost": 412,
      "throttleStatus": {
        "maximumAvailable": 1000,
        "currentlyAvailable": 588,
        "restoreRate": 50
      }
    }
  }
}

Parse this on every single call. Most developers ignore it until production breaks.

How Cost Gets Calculated

Cost is driven by two things: the number of objects you request and the depth of connections.

The problem with nested connections

The first argument multiplies cost across every level of nesting. Here is what an expensive query looks like:

{
  products(first: 250) {
    edges {
      node {
        id
        title
        variants(first: 100) {
          edges {
            node {
              id
              price
              inventoryItem {
                id
              }
            }
          }
        }
      }
    }
  }
}

This query easily hits a calculated cost of 25,000+. Shopify rejects it before execution.

The optimized version

{
  products(first: 50) {
    edges {
      node {
        id
        title
        variants(first: 10) {
          edges {
            node {
              id
              price
            }
          }
        }
      }
    }
  }
}

Smaller page sizes, fewer fields, no 3-level nesting. Cost drops by 90%+.

7 Strategies to Reduce GraphQL Points

1. Request Only the Fields You Need

GraphQL gives you field-level control. Use it.

Never pull an entire product object when you only need id and title. Unused fields carry cost weight inside nested connections.

Audit every query in your codebase. Remove what your app does not consume.

2. Reduce Connection Page Sizes

first: 250 is the single most common reason for runaway costs.

Start with first: 50. Paginate across multiple requests. Three paginated calls at first: 50 are almost always cheaper than one call at first: 250 with deep nesting.

Build pagination into your data layer from day one.

3. Split Deeply Nested Queries

Instead of fetching products, variants, and inventory in one query, split them.

# Step 1: Fetch products
{
  products(first: 50) {
    edges {
      node {
        id
        title
      }
    }
  }
}

# Step 2: Fetch variants by product ID
{
  product(id: "gid://shopify/Product/123456789") {
    variants(first: 10) {
      edges {
        node {
          id
          price
          inventoryQuantity
        }
      }
    }
  }
}

Two focused queries consistently beat one deeply nested monster query.

4. Use Query Cost Extensions for Real-Time Monitoring

Log extensions.cost on every API call. Build an alerting rule that fires when actualQueryCost exceeds a threshold you define.

Here is a minimal Node.js wrapper pattern:

async function shopifyQuery(query, variables = {}) {
  const response = await fetch(SHOPIFY_GRAPHQL_ENDPOINT, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-Shopify-Access-Token': ACCESS_TOKEN,
    },
    body: JSON.stringify({ query, variables }),
  });

  const data = await response.json();

  // Always log cost data
  if (data.extensions?.cost) {
    const { actualQueryCost, throttleStatus } = data.extensions.cost;
    console.log(`Query cost: ${actualQueryCost} | Available: ${throttleStatus.currentlyAvailable}`);

    // Alert threshold
    if (actualQueryCost > 200) {
      console.warn(`High-cost query detected: ${actualQueryCost} points`);
    }
  }

  return data;
}

5. Build Leaky Bucket Awareness Into Your Client

Check throttleStatus.currentlyAvailable before firing a query. If available points are below your expected cost, wait.

async function throttleAwareQuery(query, estimatedCost = 100) {
  const status = await getCurrentBucketStatus();

  if (status.currentlyAvailable < estimatedCost) {
    const waitSeconds = Math.ceil(
      (estimatedCost - status.currentlyAvailable) / status.restoreRate
    );
    console.log(`Waiting ${waitSeconds}s for bucket to refill...`);
    await sleep(waitSeconds * 1000);
  }

  return shopifyQuery(query);
}

This prevents burst failures without requiring exponential backoff from the start.

6. Cache Infrequently-Changing Data

Not every query needs a fresh Shopify API hit. Product catalog data, collection structures, and metafield definitions rarely change.

Use Redis with a short TTL:

async function getCachedProducts(cacheKey, ttlSeconds = 300) {
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);

  const fresh = await shopifyQuery(PRODUCTS_QUERY);
  await redis.setex(cacheKey, ttlSeconds, JSON.stringify(fresh));
  return fresh;
}

You reduce Shopify GraphQL query cost by not making the call at all when cached data is still valid.

7. Use Direct GID Lookups Over List Queries

List queries scan potentially hundreds of records. Direct lookups target exactly what you need.

# Expensive: scans all products
{
  products(first: 250) {
    edges {
      node { id title }
    }
  }
}

# Cheap: direct lookup by known GID
{
  product(id: "gid://shopify/Product/123456789") {
    id
    title
    variants(first: 5) {
      edges {
        node { id price }
      }
    }
  }
}

Store Shopify GIDs in your local database. Retrieve them with targeted queries instead of scanning lists.

Use Bulk Operations for Large Data Sets

Stop using paginated GraphQL queries for bulk data jobs. Shopify's Bulk Operations API handles this better in every way.

Bulk operations:

Run asynchronously in the background
Do not consume your standard rate limit bucket
Return results as a downloadable JSONL file

mutation {
  bulkOperationRunQuery(
    query: """
    {
      products {
        edges {
          node {
            id
            title
            variants {
              edges {
                node {
                  id
                  price
                }
              }
            }
          }
        }
      }
    }
    """
  ) {
    bulkOperation {
      id
      status
    }
    userErrors {
      field
      message
    }
  }
}

Use bulk operations for: full catalog exports, historical order analysis, large-scale metafield reads, and inventory reconciliation across locations.

Handling Throttle Errors

When you exhaust your bucket, Shopify returns HTTP 200 with an error body:

{
  "errors": [
    {
      "message": "Throttled",
      "extensions": {
        "code": "THROTTLED",
        "retryAfter": 4.5
      }
    }
  ]
}

Your retry handler:

async function queryWithRetry(query, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const data = await shopifyQuery(query);

    const throttleError = data.errors?.find(
      (e) => e.extensions?.code === 'THROTTLED'
    );

    if (throttleError) {
      const waitMs = (throttleError.extensions.retryAfter + 0.5) * 1000;
      console.log(`Throttled. Retrying in ${waitMs}ms...`);
      await sleep(waitMs);
      continue;
    }

    return data;
  }

  throw new Error('Max retries exceeded after throttle errors');
}

Always add a small buffer (0.5s) to the retryAfter value. Network latency means hitting the retry exactly at the boundary often fails again.

Common Mistakes at a Glance

Mistake	Cost Impact	Fix
Fetching all fields by default	3x to 10x waste	Select only needed fields
`first: 250` everywhere	Exponential multiplier	Start with `first: 50`, paginate
Ignoring `extensions.cost`	Zero visibility	Log cost on every call
Nested connections 3+ levels	Cost explodes	Split into separate queries
No bucket awareness	Burst failures	Read `currentlyAvailable` before querying
Polling instead of webhooks	Continuous drain	Replace polling with event-driven webhooks

Pre-Ship Checklist

Before any new GraphQL query goes to production:

[ ] Only requesting fields the app actually uses
[ ] Connection page sizes set to minimum viable first value
[ ] Nested connections flattened or split where possible
[ ] extensions.cost logged and monitored
[ ] Throttle error handler reads retryAfter
[ ] Frequently-read data cached with TTL
[ ] Bulk Operations used for large data jobs
[ ] Known IDs queried directly by GID

Wrapping Up

Shopify GraphQL query cost is not a niche concern. It directly determines whether your app stays reliable at scale or becomes a 2am debugging session.

The fix is straightforward: read your cost data, reduce page sizes, split nested queries, cache where you can, and handle throttle errors properly. None of this is complex. It just has to be built intentionally.

For the full breakdown including the GraphQL vs REST comparison table and Hydrogen-specific patterns, read the original post on the KolachiTech blog.

Originally published at kolachitech.com

How to Verify Shopify Webhooks Correctly (And Stop Getting It Wrong)

Muhammad Masad Ashraf — Fri, 05 Jun 2026 19:35:27 +0000

Originally published on KolachiTech

I have reviewed a lot of Shopify app codebases. One thing shows up constantly: webhook verification that looks correct but silently fails. The developer gets frustrated, removes the check, ships without it, and ships a vulnerability instead.

This post covers how Shopify webhook verification actually works, the implementation mistakes that trip people up, and what a production-ready setup looks like. I will share working Node.js and Python code you can drop in today.

Why Your Webhook Endpoint Is a Target

Your Shopify webhook URL is just an HTTP endpoint. It does not require authentication to reach. Anyone who knows it can POST to it.

Without verification, an attacker can:

Fake order fulfillment events
Inject customer records
Trigger inventory changes
Flood your endpoint until it falls over

This matters more than you think. If your webhooks feed an ERP, a CRM, or a fulfillment warehouse, one forged payload corrupts data in every downstream system simultaneously.

Verification is not optional. It is the lock on the front door.

What Shopify Actually Does

Shopify uses HMAC-SHA256 to sign every webhook request it sends.

Here is the exact flow:

Shopify takes the raw bytes of your request body
It runs HMAC-SHA256 using your app's shared secret as the key
It Base64-encodes the result
It drops that value into the X-Shopify-Hmac-SHA256 request header
It sends the request to your endpoint

Your job is to repeat steps 1 through 3 on your side and compare what you compute with what Shopify sent.

If they match: authentic. If they differ: reject immediately, do not process.

The Node.js Implementation

const crypto = require('crypto');

function verifyShopifyWebhook(rawBody, hmacHeader, secret) {
  const generatedHmac = crypto
    .createHmac('sha256', secret)
    .update(rawBody, 'utf8')
    .digest('base64');

  const hmacBuffer = Buffer.from(hmacHeader, 'base64');
  const generatedBuffer = Buffer.from(generatedHmac, 'base64');

  // Length check first to avoid exceptions in timingSafeEqual
  if (hmacBuffer.length !== generatedBuffer.length) {
    return false;
  }

  // Constant-time comparison — never use ===
  return crypto.timingSafeEqual(hmacBuffer, generatedBuffer);
}

Express.js setup that actually works:

app.post(
  '/webhooks/shopify',
  express.raw({ type: 'application/json' }), // Buffer raw body BEFORE any parsing
  (req, res) => {
    const hmac = req.headers['x-shopify-hmac-sha256'];
    const secret = process.env.SHOPIFY_WEBHOOK_SECRET;

    if (!verifyShopifyWebhook(req.body, hmac, secret)) {
      return res.status(401).send('Unauthorized');
    }

    // Safe to parse now
    const payload = JSON.parse(req.body);

    // Acknowledge immediately, process async
    res.status(200).send('OK');
    processWebhookAsync(payload);
  }
);

The Python Implementation

import hmac
import hashlib
import base64

def verify_shopify_webhook(raw_body: bytes, hmac_header: str, secret: str) -> bool:
    digest = hmac.new(
        secret.encode('utf-8'),
        raw_body,
        digestmod=hashlib.sha256
    ).digest()
    computed_hmac = base64.b64encode(digest).decode('utf-8')

    # Constant-time comparison
    return hmac.compare_digest(computed_hmac, hmac_header)

Django/Flask usage:

@app.route('/webhooks/shopify', methods=['POST'])
def shopify_webhook():
    raw_body = request.get_data()  # Raw bytes, not request.json
    hmac_header = request.headers.get('X-Shopify-Hmac-SHA256', '')
    secret = os.environ.get('SHOPIFY_WEBHOOK_SECRET')

    if not verify_shopify_webhook(raw_body, hmac_header, secret):
        return 'Unauthorized', 401

    payload = request.json
    return 'OK', 200

The Mistakes That Break Verification (And Ship Vulnerabilities)

Mistake 1: Parsing before verifying

This is the most common one. Your body parsing middleware runs before your verification code and transforms the raw bytes. Your computed HMAC no longer matches what Shopify signed.

The developer sees 401 everywhere. They remove verification. The app ships insecure.

Fix: Use express.raw() on your webhook route specifically, not express.json().

Mistake 2: Using `===` for comparison

// WRONG — leaks timing information
if (generatedHmac === hmacHeader) { ... }

// RIGHT — constant-time
crypto.timingSafeEqual(hmacBuffer, generatedBuffer);

An attacker can measure microsecond differences in how long === takes to compare strings and deduce the correct signature byte by byte. This is a real attack class called a timing side-channel.

Mistake 3: No replay protection

HMAC proves the payload is authentic. It does not prove it is fresh.

A payload from yesterday is still cryptographically valid today. An attacker who intercepts a legitimate webhook can replay it as many times as they want.

Fix: Validate the timestamp in the payload. Reject anything older than 5 minutes. Store event IDs and reject duplicates.

function isReplayAttack(payload) {
  const eventTime = new Date(payload.created_at).getTime();
  const now = Date.now();
  const fiveMinutes = 5 * 60 * 1000;

  return (now - eventTime) > fiveMinutes;
}

Mistake 4: No shop domain validation

All stores using your app share the same signing secret. A valid webhook from store A is also cryptographically valid when replayed against your store B handler.

Fix: Always validate X-Shopify-Shop-Domain against your database of registered shops.

const shopDomain = req.headers['x-shopify-shop-domain'];

if (!registeredShops.includes(shopDomain)) {
  return res.status(403).send('Forbidden');
}

Mistake 5: Storing the secret in code

# Never do this
SHOPIFY_WEBHOOK_SECRET = "shpss_abc123"  # Committed to git

# Do this
process.env.SHOPIFY_WEBHOOK_SECRET  # Environment variable or secrets manager

Logs are often less locked down than your codebase. Never log the secret either.

The Production Checklist

Before shipping any webhook consumer:

Transport:

HTTPS only, no HTTP fallback
TLS 1.2 or higher

Verification:

Raw body buffered before any parsing middleware
HMAC computed with correct shared secret
Constant-time comparison used
Failed verification returns 401/403 immediately

Replay protection:

Timestamp validated, reject payloads older than 5 minutes
Event IDs deduplicated (Redis or DB)
Shop domain validated against allowed list

Secrets:

Secret in environment variable or secrets manager
Secret rotation procedure documented and tested
Verification failures logged (not the secret, the failure)

Reliability:

Handler returns 200 immediately, processes async
Dead letter queue configured for processing failures

Why Acknowledging Immediately Matters

Shopify expects a 2xx response within 5 seconds. If you do slow processing synchronously, you will hit timeouts, and Shopify will retry the webhook. Retries without idempotency controls mean double-processing.

The pattern is: verify, acknowledge with 200, hand off to a queue or async worker, process there.

res.status(200).send('OK');         // Shopify gets its answer
processWebhookAsync(payload);       // You do the actual work

Scaling Shopify Webhooks to Handle Millions of Events: A Practical Guide

Muhammad Masad Ashraf — Thu, 04 Jun 2026 10:08:28 +0000

Scaling Shopify Webhooks to Handle Millions of Events: A Practical Guide

If you've built a Shopify integration that gets any real traffic, you've probably hit this issue: your webhook endpoint becomes a bottleneck.

Let me walk you through the patterns that actually work at scale, with code examples you can use today.

The Problem: Synchronous Processing Doesn't Scale

Your first webhook implementation probably looks something like this:

app.post('/webhooks/orders/create', (req, res) => {
  // Validate webhook signature
  if (!validateWebhookSignature(req)) {
    return res.status(401).send('Unauthorized');
  }

  const order = req.body;

  // Do all the work synchronously
  updateInventory(order);
  createCustomerRecord(order.customer);
  sendConfirmationEmail(order);
  syncToERP(order);
  updateAnalytics(order);

  // Finally return response
  res.status(200).send('OK');
});

This works fine until:

One slow operation blocks everything. If sendConfirmationEmail() takes 5 seconds, every webhook takes 5 seconds.
You hit Shopify's timeout. Shopify waits 28 seconds max. If your endpoint doesn't respond by then, it retries. Now you have duplicates.
Resource exhaustion. Each concurrent webhook ties up a worker process. At 1,000 webhooks/second, you need 5,000+ workers.
Cascading failures. One slow external service (email, payment processor, ERP) slows down everything.

The solution? Decouple receiving from processing.

Solution 1: Queue-Based Processing

Instead of doing all the work in the webhook handler, push events to a message queue and process them asynchronously:

const AWS = require('aws-sdk');
const sqs = new AWS.SQS();

app.post('/webhooks/orders/create', async (req, res) => {
  // Validate signature
  if (!validateWebhookSignature(req)) {
    return res.status(401).send('Unauthorized');
  }

  try {
    // Just push to queue and return immediately
    await sqs.sendMessage({
      QueueUrl: process.env.QUEUE_URL,
      MessageBody: JSON.stringify({
        type: 'order.created',
        webhookId: req.headers['x-shopify-webhook-id'],
        data: req.body,
        timestamp: new Date().toISOString()
      })
    }).promise();

    // Return 200 OK to Shopify in milliseconds
    res.status(200).send('OK');
  } catch (error) {
    console.error('Queue push failed:', error);
    // Still return 200 to Shopify - we can retry later
    res.status(202).send('Accepted');
  }
});

Now your endpoint responds in milliseconds. Shopify is happy. Workers process the queue asynchronously:

// Worker process (separate from webhook handler)
const processQueue = async () => {
  while (true) {
    const messages = await sqs.receiveMessage({
      QueueUrl: process.env.QUEUE_URL,
      MaxNumberOfMessages: 10,
      WaitTimeSeconds: 20
    }).promise();

    if (!messages.Messages) continue;

    for (const message of messages.Messages) {
      try {
        const event = JSON.parse(message.Body);

        // Now do all the work asynchronously
        await processOrder(event.data);

        // Delete from queue after successful processing
        await sqs.deleteMessage({
          QueueUrl: process.env.QUEUE_URL,
          ReceiptHandle: message.ReceiptHandle
        }).promise();
      } catch (error) {
        console.error('Processing failed:', error);
        // Leave in queue - will retry automatically
      }
    }
  }
};

processQueue();

Benefits:

Webhook endpoint responds in <10ms
Process slow operations without timeout risk
Scale workers independently
Handle spikes gracefully

Trade-offs:

Slight delay in processing (seconds to minutes, not milliseconds)
Need infrastructure for queue system
Must handle duplicates (webhooks can retry)

Solution 2: Dead Letter Queues for Failure Handling

Even with queues, events fail. Your database goes down. An API times out. You need a safety net.

Enter Dead Letter Queues (DLQ). When processing fails repeatedly, move to DLQ for investigation:

const MAX_RETRIES = 3;

const processQueue = async () => {
  while (true) {
    const messages = await sqs.receiveMessage({
      QueueUrl: process.env.QUEUE_URL,
      MaxNumberOfMessages: 10
    }).promise();

    if (!messages.Messages) continue;

    for (const message of messages.Messages) {
      let retryCount = 0;

      try {
        const event = JSON.parse(message.Body);

        // Extract retry count from message attributes
        if (message.Attributes?.ApproximateReceiveCount) {
          retryCount = parseInt(message.Attributes.ApproximateReceiveCount);
        }

        // Attempt processing with exponential backoff
        try {
          await processOrder(event.data);
        } catch (error) {
          if (retryCount < MAX_RETRIES) {
            // Requeue with backoff
            const delay = Math.min(300, Math.pow(2, retryCount) * 10);
            await sqs.changeMessageVisibility({
              QueueUrl: process.env.QUEUE_URL,
              ReceiptHandle: message.ReceiptHandle,
              VisibilityTimeout: delay
            }).promise();

            console.log(`Retry ${retryCount}/${MAX_RETRIES} after ${delay}s`, error.message);
            continue;
          } else {
            // Move to DLQ
            throw new Error(`Failed after ${MAX_RETRIES} retries: ${error.message}`);
          }
        }

        // Success - delete from queue
        await sqs.deleteMessage({
          QueueUrl: process.env.QUEUE_URL,
          ReceiptHandle: message.ReceiptHandle
        }).promise();

      } catch (error) {
        // Send to Dead Letter Queue
        console.error('Moving to DLQ:', error);

        await sqs.sendMessage({
          QueueUrl: process.env.DLQ_URL,
          MessageBody: JSON.stringify({
            originalMessage: message.Body,
            error: error.message,
            failedAt: new Date().toISOString(),
            retryCount: retryCount
          })
        }).promise();

        // Remove from main queue
        await sqs.deleteMessage({
          QueueUrl: process.env.QUEUE_URL,
          ReceiptHandle: message.ReceiptHandle
        }).promise();
      }
    }
  }
};

Key patterns:

Exponential backoff prevents overwhelming failing services
DLQ captures unrecoverable failures
Log failure reasons for debugging
Implement manual replay when issue is fixed

Solution 3: Idempotency (The Silent Killer)

Here's what nobody tells you: Shopify can send the same webhook multiple times.

Network hiccup? Retry. Your endpoint slow? Timeout and retry. You need idempotency or you'll have duplicate orders.

Track webhook IDs in your database:

const processOrder = async (order) => {
  const webhookId = order.webhookId || order.id; // Shopify webhook ID

  // Check if already processed
  const existing = await db.webhooks.findOne({ webhookId });
  if (existing) {
    console.log(`Webhook ${webhookId} already processed, skipping`);
    return; // Already done
  }

  try {
    // Do the actual processing
    await updateInventory(order);
    await createCustomerRecord(order.customer);
    await sendConfirmationEmail(order);
    await syncToERP(order);

    // Record that we processed this webhook
    await db.webhooks.insertOne({
      webhookId,
      processedAt: new Date(),
      success: true,
      orderData: order
    });

  } catch (error) {
    // Still record the attempt for debugging
    await db.webhooks.insertOne({
      webhookId,
      processedAt: new Date(),
      success: false,
      error: error.message,
      orderData: order
    });
    throw error;
  }
};

This single pattern prevents:

Duplicate orders
Duplicate charges
Duplicate inventory adjustments
Duplicate customer records

Solution 4: Event-Driven Architecture

As you scale, processing one event type directly in your worker becomes complex:

// Anti-pattern: worker does everything
const processOrder = async (order) => {
  updateInventory(order);      // slow
  createCustomer(order);       // sometimes fails
  sendEmail(order);            // unreliable
  updateAnalytics(order);      // high volume
  syncERP(order);              // complex
  assignLoyaltyPoints(order);  // business logic
  // One failure fails everything
};

Better: emit events and let services consume independently:

const EventEmitter = require('events');
const orderEvents = new EventEmitter();

// Main webhook handler - just emit event
const processOrder = async (order) => {
  // Record webhook processed
  await recordWebhook(order.id);

  // Emit event - handlers consume independently
  orderEvents.emit('order.created', {
    id: order.id,
    customerId: order.customer.id,
    amount: order.total_price,
    timestamp: new Date()
  });
};

// Separate services consume the event
orderEvents.on('order.created', async (event) => {
  try {
    await updateInventory(event);
  } catch (error) {
    // Log and queue retry - doesn't affect other consumers
    console.error('Inventory update failed:', error);
  }
});

orderEvents.on('order.created', async (event) => {
  try {
    await sendConfirmationEmail(event);
  } catch (error) {
    // Log and queue retry - inventory still updated
    console.error('Email failed:', error);
  }
});

orderEvents.on('order.created', async (event) => {
  try {
    await syncERP(event);
  } catch (error) {
    console.error('ERP sync failed:', error);
  }
});

Benefits:

Each service scales independently
One failure doesn't cascade
Easy to add new consumers
Clear separation of concerns

At scale, use Kafka, RabbitMQ, or AWS SNS/SQS for event distribution.

The Monitoring You Need

You can't manage what you don't measure. Instrument your webhook system:

const prometheus = require('prom-client');

// Create metrics
const webhookCounter = new prometheus.Counter({
  name: 'webhooks_received_total',
  help: 'Total webhooks received',
  labelNames: ['type', 'status']
});

const processingDuration = new prometheus.Histogram({
  name: 'webhook_processing_duration_seconds',
  help: 'Time to process webhook',
  labelNames: ['type'],
  buckets: [0.1, 0.5, 1, 5, 10]
});

const queueDepth = new prometheus.Gauge({
  name: 'webhook_queue_depth',
  help: 'Number of pending webhooks'
});

// Use in your code
app.post('/webhooks/orders/create', async (req, res) => {
  const startTime = Date.now();

  try {
    // ... process webhook
    webhookCounter.inc({ type: 'order.create', status: 'queued' });
    res.status(200).send('OK');
  } catch (error) {
    webhookCounter.inc({ type: 'order.create', status: 'failed' });
    res.status(202).send('Retry');
  } finally {
    const duration = (Date.now() - startTime) / 1000;
    processingDuration.observe({ type: 'order.create' }, duration);
  }
});

// Monitor queue depth
setInterval(async () => {
  const depth = await sqs.getQueueAttributes({
    QueueUrl: process.env.QUEUE_URL,
    AttributeNames: ['ApproximateNumberOfMessages']
  }).promise();

  queueDepth.set(parseInt(depth.Attributes.ApproximateNumberOfMessages));
}, 30000);

Alert on:

Failure rate > 0.5%
Queue depth > 10,000
Processing latency p99 > 5s
DLQ messages appearing

Implementation Checklist

Building scalable webhook infrastructure requires getting multiple pieces right:

Foundation
[ ] Validate webhook signatures
[ ] Use message queue (SQS, RabbitMQ, Redis)
[ ] Implement idempotency
[ ] Add basic monitoring

Reliability
[ ] Implement dead letter queues
[ ] Add exponential backoff
[ ] Log all failures with context
[ ] Implement manual replay

Scalability
[ ] Add caching layers
[ ] Optimize database indexes
[ ] Horizontal scaling of workers
[ ] Monitor queue depth

Advanced
[ ] Event-driven architecture
[ ] Multiple queue consumers
[ ] Distributed tracing
[ ] Multi-region setup

Common Gotchas

Gotcha 1: Lambda timeout with SQS
Lambda has max 15-minute timeout. For long-running webhooks, use EC2/ECS workers or Fargate instead.

Gotcha 2: Forgetting webhook signature validation
Always validate. Otherwise anyone can send fake webhooks to your system.

Gotcha 3: Not idempotent processing
The second webhook always arrives. Your code must handle it.

Gotcha 4: Monitoring too late
Add monitoring before you hit scale issues. You need baseline metrics to recognize problems.

Gotcha 5: Forgetting Shopify rate limits
Shopify has API rate limits. If you process 1M webhooks, don't make 1M API calls back to Shopify immediately.

Resources

AWS SQS - Simple queue service
RabbitMQ - Open source message broker
Shopify Webhook Documentation
Distributed Tracing with Jaeger

Read the Complete Guide

For the complete technical guide with more code examples, architecture diagrams, and a detailed implementation checklist, check out the full article on Kolachi Tech.

The complete guide includes:

Advanced scaling patterns for multi-region deployments
Detailed monitoring and observability strategies
Complete 8-step implementation checklist
Architecture diagrams and design patterns
More real-world code examples

Have you scaled webhooks to production? What patterns saved you? Drop a comment below - I'd love to hear your war stories!

Shopify Webhook Observability: Complete Implementation Guide

Muhammad Masad Ashraf — Wed, 03 Jun 2026 22:12:51 +0000

Shopify Webhook Observability: Complete Implementation Guide

Webhooks are the nervous system of your Shopify integration. When they fail silently, data corrupts and you don't know why.

This guide covers production-ready webhook observability: structured logging, health metrics, error recovery, and debugging strategies.

The Problem

Webhooks are asynchronous. Your app receives events, processes them, and moves on. When something fails:

Shopify doesn't know your app crashed
You don't know a webhook failed
Data stays out of sync
Problems compound until customers complain

Without observability, you're flying blind.

What You'll Learn

Structured logging patterns for webhooks
Health metrics and dashboards
Error categorization and retry logic
Dead letter queue implementation
Debugging production incidents

Let's build visibility.

1. Structured Logging Foundation

All observability starts with logs.

Bad logs (unstructured text):

2024-06-04 14:23:45 Webhook received
2024-06-04 14:23:46 Processing
2024-06-04 14:23:47 Done

Unstructured logs are difficult to search, filter, and aggregate.

Good logs (JSON structured format):

{
  "timestamp": "2024-06-04T14:23:45Z",
  "webhook_id": "gid://shopify/WebhookEventBridge/12345",
  "shop_id": "gid://shopify/Shop/67890",
  "event_type": "orders/create",
  "status": "success",
  "duration_ms": 245,
  "attempt": 1,
  "signature_valid": true
}

Structured logs in JSON format enable filtering, aggregation, and analysis.

Entry Point Logging

Log immediately when webhook arrives. Here's a complete Express.js implementation:

const express = require('express');
const winston = require('winston');

const logger = winston.createLogger({
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: 'webhooks.log' })
  ]
});

app.post('/webhooks/shopify', (req, res) => {
  const webhookId = req.headers['x-shopify-webhook-id'];
  const shop = req.headers['x-shopify-shop-api-version'];
  const hmac = req.headers['x-shopify-hmac-sha256'];
  const topic = req.headers['x-shopify-topic'];

  logger.info('webhook_received', {
    webhook_id: webhookId,
    shop_id: shop,
    topic,
    timestamp: new Date().toISOString(),
    body_size: req.rawBody.length
  });

  res.status(202).send('Accepted');

  // Process async
  processWebhookAsync(req.body, webhookId, hmac);
});

Why this approach: Return 202 immediately. Process asynchronously. Shopify's timeout is 5 seconds; don't waste it.

Signature Verification Logging

Verify Shopify's HMAC signature before processing any webhook:

const crypto = require('crypto');

function verifySignature(rawBody, signature, secret) {
  const hash = crypto
    .createHmac('sha256', secret)
    .update(rawBody, 'utf8')
    .digest('base64');

  return hash === signature;
}

async function processWebhookAsync(body, webhookId, signature) {
  const apiSecret = process.env.SHOPIFY_API_SECRET;

  if (!verifySignature(JSON.stringify(body), signature, apiSecret)) {
    logger.error('webhook_signature_invalid', {
      webhook_id: webhookId,
      provided_signature: signature.substring(0, 16) + '...'
    });
    return;
  }

  logger.debug('webhook_signature_verified', {
    webhook_id: webhookId
  });

  // Continue processing
}

Never process unverified webhooks. Signature verification prevents unauthorized access.

Processing Step Logging

Log each stage of webhook processing with timing information. This helps identify where delays occur:

async function processWebhookAsync(body, webhookId, signature) {
  const startTime = Date.now();

  try {
    // Verify signature
    verifySignature(JSON.stringify(body), signature, apiSecret);

    // Parse webhook
    const webhook = parseWebhook(body);
    logger.debug('webhook_parsed', {
      webhook_id: webhookId,
      resource_id: webhook.id,
      resource_type: webhook.type
    });

    // Check duplicate
    const isDuplicate = await checkDuplicate(webhookId);
    if (isDuplicate) {
      logger.info('webhook_duplicate_detected', {
        webhook_id: webhookId,
        resource_id: webhook.id
      });
      return;
    }

    // Store to database
    await storeWebhook(webhook);
    logger.info('webhook_stored', {
      webhook_id: webhookId,
      resource_id: webhook.id
    });

    // Sync with systems
    await syncDownstream(webhook);
    logger.info('webhook_processed', {
      webhook_id: webhookId,
      duration_ms: Date.now() - startTime,
      resource_id: webhook.id
    });

  } catch (error) {
    logger.error('webhook_processing_failed', {
      webhook_id: webhookId,
      error: error.message,
      stack: error.stack,
      duration_ms: Date.now() - startTime
    });

    // Retry or dead letter queue
    await handleError(error, body, webhookId);
  }
}

External Service Logging

Track interactions with third-party APIs and downstream services. Log both successful calls and failures with timing information:

async function syncDownstream(webhook) {
  const endpoints = [
    { name: 'inventory-sync', url: 'https://inventory.internal/sync' },
    { name: 'analytics-pipeline', url: 'https://analytics.internal/events' }
  ];

  for (const endpoint of endpoints) {
    const startTime = Date.now();
    try {
      const response = await fetch(endpoint.url, {
        method: 'POST',
        body: JSON.stringify(webhook),
        timeout: 5000
      });

      const duration = Date.now() - startTime;

      if (!response.ok) {
        logger.warn('external_service_error', {
          webhook_id: webhook.id,
          service: endpoint.name,
          status: response.status,
          duration_ms: duration
        });
      } else {
        logger.debug('external_service_success', {
          webhook_id: webhook.id,
          service: endpoint.name,
          duration_ms: duration
        });
      }
    } catch (error) {
      logger.error('external_service_timeout', {
        webhook_id: webhook.id,
        service: endpoint.name,
        error: error.message,
        duration_ms: Date.now() - startTime
      });

      // Store for retry
      await storeFailedSync(webhook.id, endpoint.name);
    }
  }
}

2. Health Metrics and Dashboards

Track three key metrics:

Delivery Rate

Delivery Rate = (Webhooks Successfully Processed / Total Webhooks) x 100

Target: 99.9% or higher.

Processing Latency

async function calculateLatencyMetrics() {
  const logs = await queryLogs({
    status: 'success',
    timeRange: '24h'
  });

  const durations = logs
    .map(log => log.duration_ms)
    .sort((a, b) => a - b);

  return {
    median: percentile(durations, 50),
    p95: percentile(durations, 95),
    p99: percentile(durations, 99),
    max: Math.max(...durations)
  };
}

function percentile(arr, p) {
  const index = Math.ceil((p / 100) * arr.length) - 1;
  return arr[Math.max(0, index)];
}

Target: Median under 100ms, p95 under 500ms, p99 under 2s.

Error Rate by Type

async function getErrorMetrics() {
  const errorLogs = await queryLogs({
    status: 'error',
    timeRange: '24h'
  });

  const errorsByType = {};

  for (const log of errorLogs) {
    const type = log.error_type || 'unknown';
    errorsByType[type] = (errorsByType[type] || 0) + 1;
  }

  const total = errorLogs.length;

  return Object.entries(errorsByType).reduce((acc, [type, count]) => {
    acc[type] = {
      count,
      percentage: ((count / total) * 100).toFixed(2) + '%'
    };
    return acc;
  }, {});
}

Track:

Timeout errors (slow external services)
Database errors (connection pool exhausted)
Validation errors (bad payload)
Rate limit errors (too many requests)
Service unavailable (external APIs down)

Building a Dashboard

Use your logging platform's dashboard feature (Datadog, Splunk, CloudWatch, etc.). Here are example queries to calculate key metrics:

Query 1: Calculate delivery rate percentage over the last 24 hours

SELECT count(*) as total,
       sum(case when status = 'success' then 1 else 0 end) as successful,
       (sum(case when status = 'success' then 1 else 0 end) * 100.0 / count(*)) as delivery_rate
FROM webhooks
WHERE timestamp > now() - interval '24 hours'

Query 2: Calculate latency percentiles (50th, 95th, 99th) for successful webhooks

SELECT percentile_cont(0.5) WITHIN GROUP (ORDER BY duration_ms) as p50,
       percentile_cont(0.95) WITHIN GROUP (ORDER BY duration_ms) as p95,
       percentile_cont(0.99) WITHIN GROUP (ORDER BY duration_ms) as p99
FROM webhooks
WHERE timestamp > now() - interval '24 hours'
  AND status = 'success'

Query 3: Calculate error rate by error type

       count(*) as count,
       (count(*) * 100.0 / (SELECT count(*) FROM webhooks WHERE status = 'error')) as percentage
FROM webhooks
WHERE timestamp > now() - interval '24 hours'
  AND status = 'error'
GROUP BY error_type
ORDER BY count DESC

3. Error Handling and Retry Logic

Categorize errors. Different errors need different responses.

Error Classification

Different error types require different handling strategies. This function classifies errors so you can respond appropriately:

const ErrorType = {
  TRANSIENT_TIMEOUT: 'transient_timeout',
  TRANSIENT_NETWORK: 'transient_network',
  PERMANENT_INVALID_DATA: 'permanent_invalid_data',
  PERMANENT_VALIDATION: 'permanent_validation',
  SYSTEMIC_SERVICE_DOWN: 'systemic_service_down',
  RATE_LIMITED: 'rate_limited',
  DATABASE_ERROR: 'database_error'
};

function classifyError(error, statusCode) {
  if (error.code === 'ECONNREFUSED' || error.code === 'ENOTFOUND') {
    return ErrorType.TRANSIENT_NETWORK;
  }

  if (error.code === 'ETIMEDOUT') {
    return ErrorType.TRANSIENT_TIMEOUT;
  }

  if (statusCode === 429) {
    return ErrorType.RATE_LIMITED;
  }

  if (statusCode === 503 || statusCode === 502) {
    return ErrorType.SYSTEMIC_SERVICE_DOWN;
  }

  if (statusCode === 400 || statusCode === 422) {
    return ErrorType.PERMANENT_INVALID_DATA;
  }

  if (error.type === 'ValidationError') {
    return ErrorType.PERMANENT_VALIDATION;
  }

  if (error.message.includes('database')) {
    return ErrorType.DATABASE_ERROR;
  }

  return 'unknown';
}

Exponential Backoff Retry

Implement retry logic with exponential backoff and jitter to handle transient failures gracefully:

async function processWithRetry(webhook, maxAttempts = 3) {
  const delays = [1000, 2000, 4000]; // 1s, 2s, 4s

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      await processWebhook(webhook);

      logger.info('webhook_success', {
        webhook_id: webhook.id,
        attempt
      });
      return;

    } catch (error) {
      const errorType = classifyError(error, error.statusCode);

      // Don't retry permanent errors
      if (errorType.startsWith('permanent')) {
        logger.error('webhook_permanent_failure', {
          webhook_id: webhook.id,
          error_type: errorType,
          error: error.message,
          attempt
        });
        throw error;
      }

      // Last attempt?
      if (attempt === maxAttempts) {
        logger.error('webhook_max_retries_exceeded', {
          webhook_id: webhook.id,
          error_type: errorType,
          attempts: maxAttempts
        });
        throw error;
      }

      // Retry with backoff + jitter
      const baseDelay = delays[attempt - 1] || delays[delays.length - 1];
      const jitter = Math.random() * 1000;
      const totalDelay = baseDelay + jitter;

      logger.warn('webhook_retry', {
        webhook_id: webhook.id,
        error_type: errorType,
        attempt,
        next_retry_ms: Math.round(totalDelay)
      });

      await new Promise(resolve => setTimeout(resolve, totalDelay));
    }
  }
}

Dead Letter Queue Pattern

Implement a dead letter queue to capture webhooks that fail after all retry attempts. This prevents data loss and enables async reprocessing:

// PostgreSQL schema
const schema = `
CREATE TABLE IF NOT EXISTS dead_letter_queue (
  id SERIAL PRIMARY KEY,
  webhook_id VARCHAR(255) UNIQUE NOT NULL,
  payload JSONB NOT NULL,
  error_type VARCHAR(255),
  error_message TEXT,
  attempts INT DEFAULT 1,
  first_failure_at TIMESTAMP DEFAULT NOW(),
  last_failure_at TIMESTAMP DEFAULT NOW(),
  status VARCHAR(50) DEFAULT 'pending',
  created_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_dlq_status ON dead_letter_queue(status);
CREATE INDEX idx_dlq_first_failure ON dead_letter_queue(first_failure_at);
`;

// Store failed webhook
async function storeInDeadLetterQueue(webhook, error) {
  const errorType = classifyError(error, error.statusCode);

  try {
    await db.query(
      `INSERT INTO dead_letter_queue (webhook_id, payload, error_type, error_message)
       VALUES ($1, $2, $3, $4)
       ON CONFLICT (webhook_id) DO UPDATE SET
         attempts = attempts + 1,
         last_failure_at = NOW(),
         error_message = $4`,
      [webhook.id, JSON.stringify(webhook), errorType, error.message]
    );

    logger.warn('webhook_moved_to_dlq', {
      webhook_id: webhook.id,
      error_type: errorType
    });

  } catch (dbError) {
    logger.error('dlq_storage_failed', {
      webhook_id: webhook.id,
      error: dbError.message
    });
  }
}

// Reprocess dead letter queue
async function reprocessDeadLetterQueue() {
  const pendingWebhooks = await db.query(
    `SELECT * FROM dead_letter_queue
     WHERE status = 'pending'
     AND last_failure_at < NOW() - INTERVAL '1 hour'
     ORDER BY attempts ASC
     LIMIT 100`
  );

  for (const row of pendingWebhooks.rows) {
    try {
      await processWithRetry(row.payload);

      await db.query(
        `UPDATE dead_letter_queue SET status = 'processed' WHERE id = $1`,
        [row.id]
      );

      logger.info('dlq_webhook_reprocessed', {
        webhook_id: row.webhook_id,
        attempts: row.attempts
      });

    } catch (error) {
      logger.warn('dlq_reprocess_failed', {
        webhook_id: row.webhook_id,
        error: error.message
      });

      // Update last failure time, will retry again later
      await db.query(
        `UPDATE dead_letter_queue 
         SET last_failure_at = NOW()
         WHERE id = $1`,
        [row.id]
      );
    }
  }
}

// Run periodically (every 5 minutes)
setInterval(reprocessDeadLetterQueue, 5 * 60 * 1000);

4. Detecting Duplicates

Shopify retries webhooks when it doesn't receive a 200 response within 5 seconds. You must detect and handle duplicate webhook deliveries:

// Redis store for duplicate detection
const redis = require('redis');
const client = redis.createClient();

async function checkAndMarkProcessed(webhookId) {
  const key = `webhook:${webhookId}`;

  // Set with 48-hour expiration (Shopify retry window)
  const wasProcessed = await client.set(
    key,
    Date.now(),
    'EX',
    48 * 60 * 60,
    'NX' // Only set if key doesn't exist
  );

  if (!wasProcessed) {
    return true; // Already processed
  }

  return false; // First time seeing this webhook
}

// In your webhook handler
async function processWebhookAsync(body, webhookId, signature) {
  const isDuplicate = await checkAndMarkProcessed(webhookId);

  if (isDuplicate) {
    logger.info('webhook_duplicate_ignored', {
      webhook_id: webhookId,
      topic: body.topic
    });
    return;
  }

  // Process new webhook
  await processWithRetry(body);
}

Alternative (database-based duplicate detection using PostgreSQL unique constraint):

async function checkAndMarkProcessed(webhookId) {
  try {
    await db.query(
      `INSERT INTO processed_webhooks (webhook_id, processed_at)
       VALUES ($1, NOW())`,
      [webhookId]
    );
    return false; // First time
  } catch (error) {
    if (error.code === '23505') { // Unique constraint violation
      return true; // Already processed
    }
    throw error;
  }
}

5. Production Alerts

Set up automated alerts for critical metrics. These thresholds should trigger notifications to your team:

const alertThresholds = {
  deliveryRate: 0.99, // Alert if below 99%
  errorRate: 0.01, // Alert if above 1%
  p95Latency: 2000, // Alert if above 2s (ms)
  dlqSize: 100, // Alert if more than 100 pending
  noWebhooksIn: 3600 // Alert if no webhooks in 1 hour (seconds)
};

async function checkAlerts() {
  const metrics = await getMetrics();

  if (metrics.deliveryRate < alertThresholds.deliveryRate) {
    await sendAlert({
      severity: 'critical',
      title: 'Low webhook delivery rate',
      message: `Delivery rate: ${(metrics.deliveryRate * 100).toFixed(2)}%`,
      details: metrics
    });
  }

  if (metrics.p95Latency > alertThresholds.p95Latency) {
    await sendAlert({
      severity: 'warning',
      title: 'High webhook latency',
      message: `P95 latency: ${metrics.p95Latency}ms`,
      details: metrics
    });
  }

  if (metrics.dlqSize > alertThresholds.dlqSize) {
    await sendAlert({
      severity: 'warning',
      title: 'Dead letter queue growing',
      message: `${metrics.dlqSize} pending webhooks in DLQ`,
      details: metrics
    });
  }

  if (metrics.lastWebhookAge > alertThresholds.noWebhooksIn) {
    await sendAlert({
      severity: 'critical',
      title: 'No webhooks received',
      message: `Last webhook ${metrics.lastWebhookAge}s ago`,
      details: metrics
    });
  }
}

// Run every 5 minutes
setInterval(checkAlerts, 5 * 60 * 1000);

6. Testing Your Setup

Implement health check endpoints and test webhook functionality to verify your observability infrastructure is working:

// Health check endpoint
app.get('/health/webhooks', async (req, res) => {
  const metrics = await getMetrics();

  const isHealthy = 
    metrics.deliveryRate >= 0.99 &&
    metrics.errorRate <= 0.01 &&
    metrics.p95Latency <= 2000 &&
    metrics.dlqSize <= 100;

  res.status(isHealthy ? 200 : 503).json({
    status: isHealthy ? 'healthy' : 'degraded',
    metrics,
    timestamp: new Date().toISOString()
  });
});

// Test webhook endpoint
app.post('/webhooks/test', (req, res) => {
  const testWebhook = {
    id: 'test-' + Date.now(),
    topic: 'test/event',
    shop_id: 'test-shop',
    created_at: new Date().toISOString(),
    data: { test: true }
  };

  logger.info('test_webhook_received', testWebhook);

  res.status(200).json({ success: true });
});

Key Takeaways

Log structured data - JSON with context, not raw payloads
Track three metrics - Delivery rate, latency, error rate
Categorize errors - Different errors need different handling
Implement retries - Exponential backoff with jitter
Use dead letter queues - Prevent data loss
Detect duplicates - Shopify retries; you must handle it
Alert on thresholds - Know when things break
Test thoroughly - Load test at production scale

Resources

Ready to implement? Start with structured logging, then add metrics, then implement retries and DLQ. Each piece builds on the previous.

What webhook patterns have you encountered? Share in the comments!

DEV Community: Muhammad Masad Ashraf

8 Error Recovery Patterns That Keep Shopify GraphQL Clients Alive

First, stop trusting the status code

Classify before you react

Pattern 1: Treat userErrors as a real failure

Pattern 2: Exponential backoff (with jitter)

Pattern 3: Throttle before Shopify throttles you

Pattern 4: Circuit breakers for cascading failures

Pattern 5: Use partial data, don't discard it

Pattern 6: Idempotency makes retries safe

Pattern 7: Dead letter queues for the ones that still fail

Pattern 8: You can't fix what you can't see

Putting it together

How I Stopped Shopify's GraphQL API From Throttling My App (Parallel Query Patterns That Actually Work)

Why sequential queries are the silent killer

First, understand the cost model (this is the whole game)

Pattern 1: Controlled concurrent batching

Pattern 2: Aliasing — many queries, one request

Pattern 3: Parallel pagination via segmentation

Pattern 4: When to stop parallelizing and use Bulk Operations

Pattern 5: Coalesce duplicate in-flight requests

Backpressure: mirror Shopify's bucket on your side

Handle throttles gracefully (because they'll still happen)

Design for partial failure

Measure everything

The payoff: parallelism lowers your bill too

TL;DR

How I Build Fast Shopify API Clients (Batching, Caching, Backoff)

The client is your performance ceiling

REST vs GraphQL

Respect rate limits by design

Batch to cut round trips

Cache hard, invalidate smart

Concurrency without self-throttling

Retry logic that heals itself

Paginate the right way

Go async

Architect for resilience

Quick audit checklist

Wrapping up

How Shopify's GraphQL Rate Limits Actually Work (and How to Stop Getting 429'd)

The core idea: cost, not count

How query cost is scored

Limits by plan

The portability trap

Read your usage before you get punished

Patterns that make throttling disappear

Don't forget userErrors

When to drop synchronous queries entirely

The architecture that survives production

Caching Shopify GraphQL: A Practical Guide for Developers

The Core Problem

The 4 Cache Layers

What to Cache (and What Will Burn You)

Cache Invalidation: 3 Strategies

Smart Cache Keys (don't leak data!)

Handling Personalized Data

Measure It

Common Mistakes

Wrap-Up

How I Cut Our Shopify API Usage in Half (Without Losing Features)

First, know what you are paying for

Move from REST to GraphQL

Stop polling. Use webhooks.

Use Bulk Operations for big data sets

Cache the stuff that does not change

Handle rate limits like an adult

The checklist

Start here

Shopify GraphQL Pagination: How to Handle Large Datasets Without Slowing Down Your App

What Shopify GraphQL Pagination Solves

Shopify Uses Cursor-Based Pagination

Why Cursors Work Better Than Page Numbers

Page Size Is a Performance Decision

Watch GraphQL Query Cost

A Reliable Pagination Loop

Use Filters Before Pagination

When to Use Bulk Operations

Use Webhooks for Real-Time Updates

Common Mistakes

Pattern 1: Treat `userErrors` as a real failure

Don't forget `userErrors`

Check `partialDataUrl` on failure

Mistake 2: Using `===` for comparison