What happens to a Shopify webhook when your endpoint goes down

Hydra97 — Tue, 06 Oct 2026 12:46:28 +0000

I spent the last few months building a service that sits between Shopify or WooCommerce stores and the systems that consume their webhooks: ERPs, warehouse tools, accounting. Along the way I kept a list of the failure modes behind "an order never showed up" tickets. None of them are exotic, but most integrations I looked at handle only one or two.

Here is the list, with the fix I used for each.

1. Shopify gives you five seconds

The Shopify docs are direct about it: "If your app doesn't respond within five seconds, the delivery fails."

The common mistake is doing the real work inside the request: verify, parse, call the ERP, write to the warehouse system, then return 200. When the ERP is slow, Shopify records a failure even though your code finished the job, and a retry arrives for an order you already processed.

The fix is to split receiving from delivering. The endpoint does three things only: verify the signature, store the raw body, answer 200. Delivery to the ERP happens after the response.

@app.post("/webhooks/shopify")
async def shopify_webhook(request: Request):
    raw_body = await request.body()
    signature = request.headers.get("x-shopify-hmac-sha256", "")

    if not signature_ok(SHOPIFY_SECRET, raw_body, signature):
        return JSONResponse({"ok": False}, status_code=401)

    event_id = store_event(request.headers, raw_body)
    enqueue_delivery(event_id)
    return {"ok": True}

2. Eight retries, then the subscription is gone

If your endpoint keeps failing, Shopify "retries failed webhook calls up to eight times in a four-hour period", and if failures persist, the subscription is removed.

In practice: a deploy breaks the endpoint on Friday evening. By Saturday morning every order event from that window is lost, and Shopify has stopped sending new ones. Nobody sees an error. Orders just stop arriving in the ERP.

Two things help:

Keep the receiving endpoint small and boring, so it is the last thing to break. It should not depend on the ERP being up.
Retry on your side, over a longer window than Shopify's. Once the event is stored, the destination can be down for a day and nothing is lost.

The schedule I ended up with is 12 attempts, backing off from 30 seconds to 24 hours, about three days in total:

BACKOFF_SECONDS = (30, 120, 600, 1800, 3600, 7200, 14400, 21600, 43200, 86400, 86400)


def retry_delay(failed_attempts: int) -> int:
    index = min(max(failed_attempts, 1), len(BACKOFF_SECONDS)) - 1
    return BACKOFF_SECONDS[index]

When the last attempt fails, the job is marked dead, not deleted, so it can be replayed after the destination is fixed.

3. Verify the HMAC against the raw bytes

Shopify signs the body with your secret and sends the base64 result in X-Shopify-Hmac-Sha256. The check is short:

import base64
import hashlib
import hmac


def signature_ok(secret: str, raw_body: bytes, header_value: str) -> bool:
    digest = hmac.new(secret.encode("utf-8"), raw_body, hashlib.sha256).digest()
    expected = base64.b64encode(digest).decode("ascii")
    return hmac.compare_digest(expected, header_value)

Two details break it in real code:

It has to run on the raw request body. If the framework parses the JSON and you serialize it again, key order and whitespace change and the signature never matches.
Use hmac.compare_digest, not ==.

If you forward webhooks to another system, there is a third one. The signature is valid only for the exact original bytes, so if you transform the payload before forwarding, drop the HMAC header. Otherwise the receiver rejects a perfectly good event.

WooCommerce uses the same scheme (base64 HMAC-SHA256 of the body) in X-WC-Webhook-Signature, so one function covers both.

4. The same delivery can arrive twice

If your 200 lands after the timeout, Shopify counts the attempt as failed and sends the same delivery again, even though you already processed it. Shopify's docs say to ignore duplicates using X-Shopify-Webhook-Id.

I store that ID with a unique index:

CREATE UNIQUE INDEX ix_webhook_events_project_external_id
    ON webhook_events (project_id, external_id);

The index matters more than a "select first, then insert" check, because two retries can arrive at the same moment. A duplicate hits the index, gets a 200 back and only bumps a counter on the original event. The 200 is important: an error response to a duplicate counts as one more failure on Shopify's side.

WooCommerce has a trap here. X-WC-Webhook-Delivery-ID is a hash of the webhook ID and the current time in seconds, so two different orders created in the same second get the same delivery ID. Deduplicate on that header alone and you drop real orders during a busy sale. I combine it with a hash of the body.

5. Order is not guaranteed

From Shopify's best practices page: "Shopify doesn't guarantee ordering within a topic, or across different topics for the same resource." A products/update can arrive before products/create.

So handlers should not assume a sequence. Compare updated_at before overwriting a record, or treat the webhook as a signal and fetch the current state from the Admin API. Shopify also recommends a periodic reconciliation job, because "webhook delivery isn't always guaranteed". Webhooks are the fast path, not the only path.

6. WooCommerce: disabled after five failures, and the setup ping

WooCommerce is stricter than Shopify. Its docs say webhooks "are disabled after 5 retries by default if the delivery URL returns an unsuccessful status". Store owners rarely notice the status quietly switching to Disabled.

There is also the ping. When you save a webhook in WooCommerce, it sends a test POST with the body webhook_id=123 and no signature. If your endpoint rejects unsigned requests with a 401, WooCommerce refuses to save the webhook. Recognize that body, answer 200 and don't record it as an order.

7. Know when it breaks

Everything above is invisible when it works, and just as invisible when it fails. The last piece is an alert: one Slack or Discord message when deliveries to a destination start failing, and one when they recover. Throttle it per project, or an outage during a busy hour sends one message per order.

Putting it together

Shopify / WooCommerce
        |  POST, signed
        v
  verify HMAC  --bad-->  401 + alert
        |
  dedupe on delivery ID  --seen-->  200
        |
  store raw body  -->  200 to the store, well under 5 s
        |
  queue  -->  destination (ERP, WMS, accounting)
        |  fails
  retry with backoff, 12 attempts over ~3 days
        |  still failing
  dead job + alert, replay by hand when fixed

About the project

I packaged all of this into WebhookVault, a self-hosted service in Python (FastAPI, SQLAlchemy, PostgreSQL or SQLite) that sits in front of store webhooks. It verifies signatures, drops duplicates, queues deliveries with retries, supports replay and payload transforms, sends Slack or Discord alerts, and shows orders, refunds and products in a readable dashboard. Live demo: https://webhookvault.onrender.com

I don't plan to run it as a SaaS myself, so I'm selling the full codebase and IP to one buyer, ideally an agency or team that builds Shopify integrations for clients. If that's you, email me at hydra470497@gmail.com.

Questions about any of the failure modes above are welcome in the comments.

Ask HN: Selling WebhookVault – self-hosted webhook infrastructure platform ($15k OBO) https://webhookvault.onrender.com/dashboard

Hydra97 — Thu, 25 Jun 2026 12:58:41 +0000

I'm selling WebhookVault, a self-hosted webhook infrastructure platform. FastAPI + PostgreSQL + Docker. Features: webhook capture/replay/retry, RBAC, multi-org, analytics, audit logs, Stripe billing. Full source code + deployment guide + 2 weeks support. Asking $15,000 OBO. DM me if interested.

DEV Community: Hydra97