DEV Community

Stephen Souza
Stephen Souza

Posted on

Shopify fired the webhook. My server never processed it. Here's how I catch that now.

#shopify #webdev #monitoring

Shopify's webhook delivery is reliable. That's not the problem.

The problem is what happens after the webhook lands on your side.

Shopify fires orders/create. Your Flask endpoint receives it. Returns 200. Shopify marks it delivered. And somewhere between that 200 and the database write, something silently fails. Order never recorded. No error thrown. No alert fired. Customer emails you 3 hours later asking where their order confirmation is.

That's the webhook processing gap — and it's invisible to every standard monitoring tool because nothing failed by definition.

This is the Flask boilerplate I built to close it.

What this monitors

  • Order created — real-time alert with amount, items, customer
  • Order paid — payment confirmation with amount
  • Order cancelled — immediate warning with cancellation reason
  • Refund issued — instant alert with refund amount
  • Checkout started — abandoned checkout signal

Every event sends actual numeric amounts in the meta object. NotiLens ML reads these to learn your revenue baseline automatically — your normal Wednesday order volume, your typical refund rate, your checkout-to-order conversion pattern. When reality diverges from baseline, the anomaly alert fires. No thresholds to configure.

The broader concept

This boilerplate is Shopify-specific but the pattern applies to any webhook source — Stripe, GitHub, custom systems, internal services.

The concept is simple:

  1. Receive the webhook
  2. Verify the signature — confirms it's genuinely from the source
  3. Send a notification immediately — you know it arrived and was processed
  4. Send actual business metrics in the meta — enables ML baseline learning
  5. Use force_send=True for critical events — bypasses ML, fires immediately regardless The gap this closes: webhook delivered on the sender's side, processed on your side, NotiLens knows both happened. If orders/create webhooks stop arriving during business hours — silence alert fires. That's the "webhook processing silently stopped" detection that Shopify's own dashboard will never show you.

Setup

Step 1 — Install dependencies

pip install flask notilens
Enter fullscreen mode Exit fullscreen mode

Step 2 — Get credentials

  • Shopify webhook secret — Shopify Dashboard → Settings → Notifications → Webhooks → Signing secret
  • NotiLens token + secret — app.notilens.com → Create New Topic → Token + Secret

Step 3 — Configure

SHOPIFY_WEBHOOK_SECRET = "YOUR_SHOPIFY_WEBHOOK_SECRET"
NOTILENS_TOKEN         = "YOUR_NOTILENS_TOKEN"
NOTILENS_SECRET        = "YOUR_NOTILENS_SECRET"
APP_NAME               = "shopify-monitor"
PORT                   = 5000
Enter fullscreen mode Exit fullscreen mode

Step 4 — Run

python notilens-shopify-webhook-monitor.py
Enter fullscreen mode Exit fullscreen mode

Step 5 — Expose publicly

Local development:

ngrok http 5000
Enter fullscreen mode Exit fullscreen mode

Production — deploy to Railway, Render, or any server. Your webhook endpoint:

POST https://your-domain.com/webhook/shopify
Enter fullscreen mode Exit fullscreen mode

Step 6 — Register in Shopify

Shopify Dashboard → Settings → Notifications → Webhooks → Add webhook → paste your endpoint URL → select events.

The code — section by section

Signature verification 

def verify_shopify_signature(payload: bytes, hmac_header: str) -> bool:
    """Verify webhook is genuinely from Shopify"""
    digest = hmac.new(
        SHOPIFY_WEBHOOK_SECRET.encode("utf-8"),
        payload,
        hashlib.sha256
    ).digest()
    computed = base64.b64encode(digest).decode("utf-8")
    return hmac.compare_digest(computed, hmac_header or "")
Enter fullscreen mode Exit fullscreen mode

Every incoming webhook gets verified before processing. Shopify signs every webhook with your secret using HMAC-SHA256. If the signature doesn't match — 401, no processing, no NotiLens notification.

Never skip this. Any public endpoint without signature verification is open to spoofed webhooks.

Initialize NotiLens 

nl = notilens.init(name=APP_NAME, token=NOTILENS_TOKEN, secret=NOTILENS_SECRET)
Enter fullscreen mode Exit fullscreen mode

One line. The nl object is used for all notifications throughout the handler.

Order created 

if topic == "orders/create":
    total = price(data.get("total_price"), currency)
    customer = data.get("email") or data.get("contact_email", "guest")
    items = len(data.get("line_items", []))
    nl.notify(
        "order.created",
        f"New order — {total} · {items} item(s) · {customer}",
        force_send=False,  # route through ML — reduces noise on high volume
        meta={
            "order_id":     data.get("id"),
            "order_number": data.get("order_number"),
            "total":        float(data.get("total_price", 0)),
            "currency":     currency,
            "items":        items,
            "customer":     customer,
            "is_new":       data.get("customer", {}).get("orders_count", 1) == 1,
        },
        tags="shopify,order",
    )
Enter fullscreen mode Exit fullscreen mode

force_send=False — on high volume stores, every order firing an immediate push notification becomes noise fast. With force_send=False, NotiLens routes the notification through ML. Normal order arrives — logged, no push. Anomaly detected — push fires. This is how you get alerted when order volume drops 80% from your Wednesday baseline without getting pinged on every single order.

is_new flag — tracks whether this is a first-time customer. Useful for conversion monitoring.

Refund and cancellation — force send 

elif topic == "refunds/create":
    nl.notify(
        "refund.created",
        f"Refund issued — {amt}",
        level="warning",
        meta={
            "order_id":       data.get("order_id"),
            "refund_id":      data.get("id"),
            "amount":         total_refund,
            "currency":       currency,
            "items_refunded": len(refund_lines),
        },
        tags="shopify,refund",
    )
Enter fullscreen mode Exit fullscreen mode

No force_send flag here — defaults to True for warning level events. Refund issued and order cancelled are things you want to know about immediately, not filtered through ML. Push fires regardless of baseline.

Checkout started — abandoned checkout signal 

elif topic == "checkouts/create":
    nl.notify(
        "checkout.created",
        f"Checkout started — {total} · {customer}",
        force_send=False,
        meta={
            "checkout_id": data.get("id"),
            "total":       float(data.get("total_price", 0)),
            "currency":    currency,
            "customer":    customer,
            "items":       len(data.get("line_items", [])),
        },
        tags="shopify,checkout",
    )
Enter fullscreen mode Exit fullscreen mode

Tracking checkouts/create alongside orders/create gives you a checkout-to-order signal. NotiLens learns the normal ratio between checkouts started and orders completed. When that ratio changes significantly — checkout volume stays the same but orders drop — that's the silent failure signal. Payment processing broke somewhere between checkout and order confirmation.

This is the broken flow detection concept applied to e-commerce — not monitoring individual events in isolation but the relationship between them.

The webhook handler — full flow 

@app.route("/webhook/shopify", methods=["POST"])
def shopify_webhook():
    payload    = request.data
    hmac_header = request.headers.get("X-Shopify-Hmac-Sha256", "")
    topic      = request.headers.get("X-Shopify-Topic", "")

    if not verify_shopify_signature(payload, hmac_header):
        return jsonify({"error": "Invalid signature"}), 401

    data     = request.get_json()
    currency = data.get("currency", "USD")

    # ... event handlers ...

    return jsonify({"success": True}), 200
Enter fullscreen mode Exit fullscreen mode

Always return 200 after successful processing. Returning non-200 causes Shopify to retry delivery — which creates duplicate processing if your actual handler succeeded. The 200 confirms receipt. NotiLens confirms what happened after receipt.

What the monitoring layer adds

Without this boilerplate — Shopify fires webhooks, your endpoint processes them, you find out something went wrong when a customer emails you.

With this boilerplate:

Every event processed — NotiLens logs it. You have a record of every order, payment, refund, and checkout that arrived and was processed. Not just what Shopify sent — what your backend actually handled.

Silence detection — if orders/create webhooks stop arriving during your normal business hours, NotiLens fires a silence alert. Your backend might be healthy. Your Shopify webhook registration might have expired. The endpoint URL might have changed after a deployment. NotiLens catches the gap.

Revenue anomaly detection — NotiLens ML learns your normal order volume and average order value. Sudden drop in order frequency, spike in refund volume, checkout-to-order ratio collapsing — anomaly alert fires before you check the end-of-day report.

Refund and cancellation spikes — immediate push notification. Not filtered. Not batched. Fires the moment it happens.

Adapting for other webhook sources

The same pattern works for any webhook source. Replace the Shopify signature verification with the equivalent for your source:

Stripe:

import stripe
event = stripe.Webhook.construct_event(
    payload, sig_header, stripe_webhook_secret
)
Enter fullscreen mode Exit fullscreen mode

GitHub:

signature = "sha256=" + hmac.new(
    secret.encode(), payload, hashlib.sha256
).hexdigest()
Enter fullscreen mode Exit fullscreen mode

Custom internal service:

# Use a shared secret in the request header
token = request.headers.get("X-Webhook-Token")
if token != WEBHOOK_SECRET:
    return 401
Enter fullscreen mode Exit fullscreen mode

The NotiLens notification pattern — nl.notify() with numeric meta values — works the same regardless of source. The ML baseline learning works on whatever numeric values you send. Order amounts, payment values, record counts, response times — all become signals NotiLens learns from.

Full boilerplate

Copy the complete script from the gist:

👉 gist.github.com/notilens/703cd96c1d08ec441f1f102d8b001249

Why this pattern matters

Shopify's webhook logs show delivery. Your server logs show receipt. Neither shows you whether the business logic that runs after receipt actually worked correctly.

That gap — between webhook delivered and business outcome confirmed — is where silent failures live. A webhook monitor that only tracks delivery is watching the wrong thing.

The pattern this boilerplate implements watches the outcome: order processed, payment recorded, refund logged, checkout started. When those outcomes stop arriving — that's the alert.

notilens.com — 7-day free trial, no credit card required.

Top comments (0)