Zero-Downtime Webhook Migrations: Changing Endpoints Without Data Loss

API endpoint migration
API event loss prevention
API gateway webhook migration
API integration migration
API uptime strategies
API webhook migration guide
asynchronous webhook delivery
change webhook URL without data loss
developer guide webhook migration
DevOps API reliability
DevOps webhook migration
DNS migration webhook drop
DNS propagation webhook loss
handling webhook downtime
instant webhook endpoint swap
InstaWebhook migration
prevent dropped webhooks
production webhook migration
reliable webhook delivery
replay missed webhooks
resilient webhook architecture
safe webhook URL update
seamless server migration
static webhook proxy
static webhook URL pattern
webhook architecture best practices
webhook buffer shield
webhook buffer store replay
webhook data loss prevention
webhook endpoint change
webhook event buffering
webhook event ingestion
webhook event queueing
webhook failover strategy
webhook gateway migration
webhook infrastructure engineering
webhook listener migration
webhook listener update
webhook management platform
webhook migration
webhook payload replay
webhook proxy layer
webhook reliability pattern
webhook request forwarding
webhook retry logic
webhook server cutover
webhook server migration
webhook traffic routing
webhook URL redirection
zero downtime API updates
zero downtime backend migration
zero downtime deployment webhooks
zero downtime server cutover
zero downtime webhook migration
zero payload loss migration
Zero Downtime Webhook Migrations Changing Endpoints Without Data Loss
Zero-Downtime Webhook Migrations: Changing Endpoints Without Data Loss
Every DevOps engineer and backend developer has felt the anxiety of migrating a live webhook consumer. Whether you're re-architecting a monolith into microservices, switching cloud providers, or restructuring your API domains, changing a webhook endpoint URL in production has historically been fraught with danger.

Unlike standard REST API calls — where your client controls the retry logic — webhooks invert control. Third-party providers like Stripe, GitHub, Shopify, and Twilio push events to your public endpoints on their own schedule, with their own (often very different) rules about what happens when delivery fails. Update that URL carelessly and you can end up with hours of silently dropped payments, signups, or inventory syncs.

This guide covers why direct endpoint switching fails, how a persistent webhook proxy/gateway pattern solves it, how to preserve signature verification in transit, and how to replay missed events after an outage — updated with current (2026) provider retry behavior and the real tools teams use to build this today.

1. Why Direct Webhook Endpoint Switching Fails In a naive migration, an engineer opens the provider's dashboard, changes the destination URL from https://api-v1.example.com/webhooks to https://api-v2.example.com/webhooks, and clicks save.

Code example
Copy code
+------------------+ Direct Endpoint Change +-------------------------+
| | -------------------------------------> | Old Endpoint (Dying) |
| Webhook Provider | (Connection reuse / in-flight reqs) | https://api-v1.example |
| (Stripe, GitHub) | +-------------------------+
| |
| | +-------------------------+
| | -------------------------------------> | New Endpoint (Cold) |
| | (Delayed traffic switch) | https://api-v2.example |
+------------------+ +-------------------------+
This introduces four real structural failure points.

A. HTTP Keep-Alive and Connection Reuse
The old article framed this purely as "DNS TTL caching," but the more precise (and better documented) mechanism is connection pooling. Many HTTP clients — including the ones vendor infrastructure runs on — reuse persistent keep-alive connections rather than re-resolving DNS on every request, because that avoids repeating the DNS + TCP + TLS handshake cost on each call. If a sender's dispatcher is mid-connection to your old server when you cut over, it will keep sending requests down that connection until it's forcibly closed or times out, regardless of what your DNS record now says. Node.js Agent/HttpClient-style keep-alive pools are a well-documented source of this exact problem when teams do DNS-based traffic switching.

B. Asynchronous In-Flight Payload Drops
At the instant you terminate the old server or redirect traffic, in-flight HTTP POSTs mid-flight terminate with ECONNRESET, 502 Bad Gateway, or 503 Service Unavailable. What happens next depends entirely on the provider — which is where most teams get surprised.

C. Provider Retry Policies Are Wildly Inconsistent
This is the part worth getting exactly right, because retry behavior changes over time and isn't uniform across providers. Here's the current state as of 2026:

Provider Automatic retry? Window / attempts Notes
Stripe Yes Up to ~3 days (72 hrs), exponential backoff, live mode. Test mode: 3 attempts over a few hours. Disables the endpoint and emails you after 3 days of failures. Signed with HMAC-SHA256 via Stripe-Signature.
Shopify Yes 8 attempts over a 4-hour window, exponential backoff, 5-second response timeout per attempt. This is a real, important change: Shopify shortened its retry window from the old "19 attempts over 48 hours" policy on September 10, 2024. A lot of blog posts and even some tooling docs still cite the old 19/48h figure — if your reliability code was written before that date, its timing assumptions are stale. Persistent failures get the subscription auto-removed, not just paused.
GitHub No N/A — GitHub does not automatically retry failed webhook deliveries at all. You must manually redeliver from the UI or REST API, and only within a retention window (3 days on GitHub.com/Enterprise Cloud, 7 days on GitHub Enterprise Server). After that window, a failed delivery is gone for good unless you built your own capture layer.
Twilio Partial / limited Connection-level retry (3 connection attempts, 1-second timeout) via Connection Overrides; status callbacks reportedly get a small number of retries with backoff. Voice webhooks have a hard 15-second timeout. Twilio does not keep a durable record of failed deliveries for later inspection or replay — once retries are exhausted, the event is gone from Twilio's side.
The upshot: the original assumption that "most providers retry 3–5 times and give up" undersells the risk in both directions. Stripe is actually fairly forgiving (3 days). GitHub is the opposite extreme — it doesn't retry automatically at all, which means a few minutes of downtime during a naive cutover can permanently lose events unless you're polling its Deliveries API or redelivering manually. Don't assume uniform behavior across providers; check each one's current docs before you plan a migration around them.

D. The Idempotency Gap
Without a centralized event queue, a "dual-endpoint" cutover can let both the old and new backend process the same payload simultaneously, causing double-charges, race conditions, or corrupted state. This risk doesn't go away with a proxy — it just moves to needing explicit idempotency keys, covered in Section 4.

1. The Solution: A Webhook Proxy / Gateway in Front of Your Consumer The fix is to decouple ingestion from processing. A permanent, boring, ultra-reliable proxy — a static gateway URL that never changes — insulates your application infrastructure from both provider configuration and your own internal re-architecture.

Code example
Copy code
+------------------+ Static URL (Never Changes) +------------------------+
| Webhook Provider | ----------------------------------------> | Webhook Proxy / Gateway|
| (Stripe/GitHub) | https://hooks.yourdomain.com/v1/ingest | (Buffer / Dynamic Router)
+------------------+ +------------------------+
|
+--------------------------------+--------------------------------+
| (Shadow phase / dynamic cutover) |
v v
+-----------------------------+ +-----------------------------+
| Legacy Consumer Endpoint | | Modern Consumer Endpoint |
| https://api-v1.internal/ | | https://api-v2.internal/ |
+-----------------------------+ +-----------------------------+
Build it yourself, or use an existing tool
The original version of this guide referenced a fictional "InstaWebhook" product. In practice, the 2026 landscape has real, purpose-built options, and it's worth knowing which category each one solves before picking one:

Hookdeck Event Gateway — an inbound webhook proxy: ingestion, buffering, transformation, routing, and retries for webhooks arriving at you. This is the closest match to the "proxy shield" pattern described here.
Svix — primarily an outbound webhook-sending platform (used by companies like Clerk, Brex, and Resend to deliver webhooks to their own customers), with a newer "Ingest" product for inbound receiving.
Convoy — an open-source, self-hostable gateway (written in Go) that handles both inbound and outbound webhook delivery, useful if you need to keep everything on your own infrastructure.
Hook0 — a smaller open-source option, mainly for outbound sending, with EU data residency as a differentiator.
ngrok — useful for local development tunneling, but not a production migration tool.
If you'd rather not adopt a third-party dependency, the DIY pattern below (proxy + durable buffer + DLQ) is exactly what these tools implement internally, and is a reasonable choice if your volume is modest or you have strict infrastructure constraints.

Key Components
Static ingestion URL — e.g. https://hooks.yourdomain.com/v1/stripe. You register this once with the provider and never touch it again.
Durability buffer — the proxy writes raw incoming HTTP headers and bytes to an append-only store (Redis Streams, AWS SQS, NATS, Kafka) and immediately responds 200/202 to the provider.
Dynamic forwarding router — an internal worker layer that reads from the buffer and forwards payloads to your current backend target. The target can change in real time via config, without touching the provider.
Dead letter queue (DLQ) + replay engine — an automated store for payloads that fail at your target backend, so you can replay them without asking the vendor to resend.

1. Step-by-Step: Executing a Zero-Downtime Migration Walking through migrating a payment webhook consumer from api-v1.company.internal to api-v2.company.internal.

Phase 1: Deploy the Proxy Layer
Below is a minimal Node.js/Express proxy demonstrating payload buffering and dynamic target forwarding. It's illustrative — in production you'd likely reach for one of the tools above, or harden this considerably (auth on the admin route, structured logging, backpressure handling).

Code example
Copy code
import express, { Request, Response } from 'express';
import axios from 'axios';
import { createClient } from 'redis';

const app = express();
// Capture raw body buffer to preserve signature verification integrity
app.use(express.raw({ type: '/', limit: '10mb' }));

const redis = createClient({ url: process.env.REDIS_URL });
redis.connect();

// Target destination managed dynamically (via DB, Redis key, or env)
let CURRENT_TARGET_URL = process.env.INITIAL_TARGET_URL || 'https://api-v1.company.internal/webhooks';

// Dynamic route management API — protect this behind auth in production
app.post('/admin/set-target', express.json(), (req: Request, res: Response) => {
const { newTargetUrl } = req.body;
if (!newTargetUrl) return res.status(400).json({ error: 'Missing newTargetUrl' });

CURRENT_TARGET_URL = newTargetUrl;
console.log([ROUTE CHANGED] Forwarding traffic to: ${CURRENT_TARGET_URL});
return res.status(200).json({ success: true, activeTarget: CURRENT_TARGET_URL });
});

// Primary webhook ingestion endpoint
app.all('/v1/ingest/*', async (req: Request, res: Response) => {
const eventId = evt_${Date.now()}_${Math.random().toString(36).substring(2, 9)};
const targetUrl = CURRENT_TARGET_URL;

const rawHeaders = { ...req.headers };
delete rawHeaders['host'];
delete rawHeaders['content-length'];

const payload = {
id: eventId,
path: req.params[0],
headers: rawHeaders,
body: req.body.toString('utf-8'),
targetUrl,
timestamp: new Date().toISOString(),
};

try {
// 1. Immediately buffer raw payload to a durable store
await redis.lPush('webhook_audit_log', JSON.stringify(payload));

// 2. Forward payload to the currently active consumer endpoint
const response = await axios({
  method: req.method,
  url: `${targetUrl}/${req.params[0]}`,
  data: req.body,
  headers: { ...rawHeaders, 'X-Proxy-Event-ID': eventId },
  timeout: 5000,
  validateStatus: () => true, // don't throw on non-2xx
});

if (response.status >= 200 && response.status < 300) {
  return res.status(200).send({ received: true });
} else {
  await redis.lPush('webhook_dlq', JSON.stringify({ ...payload, error: `HTTP ${response.status}` }));
  return res.status(200).send({ received: true, buffered: true });
}

} catch (error: any) {
await redis.lPush('webhook_dlq', JSON.stringify({ ...payload, error: error.message }));
// Acknowledge the vendor to avoid endpoint disablement; retain payload in DLQ
return res.status(200).send({ received: true, buffered: true });
}
});

app.listen(3000, () => console.log('Proxy Shield active on port 3000'));
Phase 2: Preserving Signature Verification
The most common technical hurdle when inserting a proxy is HMAC signature verification. Providers sign payloads with headers like Stripe-Signature or X-Hub-Signature-256; a growing number follow the open Standard Webhooks specification, which uses webhook-id, webhook-timestamp, and webhook-signature headers with HMAC-SHA256.

To avoid breaking verification:

Pass raw body bytes unmodified. Don't parse req.body into a JSON object before forwarding — re-serializing changes key ordering and whitespace, which invalidates the hash.
Preserve original headers, including the timestamp used for replay protection. The Standard Webhooks spec recommends rejecting requests with a timestamp more than 300 seconds (5 minutes) old; Stripe's libraries use a similar tolerance window.
Keep the byte stream intact end-to-end — over mTLS or private networking if your proxy terminates HTTPS and re-encrypts to the target.
Phase 3: Shadow Traffic Testing
Before cutting over fully, run shadow mode: forward 100% of production traffic to api-v1 (primary) while duplicating identical traffic to api-v2 in the background, without letting api-v2 write to production data.

Strategy Primary Receiver Shadow Receiver DB Writes Goal
Shadow Mode api-v1 api-v2 v1 real, v2 mocked/read-only Validate v2 latency, signature handling, error rate under real load
Canary Rollout api-v1 (90%) api-v2 (10%) Both write to primary DB Test incremental traffic distribution
Full Cutover None (retired) api-v2 (100%) v2 primary DB Complete transition
Phase 4: Atomic Cutover
Once shadow metrics show a 0% error rate on v2, flip the route with a single control-plane call:

Code example
Copy code
curl -X POST https://hooks.yourdomain.com/admin/set-target \
-H "Content-Type: application/json" \
-d '{"newTargetUrl": "https://api-v2.company.internal/webhooks"}'
The switch happens in memory — no DNS propagation, no provider dashboard changes. Monitor proxy queue depth and response codes on v2 closely for the first hour.

1. Replaying Missed Webhooks After an Outage Even with careful planning, target endpoints will have transient failures — connection pool exhaustion, a dependency outage, a bad deploy. You need to replay missed events without causing duplicate side effects.

Code example
Copy code
DEAD LETTER QUEUE (DLQ) REPLAY FLOW

+--------------------+ Failed Processing +-----------------------+
| Proxy Gateway | ------------------------------> | Dead Letter Queue |
| Buffer Engine | | (Storage / Redis DLQ) |
+--------------------+ +-----------------------+
|
| Manual / Automated Trigger
v
+--------------------+ Replay Payload +-----------------------+
| Target Consumer | <------------------------------ | Replay CLI / Worker |
| (api-v2.internal) | (With Idempotency Key) | Script |
+--------------------+ +-----------------------+
Idempotent Handlers
Every payload needs a unique event ID (evt_12345, or a provider-supplied X-Event-ID/X-GitHub-Delivery/MessageSid), which your handler checks before doing any real work:

Code example
Copy code
app.post('/webhooks/stripe', async (req: Request, res: Response) => {
const eventId = req.headers['x-proxy-event-id'] || req.body.id;

const isProcessed = await redis.sIsMember('processed_events', eventId);
if (isProcessed) {
console.log([DUPLICATE IGNORING] Event ${eventId} was already processed.);
return res.status(200).json({ status: 'ignored_duplicate' });
}

await processPaymentEvent(req.body);

await redis.sAdd('processed_events', eventId);
await redis.expire('processed_events', 604800); // 7-day TTL
return res.status(200).json({ status: 'success' });
});
This matters more than it sounds like it should: providers guarantee at-least-once delivery, never exactly-once, so duplicate deliveries are expected behavior even outside a migration — not a bug specific to your proxy.

Automated DLQ Replay
Code example
Copy code
// replay-dlq.ts
import axios from 'axios';
import { createClient } from 'redis';

const redis = createClient({ url: process.env.REDIS_URL });

async function replayFailedWebhooks() {
await redis.connect();
console.log('--- STARTING WEBHOOK REPLAY PROCESS ---');
let replayedCount = 0;

while (true) {
const rawItem = await redis.rPop('webhook_dlq');
if (!rawItem) break;

const event = JSON.parse(rawItem);
console.log(`[REPLAYING] Event ID: ${event.id} to Target: ${event.targetUrl}`);

try {
  const res = await axios.post(`${event.targetUrl}/${event.path}`, event.body, {
    headers: { ...event.headers, 'X-Webhook-Replay': 'true', 'X-Proxy-Event-ID': event.id },
  });

  if (res.status >= 200 && res.status < 300) {
    replayedCount++;
    console.log(`[REPLAY SUCCESS] Event ID: ${event.id}`);
  } else {
    await redis.lPush('webhook_dlq_failed_twice', JSON.stringify(event));
  }
} catch (err: any) {
  console.error(`[REPLAY FAILED] Event ID: ${event.id} Error: ${err.message}`);
  await redis.lPush('webhook_dlq_failed_twice', JSON.stringify(event));
}

}

console.log(--- REPLAY COMPLETE: ${replayedCount} events processed successfully. ---);
process.exit(0);
}

replayFailedWebhooks();

1. Direct Migration vs. Proxy Architecture Feature / Metric Direct Endpoint Update Proxy / Gateway Architecture Downtime duration Minutes to days depending on connection reuse and provider retry behavior Effectively instantaneous (in-memory cutover) Data loss risk High — depends entirely on the provider's retry policy (GitHub retries zero times automatically) Low — durable buffering before forwarding Provider re-configuration Required for every environment change Set once, permanently Testing capability Blind production cutover Shadow mode + canary dual-forwarding Failure recovery Dependent on provider's own retry schedule and retention window Local DLQ + on-demand replay Signature handling Secrets exposed per endpoint Centralized passthrough
2. Best Practices Checklist Decouple ingestion from processing. Never expose application servers directly to third-party webhooks. Capture raw payloads first, before parsing or executing business logic. Enforce idempotency using event IDs stored with a TTL in a fast key-value store. Keep ingestion response times low — well under each provider's timeout (Shopify: 5s per attempt; Twilio voice: 15s hard limit). Preserve raw bytes and headers for HMAC verification; don't re-serialize JSON in the proxy. Alert on DLQ growth via PagerDuty, Datadog, or Slack thresholds. Check each provider's current retry policy before you rely on it — these change (Shopify's 2024 policy update is a good example) and are not uniform across vendors. Test your replay path regularly, not just during an actual incident. FAQ Q: What's the core idea behind a webhook proxy/gateway? It's a static, high-availability URL between third-party dispatchers and your backend. You register the proxy's URL with providers once; internal routing changes happen in your own config, not in each vendor's dashboard.

Q: How do I keep signature verification working through a proxy? Forward the exact raw, unparsed body and original headers. Re-serializing JSON breaks the hash comparison on the receiving end.

Q: What happens if my backend goes down mid-migration? With buffering in place, incoming webhooks are acknowledged and stored in a durable buffer or DLQ. Once your backend recovers, run a replay job to resend them in order.

Q: Does every provider retry failed webhooks the same way? No — and this is the most important correction to older advice on this topic. Stripe retries for up to 3 days with exponential backoff. Shopify currently retries 8 times over 4 hours (down from 19 over 48 hours before September 2024). GitHub does not automatically retry at all — you must manually redeliver within a 3–7 day window. Always check current docs for any provider you depend on rather than assuming a shared standard.

Sources
Stripe — Webhook retry behavior (Svix review)
Shopify — Updates to webhook retry mechanism (official changelog)
Shopify — Troubleshoot webhooks (official docs)
GitHub — Redelivering webhooks (official docs)
Twilio — Webhooks (HTTP callbacks): Connection Overrides (official docs)
Standard Webhooks specification
Svix — What is a webhook signature?
Best Webhook Management APIs in 2026 — APIScout