Webhook Idempotency: Handling Duplicate Events

The Silent Killer: Why Webhook Idempotency Matters

You deploy a webhook handler that charges a customer $50 on every payment.completed event. The payment provider retries the webhook once due to a temporary network timeout. Your handler processes it twice. Now your customer is charged $100, and you're fielding an angry support ticket.

This is the core problem with webhook idempotency: duplicate events are inevitable. Network failures, timeouts, provider retries, and even your own restarts can cause the same event to be delivered multiple times. Without webhook idempotency duplicate events handling, you risk corrupting data, double-charging users, or creating orphaned records. The solution isn't to prevent duplicates—it's to make your handler idempotent, so processing the same event twice produces the same result as processing it once.

Prerequisites

• A webhook provider account (Stripe, GitHub, Shopify, or similar) with webhook delivery logs
• Familiarity with HTTP POST requests and JSON payloads
• A backend runtime: Node.js, Python, or similar
• A database or cache layer (PostgreSQL, Redis) to track processed event IDs
• Basic understanding of HTTP status codes and retry semantics

Implementing Idempotent Webhook Handling

Webhook idempotent handling relies on three principles:

1. Unique event identifiers — every webhook payload includes a stable, unique ID
2. Deduplication storage — track which event IDs you've already processed
3. Idempotent operations — structure your business logic so re-execution is safe

Step 1: Extract and Validate the Event ID

Every webhook provider includes a unique event identifier in the payload. Stripe uses id, GitHub uses delivery, Shopify uses X-Shopify-Webhook-Id. Always extract this first.

// Express.js example
const express = require('express');
const app = express();

app.post('/webhook/stripe', express.json(), async (req, res) => {
  // Extract the event ID — this is your deduplication key
  const eventId = req.body.id;

  if (!eventId) {
    return res.status(400).json({ error: 'Missing event ID' });
  }

  console.log(`Processing event: ${eventId}`);

  try {
    // Check if we've already processed this event
    const alreadyProcessed = await checkEventProcessed(eventId);

    if (alreadyProcessed) {
      console.log(`Event ${eventId} already processed, skipping`);
      return res.status(200).json({ status: 'already_processed' });
    }

    // Process the event
    await handleStripeEvent(req.body);

    // Mark as processed
    await markEventProcessed(eventId);

    res.status(200).json({ status: 'processed' });
  } catch (error) {
    console.error(`Error processing event ${eventId}:`, error);
    res.status(500).json({ error: 'Internal server error' });
  }
});

Step 2: Store Processed Event IDs

Use a database or cache to track which events you've already handled. For high-volume webhooks, Redis is ideal; for lower volume, a simple database table works.

PostgreSQL approach:

CREATE TABLE webhook_events (
  id SERIAL PRIMARY KEY,
  event_id VARCHAR(255) UNIQUE NOT NULL,
  event_type VARCHAR(100) NOT NULL,
  processed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  payload JSONB,
  INDEX idx_event_id (event_id)
);

Redis approach (Node.js):

const redis = require('redis');
const client = redis.createClient();

async function checkEventProcessed(eventId) {
  const key = `webhook:${eventId}`;
  const exists = await client.exists(key);
  return exists === 1;
}

async function markEventProcessed(eventId) {
  const key = `webhook:${eventId}`;
  // Set with 24-hour expiration to avoid unbounded memory growth
  await client.setEx(key, 86400, 'processed');
}

Step 3: Make Your Business Logic Idempotent

Even with deduplication, structure your operations to be idempotent. Avoid operations that fail if repeated:

Bad (not idempotent):

async function handlePaymentCompleted(event) {
  // This creates a duplicate charge if called twice
  const charge = await stripe.charges.create({
    amount: event.data.amount,
    currency: event.data.currency,
  });

  await db.orders.update(event.data.order_id, {
    status: 'paid',
    charge_id: charge.id,
  });
}

Good (idempotent):

async function handlePaymentCompleted(event) {
  const orderId = event.data.order_id;

  // Check if order is already marked as paid
  const order = await db.orders.findById(orderId);
  if (order.status === 'paid') {
    console.log(`Order ${orderId} already paid, skipping charge`);
    return;
  }

  // Use idempotency keys if the provider supports them
  const charge = await stripe.charges.create(
    {
      amount: event.data.amount,
      currency: event.data.currency,
    },
    {
      idempotencyKey: `order-${orderId}-${event.data.created}`,
    }
  );

  await db.orders.update(orderId, {
    status: 'paid',
    charge_id: charge.id,
  });
}

Step 4: Return 200 Immediately, Process Asynchronously

Always return HTTP 200 to the webhook provider before your business logic completes. This prevents provider retries while you're still processing:

app.post('/webhook/stripe', express.json(), async (req, res) => {
  const eventId = req.body.id;

  // Return 200 immediately
  res.status(200).json({ received: true });

  // Process asynchronously
  setImmediate(async () => {
    try {
      const alreadyProcessed = await checkEventProcessed(eventId);
      if (!alreadyProcessed) {
        await handleStripeEvent(req.body);
        await markEventProcessed(eventId);
      }
    } catch (error) {
      console.error(`Async error processing ${eventId}:`, error);
      // Log to monitoring system for manual review
    }
  });
});

Common Errors and Fixes

Error 1: "Duplicate Key Violation" in Your Event Table

Error message:

ERROR: duplicate key value violates unique constraint "webhook_events_event_id_key"

Root cause: A race condition where two requests for the same event ID hit your database simultaneously, both pass the deduplication check, and both try to insert.

Fix: Use database-level locking or atomic operations:

// PostgreSQL with advisory lock
async function processWebhookAtomic(eventId, handler) {
  const lockId = hashEventId(eventId); // Convert to numeric hash

  const client = await db.connect();
  try {
    await client.query('SELECT pg_advisory_lock($1)', [lockId]);

    const alreadyProcessed = await client.query(
      'SELECT 1 FROM webhook_events WHERE event_id = $1',
      [eventId]
    );

    if (alreadyProcessed.rows.length > 0) {
      return { status: 'already_processed' };
    }

    await handler();

    await client.query(
      'INSERT INTO webhook_events (event_id, processed_at) VALUES ($1, NOW())',
      [eventId]
    );

    return { status: 'processed' };
  } finally {
    await client.query('SELECT pg_advisory_unlock($1)', [lockId]);
    client.release();
  }
}

Error 2: "Event Processed Twice" Despite Deduplication

Symptom: You see duplicate charges or records even with deduplication in place.

Root cause: Your deduplication check happens after you've already made an external API call, or you're not using the event ID correctly.

Fix: Check deduplication before any side effects:

// WRONG: Check happens after Stripe call
const charge = await stripe.charges.create({ ... });
const alreadyProcessed = await checkEventProcessed(eventId);

// CORRECT: Check first
const alreadyProcessed = await checkEventProcessed(eventId);
if (alreadyProcessed) return;
const charge = await stripe.charges.create({ ... });

Also verify you're extracting the event ID correctly for your provider:

// Stripe: event.id
// GitHub: headers['x-github-delivery']
// Shopify: headers['x-shopify-webhook-id']
const eventId = req.body.id || req.headers['x-github-delivery'];

Frequently Asked Questions

Q: How long should I store processed event IDs?

A: At minimum, store them for the provider's maximum retry window (typically 24–72 hours). For audit purposes, keep them longer. Redis with 30-day expiration is reasonable; database retention depends on your compliance requirements. Stripe events are immutable and include a timestamp, so you can safely delete old records after 90 days.

Q: What if my database is down when a webhook arrives?

A: Return 5xx status immediately so the provider retries. Use a local queue (file, in-memory buffer, or local SQLite) as a fallback to capture the event, then process it when the database recovers. For development and testing, Anonymily's webhook inspector captures webhooks in the cloud even when localhost is down, letting you replay them once your handler is ready.

Q: Can I use just timestamps instead of event IDs for deduplication?

A: No. Multiple events can have the same timestamp (millisecond precision), and timestamps don't guarantee uniqueness across retries. Always use the provider's event ID. If a provider doesn't include one, generate a hash of the payload content, but this is fragile if the provider modifies the payload between retries.

Testing Your Idempotency

During local development, test duplicate delivery manually. Use Test Webhooks Locally Without ngrok to capture real webhook payloads, then replay them to verify your handler processes them only once.

Start by running npx @anonymilyhq/cli listen 3000 to get a stable endpoint URL. Trigger the same webhook twice from your provider's dashboard, and confirm your deduplication logic prevents double-processing. For Stripe, GitHub, or Shopify testing, the Pro tier offers provider-signed synthetic events so you can test without manual triggers.

Webhook idempotency isn't optional—it's the difference between a reliable integration and a production incident. Build it in from the start.