You'd think events fire in the order things happen. They don't.
I was building a Paddle integration last week. Subscription billing, nothing fancy. Customer clicks buy, Paddle handles checkout, my app gets webhooks and updates the database.
The flow should be simple:
- Customer completes checkout
-
subscription.createdfires -
subscription.activatedfires - My app inserts a row on
.created, updates status on.activated
That's what I built. It worked great in my head.
What actually happened
In production, subscription.activated arrived before subscription.created about 30% of the time.
My handler did a database insert on subscription.created:
case 'subscription.created':
await db.insert(subscriptions).values({
paddleId: event.data.id,
status: 'created',
customerId: event.data.customer_id,
});
break;
And an update on subscription.activated:
case 'subscription.activated':
await db.update(subscriptions)
.set({ status: 'active' })
.where(eq(subscriptions.paddleId, event.data.id));
break;
When .activated arrived first, the update found zero rows. No error, no exception. The WHERE clause just matched nothing. The update silently did nothing.
Then .created arrived and inserted the row with status created. But the .activated event was already gone. So the subscription was stuck in created status forever.
Customers had paid. Paddle showed them as active. My app showed them as pending. Support tickets started coming in.
Why this happens
Paddle does not guarantee webhook delivery order. Their docs mention it briefly but it's easy to miss when you're focused on the API endpoints.
The events are fired from different internal services. subscription.created comes from the subscription service. subscription.activated comes from the billing service after payment confirmation. They are async. They race.
This is not unique to Paddle either. Stripe has the same problem with payment_intent.created vs charge.succeeded. Most payment providers have some version of this.
The fix
The handler needs to be idempotent and order-independent. Every event should be able to create or update:
case 'subscription.created':
case 'subscription.activated':
const status = event.event_type === 'subscription.activated'
? 'active' : 'created';
await db.insert(subscriptions)
.values({
paddleId: event.data.id,
status,
customerId: event.data.customer_id,
})
.onConflictDoUpdate({
target: subscriptions.paddleId,
set: {
status: sql`CASE WHEN ${status} = 'active' THEN 'active' ELSE ${subscriptions.status} END`
},
});
break;
The key parts:
- Both events can create the row if it doesn't exist
- On conflict,
activealways wins overcreatedregardless of arrival order - No silent failures, no missing updates
Testing this is the real problem
The ordering bug is easy to fix once you know about it. The hard part is reproducing it during development.
You can't control the order Paddle sends webhooks. You can't make .activated arrive first on demand. In testing you might run through the flow 20 times and the events always arrive in order. Then in production with real network latency and load, they don't.
I ended up testing this by sending the webhook events manually in the wrong order against a local sandbox. activated first, then created. Immediately saw the bug. Fixed it in 10 minutes.
The debugging in production took 4 hours.
If you're integrating Paddle or any payment provider with webhooks, test with events arriving in every possible order. Not just the happy path order from the docs.
Test Paddle webhook ordering in a sandbox →
The takeaway
Webhook events are not a queue. They are concurrent messages from different services that happen to be about the same thing. Your handler has to treat every event as potentially the first one it sees for that resource.
If your handler has an insert for one event type and an update for another, you have this bug. You just haven't hit it in production yet.
Top comments (0)