Munchable has two webhook endpoints: one for Stripe, one for inbound email. Both are at-least-once, both have side effects that must not happen twice, and both start the same way.
const key = `webhook:${provider}:${eventId}`;
const claimed = await redis.set(key, Date.now().toString(), { nx: true, ex: ttlSeconds });
if (claimed === null) return { isFirstDelivery: false, degraded: false };
SET NX with a TTL. First delivery wins the key, every retry sees it and returns early. The TTL only has to outlast the provider's retry window, which is why this is a Redis key with an expiry rather than a table of event ids that grows forever.
The interesting question is what number goes in ttlSeconds, and the obvious answer is wrong.
The obvious TTL makes a crash permanent
Svix's retries taper off well inside a day, so 24 hours is the correct dedupe horizon. Set the claim to 24 hours and you get this failure:
- The event arrives, wins the key, and the key now says "handled" for the next day.
- The handler dies halfway. A timeout, an out of memory, a platform kill: anything that does not run your code afterwards.
- The provider retries. The key is there. The retry returns early.
- Every subsequent retry does the same, and then the retries stop.
The event is gone, nothing failed loudly, and the only artifact is a Redis key asserting that work happened which did not. The key written to prevent duplicate work has become a tombstone for work that was never done.
Three states instead of one
So the claim is written short, and promoted:
/** Svix retries taper off well inside 24h, so that is the dedupe horizon. */
const DEFAULT_TTL_SECONDS = 24 * 60 * 60;
/** Short window for the FIRST claim ... */
export const PROCESSING_TTL_SECONDS = 120;
and the route drives all three transitions:
const claim = await claimWebhookEvent('resend', svixId, PROCESSING_TTL_SECONDS);
if (!claim.isFirstDelivery) return NextResponse.json({ received: true, skipped: true });
try {
await handleEmailReceived(data);
await confirmWebhookEvent('resend', svixId); // promote to 24h
return NextResponse.json({ received: true });
} catch (error) {
await releaseWebhookEvent('resend', svixId); // delete, let the retry in
return NextResponse.json({ error: 'Webhook processing failed' }, { status: 500 });
}
Each state covers a different kind of failure, and the split exists because one of them cannot run any code at all:
-
Success:
confirmoverwrites the key with the full 24 hour TTL. NoNXon that write, so it always lands rather than silently doing nothing because the key it is extending already exists. -
A caught failure:
releasedeletes the key, so the retry that follows a 500 is treated as a first delivery rather than as a duplicate. This is the fast path back. - A crash or a timeout: no code runs, so there is nothing to delete. The short TTL is the entire mechanism. Two minutes later the key is gone and the next retry gets through.
That third case is why the first TTL exists, and the second case is why it does not have to be clever. The short window is a floor under the crash path, not the primary recovery route.
What the 120 is tied to
Not the handler's average duration. The route declares its own ceiling:
// Allow up to 30s for the fetch, parse and forward pipeline.
export const maxDuration = 30;
The claim has to outlast the longest run the platform will permit, because a handler killed at its deadline is precisely the crash case above. 120 seconds is four times a hard 30 second ceiling, which leaves room for the ceiling to be raised without the claim quietly becoming shorter than the work it is covering. Tie the number to the ceiling, not to the p99, and it stops being a number to re-tune.
It fails open, and the handlers have to be idempotent anyway
With no Redis configured, or with Redis throwing, the claim returns isFirstDelivery: true:
// Deliberately fails OPEN when Redis is unavailable: dropping a real email is
// worse than forwarding a duplicate.
Both consumers can take a duplicate, for different reasons.
The Stripe handler's write is an upsert with an ordering guard on it, so replaying an event cannot move an entitlement backwards: the update's setWhere only matches when the stored event timestamp is not newer than the one being applied. That guard has its own story, and it is what makes the idempotency key an optimisation rather than a correctness requirement.
The inbound mail handler's duplicate is a second copy of an email in an inbox, which is an annoyance. The alternative, a message that was never forwarded and never will be, is a lost customer conversation. You can see the other end of that pipeline at munchable.app/support, where the web form and inbound mail land in the same queue.
The doc comment ends with "Handlers must be idempotent", and that sentence is load bearing. If the handler is not idempotent, there is no correct TTL: short and a crash duplicates, long and a crash drops. The key choice is only interesting once the handler can survive being run twice.
A field nothing reads
The result type carries a second field:
export interface IdempotencyResult {
isFirstDelivery: boolean;
degraded: boolean;
}
degraded is true on both fail-open paths: no Redis, or Redis threw. It is set in three places and read in none. Both routes look at isFirstDelivery and nothing else.
The information is not lost, because the library logs an error on each of those paths. But the route is where it could do something: stamp a header, bump a counter, decide that an unguarded replay of a payment event deserves different treatment from an unguarded replay of a forwarded email. Until it does one of those, the field is a comment with a type annotation. I am writing it down here rather than deleting it, because the decision I actually want is a metric on it, and the honest status today is that the field is aspirational.
Related, from the same two endpoints: a Resend webhook subscription belongs to the account rather than the domain, which is a different way to process one event twice, and the Redis cache that measured one string and stored another, which is what happens when a key's lifetime is wrong in the other direction.
Top comments (0)