DEV Community

Alex Neamtu
Alex Neamtu

Posted on • Originally published at sendrec.eu

How We Made SendRec Billing Webhooks Bulletproof

#go #postgres #webhooks #opensource

We shipped SendRec v1.56.0 with idempotent webhook processing for our Creem billing integration. Here's how we made it reliable and what we learned about subscription lifecycle events.

The problem

Payment webhooks are unreliable by nature. The provider retries on failure, network issues cause duplicates, and your server might process the same event twice. Without deduplication, a retried subscription.active webhook could overwrite state that a later subscription.canceled event already set.

Creem retries webhooks 5 times with progressive backoff (30s, 1m, 5m, 1h). Each event includes a unique id field (prefixed evt_) and a created_at timestamp. We needed to use these to prevent double-processing.

Atomic deduplication with PostgreSQL

We created a creem_webhook_events table that serves as both a dedup mechanism and an audit log:

CREATE TABLE creem_webhook_events (
    event_id TEXT PRIMARY KEY,
    event_type TEXT NOT NULL,
    user_id UUID,
    payload JSONB NOT NULL,
    created_at TIMESTAMPTZ NOT NULL,
    processed_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
Enter fullscreen mode Exit fullscreen mode

The dedup uses PostgreSQL's INSERT ... ON CONFLICT DO NOTHING and checks RowsAffected():

tag, err := h.db.Exec(ctx,
    `INSERT INTO creem_webhook_events
       (event_id, event_type, user_id, payload, created_at)
     VALUES ($1, $2, $3, $4, $5)
     ON CONFLICT (event_id) DO NOTHING`,
    payload.ID, payload.EventType, userID, body, createdAt,
)
if err != nil {
    return false, err
}
return tag.RowsAffected() == 0, nil
Enter fullscreen mode Exit fullscreen mode

If RowsAffected() returns 0, the event was already recorded — it's a duplicate. This is atomic: no race conditions, no distributed locks, no Redis. Just PostgreSQL doing what it does best.

Handling the full subscription lifecycle

The initial billing integration only handled two events: subscription.active and subscription.canceled. Real subscription lifecycles are more complex. We now handle:

  • subscription.active / subscription.paid — activate or renew the Pro plan
  • subscription.canceled / subscription.scheduled_cancel — log it, but keep Pro access until expiry
  • subscription.expired — downgrade to free (the actual cutoff)
  • subscription.past_due — log a warning, Creem is retrying payment
  • refund.created — immediate downgrade to free
  • dispute.created — log for manual review

The key insight: cancellation and expiration are separate events. When a user cancels, they should keep Pro access until their billing period ends. Only subscription.expired (or a refund) triggers the actual downgrade.

Fixing the cancellation UX

We found two UX issues after testing the cancel flow:

Problem 1: After canceling, refreshing the page showed "Upgrade to Pro" instead of acknowledging the cancellation. The backend returned plan: "free" with no indication that the user was in a grace period.

Fix: We added subscriptionStatus to the billing API response, populated from Creem's subscription API. The frontend now shows three states:

  • Active Pro — cancel button and manage subscription link
  • Canceled (grace period) — "Your subscription has been canceled. You have access to Pro features until the end of your billing period."
  • Free — upgrade card

Problem 2: After checkout, Creem redirects back with query parameters in the URL (?billing=success). These persisted across page navigations.

Fix: On mount, the Settings page checks for ?billing=success, shows a success message, and cleans the URL with window.history.replaceState.

Handling edge cases

We also fixed the cancel endpoint returning a 500 error when the subscription was already canceled on Creem's side. Creem returns a 400 with "Subscription already canceled" — we now treat that as success:

if resp.StatusCode == http.StatusBadRequest &&
    strings.Contains(string(respBody), "already canceled") {
    return nil
}
Enter fullscreen mode Exit fullscreen mode

Testing

The webhook handler has 15 Go tests covering: signature verification, all event types, duplicate event rejection, and the billing API response. The cancel edge case has its own test against a mock Creem server. Frontend tests verify the three billing UI states.

What we'd do differently

If we were starting fresh, we'd add the event log table from day one. Retrofitting idempotency is straightforward with PostgreSQL's upsert, but it's even easier when you design for it upfront. The audit trail alone — being able to query every webhook event with its full payload — has already been valuable for debugging.

SendRec is open source and self-hostable. Check it out at github.com/sendrec/sendrec.

Top comments (0)