DEV Community

Daniel Pertu
Daniel Pertu

Posted on

Our settlement job and our checkout webhook can reach for the same reward row, and the one that proceeds is whoever gets it back from an UPDATE

Munchable is a barcode scanner for people with digestive conditions. When you scan something it has no ingredient data for, you can photograph the label to add it, and doing that earns a discount on your next Premium bill. The terms state the deal plainly in section 9 of munchable.app/terms: 2% off per product, up to 50 in a month, which makes that bill free. The plan it discounts is in the pricing section.

A promise like that has to be kept by something. That something is one scheduled route, 326 lines, running at 04:00 UTC on the 8th of each month for the previous calendar month, and almost all of its complexity is about being safe to run twice.

Why the 8th and not the 1st

The lag is the cheapest anti-abuse control in the system. Nothing is credited on the day it is contributed, and a row that other scanners report as junk in the meantime drops out of the count before any money moves. Settlement in arrears means a reversal is usually a row that never became a reward rather than a discount that has to be clawed back off a paid invoice.

Pass one: earning, made idempotent by a constraint

The earn pass walks the contributors with eligible work in that month and inserts one row each:

await db.insert(contributionRewards)
  .values({ id: ulid(), userId, period, percent: progress.percent, status: 'earned' })
  .onConflictDoNothing({ target: [contributionRewards.userId, contributionRewards.period] })
  .returning({ id: contributionRewards.id });
Enter fullscreen mode Exit fullscreen mode

There is a unique index on (user, period), so a second run of the same month inserts nothing and returns nothing. No read-then-write, no "have I already done this" query, and the count of what the run actually created is whatever RETURNING hands back. Re-running the earn pass by hand is free, which matters because the first thing anybody does with a monthly job is run it by hand.

Pass two: five statuses, and nobody writes blind

A reward row moves through five statuses: earned, applying, applied, no_subscription, failed. There is a transition table in the library next to them, and two sets derived from it: the statuses this job may claim, and the statuses the checkout webhook may apply.

Those two actors are the reason for everything that follows. The monthly job applies a reward to a live subscription. The checkout webhook applies one to a subscription being created right now, because somebody who earned a discount while on the free plan should see it on their first invoice. Both can touch the same row within the same second.

So the row is claimed, not read:

/** Single-actor claim: only the caller that gets the row back proceeds. */
async function claim(id: string, allowed: readonly RewardStatus[]): Promise<boolean> {
  const rows = await db.update(contributionRewards)
    .set({ status: 'applying', updatedAt: new Date() })
    .where(and(eq(contributionRewards.id, id), inArray(contributionRewards.status, [...allowed])))
    .returning({ id: contributionRewards.id });
  return rows.length > 0;
}
Enter fullscreen mode Exit fullscreen mode

One statement, conditional on the status, returning what it changed. Two callers race, one gets a row, the other gets an empty array and stops. No advisory lock, no transaction spanning a network call to Stripe, and crucially no window between the read and the write.

Every later write is the same shape, with the status it believes it is leaving passed in:

async function setStatus(
  id: string,
  from: RewardStatus,
  patch: Partial<...> & { status: RewardStatus },
): Promise<boolean>
Enter fullscreen mode Exit fullscreen mode

Calling setStatus(row.id, 'applying', { status: 'applied' }) cannot mark a row applied if something else has already moved it on. The compare-and-set is in the where, and the boolean it returns is what the summary counts. It is a small discipline that removes a whole category of question: there is no code path that can write a status based on a value read earlier in the request.

Reconcile first, retry second

The dangerous failure here is not a missing discount. It is a double one. A coupon applied twice to a subscription is money, and the customer has no reason to tell us.

So before any retry the job asks Stripe whether the thing already happened, in two places. If the row rode along on a Checkout Session, it asks whether that session consumed the reward, and if so marks it applied without applying anything. A session still in progress is left strictly alone. If the row previously got as far as naming a subscription, it asks whether the reward is already on it.

The attempt itself carries the row id as its idempotency key, so even if both reconcile reads fail and the job tries again, Stripe collapses the duplicate. Three layers, all for one thing: a given reward may discount a bill exactly once.

There is also a case that looks like an error and is not. If a reward coupon is already pending on the subscription, the row goes back to earned and waits for next month rather than stacking:

// A reward coupon is still pending on the subscription: never stack.
await setStatus(row.id, 'applying', { status: 'earned' });
summary.pending += 1;
Enter fullscreen mode Exit fullscreen mode

The reward is not lost, it is queued. A row only ever leaves the system by being applied.

The deadline is shorter than the route

The route is allowed 300 seconds. The job stops starting Stripe calls at 240:

/** Stop starting Stripe calls after this much wall time (the route has 300 s). */
const DEADLINE_MS = 240_000;
Enter fullscreen mode Exit fullscreen mode

Rows past the deadline are counted as skipped and left exactly as they were. Nothing has to be resumable in any special sense, because the state of every piece of work is in the row itself, and a row in applying that nobody finished becomes claimable again after fifteen minutes. A run that dies, is cancelled, or hits the platform timeout costs one month's worth of nothing.

The whole thing runs inside a Redis lease, and if Redis is unavailable the job does not run at all rather than running unlocked. That is the opposite of the choice made for rate limiting in the same codebase, where an outage falls back to a memory limiter and keeps serving. The difference is what a mistake costs: a few unlimited requests, or a coupon applied twice.

What it returns

Counts. contributors, earned, applied, pending, noSubscription, failed, reconciled, skipped, and the elapsed time. No barcode, no product, nothing about anybody's health. A monthly job that touches billing rows for every contributor is exactly the sort of thing whose logs quietly become a second database, so its output is a summary and its comments say so.

If you want to see the user-facing half of this, read section 9 of the terms. It is four sentences, written before the job was, and every awkward branch above exists to keep one of them true.

Top comments (0)