Duplicate auth emails are rarely caused by one big bug. More often, the API is fine, the queue is fine, and the worker logic is almost fine. Then a timeout lands between "picked job" and "marked sent", another worker retries, and a user gets two reset links. That kind of issue is annoyng because each individual component looks healthy in isolation.
When you maintain Authentication flows, the fix I trust most is not more retry logic. It is a lease on the email job itself. Before a worker sends anything, it claims the row for a short window, records who owns it, and only then continues. The result is much calmer behavior under retries, crashes, and horizontal scaling.
Why email workers still duplicate sends
Most duplicate sends happen in one of these moments:
- a worker crashes after calling the provider but before updating the database
- two workers poll the same ready job at nearly the same time
- a retry policy has no clear ownership boundary
- delivery polling from a
tempail mailinbox gets mixed with the original send timeline
This is where a lease is more useful than a plain status column. pending and sent are not enough once you have more than one consumer. You also need "claimed until" and "claimed by", so another worker can see that the job is already in progress and back off.
If you already use database-backed signup delivery, leasing is a natural next step. The outbox tells you what should be sent. The lease tells you who is allowed to send it right now.
The lease model I prefer
My baseline table usually has these fields:
idtemplaterecipientpayloadstatuslease_ownerlease_expires_atsent_atattempt_count
The key rule is simple: only rows with an expired lease, or no lease at all, are eligible to be claimed. PostgreSQL makes this clean with FOR UPDATE SKIP LOCKED, which is designed for concurrent workers that should not block each other unnecessarily (https://www.postgresql.org/docs/current/sql-select.html).
I also keep the lease short, usually 30 to 90 seconds. Long enough for one delivery attempt, short enough that a crashed worker does not stall the pipeline for ages. If your provider latency is high, measure first instead of guessing. Google's distributed systems guidance has long pushed engineers toward observable time budgets rather than wishful defaults (https://sre.google/sre-book/monitoring-distributed-systems/). Same lesson here, realy.
A PostgreSQL query that claims work safely
Here is the shape I like for claiming one job:
with next_job as (
select id
from email_jobs
where status = 'pending'
and (lease_expires_at is null or lease_expires_at < now())
order by created_at
for update skip locked
limit 1
)
update email_jobs ej
set lease_owner = $1,
lease_expires_at = now() + interval '45 seconds',
attempt_count = attempt_count + 1
from next_job
where ej.id = next_job.id
returning ej.id, ej.recipient, ej.template, ej.payload, ej.attempt_count;
Three details matter a lot:
- Claim and update in one statement.
- Increment attempts when the lease is taken, not after the send.
- Return the payload immediately so the worker does not re-read stale state.
That contract makes Node.js workers much easier to reason about. A worker either owns the lease or it does not. There is less fuzzy middle state, which is where duplicated behavior loves to live.
How to keep retries boring
After the send succeeds, update the row with status = 'sent', clear the lease, and store a provider message ID if you have one. If the send fails with a retryable error, leave status as pending and let the lease expire naturally. If it fails permanently, mark it failed and stop looping.
For Auth flows, I also recommend tying the email job to a server-side token version, not just a raw recipient. If two password reset requests arrive quickly, you want the newest token to invalidate the older one, even if an older job sneaks through later. The delivery pipeline and the token model should protect each other.
When you test this behavior, isolated inboxes help, but they are only supporting evidence. I prefer to combine the lease audit trail with contract checks around email APIs, then use a scoped inbox from temp mail so when I want to verify that exactly one human-visible message arrived. That balance keeps the design backend-first instead of mailbox-first.
A few logs make the system easier to trust:
- job id
- lease owner
- lease expiration
- token version
- provider message id
If one of those is missing, debugging gets slower than it should be. You can still fix the incident, but you will burn more team energy doing it, which is never great and kinda avoidable.
Q&A
Why not just mark rows as processing?
Because processing without an expiry becomes sticky after crashes. A lease says "processing until this exact time", which is much safer.
Should the lease duration match provider timeout?
Close to it, yes, but leave a little margin for serialization and DB update time. Do not make it huge "just in case". That tends to hide dead workers.
Does this replace idempotency keys?
No. Provider-side idempotency is still useful. The lease protects your worker coordination layer; provider idempotency protects the outbound call. Together they fail much more gracefuly than either one alone.
Top comments (1)
I particularly like how you've outlined the problem of duplicate auth emails and the solution of using PostgreSQL row leases to prevent them. The example query you provided, which claims work safely using
FOR UPDATE SKIP LOCKED, is really helpful in illustrating how to implement this approach. One thing that caught my attention was the choice of lease duration, which you've set to 30 to 90 seconds - have you found that this range works well in practice, or are there any specific considerations that would lead you to adjust this timeframe?