Short answer: use the FIFO queue's five-minute dedupe window to reduce a burst of repeated shipment events, but use a durable idempotency record and an outbox to decide whether a webhook delivery intent already exists. The queue can preserve ordering for a short interval; it cannot be the audit system for a healthtech notification that may be retried hours later.
That is the distinction I want in an architecture decision record. A shipment update may need to fan out to many subscribers, while the sender, queue, worker, and receiver all have different retry boundaries. Exactly-once is a useful mindset for the database transition, not a promise that one HTTP request crossed several independent systems once.
The data contract comes before the queue
The producer should derive one stable idempotency key from the upstream event identity, such as shipment-8472:delivered:v3, and reuse it for every retry of that same business event. It should not generate a new key because a network response was late. The consumer should atomically compare that key and a payload digest with its durable event table before creating delivery intents for subscribers.
The five-minute FIFO window is therefore an admission filter, not the source of truth. A duplicate inside the window is suppressed early. A duplicate arriving after the window is still harmless because the durable unique constraint finds the existing event. The same key paired with a different digest is a data-integrity conflict and should be held for investigation; silently treating two payloads as one event makes reconciliation guesswork.
Don't acknowledge early.
For a fanout, the event claim and the set of intended destinations belong in one transaction. A worker acknowledges the queue message only after that transaction commits. A separate delivery-attempt record then captures each outbound request, response status, and retry decision. This allows a receiver rate limit, including HTTP 429, to delay one attempt without creating another shipment event or another subscriber intent. The HTTP specification describes 429 as a rate-limit response and permits a Retry-After hint; the dispatcher should honor that hint when it is present.
A failure matrix for shipment fanout
The boundary is the consumer's commit. Before it commits, a process crash should cause the message to be delivered again. After it commits, the same event key should produce an already-seen result, while the outbox resumes any delivery intent whose attempts are incomplete. That sequence is more valuable than a boolean called processed, because an auditor needs to distinguish “accepted but not sent” from “sent and awaiting receiver confirmation.”
For healthtech data, the record should contain the event key, payload digest, subscriber identifier, intent state, attempt timestamps, response classification, and the retention policy identifier. Keep the payload itself as small as the workflow permits. Authentication material does not belong in a queue body; key access, rotation, and recovery need their own controls. OWASP's key-management guidance is a useful baseline, but it cannot select the retention period or permitted data fields for a particular organization.
The difficult example is a timeout after the receiver accepted the request. Retrying may be necessary, but it can also repeat a remote side effect. Imagine a worker sending a delivered-status update to three subscribers: the first response is lost, the worker records an unknown outcome, and the queue redelivers the message after the five-minute suppression period. If the application has only a queue key, it sees a new delivery opportunity and sends three more requests; if it has a durable intent and per-subscriber attempt history, it can identify the uncertain first attempt, apply the receiver's idempotency contract, and route the unresolved case to reconciliation. I've found that this distinction changes the incident question from “did the queue deliver?” to “which subscriber-side effect is still uncertain?” That is a much narrower investigation, even when the network gives no useful answer. The receiver therefore needs its own idempotency contract, or the application must reconcile an uncertain attempt before sending again. A local dedupe key cannot make an arbitrary remote endpoint transactional.
Record it.
Governance questions the architecture must answer
The right option depends on where ordering matters and where durable state already lives. The table is a decision aid, not a product ranking.
| Option | Good fit | Limitation to own |
|---|---|---|
| FIFO queue plus durable claim | Shipment updates need short-burst ordering and duplicate suppression | The queue window cannot cover late retries or provide an audit trail |
| Standard queue plus durable claim | Delivery order has no business meaning | At-least-once delivery still requires the same transactional claim |
| Database outbox with workers | The application already owns a relational event ledger | Polling, leases, and backpressure become application responsibilities |
| Workflow engine | Fanout includes joins, compensation, or several durable steps | It is unnecessary overhead for one delayed notification |
The catch is that a FIFO queue is not a workflow engine, and neither is it a substitute for a ledger. Use a standard queue when ordering has no business meaning. Use a workflow engine when the shipment process has a durable graph of steps. Stick with the smaller queue-plus-outbox design when the job is one event, many independent destinations, and a clear retry policy.
How can a Node.js webhook queue use a deduplication key for duplicate events?
The reader may be writing the service in Node.js, but the critical path below is Go because this article's implementation convention keeps code close to a typed transaction boundary. The language is incidental. The database contract is the point: a unique event key, a digest comparison, and outbox rows created in the same transaction.
package main
import (
"context"
"crypto/sha256"
"errors"
"fmt"
)
var ErrDuplicate = errors.New("event already claimed")
var ErrPayloadConflict = errors.New("key reused with a different payload")
type Store interface {
ClaimAndCreateIntents(ctx context.Context, key string, digest [32]byte, payload []byte, subscribers []string) error
}
func acceptShipment(ctx context.Context, store Store, key string, payload []byte, subscribers []string) (bool, error) {
digest := sha256.Sum256(payload)
err := store.ClaimAndCreateIntents(ctx, key, digest, payload, subscribers)
if err == nil {
return true, nil
}
if errors.Is(err, ErrDuplicate) {
return false, nil
}
return false, fmt.Errorf("claim shipment event %q: %w", key, err)
}
func main() {
_ = context.Background()
}
The adapter behind Store must make the unique-key conflict and digest comparison explicit. An already-seen matching event is safe to acknowledge. A payload conflict or a transient database error is not. The dispatcher later owns attempts, bounded exponential backoff, and Retry-After; it never calls ClaimAndCreateIntents again merely because a subscriber was slow.
Make the tests adversarial. Publish the same key twice inside five minutes, publish it again after five minutes, crash before acknowledgment, crash after the database commit, and deliver two different payloads under one key. Add a receiver timeout where the receiver might have committed. The expected result is one event claim, one intent per subscriber, and a visible attempt history, with ambiguous remote outcomes routed to reconciliation rather than silently erased.
When is a FIFO queue actually enough?
Reject queue-only deduplication for shipment notifications whose downstream effects matter to a patient, provider, or financial reconciliation process. It fails as soon as a retry arrives after the five-minute window, and it gives operators no durable answer when the worker dies between an outbound request and an acknowledgment.
An in-memory set is acceptable only as a best-effort optimization before the durable claim; it must never decide whether a delivery is business-new. A queue-only design is also reasonable for disposable telemetry where duplicates have no operational consequence and replay is not required. That is a different data contract.
Your mileage may vary on retention and regional controls. I'm not sure a generic healthtech checklist can determine which shipment fields are permissible, because that depends on the organization, contract, and jurisdiction. The engineering decision remains concrete: minimize sensitive payloads, preserve an immutable identity and attempt trail, and document who owns an uncertain remote side effect.
References
- MDN, “HTTP 429 Too Many Requests”: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
- OWASP, “Key Management Cheat Sheet”: https://cheatsheetseries.owasp.org/cheatsheets/Key_Management_Cheat_Sheet.html
Top comments (0)