Short answer: treat a custom sending domain as production infrastructure, rotate DKIM keys with an overlap window, keep SPF authorization narrow enough to audit, publish DMARC only after alignment is observable, and suppress permanent failures before another send is queued. The least complex reliable design is one domain-state record, one idempotent bounce consumer, and one alert tied to failed customer delivery rather than a green verification badge.
The page says transactional_email_hard_bounce_ratio crossed its threshold for a B2B SaaS tenant. The on-call sees 417 attempted password-reset messages, 38 permanent failures, a recent DNS change, and no evidence yet that the application stopped retrying invalid recipients. That is enough to act: freeze retries to confirmed permanent failures, preserve events for investigation, and inspect authentication alignment by recipient domain. A dashboard that merely says "domain verified" cannot answer the question that matters at 3 a.m.: what page fired, and which user-visible delivery path is failing?
What should the page prove?
A useful page proves impact and points toward a bounded action. It should identify the sending domain, message class, tenant, failure category, first-seen time, and whether suppression was applied. It should not fire because one DNS lookup briefly failed or because a reporting counter moved without affecting delivery.
Work backward from the permanent failures. A receiver can evaluate SPF-authenticated identifiers and DKIM-authenticated identifiers, while DMARC asks whether an authenticated identifier aligns with the domain visible to the user in the From field. RFC 7489 defines relaxed and strict alignment and describes p=none, quarantine, and reject policies. Those distinctions belong in telemetry because "DKIM passed" is incomplete when the signing domain does not align.
The earlier signal should therefore have been an alignment canary from every active selector and sending path, grouped by recipient domain, before the hard-bounce ratio changed.
DNS publication is necessary. It is not end-to-end proof.
How should Node.js verify SPF, DKIM, and DMARC during email rotation?
The application should not compress domain readiness into a Boolean. Use explicit states such as pending_dns, canary, active, rotating, and blocked, and require evidence before transitions. Keep the provider-facing API behind an adapter so that application policy does not inherit a vendor's status vocabulary. Node.js may own the workflow, but the contract is language-neutral.
| Transition | Required evidence | Failure action |
|---|---|---|
| pending_dns to canary | Expected SPF authorization, DKIM public key, and DMARC record are resolvable | Stay pending; do not send customer mail |
| canary to active | A test message shows aligned authentication through the real sending path | Keep production traffic on the previous domain |
| active to rotating | Old selector still validates and new selector is published | Continue signing with the old selector |
| rotating to active | New selector validates through the real path and old-key traffic has drained | Retire the old selector after the overlap policy |
| any state to blocked | Permanent-failure or alignment guardrail is breached | Halt affected traffic and retain suppression state |
SPF deserves separate caution. It authorizes hosts for an identity evaluated during SMTP; it does not replace DKIM, and changing it can affect every sender represented by that record. Keep ownership and review around the record, then test the actual envelope identity rather than assuming the visible From address is what SPF authenticates.
DKIM rotation is a two-key deployment. Publish the new selector first, confirm that it resolves, begin signing a canary stream with it, observe authentication and alignment, then shift traffic. Remove the old public key only after messages signed with it no longer need validation under the team's retention and delivery assumptions. No universal number of hours can be asserted here; queue lifetime, DNS behavior, and rollback requirements determine the overlap.
This state-machine approach has a limitation: it adds storage, reconciliation work, and deployment gates to a path that a small team may change only a few times per year. A low-volume internal system with no tenant-owned domains may reasonably use a reviewed runbook and recorded canary results instead. The trade-off changes once customers control DNS, multiple selectors coexist, or a rollback must happen while the on-call is handling delivery impact; then explicit state is easier to audit than a chain of chat messages and dashboard screenshots.
The consumer must suppress before it retries
Bounce handling is a data-integrity path. Accept events, authenticate them using the integration's documented mechanism, deduplicate them, classify the result, and commit suppression before acknowledging work. A temporary delivery failure may be retried under a bounded policy. A confirmed permanent recipient failure should block further sends to that recipient for the relevant scope until an explicit, audited reason clears it.
This Go example shows the core transaction boundary behind a Node.js-facing service. The event names are deliberately generic; adapters should map external payloads into this small internal contract.
package bounce
import (
"context"
"errors"
"time"
)
type Event struct {
ID string
TenantID string
Recipient string
Permanent bool
Occurred time.Time
}
type Store interface {
Seen(ctx context.Context, eventID string) (bool, error)
SuppressAndRecord(ctx context.Context, event Event) error
Record(ctx context.Context, event Event) error
}
func Consume(ctx context.Context, store Store, event Event) error {
if event.ID == "" || event.TenantID == "" || event.Recipient == "" {
return errors.New("incomplete bounce event")
}
seen, err := store.Seen(ctx, event.ID)
if err != nil {
return err
}
if seen {
return nil
}
if event.Permanent {
return store.SuppressAndRecord(ctx, event)
}
return store.Record(ctx, event)
}
The important method is SuppressAndRecord: both writes share one atomic boundary. If recording succeeds but suppression fails, a retry can send again to an address already known to be invalid. If suppression succeeds but event recording fails, investigation loses the evidence explaining why mail stopped. A queue acknowledgement comes after that method returns successfully.
Order matters.
Do not derive "permanent" from free-form text. Preserve the raw event, but map structured delivery status into a versioned internal taxonomy and quarantine unknown values for review. Also separate recipient invalidity from policy rejection and authentication failure; suppressing every rejection can hide a broken domain rollout behind an apparently clean queue.
Instrument the change that should have caught it
Add three linked observations: authentication results from controlled canaries, normalized delivery outcomes from production events, and suppression decisions from the transactional store. Correlate them with a deployment identifier and selector, but avoid recipient addresses in metric labels. High-cardinality or sensitive identifiers belong in access-controlled event records, not in the time-series key space.
The deployment gate can be small.
package gate
import "fmt"
type Evidence struct {
SPFAligned bool
DKIMAligned bool
SuppressionWritable bool
}
func Allow(e Evidence) error {
if !e.SuppressionWritable {
return fmt.Errorf("suppression store is not writable")
}
if !e.SPFAligned && !e.DKIMAligned {
return fmt.Errorf("no aligned authenticated identifier observed")
}
return nil
}
DMARC permits a message to satisfy alignment through an aligned SPF-authenticated identifier or an aligned DKIM-authenticated identifier, so the gate uses OR for those two signals. The gate does not claim that authentication guarantees inbox placement. It proves a narrower property: this sending path produced evidence consistent with the alignment mechanism before traffic moved.
Roll out by message class and tenant cohort. Password resets warrant a smaller blast radius and a faster rollback than a low-urgency digest because delayed account access is immediate user impact. Keep the previous selector and routing configuration available during the observation window, and record who approved each state transition.
Thresholds have an incident cost
A hard-bounce alert should combine a minimum event count with a ratio over a stated window, then segment by domain and message class. The count prevents one failure in a tiny sample from paging; the ratio prevents a large absolute number from being dismissed merely because overall volume is high. Choose both from historical distributions and an explicit error budget, then replay known incidents and ordinary traffic before enabling paging. The illustrative 38 failures among 417 attempts in the opening is incident context, not a recommended threshold.
Page on a condition that demands immediate human action: continuing attempts to recipients already classified as permanently invalid, or a sustained authentication-alignment failure on a critical message path.
Ticket a slow drift. Dashboard the rest.
False positives are not free. An over-sensitive threshold trains the on-call to distrust the alert, encourages broad traffic freezes, and can delay legitimate password-reset mail even when a single recipient domain is the only affected segment. An under-sensitive threshold burns sender reputation and repeats futile delivery attempts. The correct threshold is therefore an operational contract: documented impact, minimum sample, evaluation window, grouping keys, and a runbook action that can be completed without guessing.
The postmortem should ask why the system accepted a domain transition without end-to-end evidence and why suppression was not the first durable reaction to a permanent failure. Do not settle for "DNS was wrong." DNS is where the symptom surfaced; unsafe change control and missing outcome telemetry are the mechanisms that allowed customer impact.
Further reading
- RFC 7489, Domain-based Message Authentication, Reporting, and Conformance: https://datatracker.ietf.org/doc/html/rfc7489
- NIST SP 800-63B, Digital Identity Guidelines for Authentication and Lifecycle Management: https://pages.nist.gov/800-63-3/sp800-63b.html
Top comments (0)